Run the same checks on Linux that the Windows scripts run

The product could only reach a GPU through one hard-coded route: `wsl.exe -d
Ubuntu`, a device named "RTX 4090", and launch scripts that existed only as
PowerShell. On this machine `WSL_DISTRO_NAME` is already set, so the local
shell is Linux and the wsl.exe hop was unnecessary; on any other Linux host
it is impossible. Every SGLang, vLLM and KTransformers profile was reported
"missing" here even with a working 4090.

profiles.py now asks the host what it is instead of assuming: `uses_wsl()`
is false inside WSL and only true when wsl.exe is actually on PATH,
`host_shell()` runs `bash -lc` directly or through `wsl.exe -d
settings.wsl_distro`, `gpu_available()` replaced `wsl_gpu_available()`, and
`launch_script()` picks the .sh twin on a Linux host and the .ps1 on a
Windows one. The schema field `wsl_gpu` became `gpu_available` because the
old name lied on Linux.

Training no longer fails on a card name. `cuda_bf16_training()` gates on
cuda_available plus bf16_supported, which the training environment already
records, so an RTX 5090 or a 4080 passes and a non-BF16 4090 still fails.

Fourteen .sh twins cover the six launch profiles plus install, verify,
serve, download and finetune; the six WSL-hopping .ps1 files read
ELY_EYE_WSL_DISTRO instead of Ubuntu. PowerShell parsing is not verified
here because no pwsh exists on this host.

ci.yml runs pytest, ruff, go build/vet/test/gofmt, the Vite build and a
docker build-plus-curl on every push. To make that lint gate mean anything,
the 17 pre-existing ruff findings across backend/src are cleared: 14
mechanical fixes plus three probe handlers that now name the failure
families they absorb (OSError, SubprocessError, ValueError) instead of
catching Exception.

Verification: pytest 239 passed; ruff check backend/src bench clean (17 at
HEAD); mypy on the 12 touched files 14 errors, identical at HEAD; pyright on
the same files 8 errors, identical at HEAD; bash -n clean on all 15 .sh.
This commit is contained in:
ZacharyZhang-NY committed 2026-10-08 02:12:29 -04:00
1 parent 36f40c4c59
commit 922d01202f
34 files changed
+482 -88

No files matched your search

+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
for model in Qwen/Qwen3.5-9B Qwen/Qwen3.5-0.8B Qwen/Qwen3-VL-Embedding-8B; do
.venv/bin/python ./scripts/download_models.py --model "$model"
done
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
.venv/bin/python -m pip install --upgrade --no-cache-dir --index-url https://download.pytorch.org/whl/cu130 torch==2.12.0+cu130 torchvision==0.27.0+cu130
.venv/bin/python -c 'import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.version.cuda)'
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -d .venv ]; then
python3 -m venv .venv
fi
.venv/bin/python -m pip install --upgrade pip setuptools wheel
.venv/bin/python -m pip install -e ./backend
.venv/bin/python -m pip install --upgrade --no-cache-dir unsloth unsloth_zoo
.venv/bin/python -m pip install --upgrade --no-cache-dir --index-url https://download.pytorch.org/whl/cu130 torch==2.12.0+cu130 torchvision==0.27.0+cu130
.venv/bin/python -m pip install --upgrade --no-cache-dir transformers==5.9.0 datasets==4.8.5 dill==0.4.1 multiprocess==0.70.19 fsspec==2026.2.0 safetensors==0.8.0rc0
(cd frontend && npm install)
[ -f .env ] || cp .env.example .env
.venv/bin/ely-eye status
+3 -2
View File
@@ -1,12 +1,13 @@
$ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the KTransformers Linux runtime."
}
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$command = @"
set -euo pipefail
cd "$wslRoot"
@@ -20,4 +21,4 @@ python -m sglang.launch_server \
--cpu-offload-gb 48
"@
wsl.exe -d Ubuntu -- bash -lc $command
wsl.exe -d $distro -- bash -lc $command
+15
View File
@@ -0,0 +1,15 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching SGLang." >&2
exit 1
fi
source .venv-linux/bin/activate
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
exec python -m sglang.launch_server \
--model-path Qwen/Qwen3.5-9B \
--port 8002 \
--context-length 1010000 \
--kt-cpuinfer 16 \
--cpu-offload-gb 48
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ -z "${ELY_EYE_GGUF_MODEL:-}" ]; then
echo "Set ELY_EYE_GGUF_MODEL to the absolute GGUF model path." >&2
exit 1
fi
server="${LLAMA_CPP_SERVER:-llama-server}"
if ! command -v "$server" >/dev/null 2>&1; then
echo "llama-server executable is missing. Build llama.cpp or set LLAMA_CPP_SERVER." >&2
exit 1
fi
exec "$server" \
--model "$ELY_EYE_GGUF_MODEL" \
--host 127.0.0.1 \
--port 8003 \
--ctx-size 262144 \
--flash-attn \
--cache-type-k q4_0 \
--cache-type-v q4_0
+3 -2
View File
@@ -1,13 +1,14 @@
$ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the SGLang Linux runtime."
}
$override = '{"text_config":{"rope_parameters":{"mrope_interleaved":true,"mrope_section":[11,11,10],"rope_type":"yarn","rope_theta":10000000,"partial_rotary_factor":0.25,"factor":4.0,"original_max_position_embeddings":262144}}}'
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$command = @"
set -euo pipefail
cd "$wslRoot"
@@ -22,4 +23,4 @@ python -m sglang.launch_server \
--json-model-override-args '$override'
"@
wsl.exe -d Ubuntu -- bash -lc $command
wsl.exe -d $distro -- bash -lc $command
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching SGLang." >&2
exit 1
fi
source .venv-linux/bin/activate
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
exec python -m sglang.launch_server \
--model-path Qwen/Qwen3.5-9B \
--port 8000 \
--tp-size 1 \
--mem-fraction-static 0.88 \
--context-length 1010000 \
--json-model-override-args '{"text_config":{"rope_parameters":{"mrope_interleaved":true,"mrope_section":[11,11,10],"rope_type":"yarn","rope_theta":10000000,"partial_rotary_factor":0.25,"factor":4.0,"original_max_position_embeddings":262144}}}'
+3 -2
View File
@@ -1,12 +1,13 @@
$ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the SGLang Linux runtime."
}
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$command = @"
set -euo pipefail
cd "$wslRoot"
@@ -23,4 +24,4 @@ python -m sglang.launch_server \
--speculative-num-draft-tokens 4
"@
wsl.exe -d Ubuntu -- bash -lc $command
wsl.exe -d $distro -- bash -lc $command
+18
View File
@@ -0,0 +1,18 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching SGLang." >&2
exit 1
fi
source .venv-linux/bin/activate
exec python -m sglang.launch_server \
--model-path Qwen/Qwen3.5-9B \
--port 8000 \
--tp-size 1 \
--mem-fraction-static 0.82 \
--context-length 262144 \
--reasoning-parser qwen3 \
--speculative-algo NEXTN \
--speculative-num-steps 3 \
--speculative-num-draft-tokens 4
+3 -2
View File
@@ -1,12 +1,13 @@
$ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the vLLM Linux runtime."
}
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$ropeScaling = '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}'
$command = @"
set -euo pipefail
@@ -23,4 +24,4 @@ python -m vllm.entrypoints.openai.api_server \
--rope-scaling '$ropeScaling'
"@
wsl.exe -d Ubuntu -- bash -lc $command
wsl.exe -d $distro -- bash -lc $command
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux-vllm/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching vLLM." >&2
exit 1
fi
source .venv-linux-vllm/bin/activate
exec python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen3.5-9B \
--host 127.0.0.1 \
--port 8001 \
--max-model-len 1010000 \
--enable-prefix-caching \
--gpu-memory-utilization 0.88 \
--trust-remote-code \
--rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}'
+3 -2
View File
@@ -1,12 +1,13 @@
$ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the vLLM Linux runtime."
}
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$command = @"
set -euo pipefail
cd "$wslRoot"
@@ -21,4 +22,4 @@ python -m vllm.entrypoints.openai.api_server \
--trust-remote-code
"@
wsl.exe -d Ubuntu -- bash -lc $command
wsl.exe -d $distro -- bash -lc $command
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux-vllm/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching vLLM." >&2
exit 1
fi
source .venv-linux-vllm/bin/activate
exec python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen3.5-9B \
--host 127.0.0.1 \
--port 8001 \
--max-model-len 262144 \
--enable-prefix-caching \
--gpu-memory-utilization 0.82 \
--trust-remote-code
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
max_steps="${MAX_STEPS:-37}"
rank="${RANK:-16}"
alpha="${ALPHA:-32}"
max_length="${MAX_LENGTH:-8192}"
gradient_accumulation_steps="${GRADIENT_ACCUMULATION_STEPS:-4}"
cartridge_id="${CARTRIDGE_ID:-}"
dataset=.ely_eye/training/hme_prd_sft.jsonl
adapter=.ely_eye/adapters/hme_core_lora_qwen35_9b
.venv/bin/ely-eye build-prd-training-data \
--prd-path ./PRD.md \
--output-path "$dataset" \
--max-sections 48
args=(train-lora "$dataset" "$adapter"
--max-steps "$max_steps"
--kind hme_core_lora
--rank "$rank"
--alpha "$alpha"
--max-length "$max_length"
--gradient-accumulation-steps "$gradient_accumulation_steps")
if [ -n "$cartridge_id" ]; then
args+=(--cartridge-id "$cartridge_id")
fi
.venv/bin/ely-eye "${args[@]}"
echo "Adapter ready: $adapter"
echo "Use with: ELY_EYE_ADAPTER_DIR=$adapter"
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
exec .venv/bin/ely-eye serve --host 127.0.0.1 --port 8765
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
cd frontend
exec npm run dev
+10 -9
View File
@@ -1,20 +1,21 @@
$ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for SGLang, vLLM, and KTransformers runtime profiles."
}
wsl.exe -d Ubuntu -- bash -lc "command -v nvidia-smi >/dev/null && nvidia-smi -L >/dev/null"
wsl.exe -d Ubuntu -- bash -lc "command -v curl >/dev/null || (apt-get update && apt-get install -y curl)"
wsl.exe -d Ubuntu -- bash -lc "command -v /root/.local/bin/uv >/dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh"
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux --python 3.12"
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux-vllm --python 3.12"
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-sglang-linux.txt"
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux-vllm/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-vllm-linux.txt"
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-ktransformers-linux.txt"
wsl.exe -d $distro -- bash -lc "command -v nvidia-smi >/dev/null && nvidia-smi -L >/dev/null"
wsl.exe -d $distro -- bash -lc "command -v curl >/dev/null || (apt-get update && apt-get install -y curl)"
wsl.exe -d $distro -- bash -lc "command -v /root/.local/bin/uv >/dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh"
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux --python 3.12"
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux-vllm --python 3.12"
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-sglang-linux.txt"
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && source .venv-linux-vllm/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-vllm-linux.txt"
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-ktransformers-linux.txt"
if (-not (Get-Command llama-server -ErrorAction SilentlyContinue)) {
winget install --id ggml.llamacpp -e --accept-package-agreements --accept-source-agreements
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if ! command -v nvidia-smi >/dev/null 2>&1; then
echo "nvidia-smi is missing; these runtimes need a CUDA host." >&2
exit 1
fi
nvidia-smi -L >/dev/null
uv_bin="${ELY_EYE_UV:-$HOME/.local/bin/uv}"
if ! command -v "$uv_bin" >/dev/null 2>&1; then
curl -LsSf https://astral.sh/uv/install.sh | sh
uv_bin="$HOME/.local/bin/uv"
fi
for venv in .venv-linux .venv-linux-vllm; do
"$uv_bin" venv "$venv" --python 3.12
done
"$uv_bin" pip install --python .venv-linux/bin/python --prerelease=allow -r backend/requirements-sglang-linux.txt
"$uv_bin" pip install --python .venv-linux-vllm/bin/python --prerelease=allow -r backend/requirements-vllm-linux.txt
"$uv_bin" pip install --python .venv-linux/bin/python --prerelease=allow -r backend/requirements-ktransformers-linux.txt
.venv/bin/ely-eye runtime-profiles
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
.venv/bin/python -m compileall ./backend/src
.venv/bin/ely-eye status
(cd frontend && npm run build)