Run the same checks on Linux that the Windows scripts run
The product could only reach a GPU through one hard-coded route: `wsl.exe -d Ubuntu`, a device named "RTX 4090", and launch scripts that existed only as PowerShell. On this machine `WSL_DISTRO_NAME` is already set, so the local shell is Linux and the wsl.exe hop was unnecessary; on any other Linux host it is impossible. Every SGLang, vLLM and KTransformers profile was reported "missing" here even with a working 4090. profiles.py now asks the host what it is instead of assuming: `uses_wsl()` is false inside WSL and only true when wsl.exe is actually on PATH, `host_shell()` runs `bash -lc` directly or through `wsl.exe -d settings.wsl_distro`, `gpu_available()` replaced `wsl_gpu_available()`, and `launch_script()` picks the .sh twin on a Linux host and the .ps1 on a Windows one. The schema field `wsl_gpu` became `gpu_available` because the old name lied on Linux. Training no longer fails on a card name. `cuda_bf16_training()` gates on cuda_available plus bf16_supported, which the training environment already records, so an RTX 5090 or a 4080 passes and a non-BF16 4090 still fails. Fourteen .sh twins cover the six launch profiles plus install, verify, serve, download and finetune; the six WSL-hopping .ps1 files read ELY_EYE_WSL_DISTRO instead of Ubuntu. PowerShell parsing is not verified here because no pwsh exists on this host. ci.yml runs pytest, ruff, go build/vet/test/gofmt, the Vite build and a docker build-plus-curl on every push. To make that lint gate mean anything, the 17 pre-existing ruff findings across backend/src are cleared: 14 mechanical fixes plus three probe handlers that now name the failure families they absorb (OSError, SubprocessError, ValueError) instead of catching Exception. Verification: pytest 239 passed; ruff check backend/src bench clean (17 at HEAD); mypy on the 12 touched files 14 errors, identical at HEAD; pyright on the same files 8 errors, identical at HEAD; bash -n clean on all 15 .sh.
This commit is contained in:
1 parent
36f40c4c59
commit
922d01202f
34 files changed
+482
-88
No files matched your search
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
for model in Qwen/Qwen3.5-9B Qwen/Qwen3.5-0.8B Qwen/Qwen3-VL-Embedding-8B; do
|
||||
.venv/bin/python ./scripts/download_models.py --model "$model"
|
||||
done
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
.venv/bin/python -m pip install --upgrade --no-cache-dir --index-url https://download.pytorch.org/whl/cu130 torch==2.12.0+cu130 torchvision==0.27.0+cu130
|
||||
.venv/bin/python -c 'import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.version.cuda)'
|
||||
Executable
+14
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
if [ ! -d .venv ]; then
|
||||
python3 -m venv .venv
|
||||
fi
|
||||
.venv/bin/python -m pip install --upgrade pip setuptools wheel
|
||||
.venv/bin/python -m pip install -e ./backend
|
||||
.venv/bin/python -m pip install --upgrade --no-cache-dir unsloth unsloth_zoo
|
||||
.venv/bin/python -m pip install --upgrade --no-cache-dir --index-url https://download.pytorch.org/whl/cu130 torch==2.12.0+cu130 torchvision==0.27.0+cu130
|
||||
.venv/bin/python -m pip install --upgrade --no-cache-dir transformers==5.9.0 datasets==4.8.5 dill==0.4.1 multiprocess==0.70.19 fsspec==2026.2.0 safetensors==0.8.0rc0
|
||||
(cd frontend && npm install)
|
||||
[ -f .env ] || cp .env.example .env
|
||||
.venv/bin/ely-eye status
|
||||
@@ -1,12 +1,13 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the KTransformers Linux runtime."
|
||||
}
|
||||
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
cd "$wslRoot"
|
||||
@@ -20,4 +21,4 @@ python -m sglang.launch_server \
|
||||
--cpu-offload-gb 48
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
wsl.exe -d $distro -- bash -lc $command
|
||||
Executable
+15
@@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
if [ ! -f .venv-linux/bin/activate ]; then
|
||||
echo "Run scripts/setup-linux-runtimes.sh before launching SGLang." >&2
|
||||
exit 1
|
||||
fi
|
||||
source .venv-linux/bin/activate
|
||||
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
|
||||
exec python -m sglang.launch_server \
|
||||
--model-path Qwen/Qwen3.5-9B \
|
||||
--port 8002 \
|
||||
--context-length 1010000 \
|
||||
--kt-cpuinfer 16 \
|
||||
--cpu-offload-gb 48
|
||||
Executable
+20
@@ -0,0 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
if [ -z "${ELY_EYE_GGUF_MODEL:-}" ]; then
|
||||
echo "Set ELY_EYE_GGUF_MODEL to the absolute GGUF model path." >&2
|
||||
exit 1
|
||||
fi
|
||||
server="${LLAMA_CPP_SERVER:-llama-server}"
|
||||
if ! command -v "$server" >/dev/null 2>&1; then
|
||||
echo "llama-server executable is missing. Build llama.cpp or set LLAMA_CPP_SERVER." >&2
|
||||
exit 1
|
||||
fi
|
||||
exec "$server" \
|
||||
--model "$ELY_EYE_GGUF_MODEL" \
|
||||
--host 127.0.0.1 \
|
||||
--port 8003 \
|
||||
--ctx-size 262144 \
|
||||
--flash-attn \
|
||||
--cache-type-k q4_0 \
|
||||
--cache-type-v q4_0
|
||||
@@ -1,13 +1,14 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the SGLang Linux runtime."
|
||||
}
|
||||
|
||||
$override = '{"text_config":{"rope_parameters":{"mrope_interleaved":true,"mrope_section":[11,11,10],"rope_type":"yarn","rope_theta":10000000,"partial_rotary_factor":0.25,"factor":4.0,"original_max_position_embeddings":262144}}}'
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
cd "$wslRoot"
|
||||
@@ -22,4 +23,4 @@ python -m sglang.launch_server \
|
||||
--json-model-override-args '$override'
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
wsl.exe -d $distro -- bash -lc $command
|
||||
Executable
+16
@@ -0,0 +1,16 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
if [ ! -f .venv-linux/bin/activate ]; then
|
||||
echo "Run scripts/setup-linux-runtimes.sh before launching SGLang." >&2
|
||||
exit 1
|
||||
fi
|
||||
source .venv-linux/bin/activate
|
||||
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
|
||||
exec python -m sglang.launch_server \
|
||||
--model-path Qwen/Qwen3.5-9B \
|
||||
--port 8000 \
|
||||
--tp-size 1 \
|
||||
--mem-fraction-static 0.88 \
|
||||
--context-length 1010000 \
|
||||
--json-model-override-args '{"text_config":{"rope_parameters":{"mrope_interleaved":true,"mrope_section":[11,11,10],"rope_type":"yarn","rope_theta":10000000,"partial_rotary_factor":0.25,"factor":4.0,"original_max_position_embeddings":262144}}}'
|
||||
@@ -1,12 +1,13 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the SGLang Linux runtime."
|
||||
}
|
||||
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
cd "$wslRoot"
|
||||
@@ -23,4 +24,4 @@ python -m sglang.launch_server \
|
||||
--speculative-num-draft-tokens 4
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
wsl.exe -d $distro -- bash -lc $command
|
||||
Executable
+18
@@ -0,0 +1,18 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
if [ ! -f .venv-linux/bin/activate ]; then
|
||||
echo "Run scripts/setup-linux-runtimes.sh before launching SGLang." >&2
|
||||
exit 1
|
||||
fi
|
||||
source .venv-linux/bin/activate
|
||||
exec python -m sglang.launch_server \
|
||||
--model-path Qwen/Qwen3.5-9B \
|
||||
--port 8000 \
|
||||
--tp-size 1 \
|
||||
--mem-fraction-static 0.82 \
|
||||
--context-length 262144 \
|
||||
--reasoning-parser qwen3 \
|
||||
--speculative-algo NEXTN \
|
||||
--speculative-num-steps 3 \
|
||||
--speculative-num-draft-tokens 4
|
||||
@@ -1,12 +1,13 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the vLLM Linux runtime."
|
||||
}
|
||||
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
|
||||
$ropeScaling = '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}'
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
@@ -23,4 +24,4 @@ python -m vllm.entrypoints.openai.api_server \
|
||||
--rope-scaling '$ropeScaling'
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
wsl.exe -d $distro -- bash -lc $command
|
||||
Executable
+17
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
if [ ! -f .venv-linux-vllm/bin/activate ]; then
|
||||
echo "Run scripts/setup-linux-runtimes.sh before launching vLLM." >&2
|
||||
exit 1
|
||||
fi
|
||||
source .venv-linux-vllm/bin/activate
|
||||
exec python -m vllm.entrypoints.openai.api_server \
|
||||
--model Qwen/Qwen3.5-9B \
|
||||
--host 127.0.0.1 \
|
||||
--port 8001 \
|
||||
--max-model-len 1010000 \
|
||||
--enable-prefix-caching \
|
||||
--gpu-memory-utilization 0.88 \
|
||||
--trust-remote-code \
|
||||
--rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}'
|
||||
@@ -1,12 +1,13 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the vLLM Linux runtime."
|
||||
}
|
||||
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
cd "$wslRoot"
|
||||
@@ -21,4 +22,4 @@ python -m vllm.entrypoints.openai.api_server \
|
||||
--trust-remote-code
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
wsl.exe -d $distro -- bash -lc $command
|
||||
Executable
+16
@@ -0,0 +1,16 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
if [ ! -f .venv-linux-vllm/bin/activate ]; then
|
||||
echo "Run scripts/setup-linux-runtimes.sh before launching vLLM." >&2
|
||||
exit 1
|
||||
fi
|
||||
source .venv-linux-vllm/bin/activate
|
||||
exec python -m vllm.entrypoints.openai.api_server \
|
||||
--model Qwen/Qwen3.5-9B \
|
||||
--host 127.0.0.1 \
|
||||
--port 8001 \
|
||||
--max-model-len 262144 \
|
||||
--enable-prefix-caching \
|
||||
--gpu-memory-utilization 0.82 \
|
||||
--trust-remote-code
|
||||
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
max_steps="${MAX_STEPS:-37}"
|
||||
rank="${RANK:-16}"
|
||||
alpha="${ALPHA:-32}"
|
||||
max_length="${MAX_LENGTH:-8192}"
|
||||
gradient_accumulation_steps="${GRADIENT_ACCUMULATION_STEPS:-4}"
|
||||
cartridge_id="${CARTRIDGE_ID:-}"
|
||||
|
||||
dataset=.ely_eye/training/hme_prd_sft.jsonl
|
||||
adapter=.ely_eye/adapters/hme_core_lora_qwen35_9b
|
||||
|
||||
.venv/bin/ely-eye build-prd-training-data \
|
||||
--prd-path ./PRD.md \
|
||||
--output-path "$dataset" \
|
||||
--max-sections 48
|
||||
|
||||
args=(train-lora "$dataset" "$adapter"
|
||||
--max-steps "$max_steps"
|
||||
--kind hme_core_lora
|
||||
--rank "$rank"
|
||||
--alpha "$alpha"
|
||||
--max-length "$max_length"
|
||||
--gradient-accumulation-steps "$gradient_accumulation_steps")
|
||||
|
||||
if [ -n "$cartridge_id" ]; then
|
||||
args+=(--cartridge-id "$cartridge_id")
|
||||
fi
|
||||
|
||||
.venv/bin/ely-eye "${args[@]}"
|
||||
|
||||
echo "Adapter ready: $adapter"
|
||||
echo "Use with: ELY_EYE_ADAPTER_DIR=$adapter"
|
||||
Executable
+4
@@ -0,0 +1,4 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
exec .venv/bin/ely-eye serve --host 127.0.0.1 --port 8765
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
cd frontend
|
||||
exec npm run dev
|
||||
@@ -1,20 +1,21 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for SGLang, vLLM, and KTransformers runtime profiles."
|
||||
}
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc "command -v nvidia-smi >/dev/null && nvidia-smi -L >/dev/null"
|
||||
wsl.exe -d Ubuntu -- bash -lc "command -v curl >/dev/null || (apt-get update && apt-get install -y curl)"
|
||||
wsl.exe -d Ubuntu -- bash -lc "command -v /root/.local/bin/uv >/dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh"
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux --python 3.12"
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux-vllm --python 3.12"
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-sglang-linux.txt"
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux-vllm/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-vllm-linux.txt"
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-ktransformers-linux.txt"
|
||||
wsl.exe -d $distro -- bash -lc "command -v nvidia-smi >/dev/null && nvidia-smi -L >/dev/null"
|
||||
wsl.exe -d $distro -- bash -lc "command -v curl >/dev/null || (apt-get update && apt-get install -y curl)"
|
||||
wsl.exe -d $distro -- bash -lc "command -v /root/.local/bin/uv >/dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh"
|
||||
$wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
|
||||
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux --python 3.12"
|
||||
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux-vllm --python 3.12"
|
||||
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-sglang-linux.txt"
|
||||
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && source .venv-linux-vllm/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-vllm-linux.txt"
|
||||
wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-ktransformers-linux.txt"
|
||||
|
||||
if (-not (Get-Command llama-server -ErrorAction SilentlyContinue)) {
|
||||
winget install --id ggml.llamacpp -e --accept-package-agreements --accept-source-agreements
|
||||
|
||||
Executable
+20
@@ -0,0 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
if ! command -v nvidia-smi >/dev/null 2>&1; then
|
||||
echo "nvidia-smi is missing; these runtimes need a CUDA host." >&2
|
||||
exit 1
|
||||
fi
|
||||
nvidia-smi -L >/dev/null
|
||||
uv_bin="${ELY_EYE_UV:-$HOME/.local/bin/uv}"
|
||||
if ! command -v "$uv_bin" >/dev/null 2>&1; then
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
uv_bin="$HOME/.local/bin/uv"
|
||||
fi
|
||||
for venv in .venv-linux .venv-linux-vllm; do
|
||||
"$uv_bin" venv "$venv" --python 3.12
|
||||
done
|
||||
"$uv_bin" pip install --python .venv-linux/bin/python --prerelease=allow -r backend/requirements-sglang-linux.txt
|
||||
"$uv_bin" pip install --python .venv-linux-vllm/bin/python --prerelease=allow -r backend/requirements-vllm-linux.txt
|
||||
"$uv_bin" pip install --python .venv-linux/bin/python --prerelease=allow -r backend/requirements-ktransformers-linux.txt
|
||||
.venv/bin/ely-eye runtime-profiles
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
.venv/bin/python -m compileall ./backend/src
|
||||
.venv/bin/ely-eye status
|
||||
(cd frontend && npm run build)
|
||||
Reference in new issue
Block a user