Run the same checks on Linux that the Windows scripts run

The product could only reach a GPU through one hard-coded route: `wsl.exe -d
Ubuntu`, a device named "RTX 4090", and launch scripts that existed only as
PowerShell. On this machine `WSL_DISTRO_NAME` is already set, so the local
shell is Linux and the wsl.exe hop was unnecessary; on any other Linux host
it is impossible. Every SGLang, vLLM and KTransformers profile was reported
"missing" here even with a working 4090.

profiles.py now asks the host what it is instead of assuming: `uses_wsl()`
is false inside WSL and only true when wsl.exe is actually on PATH,
`host_shell()` runs `bash -lc` directly or through `wsl.exe -d
settings.wsl_distro`, `gpu_available()` replaced `wsl_gpu_available()`, and
`launch_script()` picks the .sh twin on a Linux host and the .ps1 on a
Windows one. The schema field `wsl_gpu` became `gpu_available` because the
old name lied on Linux.

Training no longer fails on a card name. `cuda_bf16_training()` gates on
cuda_available plus bf16_supported, which the training environment already
records, so an RTX 5090 or a 4080 passes and a non-BF16 4090 still fails.

Fourteen .sh twins cover the six launch profiles plus install, verify,
serve, download and finetune; the six WSL-hopping .ps1 files read
ELY_EYE_WSL_DISTRO instead of Ubuntu. PowerShell parsing is not verified
here because no pwsh exists on this host.

ci.yml runs pytest, ruff, go build/vet/test/gofmt, the Vite build and a
docker build-plus-curl on every push. To make that lint gate mean anything,
the 17 pre-existing ruff findings across backend/src are cleared: 14
mechanical fixes plus three probe handlers that now name the failure
families they absorb (OSError, SubprocessError, ValueError) instead of
catching Exception.

Verification: pytest 239 passed; ruff check backend/src bench clean (17 at
HEAD); mypy on the 12 touched files 14 errors, identical at HEAD; pyright on
the same files 8 errors, identical at HEAD; bash -n clean on all 15 .sh.
This commit is contained in:
ZacharyZhang-NY committed 2026-10-08 02:12:29 -04:00
1 parent 36f40c4c59
commit 922d01202f
34 files changed
+482 -88

No files matched your search

+67
View File
@@ -0,0 +1,67 @@
name: ci
on:
push:
branches: [main]
pull_request:
permissions:
contents: read
jobs:
backend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
- run: pip install -e ./backend
- run: python -m pytest -q
- run: pip install ruff==0.16.10 && ruff check backend/src bench
mcp:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.25"
cache-dependency-path: ely-eye-mcp/go.sum
- name: Build, vet and test
working-directory: ely-eye-mcp
run: go build ./... && go vet ./... && go test ./...
- name: gofmt is clean
working-directory: ely-eye-mcp
run: test -z "$(gofmt -l .)"
interface:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "22"
cache: npm
cache-dependency-path: frontend/package-lock.json
- run: npm ci && npm run build
working-directory: frontend
image:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build the image from a clean clone
run: docker build -t ely-eye:ci .
- name: The container answers on both faces
run: |
docker run -d --name elyeye -p 8765:8765 ely-eye:ci
url=http://127.0.0.1:8765/api/status
for attempt in $(seq 1 30); do
curl -fsS "$url" -o /tmp/status.json && break
sleep 2
done
cat /tmp/status.json
curl -fsS http://127.0.0.1:8765/ | grep -q '<div id="root">'
docker logs elyeye
+4 -2
View File
@@ -277,8 +277,10 @@ def create_app() -> FastAPI:
return benchmark_prediction_report(settings, settings.data_dir / "benchmark_predictions") return benchmark_prediction_report(settings, settings.data_dir / "benchmark_predictions")
@app.get("/api/runtime/profiles", response_model=RuntimeProfileReport) @app.get("/api/runtime/profiles", response_model=RuntimeProfileReport)
def runtime_profiles() -> RuntimeProfileReport: def runtime_profiles(
return runtime_profile_report() settings: Annotated[Settings, Depends(settings_dep)],
) -> RuntimeProfileReport:
return runtime_profile_report(settings)
@app.get("/api/runtime/adapters", response_model=AdapterLoadReport) @app.get("/api/runtime/adapters", response_model=AdapterLoadReport)
def runtime_adapters(settings: Annotated[Settings, Depends(settings_dep)]) -> AdapterLoadReport: def runtime_adapters(settings: Annotated[Settings, Depends(settings_dep)]) -> AdapterLoadReport:
+1 -1
View File
@@ -325,7 +325,7 @@ def run_benchmark(
@app.command("runtime-profiles") @app.command("runtime-profiles")
def runtime_profiles() -> None: def runtime_profiles() -> None:
typer.echo(runtime_profile_report().model_dump_json(indent=2)) typer.echo(runtime_profile_report(get_settings()).model_dump_json(indent=2))
@app.command("research-modules") @app.command("research-modules")
+1
View File
@@ -27,6 +27,7 @@ class Settings(BaseSettings):
sglang_base_url: str = "http://127.0.0.1:8000/v1" sglang_base_url: str = "http://127.0.0.1:8000/v1"
sglang_api_key: str = "ely-eye-local" sglang_api_key: str = "ely-eye-local"
ui_dir: Path | None = None ui_dir: Path | None = None
wsl_distro: str = "Ubuntu"
dense_embeddings_enabled: bool = False dense_embeddings_enabled: bool = False
live_context_tokens: int = 262_144 live_context_tokens: int = 262_144
extreme_context_tokens: int = 1_010_000 extreme_context_tokens: int = 1_010_000
-1
View File
@@ -2,7 +2,6 @@ from __future__ import annotations
import json import json
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path
from typing import Any from typing import Any
import numpy as np import numpy as np
+1 -1
View File
@@ -64,5 +64,5 @@ class EvidenceGraphService:
def claim_node_id(question: str, answer: str) -> str: def claim_node_id(question: str, answer: str) -> str:
digest = hashlib.sha256(f"{question}\n{answer}".encode("utf-8")).hexdigest()[:20] digest = hashlib.sha256(f"{question}\n{answer}".encode()).hexdigest()[:20]
return f"claim_{digest}" return f"claim_{digest}"
+64 -42
View File
@@ -6,9 +6,9 @@ import shutil
import subprocess import subprocess
from pathlib import Path from pathlib import Path
from .config import Settings
from .schemas import RuntimeLaunchProfile, RuntimeProfileReport, RuntimeProfileValidation from .schemas import RuntimeLaunchProfile, RuntimeProfileReport, RuntimeProfileValidation
EXTREME_YARN_OVERRIDE = { EXTREME_YARN_OVERRIDE = {
"text_config": { "text_config": {
"rope_parameters": { "rope_parameters": {
@@ -24,7 +24,7 @@ EXTREME_YARN_OVERRIDE = {
} }
def runtime_profile_report(project_root: Path | None = None) -> RuntimeProfileReport: def runtime_profile_report(settings: Settings, project_root: Path | None = None) -> RuntimeProfileReport:
root = project_root or Path.cwd() root = project_root or Path.cwd()
profiles = [ profiles = [
RuntimeLaunchProfile( RuntimeLaunchProfile(
@@ -185,29 +185,55 @@ def runtime_profile_report(project_root: Path | None = None) -> RuntimeProfileRe
required_features=["GGUF", "offline demo", "quantized KV"], required_features=["GGUF", "offline demo", "quantized KV"],
), ),
] ]
validations = validate_runtime_profiles(root, profiles) validations = validate_runtime_profiles(root, profiles, settings)
ready = all(validation.status == "ready" for validation in validations) ready = all(validation.status == "ready" for validation in validations)
return RuntimeProfileReport( return RuntimeProfileReport(
status="ready" if ready else "incomplete", status="ready" if ready else "incomplete",
profiles=profiles, profiles=[profile.model_copy(update={"script": launch_script(profile.script)}) for profile in profiles],
validations=validations, validations=validations,
) )
def uses_wsl() -> bool:
"""A Windows process reaches Linux through wsl.exe; inside WSL the local shell already is Linux."""
if os.environ.get("WSL_DISTRO_NAME"):
return False
return shutil.which("wsl.exe") is not None
def launch_script(script: str) -> str:
"""A Windows host runs the .ps1; a Linux host, WSL included, runs the .sh twin."""
return script if uses_wsl() else str(Path(script).with_suffix(".sh"))
def host_shell(command: str, settings: Settings) -> list[str]:
"""Run a shell command where the launch environments live."""
if uses_wsl():
return ["wsl.exe", "-d", settings.wsl_distro, "bash", "-lc", command]
return ["bash", "-lc", command]
def host_path(root: Path) -> str:
"""The path the host shell can cd into; a Windows path is reached through the WSL mount."""
resolved = root.resolve()
return windows_path_to_wsl(resolved) if uses_wsl() else str(resolved)
def validate_runtime_profiles( def validate_runtime_profiles(
root: Path, root: Path,
profiles: list[RuntimeLaunchProfile], profiles: list[RuntimeLaunchProfile],
settings: Settings,
) -> list[RuntimeProfileValidation]: ) -> list[RuntimeProfileValidation]:
wsl_gpu = wsl_gpu_available() gpu = gpu_available(settings)
package_cache = { package_cache = {
".venv-linux": linux_package_versions(root, ".venv-linux"), ".venv-linux": linux_package_versions(root, ".venv-linux", settings),
".venv-linux-vllm": linux_package_versions(root, ".venv-linux-vllm"), ".venv-linux-vllm": linux_package_versions(root, ".venv-linux-vllm", settings),
} }
probe_cache: dict[tuple[str, str, str | None], dict[str, object]] = {} probe_cache: dict[tuple[str, str, str | None], dict[str, object]] = {}
llama_server = resolve_llama_server() llama_server = resolve_llama_server()
validations: list[RuntimeProfileValidation] = [] validations: list[RuntimeProfileValidation] = []
for profile in profiles: for profile in profiles:
script_exists = (root / profile.script).exists() script_exists = (root / launch_script(profile.script)).exists()
package = package_for_runtime(profile.runtime) package = package_for_runtime(profile.runtime)
env_name = linux_env_for_runtime(profile.runtime) env_name = linux_env_for_runtime(profile.runtime)
linux_packages = package_cache.get(env_name, {}) linux_packages = package_cache.get(env_name, {})
@@ -216,24 +242,24 @@ def validate_runtime_profiles(
executable = llama_server if profile.runtime == "llama.cpp" else None executable = llama_server if profile.runtime == "llama.cpp" else None
probe_key = (profile.runtime, env_name, executable) probe_key = (profile.runtime, env_name, executable)
if probe_key not in probe_cache: if probe_key not in probe_cache:
probe_cache[probe_key] = runtime_launcher_probe(root, profile.runtime, env_name, executable) probe_cache[probe_key] = runtime_launcher_probe(root, profile.runtime, env_name, executable, settings)
probe = probe_cache[probe_key] probe = probe_cache[probe_key]
probe_status = str(probe.get("status", "skipped")) probe_status = str(probe.get("status", "skipped"))
probe_passed = probe_status == "passed" probe_passed = probe_status == "passed"
if profile.runtime in {"SGLang", "vLLM", "KTransformers"}: if profile.runtime in {"SGLang", "vLLM", "KTransformers"}:
ready = script_exists and bool(package_version) and wsl_gpu and probe_passed ready = script_exists and bool(package_version) and gpu and probe_passed
if profile.runtime == "KTransformers": if profile.runtime == "KTransformers":
ready = ready and bool(kt_sglang_version) ready = ready and bool(kt_sglang_version)
detail = ( detail = (
( (
f"WSL GPU ready; {package} {package_version}; " f"GPU ready; {package} {package_version}; "
f"sglang-kt entry {kt_sglang_version}" f"sglang-kt entry {kt_sglang_version}"
) )
if profile.runtime == "KTransformers" and ready if profile.runtime == "KTransformers" and ready
else f"WSL GPU ready; {package} {package_version}; env {env_name}" else f"GPU ready; {package} {package_version}; env {env_name}"
if ready if ready
else ( else (
f"script={script_exists}, wsl_gpu={wsl_gpu}, env={env_name}, " f"script={script_exists}, gpu={gpu}, env={env_name}, "
f"{package}={package_version}, sglang={kt_sglang_version}" f"{package}={package_version}, sglang={kt_sglang_version}"
) )
) )
@@ -253,7 +279,7 @@ def validate_runtime_profiles(
package=package, package=package,
package_version=package_version, package_version=package_version,
executable=executable, executable=executable,
wsl_gpu=wsl_gpu, gpu_available=gpu,
probe_command=[str(item) for item in probe.get("command", [])], probe_command=[str(item) for item in probe.get("command", [])],
probe_status=probe_status, probe_status=probe_status,
probe_exit_code=probe.get("exit_code") if isinstance(probe.get("exit_code"), int) else None, probe_exit_code=probe.get("exit_code") if isinstance(probe.get("exit_code"), int) else None,
@@ -278,7 +304,7 @@ def linux_env_for_runtime(runtime: str) -> str:
return ".venv-linux" return ".venv-linux"
def linux_package_versions(root: Path, env_name: str) -> dict[str, str]: def linux_package_versions(root: Path, env_name: str, settings: Settings) -> dict[str, str]:
script = ( script = (
"import importlib.metadata as md, json;" "import importlib.metadata as md, json;"
"names=['sglang','vllm','ktransformers','torch','torchvision','qwen-vl-utils'];" "names=['sglang','vllm','ktransformers','torch','torchvision','qwen-vl-utils'];"
@@ -290,25 +316,21 @@ def linux_package_versions(root: Path, env_name: str) -> dict[str, str]:
) )
try: try:
output = subprocess.check_output( output = subprocess.check_output(
[ host_shell(
"wsl.exe", f"cd {wsl_quote(host_path(root))} "
"-d", f"&& test -x {env_name}/bin/python "
"Ubuntu", f"&& {env_name}/bin/python -c {wsl_quote(script)}",
"bash", settings,
"-lc", ),
(
f"cd {wsl_quote(windows_path_to_wsl(root))} "
f"&& test -x {env_name}/bin/python "
f"&& {env_name}/bin/python -c {wsl_quote(script)}"
),
],
text=True, text=True,
stderr=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
timeout=20, timeout=20,
) )
data = json.loads(output.strip()) payload = json.loads(output.strip())
return {str(key): str(value) for key, value in data.items()} if not isinstance(payload, dict):
except Exception: return {}
return {str(key): str(value) for key, value in payload.items()}
except (OSError, subprocess.SubprocessError, ValueError):
return {} return {}
@@ -317,6 +339,7 @@ def runtime_launcher_probe(
runtime: str, runtime: str,
env_name: str, env_name: str,
executable: str | None, executable: str | None,
settings: Settings,
) -> dict[str, object]: ) -> dict[str, object]:
if runtime == "SGLang": if runtime == "SGLang":
probe_script = ( probe_script = (
@@ -325,7 +348,7 @@ def runtime_launcher_probe(
"print(spec.origin if spec else 'missing'); " "print(spec.origin if spec else 'missing'); "
"sys.exit(0 if spec else 1)" "sys.exit(0 if spec else 1)"
) )
command = wsl_python_command(root, env_name, f"-c {wsl_quote(probe_script)}") command = host_python_command(root, env_name, f"-c {wsl_quote(probe_script)}", settings)
elif runtime == "vLLM": elif runtime == "vLLM":
probe_script = ( probe_script = (
"import importlib.util, sys; " "import importlib.util, sys; "
@@ -333,10 +356,10 @@ def runtime_launcher_probe(
"print(spec.origin if spec else 'missing'); " "print(spec.origin if spec else 'missing'); "
"sys.exit(0 if spec else 1)" "sys.exit(0 if spec else 1)"
) )
command = wsl_python_command(root, env_name, f"-c {wsl_quote(probe_script)}") command = host_python_command(root, env_name, f"-c {wsl_quote(probe_script)}", settings)
elif runtime == "KTransformers": elif runtime == "KTransformers":
probe_script = "import ktransformers, sglang; print('ktransformers and sglang import ok')" probe_script = "import ktransformers, sglang; print('ktransformers and sglang import ok')"
command = wsl_python_command(root, env_name, f"-c {wsl_quote(probe_script)}") command = host_python_command(root, env_name, f"-c {wsl_quote(probe_script)}", settings)
elif runtime == "llama.cpp" and executable: elif runtime == "llama.cpp" and executable:
command = [executable, "--help"] command = [executable, "--help"]
else: else:
@@ -349,7 +372,7 @@ def runtime_launcher_probe(
timeout=30, timeout=30,
check=False, check=False,
) )
except Exception as exc: except (OSError, subprocess.SubprocessError) as exc:
return { return {
"status": "failed", "status": "failed",
"command": command, "command": command,
@@ -365,13 +388,13 @@ def runtime_launcher_probe(
} }
def wsl_python_command(root: Path, env_name: str, python_args: str) -> list[str]: def host_python_command(root: Path, env_name: str, python_args: str, settings: Settings) -> list[str]:
shell_command = ( shell_command = (
f"cd {wsl_quote(windows_path_to_wsl(root))} " f"cd {wsl_quote(host_path(root))} "
f"&& test -x {env_name}/bin/python " f"&& test -x {env_name}/bin/python "
f"&& {env_name}/bin/python {python_args}" f"&& {env_name}/bin/python {python_args}"
) )
return ["wsl.exe", "-d", "Ubuntu", "bash", "-lc", shell_command] return host_shell(shell_command, settings)
def compact_output(stdout: str, stderr: str, limit: int = 800) -> str: def compact_output(stdout: str, stderr: str, limit: int = 800) -> str:
@@ -379,18 +402,17 @@ def compact_output(stdout: str, stderr: str, limit: int = 800) -> str:
return text[:limit] return text[:limit]
def wsl_gpu_available() -> bool: def gpu_available(settings: Settings) -> bool:
if shutil.which("wsl.exe") is None: """Ask the host for a CUDA-capable GPU without naming a card: any GPU the driver lists will do."""
return False
try: try:
output = subprocess.check_output( output = subprocess.check_output(
["wsl.exe", "-d", "Ubuntu", "nvidia-smi", "-L"], host_shell("nvidia-smi -L", settings),
text=True, text=True,
stderr=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
timeout=10, timeout=10,
) )
return "GPU" in output return "GPU" in output
except Exception: except (OSError, subprocess.SubprocessError):
return False return False
+1 -1
View File
@@ -232,7 +232,7 @@ class RuntimeChecks:
runtime.unload() runtime.unload()
def _runtime_profile_check(self, proof_root: Path) -> ProofCheck: def _runtime_profile_check(self, proof_root: Path) -> ProofCheck:
report = runtime_profile_report() report = runtime_profile_report(self.settings)
artifact = proof_root / "runtime_profiles.json" artifact = proof_root / "runtime_profiles.json"
artifact.write_text(report.model_dump_json(indent=2), encoding="utf-8") artifact.write_text(report.model_dump_json(indent=2), encoding="utf-8")
required_runtimes = {"SGLang", "vLLM", "KTransformers", "llama.cpp"} required_runtimes = {"SGLang", "vLLM", "KTransformers", "llama.cpp"}
+7 -7
View File
@@ -24,6 +24,11 @@ from ..training import (
) )
def cuda_bf16_training(environment: dict[str, Any]) -> bool:
"""The requirement is CUDA plus BF16 support, not one card name written down in 2025."""
return bool(environment.get("cuda_available")) and bool(environment.get("bf16_supported"))
def count_safetensor_keys(path: Path) -> int: def count_safetensor_keys(path: Path) -> int:
return int(safetensor_weight_stats(path)["tensor_count"]) return int(safetensor_weight_stats(path)["tensor_count"])
@@ -173,8 +178,7 @@ def validate_training_trace(manifest: AdapterManifest, adapter_root: Path) -> di
} }
) )
environment = summary.get("environment") if isinstance(summary.get("environment"), dict) else {} environment = summary.get("environment") if isinstance(summary.get("environment"), dict) else {}
device_name = str(environment.get("device_name") or "") report["cuda_training"] = cuda_bf16_training(environment)
report["cuda_training"] = bool(environment.get("cuda_available")) and "RTX 4090" in device_name
report["bf16_supported"] = bool(environment.get("bf16_supported")) report["bf16_supported"] = bool(environment.get("bf16_supported"))
report["valid"] = ( report["valid"] = (
report["trace_sha256_ok"] report["trace_sha256_ok"]
@@ -282,11 +286,7 @@ def validate_training_proof(manifest: AdapterManifest, adapter_root: Path) -> di
and int(proof_weight_stats.get("nonzero_elements") or 0) > 0 and int(proof_weight_stats.get("nonzero_elements") or 0) > 0
and bool(proof_weight_stats.get("all_finite")) and bool(proof_weight_stats.get("all_finite"))
), ),
"proof_cuda_bf16": ( "proof_cuda_bf16": bool(proof.get("bf16")) and cuda_bf16_training(proof_environment),
bool(proof.get("bf16"))
and bool(proof_environment.get("cuda_available"))
and "RTX 4090" in str(proof_environment.get("device_name") or "")
),
"proof_bf16_supported": bool(proof_environment.get("bf16_supported")), "proof_bf16_supported": bool(proof_environment.get("bf16_supported")),
"proof_contract_ok": training_payload_contract_ok(proof, manifest), "proof_contract_ok": training_payload_contract_ok(proof, manifest),
"proof_optimizer_ok": proof.get("optimizer") == "paged_adamw_8bit", "proof_optimizer_ok": proof.get("optimizer") == "paged_adamw_8bit",
+1 -1
View File
@@ -155,7 +155,7 @@ class ResearchModuleService:
"entries": layer.entries, "entries": layer.entries,
"hit_rate": hit_rate, "hit_rate": hit_rate,
"evict": hit_rate < 0.25 and layer.entries > 0, "evict": hit_rate < 0.25 and layer.entries > 0,
"replay_token": hashlib.sha256(f"{layer.layer}:{layer.entries}".encode("utf-8")).hexdigest()[:16], "replay_token": hashlib.sha256(f"{layer.layer}:{layer.entries}".encode()).hexdigest()[:16],
} }
) )
return {"policy": "hit-rate weighted replay buffer", "layers": rows} return {"policy": "hit-rate weighted replay buffer", "layers": rows}
+1 -1
View File
@@ -439,7 +439,7 @@ class RuntimeProfileValidation(BaseModel):
package: str | None = None package: str | None = None
package_version: str | None = None package_version: str | None = None
executable: str | None = None executable: str | None = None
wsl_gpu: bool = False gpu_available: bool = False
probe_command: list[str] = Field(default_factory=list) probe_command: list[str] = Field(default_factory=list)
probe_status: Literal["passed", "failed", "skipped"] = "skipped" probe_status: Literal["passed", "failed", "skipped"] = "skipped"
probe_exit_code: int | None = None probe_exit_code: int | None = None
+10 -11
View File
@@ -9,7 +9,7 @@ import shutil
import subprocess import subprocess
import time import time
from contextlib import contextmanager from contextlib import contextmanager
from datetime import datetime, timezone from datetime import UTC, datetime
from importlib.metadata import PackageNotFoundError, version from importlib.metadata import PackageNotFoundError, version
from pathlib import Path from pathlib import Path
from typing import Any, TextIO from typing import Any, TextIO
@@ -20,7 +20,6 @@ from .config import Settings, get_settings
from .db import Database from .db import Database
from .schemas import AdapterManifest, EvidenceAtom, TrainingAdapterKind from .schemas import AdapterManifest, EvidenceAtom, TrainingAdapterKind
SYSTEM_PROMPT = ( SYSTEM_PROMPT = (
"You are Ely-Eye. Answer from supplied evidence, return compact JSON, " "You are Ely-Eye. Answer from supplied evidence, return compact JSON, "
"and cite atom ids exactly." "and cite atom ids exactly."
@@ -433,7 +432,7 @@ class TrainingService:
gradient_accumulation_steps or self.settings.train_gradient_accumulation_steps gradient_accumulation_steps or self.settings.train_gradient_accumulation_steps
) )
learning_rate = learning_rate or self.settings.train_learning_rate learning_rate = learning_rate or self.settings.train_learning_rate
started_at = datetime.now(timezone.utc) started_at = datetime.now(UTC)
wall_start = time.perf_counter() wall_start = time.perf_counter()
trace_path = output_dir / "training_trace.jsonl" trace_path = output_dir / "training_trace.jsonl"
@@ -575,7 +574,7 @@ class TrainingService:
processor.save_pretrained(output_dir) processor.save_pretrained(output_dir)
self.write_lora_config(output_dir, kind, rank, alpha, model_id) self.write_lora_config(output_dir, kind, rank, alpha, model_id)
weight_stats = safetensor_weight_stats(output_dir / "adapter_model.safetensors") weight_stats = safetensor_weight_stats(output_dir / "adapter_model.safetensors")
finished_at = datetime.now(timezone.utc) finished_at = datetime.now(UTC)
wall_seconds = time.perf_counter() - wall_start wall_seconds = time.perf_counter() - wall_start
summary_path = output_dir / "training_summary.json" summary_path = output_dir / "training_summary.json"
proof_path = output_dir / "training_proof.json" proof_path = output_dir / "training_proof.json"
@@ -671,7 +670,7 @@ class TrainingService:
manifest = AdapterManifest( manifest = AdapterManifest(
adapter_id=adapter_id(kind, dataset_path, model_id), adapter_id=adapter_id(kind, dataset_path, model_id),
kind=kind, kind=kind,
created_at=datetime.now(timezone.utc), created_at=datetime.now(UTC),
proof_version=TRAINING_PROOF_VERSION, proof_version=TRAINING_PROOF_VERSION,
training_method=TRAINING_METHOD, training_method=TRAINING_METHOD,
precision=TRAINING_PRECISION, precision=TRAINING_PRECISION,
@@ -756,7 +755,7 @@ class TrainingService:
output_dir.mkdir(parents=True, exist_ok=True) output_dir.mkdir(parents=True, exist_ok=True)
max_length = max_length or self.settings.train_max_length max_length = max_length or self.settings.train_max_length
learning_rate = learning_rate or self.settings.train_learning_rate learning_rate = learning_rate or self.settings.train_learning_rate
started_at = datetime.now(timezone.utc) started_at = datetime.now(UTC)
wall_start = time.perf_counter() wall_start = time.perf_counter()
trace_path = output_dir / "training_trace.jsonl" trace_path = output_dir / "training_trace.jsonl"
@@ -882,7 +881,7 @@ class TrainingService:
processor.save_pretrained(output_dir) processor.save_pretrained(output_dir)
self.write_lora_config(output_dir, TrainingAdapterKind.hme_retrieval, rank, alpha, model_id) self.write_lora_config(output_dir, TrainingAdapterKind.hme_retrieval, rank, alpha, model_id)
weight_stats = safetensor_weight_stats(output_dir / "adapter_model.safetensors") weight_stats = safetensor_weight_stats(output_dir / "adapter_model.safetensors")
finished_at = datetime.now(timezone.utc) finished_at = datetime.now(UTC)
wall_seconds = time.perf_counter() - wall_start wall_seconds = time.perf_counter() - wall_start
summary_path = output_dir / "training_summary.json" summary_path = output_dir / "training_summary.json"
proof_path = output_dir / "training_proof.json" proof_path = output_dir / "training_proof.json"
@@ -977,7 +976,7 @@ class TrainingService:
manifest = AdapterManifest( manifest = AdapterManifest(
adapter_id=adapter_id(TrainingAdapterKind.hme_retrieval, dataset_path, model_id), adapter_id=adapter_id(TrainingAdapterKind.hme_retrieval, dataset_path, model_id),
kind=TrainingAdapterKind.hme_retrieval, kind=TrainingAdapterKind.hme_retrieval,
created_at=datetime.now(timezone.utc), created_at=datetime.now(UTC),
proof_version=TRAINING_PROOF_VERSION, proof_version=TRAINING_PROOF_VERSION,
training_method=TRAINING_METHOD, training_method=TRAINING_METHOD,
precision=TRAINING_PRECISION, precision=TRAINING_PRECISION,
@@ -1269,7 +1268,7 @@ def trainable_update_l2(before_update: dict[str, Any], named_parameters: list[tu
def require_cuda_bf16(torch_module: Any) -> None: def require_cuda_bf16(torch_module: Any) -> None:
if not torch_module.cuda.is_available(): if not torch_module.cuda.is_available():
raise RuntimeError("PRD 12.2 training requires CUDA BF16 on the local RTX 4090.") raise RuntimeError("training needs a CUDA device; none is available.")
bf16_supported = getattr(torch_module.cuda, "is_bf16_supported", lambda: False) bf16_supported = getattr(torch_module.cuda, "is_bf16_supported", lambda: False)
if not bool(bf16_supported()): if not bool(bf16_supported()):
raise RuntimeError("PRD 12.2 training requires BF16-capable CUDA hardware.") raise RuntimeError("PRD 12.2 training requires BF16-capable CUDA hardware.")
@@ -1598,7 +1597,7 @@ def slugify(value: str) -> str:
def adapter_id(kind: TrainingAdapterKind, dataset_path: Path, model_id: str) -> str: def adapter_id(kind: TrainingAdapterKind, dataset_path: Path, model_id: str) -> str:
raw = f"{kind.value}:{model_id}:{dataset_path.resolve()}:{datetime.now(timezone.utc).isoformat()}" raw = f"{kind.value}:{model_id}:{dataset_path.resolve()}:{datetime.now(UTC).isoformat()}"
return f"{kind.value}_{sha256_bytes(raw.encode('utf-8'))[:12]}" return f"{kind.value}_{sha256_bytes(raw.encode('utf-8'))[:12]}"
@@ -1632,7 +1631,7 @@ def sha256_bytes(data: bytes) -> str:
class _DigestWriter: class _DigestWriter:
def __init__(self, digest: "hashlib._Hash") -> None: def __init__(self, digest: hashlib._Hash) -> None:
self.digest = digest self.digest = digest
def write(self, data: bytes) -> int: def write(self, data: bytes) -> int:
-1
View File
@@ -7,7 +7,6 @@ from typing import Any
from .schemas import CitationReport, CompiledContext from .schemas import CitationReport, CompiledContext
ATOM_PATTERN = re.compile(r"[A-Za-z0-9_./\\:-]+:(?:text|image|page|frame|region)[A-Za-z0-9_./\\:#=-]*") ATOM_PATTERN = re.compile(r"[A-Za-z0-9_./\\:-]+:(?:text|image|page|frame|region)[A-Za-z0-9_./\\:#=-]*")
+102
View File
@@ -0,0 +1,102 @@
"""The launch layer must describe the machine it runs on, not the one it was written on."""
from __future__ import annotations
import os
import stat
from pathlib import Path
import pytest
from ely_eye import profiles
from ely_eye.config import Settings
from ely_eye.proofs.training_trace import cuda_bf16_training
from ely_eye.training import require_cuda_bf16
def _stub_nvidia_smi(tmp_path: Path, monkeypatch, output: str, exit_code: int = 0) -> None:
bin_dir = tmp_path / "bin"
bin_dir.mkdir()
stub = bin_dir / "nvidia-smi"
stub.write_text(f"#!/bin/sh\nprintf '%s\\n' '{output}'\nexit {exit_code}\n", encoding="utf-8")
stub.chmod(stub.stat().st_mode | stat.S_IEXEC)
monkeypatch.setenv("PATH", f"{bin_dir}{os.pathsep}{os.environ['PATH']}")
def test_a_gpu_that_is_not_the_written_down_card_still_counts(tmp_path: Path, monkeypatch) -> None:
_stub_nvidia_smi(tmp_path, monkeypatch, "GPU 0: NVIDIA GeForce RTX 5090 (UUID: GPU-1234)")
monkeypatch.setattr(profiles, "uses_wsl", lambda: False)
assert profiles.gpu_available(Settings(home=tmp_path / "home")) is True
def test_a_machine_the_driver_cannot_see_reports_no_gpu(tmp_path: Path, monkeypatch) -> None:
_stub_nvidia_smi(tmp_path, monkeypatch, "nvidia-smi: command not found", exit_code=127)
monkeypatch.setattr(profiles, "uses_wsl", lambda: False)
assert profiles.gpu_available(Settings(home=tmp_path / "home")) is False
def test_the_distro_name_comes_from_settings(monkeypatch) -> None:
monkeypatch.setattr(profiles, "uses_wsl", lambda: True)
command = profiles.host_shell("true", Settings(home=Path("/tmp"), wsl_distro="Ubuntu-24.04"))
assert command == ["wsl.exe", "-d", "Ubuntu-24.04", "bash", "-lc", "true"]
def test_inside_wsl_the_local_shell_runs(monkeypatch, tmp_path: Path) -> None:
monkeypatch.setenv("WSL_DISTRO_NAME", "Ubuntu")
monkeypatch.setattr(profiles.shutil, "which", lambda name: "/mnt/c/WINDOWS/system32/wsl.exe")
assert profiles.uses_wsl() is False
monkeypatch.delenv("WSL_DISTRO_NAME")
assert profiles.uses_wsl() is True
assert profiles.host_shell("true", Settings(home=tmp_path / "home", wsl_distro="Ubuntu"))[0] == "wsl.exe"
def test_a_linux_host_is_pointed_at_the_shell_twin(monkeypatch) -> None:
monkeypatch.setattr(profiles, "uses_wsl", lambda: False)
assert profiles.launch_script("scripts/launch-sglang-live.ps1") == "scripts/launch-sglang-live.sh"
monkeypatch.setattr(profiles, "uses_wsl", lambda: True)
assert profiles.launch_script("scripts/launch-sglang-live.ps1") == "scripts/launch-sglang-live.ps1"
def test_the_training_verdict_does_not_name_a_card() -> None:
newer = {"cuda_available": True, "bf16_supported": True, "device_name": "NVIDIA GeForce RTX 5090"}
assert cuda_bf16_training(newer) is True
older = {"cuda_available": True, "bf16_supported": True, "device_name": "NVIDIA GeForce RTX 4080"}
assert cuda_bf16_training(older) is True
without_bf16 = {"cuda_available": True, "bf16_supported": False, "device_name": "NVIDIA GeForce RTX 4090"}
assert cuda_bf16_training(without_bf16) is False
class _FakeCuda:
def __init__(self, available: bool, bf16: bool) -> None:
self._available = available
self._bf16 = bf16
def is_available(self) -> bool:
return self._available
def is_bf16_supported(self) -> bool:
return self._bf16
class _FakeTorch:
def __init__(self, available: bool, bf16: bool) -> None:
self.cuda = _FakeCuda(available, bf16)
def test_training_refuses_a_machine_without_cuda_by_capability_not_card() -> None:
with pytest.raises(RuntimeError) as refused:
require_cuda_bf16(_FakeTorch(available=False, bf16=False))
assert "4090" not in str(refused.value)
require_cuda_bf16(_FakeTorch(available=True, bf16=True))
def test_every_profile_ships_a_script_for_this_host(tmp_path: Path, monkeypatch) -> None:
root = Path(__file__).resolve().parents[2]
monkeypatch.setattr(profiles, "uses_wsl", lambda: False)
linux = profiles.runtime_profile_report(Settings(home=tmp_path / "home"), root)
assert [item.name for item in linux.validations if not item.script_exists] == []
assert all(item.script.endswith(".sh") for item in linux.profiles), linux.profiles
monkeypatch.setattr(profiles, "uses_wsl", lambda: True)
windows = profiles.runtime_profile_report(Settings(home=tmp_path / "home"), root)
assert [item.name for item in windows.validations if not item.script_exists] == []
assert all(item.script.endswith(".ps1") for item in windows.profiles), windows.profiles
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
for model in Qwen/Qwen3.5-9B Qwen/Qwen3.5-0.8B Qwen/Qwen3-VL-Embedding-8B; do
.venv/bin/python ./scripts/download_models.py --model "$model"
done
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
.venv/bin/python -m pip install --upgrade --no-cache-dir --index-url https://download.pytorch.org/whl/cu130 torch==2.12.0+cu130 torchvision==0.27.0+cu130
.venv/bin/python -c 'import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.version.cuda)'
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -d .venv ]; then
python3 -m venv .venv
fi
.venv/bin/python -m pip install --upgrade pip setuptools wheel
.venv/bin/python -m pip install -e ./backend
.venv/bin/python -m pip install --upgrade --no-cache-dir unsloth unsloth_zoo
.venv/bin/python -m pip install --upgrade --no-cache-dir --index-url https://download.pytorch.org/whl/cu130 torch==2.12.0+cu130 torchvision==0.27.0+cu130
.venv/bin/python -m pip install --upgrade --no-cache-dir transformers==5.9.0 datasets==4.8.5 dill==0.4.1 multiprocess==0.70.19 fsspec==2026.2.0 safetensors==0.8.0rc0
(cd frontend && npm install)
[ -f .env ] || cp .env.example .env
.venv/bin/ely-eye status
+3 -2
View File
@@ -1,12 +1,13 @@
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot) Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) { if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the KTransformers Linux runtime." throw "WSL is required for the KTransformers Linux runtime."
} }
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim() $wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$command = @" $command = @"
set -euo pipefail set -euo pipefail
cd "$wslRoot" cd "$wslRoot"
@@ -20,4 +21,4 @@ python -m sglang.launch_server \
--cpu-offload-gb 48 --cpu-offload-gb 48
"@ "@
wsl.exe -d Ubuntu -- bash -lc $command wsl.exe -d $distro -- bash -lc $command
+15
View File
@@ -0,0 +1,15 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching SGLang." >&2
exit 1
fi
source .venv-linux/bin/activate
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
exec python -m sglang.launch_server \
--model-path Qwen/Qwen3.5-9B \
--port 8002 \
--context-length 1010000 \
--kt-cpuinfer 16 \
--cpu-offload-gb 48
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ -z "${ELY_EYE_GGUF_MODEL:-}" ]; then
echo "Set ELY_EYE_GGUF_MODEL to the absolute GGUF model path." >&2
exit 1
fi
server="${LLAMA_CPP_SERVER:-llama-server}"
if ! command -v "$server" >/dev/null 2>&1; then
echo "llama-server executable is missing. Build llama.cpp or set LLAMA_CPP_SERVER." >&2
exit 1
fi
exec "$server" \
--model "$ELY_EYE_GGUF_MODEL" \
--host 127.0.0.1 \
--port 8003 \
--ctx-size 262144 \
--flash-attn \
--cache-type-k q4_0 \
--cache-type-v q4_0
+3 -2
View File
@@ -1,13 +1,14 @@
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot) Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) { if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the SGLang Linux runtime." throw "WSL is required for the SGLang Linux runtime."
} }
$override = '{"text_config":{"rope_parameters":{"mrope_interleaved":true,"mrope_section":[11,11,10],"rope_type":"yarn","rope_theta":10000000,"partial_rotary_factor":0.25,"factor":4.0,"original_max_position_embeddings":262144}}}' $override = '{"text_config":{"rope_parameters":{"mrope_interleaved":true,"mrope_section":[11,11,10],"rope_type":"yarn","rope_theta":10000000,"partial_rotary_factor":0.25,"factor":4.0,"original_max_position_embeddings":262144}}}'
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim() $wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$command = @" $command = @"
set -euo pipefail set -euo pipefail
cd "$wslRoot" cd "$wslRoot"
@@ -22,4 +23,4 @@ python -m sglang.launch_server \
--json-model-override-args '$override' --json-model-override-args '$override'
"@ "@
wsl.exe -d Ubuntu -- bash -lc $command wsl.exe -d $distro -- bash -lc $command
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching SGLang." >&2
exit 1
fi
source .venv-linux/bin/activate
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
exec python -m sglang.launch_server \
--model-path Qwen/Qwen3.5-9B \
--port 8000 \
--tp-size 1 \
--mem-fraction-static 0.88 \
--context-length 1010000 \
--json-model-override-args '{"text_config":{"rope_parameters":{"mrope_interleaved":true,"mrope_section":[11,11,10],"rope_type":"yarn","rope_theta":10000000,"partial_rotary_factor":0.25,"factor":4.0,"original_max_position_embeddings":262144}}}'
+3 -2
View File
@@ -1,12 +1,13 @@
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot) Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) { if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the SGLang Linux runtime." throw "WSL is required for the SGLang Linux runtime."
} }
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim() $wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$command = @" $command = @"
set -euo pipefail set -euo pipefail
cd "$wslRoot" cd "$wslRoot"
@@ -23,4 +24,4 @@ python -m sglang.launch_server \
--speculative-num-draft-tokens 4 --speculative-num-draft-tokens 4
"@ "@
wsl.exe -d Ubuntu -- bash -lc $command wsl.exe -d $distro -- bash -lc $command
+18
View File
@@ -0,0 +1,18 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching SGLang." >&2
exit 1
fi
source .venv-linux/bin/activate
exec python -m sglang.launch_server \
--model-path Qwen/Qwen3.5-9B \
--port 8000 \
--tp-size 1 \
--mem-fraction-static 0.82 \
--context-length 262144 \
--reasoning-parser qwen3 \
--speculative-algo NEXTN \
--speculative-num-steps 3 \
--speculative-num-draft-tokens 4
+3 -2
View File
@@ -1,12 +1,13 @@
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot) Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) { if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the vLLM Linux runtime." throw "WSL is required for the vLLM Linux runtime."
} }
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim() $wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$ropeScaling = '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}' $ropeScaling = '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}'
$command = @" $command = @"
set -euo pipefail set -euo pipefail
@@ -23,4 +24,4 @@ python -m vllm.entrypoints.openai.api_server \
--rope-scaling '$ropeScaling' --rope-scaling '$ropeScaling'
"@ "@
wsl.exe -d Ubuntu -- bash -lc $command wsl.exe -d $distro -- bash -lc $command
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux-vllm/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching vLLM." >&2
exit 1
fi
source .venv-linux-vllm/bin/activate
exec python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen3.5-9B \
--host 127.0.0.1 \
--port 8001 \
--max-model-len 1010000 \
--enable-prefix-caching \
--gpu-memory-utilization 0.88 \
--trust-remote-code \
--rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}'
+3 -2
View File
@@ -1,12 +1,13 @@
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot) Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) { if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for the vLLM Linux runtime." throw "WSL is required for the vLLM Linux runtime."
} }
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim() $wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
$command = @" $command = @"
set -euo pipefail set -euo pipefail
cd "$wslRoot" cd "$wslRoot"
@@ -21,4 +22,4 @@ python -m vllm.entrypoints.openai.api_server \
--trust-remote-code --trust-remote-code
"@ "@
wsl.exe -d Ubuntu -- bash -lc $command wsl.exe -d $distro -- bash -lc $command
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [ ! -f .venv-linux-vllm/bin/activate ]; then
echo "Run scripts/setup-linux-runtimes.sh before launching vLLM." >&2
exit 1
fi
source .venv-linux-vllm/bin/activate
exec python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen3.5-9B \
--host 127.0.0.1 \
--port 8001 \
--max-model-len 262144 \
--enable-prefix-caching \
--gpu-memory-utilization 0.82 \
--trust-remote-code
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
max_steps="${MAX_STEPS:-37}"
rank="${RANK:-16}"
alpha="${ALPHA:-32}"
max_length="${MAX_LENGTH:-8192}"
gradient_accumulation_steps="${GRADIENT_ACCUMULATION_STEPS:-4}"
cartridge_id="${CARTRIDGE_ID:-}"
dataset=.ely_eye/training/hme_prd_sft.jsonl
adapter=.ely_eye/adapters/hme_core_lora_qwen35_9b
.venv/bin/ely-eye build-prd-training-data \
--prd-path ./PRD.md \
--output-path "$dataset" \
--max-sections 48
args=(train-lora "$dataset" "$adapter"
--max-steps "$max_steps"
--kind hme_core_lora
--rank "$rank"
--alpha "$alpha"
--max-length "$max_length"
--gradient-accumulation-steps "$gradient_accumulation_steps")
if [ -n "$cartridge_id" ]; then
args+=(--cartridge-id "$cartridge_id")
fi
.venv/bin/ely-eye "${args[@]}"
echo "Adapter ready: $adapter"
echo "Use with: ELY_EYE_ADAPTER_DIR=$adapter"
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
exec .venv/bin/ely-eye serve --host 127.0.0.1 --port 8765
+5
View File
@@ -0,0 +1,5 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
cd frontend
exec npm run dev
+10 -9
View File
@@ -1,20 +1,21 @@
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
Set-Location (Split-Path -Parent $PSScriptRoot) Set-Location (Split-Path -Parent $PSScriptRoot)
$distro = if ($env:ELY_EYE_WSL_DISTRO) { $env:ELY_EYE_WSL_DISTRO } else { "Ubuntu" }
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) { if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
throw "WSL is required for SGLang, vLLM, and KTransformers runtime profiles." throw "WSL is required for SGLang, vLLM, and KTransformers runtime profiles."
} }
wsl.exe -d Ubuntu -- bash -lc "command -v nvidia-smi >/dev/null && nvidia-smi -L >/dev/null" wsl.exe -d $distro -- bash -lc "command -v nvidia-smi >/dev/null && nvidia-smi -L >/dev/null"
wsl.exe -d Ubuntu -- bash -lc "command -v curl >/dev/null || (apt-get update && apt-get install -y curl)" wsl.exe -d $distro -- bash -lc "command -v curl >/dev/null || (apt-get update && apt-get install -y curl)"
wsl.exe -d Ubuntu -- bash -lc "command -v /root/.local/bin/uv >/dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh" wsl.exe -d $distro -- bash -lc "command -v /root/.local/bin/uv >/dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh"
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim() $wslRoot = (wsl.exe -d $distro wslpath -a (Get-Location).Path).Trim()
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux --python 3.12" wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux --python 3.12"
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux-vllm --python 3.12" wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux-vllm --python 3.12"
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-sglang-linux.txt" wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-sglang-linux.txt"
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux-vllm/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-vllm-linux.txt" wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && source .venv-linux-vllm/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-vllm-linux.txt"
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-ktransformers-linux.txt" wsl.exe -d $distro -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-ktransformers-linux.txt"
if (-not (Get-Command llama-server -ErrorAction SilentlyContinue)) { if (-not (Get-Command llama-server -ErrorAction SilentlyContinue)) {
winget install --id ggml.llamacpp -e --accept-package-agreements --accept-source-agreements winget install --id ggml.llamacpp -e --accept-package-agreements --accept-source-agreements
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if ! command -v nvidia-smi >/dev/null 2>&1; then
echo "nvidia-smi is missing; these runtimes need a CUDA host." >&2
exit 1
fi
nvidia-smi -L >/dev/null
uv_bin="${ELY_EYE_UV:-$HOME/.local/bin/uv}"
if ! command -v "$uv_bin" >/dev/null 2>&1; then
curl -LsSf https://astral.sh/uv/install.sh | sh
uv_bin="$HOME/.local/bin/uv"
fi
for venv in .venv-linux .venv-linux-vllm; do
"$uv_bin" venv "$venv" --python 3.12
done
"$uv_bin" pip install --python .venv-linux/bin/python --prerelease=allow -r backend/requirements-sglang-linux.txt
"$uv_bin" pip install --python .venv-linux-vllm/bin/python --prerelease=allow -r backend/requirements-vllm-linux.txt
"$uv_bin" pip install --python .venv-linux/bin/python --prerelease=allow -r backend/requirements-ktransformers-linux.txt
.venv/bin/ely-eye runtime-profiles
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
.venv/bin/python -m compileall ./backend/src
.venv/bin/ely-eye status
(cd frontend && npm run build)