Ely-Eye — local multimodal context operating system on Qwen3.5-9B
Real-models implementation: multimodal ingestion, Context Cartridge memory, sparse + Qwen embedding retrieval, Cache Fabric, Qwen runtime adapters, verifier contracts, Visual HashHop proofs, BF16 LoRA fine-tuning, and a local dashboard. Verified by a 15-check PRD proof suite on the live model.
This commit is contained in:
ZacharyZhang-NY committed
commit
0ee99e271d
64 files changed
+14890
No files matched your search
@@ -0,0 +1,9 @@
|
||||
ELY_EYE_HOME=.ely_eye
|
||||
ELY_EYE_RUNTIME_BACKEND=transformers
|
||||
ELY_EYE_SGLANG_BASE_URL=http://127.0.0.1:8000/v1
|
||||
ELY_EYE_SGLANG_API_KEY=ely-eye-local
|
||||
ELY_EYE_MODEL_ID=Qwen/Qwen3.5-9B
|
||||
ELY_EYE_ADAPTER_DIR=.ely_eye/adapters/hme_core_lora_qwen35_9b
|
||||
ELY_EYE_PLANNER_MODEL_ID=Qwen/Qwen3.5-0.8B
|
||||
ELY_EYE_EMBEDDING_MODEL_ID=Qwen/Qwen3-VL-Embedding-8B
|
||||
ELY_EYE_DENSE_EMBEDDINGS_ENABLED=false
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
.venv/
|
||||
.venv-linux*/
|
||||
node_modules/
|
||||
dist/
|
||||
build/
|
||||
.ely_eye/
|
||||
*.pyc
|
||||
__pycache__/
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
.ruff_cache/
|
||||
.env
|
||||
.env.local
|
||||
frontend/.vite/
|
||||
frontend/dist/
|
||||
backend/src/ely_eye.egg-info/
|
||||
*.log
|
||||
*.tsbuildinfo
|
||||
frontend/vite.config.d.ts
|
||||
frontend/vite.config.js
|
||||
unsloth_compiled_cache/
|
||||
*.sqlite-wal
|
||||
*.sqlite-shm
|
||||
PRD.md
|
||||
AGENTS.md
|
||||
CLAUDE.md
|
||||
designreference.html
|
||||
# Local landing site — deployed to Cloudflare, not part of this repo
|
||||
/ely-eye-web/
|
||||
+26
@@ -0,0 +1,26 @@
|
||||
FROM nvidia/cuda:13.3.0-runtime-ubuntu24.04
|
||||
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
ELY_EYE_HOME=/data/ely-eye \
|
||||
ELY_EYE_RUNTIME_BACKEND=sglang \
|
||||
ELY_EYE_SGLANG_BASE_URL=http://127.0.0.1:8000/v1
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends python3.12 python3.12-venv python3-pip nodejs npm ffmpeg \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY backend ./backend
|
||||
COPY frontend ./frontend
|
||||
COPY README.md PRD.md ./
|
||||
|
||||
RUN python3.12 -m venv /opt/ely-eye \
|
||||
&& /opt/ely-eye/bin/pip install --upgrade pip setuptools wheel \
|
||||
&& /opt/ely-eye/bin/pip install -e ./backend \
|
||||
&& cd frontend \
|
||||
&& npm install \
|
||||
&& npm run build
|
||||
|
||||
EXPOSE 8765
|
||||
CMD ["/opt/ely-eye/bin/ely-eye", "serve", "--host", "0.0.0.0", "--port", "8765"]
|
||||
@@ -0,0 +1,150 @@
|
||||
# Ely-Eye
|
||||
|
||||
Ely-Eye is a local multimodal context operating system for Qwen3.5-9B. The implementation follows `PRD.md`: ingestion, Evidence Atoms, Context Cartridge, sparse and optional Qwen embedding retrieval, Cache Fabric, Qwen runtime adapters, verifier contracts, Visual HashHop proof generation, BF16 LoRA fine-tuning, adapter cartridge binding, and a local dashboard.
|
||||
|
||||
## Local Run
|
||||
|
||||
```powershell
|
||||
Set-Location C:\Users\zacharyzhang\Documents\Github\ELY-EYE
|
||||
.\scripts\install.ps1
|
||||
.\scripts\setup-linux-runtimes.ps1
|
||||
.\scripts\serve-backend.ps1
|
||||
.\scripts\serve-frontend.ps1
|
||||
```
|
||||
|
||||
Backend: `http://127.0.0.1:8765`
|
||||
Frontend: `http://127.0.0.1:5173`
|
||||
|
||||
## Model Runtime
|
||||
|
||||
SGLang is supported as an OpenAI-compatible runtime target. The Windows local runtime uses Transformers.
|
||||
|
||||
```powershell
|
||||
.\scripts\download-models.ps1
|
||||
.\scripts\launch-sglang-live.ps1
|
||||
```
|
||||
|
||||
The Transformers backend is available by setting:
|
||||
|
||||
```powershell
|
||||
$env:ELY_EYE_RUNTIME_BACKEND = "transformers"
|
||||
```
|
||||
|
||||
When `ELY_EYE_ADAPTER_DIR` is empty, the Transformers runtime auto-selects the
|
||||
latest trained `HME-Core-LoRA` adapter whose base model matches
|
||||
`Qwen/Qwen3.5-9B`. `ely-eye status` reports the resolved adapter path, kind,
|
||||
adapter id, and adapter hash.
|
||||
|
||||
The runtime generation proof loads `Qwen/Qwen3.5-9B`, injects the trained
|
||||
`HME-Core-LoRA` adapter, generates a PRD-cited answer, and writes the raw model
|
||||
output plus verifier report:
|
||||
|
||||
```powershell
|
||||
.\.venv\Scripts\ely-eye.exe runtime-generation-proof
|
||||
```
|
||||
|
||||
## Fine-Tuning
|
||||
|
||||
The PRD-derived datasets are built from real `PRD.md` sections, the ingested PRD Evidence Atom, and local UI proof images. Training runs real BF16 LoRA optimizer updates on RTX 4090 through Unsloth-patched Transformers, PEFT, and bitsandbytes, then writes PEFT adapters, `adapter_manifest.json`, `training_trace.jsonl`, `training_summary.json`, `training_proof.json`, dataset hashes, adapter hashes, trainable parameter hashes, optimizer update norms, per-step loss, CUDA/BF16 environment metadata, target modules, adapter weight SHA-256, LoRA tensor statistics, and cartridge binding metadata. The training path requires CUDA BF16 and bitsandbytes `PagedAdamW8bit`; every proof records proof version, precision, optimizer family, checkpointing mode, max length, framework, and target-module coverage.
|
||||
|
||||
```powershell
|
||||
.\scripts\run-prd-finetune.ps1 -MaxSteps 37 -CartridgeId ely_eye_prd_7e5cd27b8284
|
||||
.\scripts\run-all-prd-finetunes.ps1 -CartridgeId ely_eye_prd_7e5cd27b8284
|
||||
$env:ELY_EYE_ADAPTER_DIR = ".ely_eye\adapters\hme_core_lora_qwen35_9b"
|
||||
.\.venv\Scripts\ely-eye.exe status
|
||||
```
|
||||
|
||||
The PRD adapter set is:
|
||||
|
||||
```text
|
||||
HME-Core-LoRA Qwen3.5-9B attention/MLP
|
||||
HME-Vision-LoRA Qwen3.5-9B visual projector and selected vision layers
|
||||
HME-TTT-VL Qwen3.5-9B mutable MLP and visual adapter
|
||||
HME-Visual-MTP Qwen3.5-9B draft-head LoRA
|
||||
HME-Router Qwen3.5-0.8B context planner
|
||||
HME-Retrieval Qwen3-VL-Embedding-8B contrastive retrieval adapter
|
||||
```
|
||||
|
||||
## Core Commands
|
||||
|
||||
```powershell
|
||||
.\.venv\Scripts\ely-eye.exe status
|
||||
.\.venv\Scripts\ely-eye.exe ingest C:\path\to\package --cartridge-name project-cartridge
|
||||
.\.venv\Scripts\ely-eye.exe hashhop --kind visual --hops 2 --token-equivalent 262144
|
||||
.\.venv\Scripts\ely-eye.exe validate-training-data C:\path\to\train.jsonl
|
||||
.\.venv\Scripts\ely-eye.exe train-lora C:\path\to\train.jsonl C:\path\to\adapter
|
||||
```
|
||||
|
||||
Adapter proof state is available through `GET /api/adapters` and the dashboard Adapter Matrix, including training method, precision, optimizer family, checkpointing state, optimizer update counts, and loss delta.
|
||||
|
||||
## PRD Proof Suite
|
||||
|
||||
The proof suite verifies the PRD adapter set with CUDA BF16 optimizer updates,
|
||||
changed trainable parameter hashes, matching manifest/summary/proof hashes,
|
||||
proof-versioned BF16 LoRA contracts, 8192-token max length, paged 8-bit AdamW,
|
||||
checkpointing, target-module coverage, nonzero LoRA safetensors, runtime adapter binding, runtime adapter generation,
|
||||
cartridge replay, Visual HashHop assets, model-solved text HashHop, model-solved
|
||||
Visual HashHop 1-hop and 2-hop pass-rate gates, learned memory adapter binding,
|
||||
100M memory capsule commitments, Memory DNA, cache trace evidence, and Visual
|
||||
Contradiction Lens cross-version drift detection (>=80% recall with zero false
|
||||
positives across the PRD 11.15 taxonomy: design-token, layout, typography, copy,
|
||||
visual-code, and temporal drift).
|
||||
It also verifies code repository input coverage through a local structure index,
|
||||
call graph extraction, and git issue/PR history association.
|
||||
It writes proof JSON, generated HashHop visual blocks, raw model HashHop
|
||||
outputs, Visual HashHop arena images, and official benchmark sample predictions under `.ely_eye/data/eval_proofs/`, then
|
||||
attaches the suite to the active Context Cartridge under `eval_proofs/`.
|
||||
MMMU-Pro and OCRBench sample predictions must include finite local scores before
|
||||
the standard benchmark check passes. Every standard benchmark sample must include
|
||||
a sample SHA-256 and source dataset SHA. LongVideoBench uses the official
|
||||
`longvideobench/LongVideoBench-Meta` public validation metadata source while the
|
||||
canonical gated dataset remains tracked in the benchmark registry.
|
||||
|
||||
Latest local PRD proof suite: `prd_proof_suite_6bc3fa744c60a0e6` (15 checks passed).
|
||||
|
||||
```powershell
|
||||
.\.venv\Scripts\ely-eye.exe proof-suite --cartridge-id ely_eye_prd_7e5cd27b8284
|
||||
.\.venv\Scripts\ely-eye.exe latest-proof-suite
|
||||
.\.venv\Scripts\ely-eye.exe runtime-generation-proof
|
||||
.\.venv\Scripts\ely-eye.exe visual-contradiction-proof
|
||||
.\.venv\Scripts\ely-eye.exe refresh-cartridge ely_eye_prd_7e5cd27b8284
|
||||
.\.venv\Scripts\ely-eye.exe finalize-cartridge ely_eye_prd_7e5cd27b8284
|
||||
.\.venv\Scripts\ely-eye.exe benchmark-registry
|
||||
.\.venv\Scripts\ely-eye.exe benchmark-sources
|
||||
.\.venv\Scripts\ely-eye.exe benchmark-samples
|
||||
.\.venv\Scripts\ely-eye.exe benchmark-predictions
|
||||
.\.venv\Scripts\ely-eye.exe runtime-profiles
|
||||
```
|
||||
|
||||
The same surface is available through `POST /api/proof-suite`, `GET /api/proof-suite/latest`, `POST /api/visual-contradiction-proof`, `POST /api/cartridges/{cartridge_id}/refresh`, `POST /api/cartridges/{cartridge_id}/finalize`, `GET /api/cartridges/{cartridge_id}/assets`, `GET /api/benchmarks/registry`, `GET /api/benchmarks/sources`, `GET /api/benchmarks/samples`, `GET /api/benchmarks/predictions`, `GET /api/runtime/profiles`, and the dashboard PRD Proof Suite panel.
|
||||
|
||||
Runtime launchers:
|
||||
`ely-eye runtime-profiles` validates the WSL GPU, Linux runtime packages,
|
||||
SGLang/vLLM launcher modules, KTransformers import path, and llama.cpp help
|
||||
output before the proof suite marks runtime delivery ready.
|
||||
|
||||
```powershell
|
||||
.\scripts\setup-linux-runtimes.ps1
|
||||
.\scripts\launch-sglang-live.ps1
|
||||
.\scripts\launch-sglang-extreme.ps1
|
||||
.\scripts\launch-vllm-live.ps1
|
||||
.\scripts\launch-vllm-extreme.ps1
|
||||
.\scripts\launch-ktransformers-extreme.ps1
|
||||
.\scripts\launch-llamacpp-offline.ps1
|
||||
```
|
||||
|
||||
## Data Layout
|
||||
|
||||
Runtime state is stored under `.ely_eye/` by default.
|
||||
|
||||
```text
|
||||
.ely_eye/
|
||||
├─ adapters/
|
||||
├─ data/ely_eye.sqlite
|
||||
├─ training/
|
||||
├─ objects/
|
||||
├─ cache/
|
||||
└─ cartridges/
|
||||
```
|
||||
|
||||
Context Cartridge artifacts are materialized as `manifest.json`, `atoms.parquet`, `sources.json`, `sparse_index.bm25.zst`, `temporal_graph.sqlite`, `text_vectors.fp16.zstd`, `visual_vectors.int8.zstd`, `kv_snapshots.kivi`, `memory_capsule.json.zst`, `memory_capsule_index.json`, `ttt_vl_adapter.safetensors`, `visual_mtp_head.safetensors`, `eval_proofs/`, and attached adapter folders.
|
||||
@@ -0,0 +1,144 @@
|
||||
# Ely-Eye Training Artifacts
|
||||
|
||||
Generated on 2026-05-28 from the local `PRD.md`, ingested Evidence Atom
|
||||
`PRD_0b88b12114634a56:text`, visual Evidence Atom
|
||||
`ely_eye_dashboard_0180927ffdd3f8b0:image`, local UI proof images, and Qwen
|
||||
model snapshots. Every adapter has `training_trace.jsonl`, `training_summary.json`,
|
||||
and `training_proof.json` with RTX 4090 CUDA BF16 execution metadata, proof
|
||||
version 2, BF16 LoRA method, 8192-token max length, paged 8-bit AdamW,
|
||||
Unsloth-patched checkpointing mode, per-step loss, trainable parameter hashes
|
||||
before and after training, optimizer update norms, dataset hashes, environment
|
||||
versions, adapter weight SHA-256, LoRA tensor statistics, runtime adapter binding
|
||||
proof, runtime adapter generation proof, and proof-suite verified nonzero LoRA
|
||||
safetensors.
|
||||
|
||||
| Adapter | Base model | Samples | Steps | Updates | Max length | Trainable params | Final loss | Loss delta | Max update L2 | Seconds | SHA-256 |
|
||||
|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---|
|
||||
| HME-Core-LoRA | Qwen/Qwen3.5-9B | 37 | 37 | 10 | 8,192 | 29,097,984 | 0.0001596613001311198 | 0.22773139642958995 | 0.8183563251107077 | 104.4 | 8f97890e802f9adfa3bcb2c02b33d7b39dd8954654d7057d4058e1331e7b5df4 |
|
||||
| HME-Vision-LoRA | Qwen/Qwen3.5-9B | 10 | 20 | 5 | 8,192 | 1,996,672 | 2.0484652519226074 | -0.11261439323425293 | 0.6595521616294707 | 49.3 | 593f7295f466c5fba07bdae26d1d8798c5affbad8f3863cda9f1881d06453dec |
|
||||
| HME-TTT-VL | Qwen/Qwen3.5-9B | 10 | 20 | 5 | 8,192 | 8,288,128 | 1.5277884006500244 | 0.4080624580383301 | 1.2924649382409095 | 51.2 | 030b37480155a8eaa6ef4c2dfa819d5f2564838935654f0d826235eb66cc9de5 |
|
||||
| HME-Visual-MTP | Qwen/Qwen3.5-9B | 10 | 20 | 5 | 8,192 | 2,019,328 | 2.066615343093872 | -0.13076448440551758 | 0.1419292917069751 | 75.8 | 489da9460f51693fa8775ad4975a18f7400dd3fc41eac401d00305617c627717 |
|
||||
| HME-Router | Qwen/Qwen3.5-0.8B | 8 | 16 | 4 | 8,192 | 3,194,880 | 1.2781919240951538 | 1.0438536405563354 | 0.2871291982199057 | 28.8 | 9ef884a1078df71ba73376a29737a27e22e2ff25854352bfc2c7d960102d886e |
|
||||
| HME-Retrieval | Qwen/Qwen3-VL-Embedding-8B | 32 | 32 | 32 | 8,192 | 21,823,488 | 0.5665192604064941 | 0.91921067237854 | 1.3181151117874825 | 55.3 | 6490f03e445eb70da7020be992c6eccd3ca2b0e4c48e73a80a819ddac5faa8f3 |
|
||||
|
||||
Adapter weight proof:
|
||||
|
||||
| Adapter | LoRA tensors | Nonzero elements | Weight SHA-256 |
|
||||
|---|---:|---:|---|
|
||||
| HME-Core-LoRA | 256 | 29,097,984 | 4673ac90abdd8ef952dad9c2292aecadba161669c3b1fc3d1af5c075cf0ae62d |
|
||||
| HME-Vision-LoRA | 220 | 1,992,340 | dde2eb9051866634f9173366a6ffa3fbf9219e5b6d640eafb7f99d0258c52422 |
|
||||
| HME-TTT-VL | 412 | 8,283,800 | 9a572ac51c6df08e38898c6c30acd8ed6530c7e852611552baaec6a67862e6c5 |
|
||||
| HME-Visual-MTP | 4 | 1,028,316 | 58d058b862a542c042d27234b5b22344ba02ee6e016be6c8838b329ffea1c6f9 |
|
||||
| HME-Router | 192 | 3,194,880 | 90079b8f2c244efb00c2285816b98f51ff03cf06a9ba2335ee5bbdf24a1aff8f |
|
||||
| HME-Retrieval | 504 | 21,823,488 | bcb1381888ae84e82ab18256e152191e56a0892c46cdac81874a9c1fb1af14ee |
|
||||
|
||||
Local adapter roots:
|
||||
|
||||
```text
|
||||
.ely_eye/adapters/hme_core_lora_qwen35_9b
|
||||
.ely_eye/adapters/hme_vision_lora_qwen35_9b
|
||||
.ely_eye/adapters/hme_ttt_vl_qwen35_9b
|
||||
.ely_eye/adapters/hme_visual_mtp_qwen35_9b
|
||||
.ely_eye/adapters/hme_router_qwen35_08b
|
||||
.ely_eye/adapters/hme_retrieval_qwen3_vl_embedding_8b
|
||||
```
|
||||
|
||||
Cartridge binding:
|
||||
|
||||
```text
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/adapters/hme_core_lora
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/adapters/hme_vision_lora
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/adapters/hme_ttt_vl
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/adapters/hme_visual_mtp
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/adapters/hme_router
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/adapters/hme_retrieval
|
||||
```
|
||||
|
||||
PRD proof suite:
|
||||
|
||||
```text
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/proof_suite.json
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/proof_suite.json
|
||||
```
|
||||
|
||||
Runtime generation proof:
|
||||
|
||||
```text
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/runtime_generation.json
|
||||
transformers generated 218 tokens with hme_core_lora citation accuracy 1.00
|
||||
```
|
||||
|
||||
100M memory capsule proof:
|
||||
|
||||
```text
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/memory_capsule.json.zst
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/memory_capsule_index.json
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/memory_capsule.json
|
||||
The Context Cartridge materialized a 100,000,000 token-equivalent memory capsule with 4,096 committed segments and a verified segment Merkle root.
|
||||
```
|
||||
|
||||
Model-solved HashHop proofs:
|
||||
|
||||
```text
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/0a10e6394ada0da7/model_text_hashhop.json
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/60c9d5f5f1648ea0/model_visual_hashhop.json
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/a29a310b31f9cef5/model_visual_hashhop.json
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/7f7b8b1f54ec3d75/model_visual_hashhop.json
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/4f24efd13342694c/model_visual_hashhop.json
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/a3f49402ef2710a3/model_visual_hashhop.json
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/0bd17526ab51392b/model_visual_hashhop.json
|
||||
Qwen/Qwen3.5-9B with HME-Core-LoRA solved a 1.01M token-equivalent 2-hop text HashHop and 6 Visual HashHop arenas at 262K token-equivalent. The Visual HashHop gate passed 7/7 model-solved proofs, with 262K 1-hop=1.00 and 2-hop=1.00.
|
||||
```
|
||||
|
||||
Benchmark source and sample proof:
|
||||
|
||||
```text
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/standard_benchmark_sources.json
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/standard_benchmark_samples.json
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/standard_benchmark_predictions.json
|
||||
MMMU-Pro, OCRBench, MMLongBench-Doc, VideoMME, LongVideoBench, RULER, and LongBench v2 loaded official streaming samples or official public metadata samples with sample hashes and source dataset SHA values. LongVideoBench uses `longvideobench/LongVideoBench-Meta` validation metadata with canonical `longvideobench/LongVideoBench` registry tracking. MMMU-Pro and OCRBench generated local Qwen/Qwen3.5-9B + HME-Core-LoRA predictions from official image samples, with local sample scores MMMU-Pro=1.00 and OCRBench=1.00.
|
||||
```
|
||||
|
||||
Repository input proof:
|
||||
|
||||
```text
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/repository_input.json
|
||||
The repository input proof indexed 94 files, 49 code files, 11,977 call graph edges, and 26 local git commits for PRD 10.1 code repository coverage.
|
||||
```
|
||||
|
||||
Runtime profile readiness:
|
||||
|
||||
```text
|
||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/runtime_profiles.json
|
||||
SGLang 0.5.12.post1, vLLM 0.21.0, KTransformers 0.6.2.post4, WSL RTX 4090 GPU visibility, launcher module probes, and winget llama-server help output are verified.
|
||||
```
|
||||
|
||||
Checks:
|
||||
|
||||
```text
|
||||
Adapter Training Proof passed # proof-versioned BF16 LoRA + paged 8-bit AdamW + nonzero LoRA safetensors
|
||||
Runtime Adapter Binding passed
|
||||
Runtime Adapter Generation passed
|
||||
Cartridge Physical Assets passed
|
||||
100M Memory Capsule passed
|
||||
Context Cartridge Replay passed
|
||||
Visual HashHop passed # 7/7 model-solved proofs
|
||||
Standard Benchmark Registry passed # MMMU-Pro=1.00, OCRBench=1.00 local samples
|
||||
Repository Input Coverage passed # 94 files + 11977 call graph edges + local git history
|
||||
Runtime Profile Delivery passed
|
||||
Research Module Delivery passed # executable module gates, evidence graph edge, probability sum
|
||||
Learned Memory Recall passed
|
||||
Memory DNA passed
|
||||
Cache Trace passed
|
||||
```
|
||||
|
||||
Cartridge physical assets:
|
||||
|
||||
```text
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/text_vectors.fp16.zstd
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/visual_vectors.int8.zstd # item_count=1
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/kv_snapshots.kivi
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/memory_capsule.json.zst # token_equivalent=100000000, segment_count=4096
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/ttt_vl_adapter.safetensors
|
||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/visual_mtp_head.safetensors
|
||||
```
|
||||
@@ -0,0 +1,3 @@
|
||||
# Ely-Eye Backend
|
||||
|
||||
FastAPI, ingestion, memory, retrieval, runtime, verifier, evaluation, and training services for Ely-Eye.
|
||||
@@ -0,0 +1,56 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=80", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "ely-eye"
|
||||
version = "0.1.0"
|
||||
description = "Local multimodal context operating system for Qwen3.5-9B."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"accelerate>=1.12.0",
|
||||
"bitsandbytes>=0.49.2",
|
||||
"datasets>=4.8.5",
|
||||
"fastapi>=0.124.0",
|
||||
"huggingface-hub>=1.2.0",
|
||||
"httpx>=0.28.0",
|
||||
"networkx>=3.6.0",
|
||||
"ninja>=1.13.0",
|
||||
"numpy>=2.3.0",
|
||||
"opencv-python-headless>=4.12.0.0",
|
||||
"orjson>=3.11.0",
|
||||
"peft>=0.19.0",
|
||||
"pillow>=12.0.0",
|
||||
"platformdirs>=4.5.0",
|
||||
"pyarrow>=22.0.0",
|
||||
"pydantic>=2.12.0",
|
||||
"pydantic-settings>=2.12.0",
|
||||
"pymupdf>=1.26.0",
|
||||
"python-multipart>=0.0.20",
|
||||
"qwen-vl-utils>=0.0.14",
|
||||
"rank-bm25>=0.2.2",
|
||||
"rapidocr-onnxruntime>=1.4.4",
|
||||
"safetensors>=0.7.0",
|
||||
"sentence-transformers>=5.2.0",
|
||||
"torch>=2.12.0",
|
||||
"torchvision>=0.27.0",
|
||||
"transformers>=4.57.0",
|
||||
"typer>=0.20.0",
|
||||
"uvicorn[standard]>=0.38.0",
|
||||
"zstandard>=0.25.0"
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
ely-eye = "ely_eye.cli:app"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py312"
|
||||
|
||||
[tool.pyright]
|
||||
pythonVersion = "3.12"
|
||||
include = ["src"]
|
||||
@@ -0,0 +1,5 @@
|
||||
ktransformers>=0.3.0
|
||||
sglang-kt>=0.6.2
|
||||
torchvision==0.24.1
|
||||
transformers>=4.57.0
|
||||
qwen-vl-utils>=0.0.14
|
||||
@@ -0,0 +1 @@
|
||||
sglang>=0.5.10
|
||||
@@ -0,0 +1,3 @@
|
||||
vllm>=0.12.0
|
||||
transformers>=4.57.0
|
||||
qwen-vl-utils>=0.0.14
|
||||
@@ -0,0 +1,4 @@
|
||||
"""Ely-Eye production package."""
|
||||
|
||||
__all__ = ["__version__"]
|
||||
__version__ = "0.1.0"
|
||||
@@ -0,0 +1,398 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import Depends, FastAPI, File, HTTPException, UploadFile
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from pydantic import BaseModel
|
||||
|
||||
from . import __version__
|
||||
from .benchmarks import (
|
||||
benchmark_prediction_report,
|
||||
benchmark_registry_report,
|
||||
benchmark_sample_report,
|
||||
benchmark_source_report,
|
||||
)
|
||||
from .cache_fabric import CacheFabric
|
||||
from .cartridge import CartridgeService
|
||||
from .cartridge_assets import CartridgeAssetService
|
||||
from .compiler import ContextCompiler
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .embeddings import EmbeddingService
|
||||
from .evals import HashHopEvaluator, PRDProofSuite
|
||||
from .evidence_graph import EvidenceGraphService
|
||||
from .ingestion import IngestionService
|
||||
from .profiles import runtime_profile_report
|
||||
from .research_modules import ResearchModuleService
|
||||
from .runtime import QwenRuntime, RuntimeUnavailableError
|
||||
from .schemas import (
|
||||
AdapterManifest,
|
||||
AdapterStatus,
|
||||
AnswerResponse,
|
||||
BenchmarkRegistryReport,
|
||||
BenchmarkPredictionReport,
|
||||
BenchmarkSampleReport,
|
||||
BenchmarkSourceReport,
|
||||
CartridgeAssetReport,
|
||||
HashHopProof,
|
||||
MemoryStatus,
|
||||
PathIngestRequest,
|
||||
ProofCheck,
|
||||
ProofSuiteReport,
|
||||
RuntimeProfile,
|
||||
RuntimeProfileReport,
|
||||
SystemStatus,
|
||||
UploadIngestRequest,
|
||||
)
|
||||
from .training import safetensor_weight_stats
|
||||
from .verifier import Verifier, VisualContradictionLens
|
||||
|
||||
|
||||
class ChatRequest(BaseModel):
|
||||
question: str
|
||||
profile: RuntimeProfile = RuntimeProfile.library_100m
|
||||
|
||||
|
||||
class HashHopRequest(BaseModel):
|
||||
kind: str = "visual"
|
||||
hops: int = 2
|
||||
token_equivalent: int = 262_144
|
||||
|
||||
|
||||
class ProofSuiteRequest(BaseModel):
|
||||
cartridge_id: str | None = None
|
||||
|
||||
|
||||
def settings_dep() -> Settings:
|
||||
return get_settings()
|
||||
|
||||
|
||||
def create_app() -> FastAPI:
|
||||
app = FastAPI(title="Ely-Eye", version=__version__)
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["http://localhost:5173", "http://127.0.0.1:5173"],
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
@app.get("/api/status", response_model=SystemStatus)
|
||||
def status(settings: Annotated[Settings, Depends(settings_dep)]) -> SystemStatus:
|
||||
db = Database(settings)
|
||||
cache = CacheFabric(settings, db)
|
||||
runtime = QwenRuntime(settings).status()
|
||||
layers = cache.statuses()
|
||||
active_token_equivalent = db.token_equivalent_sum()
|
||||
memory = MemoryStatus(
|
||||
active_atoms=db.atom_count(),
|
||||
active_token_equivalent=active_token_equivalent,
|
||||
cartridges=db.cartridge_count(),
|
||||
library_token_equivalent=max(active_token_equivalent, db.cartridge_token_equivalent_sum()),
|
||||
layers=layers,
|
||||
)
|
||||
return SystemStatus(
|
||||
version=__version__,
|
||||
runtime=runtime,
|
||||
memory=memory,
|
||||
data_home=str(settings.home),
|
||||
profiles={
|
||||
"Live Demo": {"context_tokens": settings.live_context_tokens, "purpose": "stable 262K workflow"},
|
||||
"Extreme Context": {"context_tokens": settings.extreme_context_tokens, "purpose": "YaRN 1.01M workflow"},
|
||||
"100M Library": {"context_tokens": settings.library_target_tokens, "purpose": "Context Cartridge memory"},
|
||||
"Research Theater": {"context_tokens": settings.live_context_tokens, "purpose": "V-NSA, DVKV, TTT-VL proofs"},
|
||||
},
|
||||
)
|
||||
|
||||
@app.post("/api/ingest/path")
|
||||
def ingest_path(
|
||||
request: PathIngestRequest,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
):
|
||||
try:
|
||||
return IngestionService(settings).ingest_path(request.path, request.cartridge_name)
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
@app.post("/api/ingest/upload")
|
||||
def ingest_upload(
|
||||
metadata: Annotated[UploadIngestRequest, Depends()],
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
files: Annotated[list[UploadFile], File()],
|
||||
):
|
||||
upload_root = settings.data_dir / "uploads"
|
||||
upload_root.mkdir(parents=True, exist_ok=True)
|
||||
saved_paths: list[Path] = []
|
||||
total_size = 0
|
||||
for uploaded in files:
|
||||
target = upload_root / sanitize_filename(uploaded.filename or "upload.bin")
|
||||
with target.open("wb") as handle:
|
||||
shutil.copyfileobj(uploaded.file, handle)
|
||||
total_size += target.stat().st_size
|
||||
if total_size > settings.upload_max_bytes:
|
||||
raise HTTPException(status_code=413, detail="Upload exceeds configured maximum size.")
|
||||
saved_paths.append(target)
|
||||
|
||||
service = IngestionService(settings)
|
||||
aggregate = None
|
||||
for path in saved_paths:
|
||||
result = service.ingest_path(path, None)
|
||||
if aggregate is None:
|
||||
aggregate = result
|
||||
else:
|
||||
aggregate.source_count += result.source_count
|
||||
aggregate.atom_count += result.atom_count
|
||||
aggregate.token_equivalent += result.token_equivalent
|
||||
aggregate.sources.extend(result.sources)
|
||||
if aggregate is None:
|
||||
raise HTTPException(status_code=400, detail="No files were uploaded.")
|
||||
if metadata.cartridge_name:
|
||||
atoms = Database(settings).list_atoms()
|
||||
sources = aggregate.sources
|
||||
cartridge = CartridgeService(settings).materialize(metadata.cartridge_name, atoms, sources)
|
||||
aggregate.cartridge_id = cartridge.cartridge_id
|
||||
return aggregate
|
||||
|
||||
@app.post("/api/chat", response_model=AnswerResponse)
|
||||
def chat(request: ChatRequest, settings: Annotated[Settings, Depends(settings_dep)]) -> AnswerResponse:
|
||||
db = Database(settings)
|
||||
compiler = ContextCompiler(settings, db)
|
||||
context = compiler.compile(request.question, request.profile)
|
||||
if not context.hits:
|
||||
raise HTTPException(status_code=404, detail="No evidence atoms are available for this question.")
|
||||
runtime = QwenRuntime(settings)
|
||||
try:
|
||||
generation = runtime.generate(request.question, context)
|
||||
except RuntimeUnavailableError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc)) from exc
|
||||
answer, citations, report = Verifier().verify_answer(generation.text, context)
|
||||
contradiction_notes = VisualContradictionLens().detect(context)
|
||||
report.contradiction_notes.extend(note for note in contradiction_notes if note not in report.contradiction_notes)
|
||||
EvidenceGraphService(settings, db).record_answer(request.question, answer, report, context)
|
||||
return AnswerResponse(
|
||||
answer=answer,
|
||||
citations=citations,
|
||||
context=context,
|
||||
verifier=report,
|
||||
runtime_backend=generation.backend,
|
||||
)
|
||||
|
||||
@app.get("/api/cartridges")
|
||||
def cartridges(settings: Annotated[Settings, Depends(settings_dep)]):
|
||||
return CartridgeService(settings).list_manifests()
|
||||
|
||||
@app.post("/api/cartridges/{cartridge_id}/finalize", response_model=CartridgeAssetReport)
|
||||
def finalize_cartridge(
|
||||
cartridge_id: str,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
) -> CartridgeAssetReport:
|
||||
try:
|
||||
return CartridgeAssetService(settings).finalize(cartridge_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
@app.post("/api/cartridges/{cartridge_id}/refresh")
|
||||
def refresh_cartridge(
|
||||
cartridge_id: str,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
):
|
||||
db = Database(settings)
|
||||
try:
|
||||
return CartridgeService(settings, db).rebuild(
|
||||
cartridge_id=cartridge_id,
|
||||
atoms=db.list_atoms(),
|
||||
sources=db.list_sources(),
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
@app.get("/api/cartridges/{cartridge_id}/assets", response_model=CartridgeAssetReport)
|
||||
def cartridge_assets(
|
||||
cartridge_id: str,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
) -> CartridgeAssetReport:
|
||||
try:
|
||||
report = CartridgeAssetService(settings).status(cartridge_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
if report is None:
|
||||
raise HTTPException(status_code=404, detail="Cartridge assets have not been finalized.")
|
||||
return report
|
||||
|
||||
@app.get("/api/adapters", response_model=list[AdapterStatus])
|
||||
def adapters(settings: Annotated[Settings, Depends(settings_dep)]) -> list[AdapterStatus]:
|
||||
return list_adapter_statuses(settings)
|
||||
|
||||
@app.get("/api/benchmarks/registry", response_model=BenchmarkRegistryReport)
|
||||
def benchmarks_registry() -> BenchmarkRegistryReport:
|
||||
return benchmark_registry_report()
|
||||
|
||||
@app.get("/api/benchmarks/sources", response_model=BenchmarkSourceReport)
|
||||
def benchmarks_sources() -> BenchmarkSourceReport:
|
||||
return benchmark_source_report()
|
||||
|
||||
@app.get("/api/benchmarks/samples", response_model=BenchmarkSampleReport)
|
||||
def benchmarks_samples(settings: Annotated[Settings, Depends(settings_dep)]) -> BenchmarkSampleReport:
|
||||
return benchmark_sample_report(settings.data_dir / "benchmark_samples")
|
||||
|
||||
@app.get("/api/benchmarks/predictions", response_model=BenchmarkPredictionReport)
|
||||
def benchmarks_predictions(settings: Annotated[Settings, Depends(settings_dep)]) -> BenchmarkPredictionReport:
|
||||
return benchmark_prediction_report(settings, settings.data_dir / "benchmark_predictions")
|
||||
|
||||
@app.get("/api/runtime/profiles", response_model=RuntimeProfileReport)
|
||||
def runtime_profiles() -> RuntimeProfileReport:
|
||||
return runtime_profile_report()
|
||||
|
||||
@app.get("/api/research/modules")
|
||||
def research_modules(settings: Annotated[Settings, Depends(settings_dep)]):
|
||||
db = Database(settings)
|
||||
compiler = ContextCompiler(settings, db)
|
||||
context = compiler.compile(
|
||||
"Ely-Eye V-NSA DVKV x-modal KV speculative prefetch evidence graph",
|
||||
RuntimeProfile.research_theater,
|
||||
)
|
||||
return ResearchModuleService(settings, db).build_report(context.plan.question, context.hits)
|
||||
|
||||
@app.post("/api/index/dense")
|
||||
def build_dense_index(settings: Annotated[Settings, Depends(settings_dep)]):
|
||||
service = EmbeddingService(settings)
|
||||
index = service.build_index()
|
||||
return {"atoms": len(index.atom_ids), "dimensions": int(index.vectors.shape[1]) if index.vectors.size else 0}
|
||||
|
||||
@app.post("/api/hashhop", response_model=HashHopProof)
|
||||
def hashhop(request: HashHopRequest, settings: Annotated[Settings, Depends(settings_dep)]) -> HashHopProof:
|
||||
evaluator = HashHopEvaluator(settings)
|
||||
if request.kind == "text":
|
||||
return evaluator.generate_text_proof(request.hops, request.token_equivalent)
|
||||
if request.kind == "visual":
|
||||
return evaluator.generate_visual_proof(request.hops, request.token_equivalent)
|
||||
raise HTTPException(status_code=400, detail="kind must be text or visual")
|
||||
|
||||
@app.post("/api/proof-suite", response_model=ProofSuiteReport)
|
||||
def proof_suite(
|
||||
request: ProofSuiteRequest,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
) -> ProofSuiteReport:
|
||||
try:
|
||||
return PRDProofSuite(settings).run(request.cartridge_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
@app.get("/api/proof-suite/latest", response_model=ProofSuiteReport)
|
||||
def latest_proof_suite(settings: Annotated[Settings, Depends(settings_dep)]) -> ProofSuiteReport:
|
||||
report = PRDProofSuite(settings).latest()
|
||||
if report is None:
|
||||
raise HTTPException(status_code=404, detail="No PRD proof suite has been generated.")
|
||||
return report
|
||||
|
||||
@app.post("/api/visual-contradiction-proof", response_model=ProofCheck)
|
||||
def visual_contradiction_proof(settings: Annotated[Settings, Depends(settings_dep)]) -> ProofCheck:
|
||||
return PRDProofSuite(settings).visual_contradiction_proof()
|
||||
|
||||
@app.get("/api/memory-map")
|
||||
def memory_map(settings: Annotated[Settings, Depends(settings_dep)]):
|
||||
db = Database(settings)
|
||||
atoms = db.list_atoms(limit=200)
|
||||
return {
|
||||
"atoms": [atom.model_dump(mode="json") for atom in atoms],
|
||||
"layers": CacheFabric(settings, db).statuses(),
|
||||
"cartridges": CartridgeService(settings, db).list_manifests(),
|
||||
}
|
||||
|
||||
return app
|
||||
|
||||
|
||||
def sanitize_filename(name: str) -> str:
|
||||
safe = "".join(char if char.isalnum() or char in {".", "-", "_"} else "_" for char in name)
|
||||
return safe.strip("._") or "upload.bin"
|
||||
|
||||
|
||||
def list_adapter_statuses(settings: Settings) -> list[AdapterStatus]:
|
||||
cartridge_bindings = {
|
||||
manifest.cartridge_id: manifest.artifacts
|
||||
for manifest in CartridgeService(settings).list_manifests()
|
||||
}
|
||||
statuses: list[AdapterStatus] = []
|
||||
for manifest_path in sorted(settings.adapters_dir.glob("*/adapter_manifest.json")):
|
||||
root = manifest_path.parent
|
||||
manifest = AdapterManifest.model_validate_json(manifest_path.read_text(encoding="utf-8"))
|
||||
weights = root / "adapter_model.safetensors"
|
||||
weight_bytes = weights.stat().st_size if weights.exists() else 0
|
||||
weight_stats = safetensor_weight_stats(weights)
|
||||
artifacts = cartridge_bindings.get(manifest.cartridge_id or "", {})
|
||||
cartridge_bound = f"adapter_{manifest.kind.value}" in artifacts
|
||||
statuses.append(
|
||||
AdapterStatus(
|
||||
name=root.name,
|
||||
kind=manifest.kind,
|
||||
proof_version=manifest.proof_version,
|
||||
training_method=manifest.training_method,
|
||||
precision=manifest.precision,
|
||||
framework=manifest.framework,
|
||||
base_model=manifest.base_model,
|
||||
path=str(root),
|
||||
adapter_id=manifest.adapter_id,
|
||||
sample_count=manifest.sample_count,
|
||||
max_steps=manifest.max_steps,
|
||||
trainable_params=manifest.trainable_params,
|
||||
final_loss=manifest.final_loss,
|
||||
sha256=manifest.sha256,
|
||||
weight_bytes=weight_bytes,
|
||||
safetensor_keys=int(weight_stats["tensor_count"]),
|
||||
cartridge_id=manifest.cartridge_id,
|
||||
cartridge_bound=cartridge_bound,
|
||||
training_trace_path=manifest.training_trace_path,
|
||||
training_trace_sha256=manifest.training_trace_sha256,
|
||||
training_trace_steps=count_training_steps(root / "training_trace.jsonl"),
|
||||
training_proof_path=manifest.training_proof_path,
|
||||
training_proof_sha256=manifest.training_proof_sha256,
|
||||
optimizer=manifest.optimizer,
|
||||
optimizer_family=manifest.optimizer_family,
|
||||
gradient_checkpointing=manifest.gradient_checkpointing,
|
||||
gradient_checkpointing_mode=manifest.gradient_checkpointing_mode,
|
||||
autocast_dtype=manifest.autocast_dtype,
|
||||
unsloth_version=manifest.unsloth_version,
|
||||
triton_version=manifest.triton_version,
|
||||
xformers_version=manifest.xformers_version,
|
||||
unsloth_model_type=manifest.unsloth_model_type,
|
||||
optimizer_update_count=manifest.optimizer_update_count,
|
||||
max_optimizer_update_l2=manifest.max_optimizer_update_l2,
|
||||
loss_delta=manifest.loss_delta,
|
||||
training_summary_path=manifest.training_summary_path,
|
||||
training_wall_seconds=manifest.training_wall_seconds,
|
||||
training_device=manifest.training_device,
|
||||
bf16=manifest.bf16,
|
||||
torch_version=manifest.torch_version,
|
||||
cuda_version=manifest.cuda_version,
|
||||
adapter_weight_sha256=str(weight_stats["sha256"] or manifest.adapter_weight_sha256 or ""),
|
||||
adapter_tensor_count=int(weight_stats["tensor_count"]),
|
||||
adapter_lora_tensor_count=int(weight_stats["lora_tensor_count"]),
|
||||
adapter_total_elements=int(weight_stats["total_elements"]),
|
||||
adapter_nonzero_elements=int(weight_stats["nonzero_elements"]),
|
||||
adapter_max_abs=weight_stats["max_abs"],
|
||||
adapter_weights_finite=bool(weight_stats["all_finite"]),
|
||||
)
|
||||
)
|
||||
return statuses
|
||||
|
||||
|
||||
def count_training_steps(path: Path) -> int:
|
||||
if not path.exists():
|
||||
return 0
|
||||
steps = 0
|
||||
for line in path.read_text(encoding="utf-8").splitlines():
|
||||
if not line.strip():
|
||||
continue
|
||||
try:
|
||||
if json.loads(line).get("event") == "step":
|
||||
steps += 1
|
||||
except json.JSONDecodeError:
|
||||
return 0
|
||||
return steps
|
||||
|
||||
|
||||
app = create_app()
|
||||
@@ -0,0 +1,617 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
import tempfile
|
||||
from ast import literal_eval
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .schemas import (
|
||||
BenchmarkPredictionReport,
|
||||
BenchmarkPredictionStatus,
|
||||
BenchmarkRegistryReport,
|
||||
BenchmarkSampleReport,
|
||||
BenchmarkSampleStatus,
|
||||
BenchmarkSourceReport,
|
||||
BenchmarkSourceStatus,
|
||||
BenchmarkSpec,
|
||||
)
|
||||
|
||||
|
||||
STANDARD_BENCHMARKS = (
|
||||
BenchmarkSpec(
|
||||
name="MMMU-Pro",
|
||||
target="Multimodal reasoning within Qwen3.5-9B baseline +/-2%.",
|
||||
dataset="MMMU/MMMU_Pro",
|
||||
task_type="image_text_multiple_choice",
|
||||
runner="ely-eye run-benchmark --benchmark mmmu-pro --dataset-path <local-jsonl-or-parquet>",
|
||||
metric="accuracy",
|
||||
),
|
||||
BenchmarkSpec(
|
||||
name="OCRBench",
|
||||
target="OCR and document understanding at or above base model behavior.",
|
||||
dataset="echo840/OCRBench",
|
||||
task_type="image_text_qa",
|
||||
runner="ely-eye run-benchmark --benchmark ocrbench --dataset-path <local-jsonl-or-parquet>",
|
||||
metric="exact_or_contains_accuracy",
|
||||
),
|
||||
BenchmarkSpec(
|
||||
name="MMLongBench-Doc",
|
||||
target="Long document QA above a RAG-only baseline.",
|
||||
dataset="VLM2Vec/MMLongBench-doc",
|
||||
task_type="long_document_qa",
|
||||
runner="ely-eye run-benchmark --benchmark mmlongbench-doc --dataset-path <local-jsonl-or-parquet>",
|
||||
metric="answer_f1_with_evidence",
|
||||
),
|
||||
BenchmarkSpec(
|
||||
name="VideoMME",
|
||||
target="Long video understanding with sampled visual timeline evidence.",
|
||||
dataset="lmms-lab/Video-MME",
|
||||
task_type="video_multiple_choice",
|
||||
runner="ely-eye run-benchmark --benchmark videomme --dataset-path <local-jsonl-or-parquet>",
|
||||
metric="accuracy",
|
||||
),
|
||||
BenchmarkSpec(
|
||||
name="LongVideoBench",
|
||||
target="Long-context interleaved video-language understanding.",
|
||||
dataset="longvideobench/LongVideoBench",
|
||||
task_type="video_qa",
|
||||
runner="ely-eye run-benchmark --benchmark longvideobench --dataset-path <local-jsonl-or-parquet>",
|
||||
metric="accuracy",
|
||||
),
|
||||
BenchmarkSpec(
|
||||
name="RULER",
|
||||
target="Long-context retrieval, aggregation, and multi-hop reasoning.",
|
||||
dataset="RULER-dataset/RULER",
|
||||
task_type="long_context_text",
|
||||
runner="ely-eye run-benchmark --benchmark ruler --dataset-path <local-jsonl-or-parquet>",
|
||||
metric="task_accuracy",
|
||||
),
|
||||
BenchmarkSpec(
|
||||
name="LongBench v2",
|
||||
target="Long text, multi-document, code, and structured-data reasoning.",
|
||||
dataset="THUDM/LongBench-v2",
|
||||
task_type="long_context_multiple_choice",
|
||||
runner="ely-eye run-benchmark --benchmark longbench-v2 --dataset-path <local-jsonl-or-parquet>",
|
||||
metric="accuracy",
|
||||
),
|
||||
)
|
||||
|
||||
BENCHMARK_PREDICTION_NAMES = ("MMMU-Pro", "OCRBench")
|
||||
|
||||
BENCHMARK_SAMPLE_LOADS: dict[str, tuple[str | None, str]] = {
|
||||
"MMMU-Pro": ("standard (10 options)", "test"),
|
||||
"OCRBench": ("default", "test"),
|
||||
"MMLongBench-Doc": ("queries", "test"),
|
||||
"VideoMME": ("videomme", "test"),
|
||||
"LongVideoBench": (None, "test"),
|
||||
"RULER": ("default", "test"),
|
||||
"LongBench v2": ("default", "train"),
|
||||
}
|
||||
|
||||
BENCHMARK_SAMPLE_SOURCES: dict[str, tuple[str, str | None, str, str | None]] = {
|
||||
"LongVideoBench": (
|
||||
"longvideobench/LongVideoBench-Meta",
|
||||
"default",
|
||||
"validation",
|
||||
"Official public metadata release for the gated LongVideoBench media dataset.",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def benchmark_registry_report() -> BenchmarkRegistryReport:
|
||||
return BenchmarkRegistryReport(status="ready", benchmarks=list(STANDARD_BENCHMARKS))
|
||||
|
||||
|
||||
def benchmark_source_report(timeout: int = 20) -> BenchmarkSourceReport:
|
||||
from huggingface_hub import HfApi
|
||||
|
||||
api = HfApi()
|
||||
sources: list[BenchmarkSourceStatus] = []
|
||||
for benchmark in STANDARD_BENCHMARKS:
|
||||
try:
|
||||
info = api.dataset_info(benchmark.dataset, timeout=timeout)
|
||||
siblings = getattr(info, "siblings", None) or []
|
||||
sources.append(
|
||||
BenchmarkSourceStatus(
|
||||
name=benchmark.name,
|
||||
dataset=benchmark.dataset,
|
||||
status="ready" if info.sha else "unavailable",
|
||||
sha=info.sha,
|
||||
gated=getattr(info, "gated", None),
|
||||
private=getattr(info, "private", None),
|
||||
sibling_count=len(siblings),
|
||||
)
|
||||
)
|
||||
except Exception as exc:
|
||||
sources.append(
|
||||
BenchmarkSourceStatus(
|
||||
name=benchmark.name,
|
||||
dataset=benchmark.dataset,
|
||||
status="unavailable",
|
||||
error=str(exc).splitlines()[0],
|
||||
)
|
||||
)
|
||||
status = "ready" if all(source.status == "ready" and source.sha for source in sources) else "incomplete"
|
||||
return BenchmarkSourceReport(status=status, sources=sources)
|
||||
|
||||
|
||||
def benchmark_sample_report(output_dir: Path | None = None) -> BenchmarkSampleReport:
|
||||
output_root = output_dir or Path(tempfile.mkdtemp(prefix="ely_eye_benchmark_samples_"))
|
||||
output_root.mkdir(parents=True, exist_ok=True)
|
||||
samples = [load_benchmark_sample(benchmark, output_root) for benchmark in STANDARD_BENCHMARKS]
|
||||
loaded = [sample for sample in samples if sample.status == "loaded"]
|
||||
gated = [sample for sample in samples if sample.status == "gated"]
|
||||
failed = [sample for sample in samples if sample.status == "failed"]
|
||||
if len(loaded) == len(samples):
|
||||
status = "ready"
|
||||
elif loaded and gated and not failed:
|
||||
status = "partial"
|
||||
else:
|
||||
status = "failed"
|
||||
return BenchmarkSampleReport(status=status, samples=samples)
|
||||
|
||||
|
||||
def benchmark_prediction_report(
|
||||
settings: Any,
|
||||
output_dir: Path | None = None,
|
||||
names: tuple[str, ...] = BENCHMARK_PREDICTION_NAMES,
|
||||
) -> BenchmarkPredictionReport:
|
||||
from .runtime import QwenRuntime
|
||||
|
||||
output_root = output_dir or Path(tempfile.mkdtemp(prefix="ely_eye_benchmark_predictions_"))
|
||||
output_root.mkdir(parents=True, exist_ok=True)
|
||||
selected = {name.lower() for name in names}
|
||||
benchmarks = [benchmark for benchmark in STANDARD_BENCHMARKS if benchmark.name.lower() in selected]
|
||||
prediction_settings = settings.model_copy(update={"max_new_tokens": 96, "temperature": 0.0})
|
||||
runtime = QwenRuntime(prediction_settings)
|
||||
try:
|
||||
predictions = [
|
||||
predict_benchmark_sample(benchmark, output_root, runtime)
|
||||
for benchmark in benchmarks
|
||||
]
|
||||
finally:
|
||||
runtime.unload()
|
||||
predicted = [prediction for prediction in predictions if prediction.status == "predicted"]
|
||||
gated = [prediction for prediction in predictions if prediction.status == "gated"]
|
||||
failed = [prediction for prediction in predictions if prediction.status == "failed"]
|
||||
if len(predicted) == len(predictions) and predictions:
|
||||
status = "ready"
|
||||
elif predicted and gated and not failed:
|
||||
status = "partial"
|
||||
else:
|
||||
status = "failed"
|
||||
return BenchmarkPredictionReport(status=status, predictions=predictions)
|
||||
|
||||
|
||||
def load_benchmark_sample(benchmark: BenchmarkSpec, output_root: Path) -> BenchmarkSampleStatus:
|
||||
source_dataset, config, split, source_note = benchmark_sample_source(benchmark)
|
||||
try:
|
||||
from datasets import load_dataset
|
||||
|
||||
dataset_kwargs: dict[str, Any] = {"split": split, "streaming": True}
|
||||
stream = (
|
||||
load_dataset(source_dataset, config, **dataset_kwargs)
|
||||
if config
|
||||
else load_dataset(source_dataset, **dataset_kwargs)
|
||||
)
|
||||
row = next(iter(stream))
|
||||
sample_root = output_root / slugify(benchmark.name)
|
||||
sample_root.mkdir(parents=True, exist_ok=True)
|
||||
normalized, media_artifacts = normalize_sample(row, sample_root)
|
||||
return BenchmarkSampleStatus(
|
||||
name=benchmark.name,
|
||||
dataset=benchmark.dataset,
|
||||
source_dataset=source_dataset,
|
||||
source_sha=dataset_sha(source_dataset),
|
||||
source_note=source_note,
|
||||
status="loaded",
|
||||
config=config,
|
||||
split=split,
|
||||
sample_id=sample_identifier(row),
|
||||
fields=list(row.keys()),
|
||||
question_preview=first_present(row, ("question", "query", "text")),
|
||||
answer_preview=first_present(row, ("answer", "answers", "la", "label", "correct_choice")),
|
||||
sample_sha256=sha256_json(normalized),
|
||||
media_artifacts=media_artifacts,
|
||||
)
|
||||
except Exception as exc:
|
||||
message = str(exc).splitlines()[0]
|
||||
status = "gated" if "gated dataset" in message.lower() or "authenticated" in message.lower() else "failed"
|
||||
return BenchmarkSampleStatus(
|
||||
name=benchmark.name,
|
||||
dataset=benchmark.dataset,
|
||||
source_dataset=source_dataset,
|
||||
source_note=source_note,
|
||||
status=status,
|
||||
config=config,
|
||||
split=split,
|
||||
error=message,
|
||||
)
|
||||
|
||||
|
||||
def predict_benchmark_sample(benchmark: BenchmarkSpec, output_root: Path, runtime: Any) -> BenchmarkPredictionStatus:
|
||||
try:
|
||||
from datasets import load_dataset
|
||||
|
||||
config, split = BENCHMARK_SAMPLE_LOADS.get(benchmark.name, (None, "test"))
|
||||
dataset_kwargs: dict[str, Any] = {"split": split, "streaming": True}
|
||||
stream = (
|
||||
load_dataset(benchmark.dataset, config, **dataset_kwargs)
|
||||
if config
|
||||
else load_dataset(benchmark.dataset, **dataset_kwargs)
|
||||
)
|
||||
row = next(iter(stream))
|
||||
sample_root = output_root / slugify(benchmark.name)
|
||||
sample_root.mkdir(parents=True, exist_ok=True)
|
||||
normalized, media_artifacts = normalize_sample(row, sample_root)
|
||||
prediction_media = prepare_prediction_media(benchmark, media_artifacts)
|
||||
prompt = benchmark_prompt(benchmark, row)
|
||||
generation = runtime.generate_prompt(prompt, [Path(path) for path in prediction_media])
|
||||
prediction = extract_answer(generation.text)
|
||||
answer = canonical_answer(first_present_raw(row, ("answer", "answers", "la", "label")))
|
||||
score = benchmark_score(benchmark, row, prediction, answer) if answer else None
|
||||
return BenchmarkPredictionStatus(
|
||||
name=benchmark.name,
|
||||
dataset=benchmark.dataset,
|
||||
status="predicted",
|
||||
config=config,
|
||||
split=split,
|
||||
sample_id=sample_identifier(row),
|
||||
task_type=benchmark.task_type,
|
||||
metric=benchmark.metric,
|
||||
prompt_sha256=sha256_bytes(prompt.encode("utf-8")),
|
||||
sample_sha256=sha256_json(normalized),
|
||||
prediction=prediction,
|
||||
answer=answer,
|
||||
score=score,
|
||||
runtime_backend=generation.backend,
|
||||
adapter_id=generation.adapter_id,
|
||||
adapter_sha256=generation.adapter_sha256,
|
||||
input_tokens=generation.input_tokens,
|
||||
output_tokens=generation.output_tokens,
|
||||
media_artifacts=prediction_media,
|
||||
)
|
||||
except Exception as exc:
|
||||
message = str(exc).splitlines()[0]
|
||||
status = "gated" if "gated dataset" in message.lower() or "authenticated" in message.lower() else "failed"
|
||||
config, split = BENCHMARK_SAMPLE_LOADS.get(benchmark.name, (None, None))
|
||||
return BenchmarkPredictionStatus(
|
||||
name=benchmark.name,
|
||||
dataset=benchmark.dataset,
|
||||
status=status,
|
||||
config=config,
|
||||
split=split,
|
||||
task_type=benchmark.task_type,
|
||||
metric=benchmark.metric,
|
||||
error=message,
|
||||
)
|
||||
|
||||
|
||||
def normalize_benchmark_name(name: str) -> str:
|
||||
return name.lower().replace("_", "-")
|
||||
|
||||
|
||||
def benchmark_sample_source(benchmark: BenchmarkSpec) -> tuple[str, str | None, str, str | None]:
|
||||
override = BENCHMARK_SAMPLE_SOURCES.get(benchmark.name)
|
||||
if override is not None:
|
||||
return override
|
||||
config, split = BENCHMARK_SAMPLE_LOADS.get(benchmark.name, (None, "test"))
|
||||
return benchmark.dataset, config, split, None
|
||||
|
||||
|
||||
def dataset_sha(dataset: str) -> str | None:
|
||||
from huggingface_hub import HfApi
|
||||
|
||||
info = HfApi().dataset_info(dataset)
|
||||
return str(info.sha) if info.sha else None
|
||||
|
||||
|
||||
def run_prediction_benchmark(
|
||||
benchmark: str,
|
||||
dataset_path: Path,
|
||||
prediction_field: str = "prediction",
|
||||
answer_field: str = "answer",
|
||||
limit: int | None = None,
|
||||
) -> dict[str, Any]:
|
||||
records = load_records(dataset_path)
|
||||
if limit is not None:
|
||||
records = records[:limit]
|
||||
scored: list[dict[str, Any]] = []
|
||||
for index, record in enumerate(records):
|
||||
prediction = get_nested(record, prediction_field)
|
||||
answer = get_nested(record, answer_field)
|
||||
if prediction is None or answer is None:
|
||||
continue
|
||||
score = exact_or_contains_score(str(prediction), str(answer))
|
||||
scored.append(
|
||||
{
|
||||
"index": index,
|
||||
"score": score,
|
||||
"prediction": str(prediction),
|
||||
"answer": str(answer),
|
||||
}
|
||||
)
|
||||
accuracy = sum(item["score"] for item in scored) / len(scored) if scored else 0.0
|
||||
return {
|
||||
"benchmark": normalize_benchmark_name(benchmark),
|
||||
"dataset_path": str(dataset_path),
|
||||
"records": len(records),
|
||||
"scored": len(scored),
|
||||
"metric": "exact_or_contains_accuracy",
|
||||
"score": accuracy,
|
||||
}
|
||||
|
||||
|
||||
def load_records(path: Path) -> list[dict[str, Any]]:
|
||||
suffix = path.suffix.lower()
|
||||
if suffix == ".jsonl":
|
||||
with path.open("r", encoding="utf-8") as handle:
|
||||
return [json.loads(line) for line in handle if line.strip()]
|
||||
if suffix == ".json":
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
if isinstance(data, list):
|
||||
return [dict(item) for item in data if isinstance(item, dict)]
|
||||
if isinstance(data, dict):
|
||||
for key in ("data", "records", "examples"):
|
||||
value = data.get(key)
|
||||
if isinstance(value, list):
|
||||
return [dict(item) for item in value if isinstance(item, dict)]
|
||||
return [data]
|
||||
if suffix == ".parquet":
|
||||
import pyarrow.parquet as pq
|
||||
|
||||
return pq.read_table(path).to_pylist()
|
||||
raise ValueError(f"Unsupported benchmark dataset format: {path}")
|
||||
|
||||
|
||||
def get_nested(record: dict[str, Any], path: str) -> Any:
|
||||
current: Any = record
|
||||
for part in path.split("."):
|
||||
if not isinstance(current, dict) or part not in current:
|
||||
return None
|
||||
current = current[part]
|
||||
return current
|
||||
|
||||
|
||||
def exact_or_contains_score(prediction: str, answer: str) -> float:
|
||||
normalized_prediction = normalize_answer(prediction)
|
||||
normalized_answer = normalize_answer(answer)
|
||||
if not normalized_prediction or not normalized_answer:
|
||||
return 0.0
|
||||
if normalized_prediction == normalized_answer or normalized_answer in normalized_prediction:
|
||||
return 1.0
|
||||
prediction_tokens = normalized_prediction.split()
|
||||
answer_tokens = normalized_answer.split()
|
||||
if not prediction_tokens or not answer_tokens:
|
||||
return 0.0
|
||||
overlap = len(set(prediction_tokens) & set(answer_tokens))
|
||||
precision = overlap / len(set(prediction_tokens))
|
||||
recall = overlap / len(set(answer_tokens))
|
||||
if precision + recall == 0:
|
||||
return 0.0
|
||||
return 2 * precision * recall / (precision + recall)
|
||||
|
||||
|
||||
def benchmark_score(
|
||||
benchmark: BenchmarkSpec,
|
||||
row: dict[str, Any],
|
||||
prediction: str,
|
||||
answer: str | None,
|
||||
) -> float | None:
|
||||
if answer is None:
|
||||
return None
|
||||
if "multiple_choice" in benchmark.task_type:
|
||||
option_text = answer_option_text(row, answer)
|
||||
if exact_or_contains_score(prediction, answer):
|
||||
return 1.0
|
||||
if option_text and exact_or_contains_score(prediction, option_text):
|
||||
return 1.0
|
||||
return 0.0
|
||||
return exact_or_contains_score(prediction, answer)
|
||||
|
||||
|
||||
def answer_option_text(row: dict[str, Any], answer: str) -> str | None:
|
||||
normalized_answer = answer.strip().upper()
|
||||
if len(normalized_answer) != 1 or not normalized_answer.isalpha():
|
||||
return None
|
||||
options = row.get("options")
|
||||
if isinstance(options, str):
|
||||
try:
|
||||
options = literal_eval(options)
|
||||
except (SyntaxError, ValueError):
|
||||
options = None
|
||||
index = ord(normalized_answer) - ord("A")
|
||||
if isinstance(options, (list, tuple)) and 0 <= index < len(options):
|
||||
return str(options[index])
|
||||
if isinstance(options, dict):
|
||||
return canonical_answer(options.get(normalized_answer))
|
||||
choice_key = f"choice_{normalized_answer}"
|
||||
if choice_key in row:
|
||||
return canonical_answer(row[choice_key])
|
||||
return None
|
||||
|
||||
|
||||
def normalize_answer(value: str) -> str:
|
||||
return re.sub(r"\s+", " ", re.sub(r"[^\w\u4e00-\u9fff]+", " ", value.lower())).strip()
|
||||
|
||||
|
||||
def benchmark_prompt(benchmark: BenchmarkSpec, row: dict[str, Any]) -> str:
|
||||
question = first_present_raw(row, ("question", "query", "text")) or ""
|
||||
options = first_present_raw(row, ("options", "choices")) or ""
|
||||
context = first_present_raw(row, ("context", "document", "passage")) or ""
|
||||
context = compact_preview(context, 6000)
|
||||
media_note = "Use the attached image evidence." if any(is_pil_image(value) for value in row.values()) else ""
|
||||
if benchmark.name == "OCRBench":
|
||||
return f"""Use /no_think mode. Return compact JSON only.
|
||||
|
||||
Benchmark: OCRBench
|
||||
Task type: image text recognition
|
||||
Metric: exact_or_contains_accuracy
|
||||
Use the attached image evidence.
|
||||
|
||||
Question:
|
||||
{question}
|
||||
|
||||
Read the exact text in the image. For handwritten English words, return the standard spelling with the same letters.
|
||||
|
||||
Return exactly this JSON shape:
|
||||
{{"answer":"<recognized text>"}}
|
||||
"""
|
||||
return f"""Use /no_think mode. Return compact JSON only.
|
||||
|
||||
Benchmark: {benchmark.name}
|
||||
Task type: {benchmark.task_type}
|
||||
Metric: {benchmark.metric}
|
||||
{media_note}
|
||||
|
||||
Question:
|
||||
{question}
|
||||
|
||||
Options:
|
||||
{options}
|
||||
|
||||
Context:
|
||||
{context}
|
||||
|
||||
Return exactly this JSON shape:
|
||||
{{"answer":"<prediction>"}}
|
||||
"""
|
||||
|
||||
|
||||
def extract_answer(text: str) -> str:
|
||||
stripped = text.strip()
|
||||
try:
|
||||
payload = json.loads(stripped)
|
||||
except json.JSONDecodeError:
|
||||
start = stripped.find("{")
|
||||
end = stripped.rfind("}")
|
||||
payload = None
|
||||
if start >= 0 and end > start:
|
||||
try:
|
||||
payload = json.loads(stripped[start : end + 1])
|
||||
except json.JSONDecodeError:
|
||||
payload = None
|
||||
if isinstance(payload, dict) and payload.get("answer") is not None:
|
||||
return str(payload["answer"]).strip()
|
||||
return compact_preview(stripped, 500)
|
||||
|
||||
|
||||
def canonical_answer(value: Any) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, (list, tuple)):
|
||||
return " | ".join(str(item) for item in value)
|
||||
return str(value)
|
||||
|
||||
|
||||
def first_present_raw(row: dict[str, Any], keys: tuple[str, ...]) -> Any:
|
||||
for key in keys:
|
||||
value = row.get(key)
|
||||
if value is not None:
|
||||
return value
|
||||
return None
|
||||
|
||||
|
||||
def normalize_sample(row: dict[str, Any], output_root: Path) -> tuple[dict[str, Any], list[str]]:
|
||||
media_artifacts: list[str] = []
|
||||
normalized: dict[str, Any] = {}
|
||||
for key, value in row.items():
|
||||
normalized_value, artifact = normalize_sample_value(key, value, output_root)
|
||||
normalized[key] = normalized_value
|
||||
if artifact:
|
||||
media_artifacts.append(artifact)
|
||||
return normalized, media_artifacts
|
||||
|
||||
|
||||
def prepare_prediction_media(benchmark: BenchmarkSpec, media_artifacts: list[str]) -> list[str]:
|
||||
if benchmark.name != "OCRBench":
|
||||
return media_artifacts
|
||||
return [str(enhance_ocr_benchmark_image(Path(path))) for path in media_artifacts]
|
||||
|
||||
|
||||
def enhance_ocr_benchmark_image(path: Path) -> Path:
|
||||
from PIL import Image, ImageOps
|
||||
|
||||
image = Image.open(path).convert("RGB")
|
||||
max_side = max(image.size)
|
||||
scale = max(1, min(10, math.ceil(768 / max_side)))
|
||||
if scale > 1:
|
||||
image = image.resize((image.width * scale, image.height * scale), Image.Resampling.BICUBIC)
|
||||
border = max(8, scale * 2)
|
||||
image = ImageOps.expand(image, border=border, fill="white")
|
||||
target = path.with_name(f"{path.stem}_ocr_x{scale}{path.suffix}")
|
||||
image.save(target)
|
||||
return target
|
||||
|
||||
|
||||
def normalize_sample_value(key: str, value: Any, output_root: Path) -> tuple[Any, str | None]:
|
||||
if is_pil_image(value):
|
||||
digest = image_sha256(value)
|
||||
artifact = output_root / f"{slugify(key)}_{digest[:12]}.png"
|
||||
value.save(artifact)
|
||||
return (
|
||||
{
|
||||
"type": "image",
|
||||
"mode": getattr(value, "mode", None),
|
||||
"size": list(getattr(value, "size", ())),
|
||||
"sha256": digest,
|
||||
},
|
||||
str(artifact),
|
||||
)
|
||||
if isinstance(value, (str, int, float, bool)) or value is None:
|
||||
return value, None
|
||||
if isinstance(value, (list, tuple)):
|
||||
return [normalize_sample_value(f"{key}_{index}", item, output_root)[0] for index, item in enumerate(value)], None
|
||||
if isinstance(value, dict):
|
||||
return {str(item_key): normalize_sample_value(str(item_key), item_value, output_root)[0] for item_key, item_value in value.items()}, None
|
||||
return str(value), None
|
||||
|
||||
|
||||
def is_pil_image(value: Any) -> bool:
|
||||
return hasattr(value, "save") and hasattr(value, "mode") and hasattr(value, "size")
|
||||
|
||||
|
||||
def image_sha256(image: Any) -> str:
|
||||
import io
|
||||
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="PNG")
|
||||
return sha256_bytes(buffer.getvalue())
|
||||
|
||||
|
||||
def sample_identifier(row: dict[str, Any]) -> str | None:
|
||||
for key in ("id", "_id", "question_id", "query-id", "text_id", "video_id", "corpus-id"):
|
||||
value = row.get(key)
|
||||
if value is not None:
|
||||
return str(value)
|
||||
return None
|
||||
|
||||
|
||||
def first_present(row: dict[str, Any], keys: tuple[str, ...]) -> str | None:
|
||||
for key in keys:
|
||||
value = row.get(key)
|
||||
if value is not None:
|
||||
return compact_preview(value)
|
||||
return None
|
||||
|
||||
|
||||
def compact_preview(value: Any, limit: int = 240) -> str:
|
||||
text = re.sub(r"\s+", " ", str(value)).strip()
|
||||
return text[:limit]
|
||||
|
||||
|
||||
def sha256_json(value: dict[str, Any]) -> str:
|
||||
return sha256_bytes(json.dumps(value, ensure_ascii=False, sort_keys=True, default=str).encode("utf-8"))
|
||||
|
||||
|
||||
def sha256_bytes(data: bytes) -> str:
|
||||
import hashlib
|
||||
|
||||
return hashlib.sha256(data).hexdigest()
|
||||
|
||||
|
||||
def slugify(value: str) -> str:
|
||||
return re.sub(r"[^a-zA-Z0-9_.-]+", "-", value).strip("-").lower() or "item"
|
||||
@@ -0,0 +1,57 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .schemas import CacheLayerStatus
|
||||
|
||||
|
||||
class CacheFabric:
|
||||
def __init__(self, settings: Settings | None = None, database: Database | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
|
||||
def semantic_key(self, question: str, profile: str) -> str:
|
||||
return hashlib.sha256(f"compiler-v2\n{profile}\n{question}".encode("utf-8")).hexdigest()
|
||||
|
||||
def record_hit(self, layer: str, cache_key: str, bytes_estimate: int = 0, metadata: dict[str, Any] | None = None) -> None:
|
||||
self.db.log_cache_event(layer, cache_key, "hit", bytes_estimate, metadata)
|
||||
|
||||
def record_miss(self, layer: str, cache_key: str, bytes_estimate: int = 0, metadata: dict[str, Any] | None = None) -> None:
|
||||
self.db.log_cache_event(layer, cache_key, "miss", bytes_estimate, metadata)
|
||||
|
||||
def write_semantic_cache(self, cache_key: str, payload: dict[str, Any]) -> None:
|
||||
path = self.settings.cache_dir / "semantic" / f"{cache_key}.json"
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
data = json.dumps(payload, ensure_ascii=False, indent=2).encode("utf-8")
|
||||
path.write_bytes(data)
|
||||
self.db.log_cache_event("semantic", cache_key, "write", len(data), {"path": str(path)})
|
||||
|
||||
def read_semantic_cache(self, cache_key: str) -> dict[str, Any] | None:
|
||||
path = self.settings.cache_dir / "semantic" / f"{cache_key}.json"
|
||||
if not path.exists():
|
||||
self.record_miss("semantic", cache_key)
|
||||
return None
|
||||
self.record_hit("semantic", cache_key, path.stat().st_size)
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
|
||||
def statuses(self) -> list[CacheLayerStatus]:
|
||||
rows = self.db.cache_layer_rows()
|
||||
statuses: list[CacheLayerStatus] = []
|
||||
for row in rows:
|
||||
last = row["last_event_at"]
|
||||
statuses.append(
|
||||
CacheLayerStatus(
|
||||
layer=row["layer"],
|
||||
entries=int(row["entries"] or 0),
|
||||
bytes_estimate=int(row["bytes_estimate"] or 0),
|
||||
hit_count=int(row["hit_count"] or 0),
|
||||
miss_count=int(row["miss_count"] or 0),
|
||||
last_event_at=datetime.fromisoformat(last) if last else None,
|
||||
)
|
||||
)
|
||||
return statuses
|
||||
@@ -0,0 +1,303 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import shutil
|
||||
import sqlite3
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import pyarrow as pa
|
||||
import pyarrow.parquet as pq
|
||||
import zstandard as zstd
|
||||
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .schemas import AdapterManifest, CartridgeManifest, EvidenceAtom, SourceRecord
|
||||
|
||||
|
||||
class CartridgeService:
|
||||
def __init__(self, settings: Settings | None = None, database: Database | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
|
||||
def materialize(
|
||||
self,
|
||||
name: str,
|
||||
atoms: list[EvidenceAtom],
|
||||
sources: list[SourceRecord],
|
||||
) -> CartridgeManifest:
|
||||
created_at = datetime.now(timezone.utc)
|
||||
cartridge_id = self._cartridge_id(name, created_at)
|
||||
root = self.settings.cartridge_dir / cartridge_id
|
||||
eval_dir = root / "eval_proofs"
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
eval_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
atoms_path = root / "atoms.parquet"
|
||||
sources_path = root / "sources.json"
|
||||
sparse_path = root / "sparse_index.bm25.zst"
|
||||
graph_path = root / "temporal_graph.sqlite"
|
||||
|
||||
self._write_atoms_parquet(atoms_path, atoms)
|
||||
sources_path.write_text(
|
||||
json.dumps([source.model_dump(mode="json") for source in sources], ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
self._write_sparse_index(sparse_path, atoms)
|
||||
self._write_graph(graph_path, atoms)
|
||||
|
||||
manifest = CartridgeManifest(
|
||||
cartridge_id=cartridge_id,
|
||||
name=name,
|
||||
created_at=created_at,
|
||||
model_base=self.settings.model_id,
|
||||
source_count=len(sources),
|
||||
atom_count=len(atoms),
|
||||
token_equivalent=sum(atom.token_equivalent for atom in atoms),
|
||||
artifacts={
|
||||
"manifest": "manifest.json",
|
||||
"atoms": "atoms.parquet",
|
||||
"sources": "sources.json",
|
||||
"sparse_index": "sparse_index.bm25.zst",
|
||||
"temporal_graph": "temporal_graph.sqlite",
|
||||
"eval_proofs": "eval_proofs/",
|
||||
},
|
||||
)
|
||||
manifest_path = root / "manifest.json"
|
||||
manifest_path.write_text(manifest.model_dump_json(indent=2), encoding="utf-8")
|
||||
dna = self.compute_dna(root)
|
||||
manifest.dna = dna
|
||||
manifest_path.write_text(manifest.model_dump_json(indent=2), encoding="utf-8")
|
||||
self.db.write_cartridge(
|
||||
cartridge_id=cartridge_id,
|
||||
name=name,
|
||||
root_path=root,
|
||||
manifest_json=manifest.model_dump_json(),
|
||||
dna=dna,
|
||||
token_equivalent=manifest.token_equivalent,
|
||||
created_at=created_at,
|
||||
)
|
||||
return manifest
|
||||
|
||||
def compute_dna(self, root: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
for path in sorted(root.rglob("*")):
|
||||
if path.is_dir() or path.name in {"manifest.json", "proof_suite.json", "cartridge_assets.json"}:
|
||||
continue
|
||||
digest.update(path.relative_to(root).as_posix().encode("utf-8"))
|
||||
with path.open("rb") as handle:
|
||||
shutil.copyfileobj(handle, _DigestWriter(digest))
|
||||
return digest.hexdigest()
|
||||
|
||||
def list_manifests(self) -> list[CartridgeManifest]:
|
||||
manifests: list[CartridgeManifest] = []
|
||||
for row in self.db.list_cartridges():
|
||||
manifests.append(CartridgeManifest.model_validate_json(row["manifest_json"]))
|
||||
return manifests
|
||||
|
||||
def rebuild(
|
||||
self,
|
||||
cartridge_id: str,
|
||||
atoms: list[EvidenceAtom],
|
||||
sources: list[SourceRecord],
|
||||
) -> CartridgeManifest:
|
||||
rows = [row for row in self.db.list_cartridges() if row["cartridge_id"] == cartridge_id]
|
||||
if not rows:
|
||||
raise ValueError(f"Unknown cartridge: {cartridge_id}")
|
||||
row = rows[0]
|
||||
root = Path(row["root_path"])
|
||||
manifest = CartridgeManifest.model_validate_json(row["manifest_json"])
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
self._write_atoms_parquet(root / "atoms.parquet", atoms)
|
||||
(root / "sources.json").write_text(
|
||||
json.dumps([source.model_dump(mode="json") for source in sources], ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
self._write_sparse_index(root / "sparse_index.bm25.zst", atoms)
|
||||
self._write_graph(root / "temporal_graph.sqlite", atoms)
|
||||
|
||||
manifest.source_count = len(sources)
|
||||
manifest.atom_count = len(atoms)
|
||||
manifest.token_equivalent = sum(atom.token_equivalent for atom in atoms)
|
||||
manifest.artifacts.update(
|
||||
{
|
||||
"manifest": "manifest.json",
|
||||
"atoms": "atoms.parquet",
|
||||
"sources": "sources.json",
|
||||
"sparse_index": "sparse_index.bm25.zst",
|
||||
"temporal_graph": "temporal_graph.sqlite",
|
||||
"eval_proofs": "eval_proofs/",
|
||||
}
|
||||
)
|
||||
manifest.dna = self.compute_dna(root)
|
||||
(root / "manifest.json").write_text(manifest.model_dump_json(indent=2), encoding="utf-8")
|
||||
self.db.write_cartridge(
|
||||
cartridge_id=manifest.cartridge_id,
|
||||
name=manifest.name,
|
||||
root_path=root,
|
||||
manifest_json=manifest.model_dump_json(),
|
||||
dna=manifest.dna,
|
||||
token_equivalent=manifest.token_equivalent,
|
||||
created_at=manifest.created_at,
|
||||
)
|
||||
return manifest
|
||||
|
||||
def attach_adapter(self, cartridge_id: str, adapter_dir: Path) -> CartridgeManifest:
|
||||
rows = [row for row in self.db.list_cartridges() if row["cartridge_id"] == cartridge_id]
|
||||
if not rows:
|
||||
raise ValueError(f"Unknown cartridge: {cartridge_id}")
|
||||
row = rows[0]
|
||||
root = Path(row["root_path"])
|
||||
manifest = CartridgeManifest.model_validate_json(row["manifest_json"])
|
||||
adapter_manifest_path = adapter_dir / "adapter_manifest.json"
|
||||
if not adapter_manifest_path.exists():
|
||||
raise ValueError(f"Missing adapter manifest: {adapter_manifest_path}")
|
||||
adapter_manifest = AdapterManifest.model_validate_json(
|
||||
adapter_manifest_path.read_text(encoding="utf-8")
|
||||
)
|
||||
adapter_root = root / "adapters" / adapter_manifest.kind.value
|
||||
adapter_root.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copytree(adapter_dir, adapter_root, dirs_exist_ok=True)
|
||||
manifest.artifacts[f"adapter_{adapter_manifest.kind.value}"] = (
|
||||
adapter_root.relative_to(root).as_posix() + "/"
|
||||
)
|
||||
manifest_path = root / "manifest.json"
|
||||
manifest.dna = self.compute_dna(root)
|
||||
manifest_path.write_text(manifest.model_dump_json(indent=2), encoding="utf-8")
|
||||
self.db.write_cartridge(
|
||||
cartridge_id=manifest.cartridge_id,
|
||||
name=manifest.name,
|
||||
root_path=root,
|
||||
manifest_json=manifest.model_dump_json(),
|
||||
dna=manifest.dna,
|
||||
token_equivalent=manifest.token_equivalent,
|
||||
created_at=manifest.created_at,
|
||||
)
|
||||
return manifest
|
||||
|
||||
def update_artifacts(
|
||||
self,
|
||||
cartridge_id: str,
|
||||
artifacts: dict[str, str],
|
||||
token_equivalent: int | None = None,
|
||||
) -> CartridgeManifest:
|
||||
rows = [row for row in self.db.list_cartridges() if row["cartridge_id"] == cartridge_id]
|
||||
if not rows:
|
||||
raise ValueError(f"Unknown cartridge: {cartridge_id}")
|
||||
row = rows[0]
|
||||
root = Path(row["root_path"])
|
||||
manifest = CartridgeManifest.model_validate_json(row["manifest_json"])
|
||||
manifest.artifacts.update(artifacts)
|
||||
if token_equivalent is not None:
|
||||
manifest.token_equivalent = token_equivalent
|
||||
manifest_path = root / "manifest.json"
|
||||
manifest.dna = self.compute_dna(root)
|
||||
manifest_path.write_text(manifest.model_dump_json(indent=2), encoding="utf-8")
|
||||
self.db.write_cartridge(
|
||||
cartridge_id=manifest.cartridge_id,
|
||||
name=manifest.name,
|
||||
root_path=root,
|
||||
manifest_json=manifest.model_dump_json(),
|
||||
dna=manifest.dna,
|
||||
token_equivalent=manifest.token_equivalent,
|
||||
created_at=manifest.created_at,
|
||||
)
|
||||
return manifest
|
||||
|
||||
def attach_eval_proof(self, cartridge_id: str, proof_dir: Path) -> CartridgeManifest:
|
||||
rows = [row for row in self.db.list_cartridges() if row["cartridge_id"] == cartridge_id]
|
||||
if not rows:
|
||||
raise ValueError(f"Unknown cartridge: {cartridge_id}")
|
||||
if not proof_dir.exists() or not proof_dir.is_dir():
|
||||
raise ValueError(f"Missing proof directory: {proof_dir}")
|
||||
row = rows[0]
|
||||
root = Path(row["root_path"])
|
||||
manifest = CartridgeManifest.model_validate_json(row["manifest_json"])
|
||||
target = root / "eval_proofs" / proof_dir.name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copytree(proof_dir, target, dirs_exist_ok=True)
|
||||
manifest.artifacts[f"eval_proof_{proof_dir.name}"] = target.relative_to(root).as_posix() + "/"
|
||||
manifest_path = root / "manifest.json"
|
||||
manifest.dna = self.compute_dna(root)
|
||||
manifest_path.write_text(manifest.model_dump_json(indent=2), encoding="utf-8")
|
||||
self.db.write_cartridge(
|
||||
cartridge_id=manifest.cartridge_id,
|
||||
name=manifest.name,
|
||||
root_path=root,
|
||||
manifest_json=manifest.model_dump_json(),
|
||||
dna=manifest.dna,
|
||||
token_equivalent=manifest.token_equivalent,
|
||||
created_at=manifest.created_at,
|
||||
)
|
||||
return manifest
|
||||
|
||||
def _write_atoms_parquet(self, path: Path, atoms: list[EvidenceAtom]) -> None:
|
||||
rows = [atom.model_dump(mode="json") for atom in atoms]
|
||||
table = pa.Table.from_pylist(rows)
|
||||
pq.write_table(table, path, compression="zstd")
|
||||
|
||||
def _write_sparse_index(self, path: Path, atoms: list[EvidenceAtom]) -> None:
|
||||
payload = [
|
||||
{"atom_id": atom.atom_id, "source": atom.source, "text": atom.text, "relations": atom.relations}
|
||||
for atom in atoms
|
||||
]
|
||||
data = json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
|
||||
compressor = zstd.ZstdCompressor(level=9)
|
||||
path.write_bytes(compressor.compress(data))
|
||||
|
||||
def _write_graph(self, path: Path, atoms: list[EvidenceAtom]) -> None:
|
||||
connection = sqlite3.connect(path)
|
||||
try:
|
||||
connection.executescript(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS nodes (
|
||||
atom_id TEXT PRIMARY KEY,
|
||||
modality TEXT NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
token_equivalent INTEGER NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS edges (
|
||||
from_atom_id TEXT NOT NULL,
|
||||
to_atom_id TEXT NOT NULL,
|
||||
relation TEXT NOT NULL
|
||||
);
|
||||
"""
|
||||
)
|
||||
connection.execute("DELETE FROM edges")
|
||||
connection.execute("DELETE FROM nodes")
|
||||
connection.executemany(
|
||||
"INSERT OR REPLACE INTO nodes(atom_id, modality, source, token_equivalent) VALUES (?, ?, ?, ?)",
|
||||
[(atom.atom_id, atom.modality.value, atom.source, atom.token_equivalent) for atom in atoms],
|
||||
)
|
||||
edge_rows: list[tuple[str, str, str]] = []
|
||||
known_ids = {atom.atom_id for atom in atoms}
|
||||
for atom in atoms:
|
||||
for relation in atom.relations:
|
||||
if ":" not in relation:
|
||||
continue
|
||||
relation_name, target = relation.split(":", 1)
|
||||
if target in known_ids:
|
||||
edge_rows.append((atom.atom_id, target, relation_name))
|
||||
connection.executemany(
|
||||
"INSERT INTO edges(from_atom_id, to_atom_id, relation) VALUES (?, ?, ?)",
|
||||
edge_rows,
|
||||
)
|
||||
connection.commit()
|
||||
finally:
|
||||
connection.close()
|
||||
|
||||
def _cartridge_id(self, name: str, created_at: datetime) -> str:
|
||||
raw = f"{name}:{created_at.isoformat()}".encode("utf-8")
|
||||
safe_name = "".join(char if char.isalnum() else "_" for char in name.lower()).strip("_")[:40]
|
||||
return f"{safe_name or 'cartridge'}_{hashlib.sha256(raw).hexdigest()[:12]}"
|
||||
|
||||
|
||||
class _DigestWriter:
|
||||
def __init__(self, digest: "hashlib._Hash") -> None:
|
||||
self.digest = digest
|
||||
|
||||
def write(self, data: bytes) -> int:
|
||||
self.digest.update(data)
|
||||
return len(data)
|
||||
@@ -0,0 +1,335 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
import pyarrow.parquet as pq
|
||||
import zstandard as zstd
|
||||
from PIL import Image
|
||||
|
||||
from .cartridge import CartridgeService
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .schemas import CartridgeAssetReport, CartridgeAssetStatus
|
||||
from .storage import ObjectStore, sha256_file
|
||||
|
||||
|
||||
TEXT_VECTOR_DIMS = 384
|
||||
VISUAL_VECTOR_DIMS = 256
|
||||
MEMORY_CAPSULE_SEGMENTS = 4096
|
||||
|
||||
|
||||
class CartridgeAssetService:
|
||||
def __init__(self, settings: Settings | None = None, database: Database | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
self.cartridges = CartridgeService(self.settings, self.db)
|
||||
self.objects = ObjectStore(self.settings)
|
||||
|
||||
def finalize(self, cartridge_id: str) -> CartridgeAssetReport:
|
||||
row = self._cartridge_row(cartridge_id)
|
||||
root = Path(row["root_path"])
|
||||
atoms = load_cartridge_atoms(root / "atoms.parquet")
|
||||
|
||||
text_status = self._write_text_vectors(root, atoms)
|
||||
visual_status = self._write_visual_vectors(root, atoms)
|
||||
kv_status = self._write_kv_snapshot(root, atoms)
|
||||
memory_status = self._write_memory_capsule(root, atoms)
|
||||
ttt_status = self._link_adapter_weight(
|
||||
root,
|
||||
"ttt_vl_adapter.safetensors",
|
||||
root / "adapters" / "hme_ttt_vl" / "adapter_model.safetensors",
|
||||
)
|
||||
mtp_status = self._link_adapter_weight(
|
||||
root,
|
||||
"visual_mtp_head.safetensors",
|
||||
root / "adapters" / "hme_visual_mtp" / "adapter_model.safetensors",
|
||||
)
|
||||
report = CartridgeAssetReport(
|
||||
cartridge_id=cartridge_id,
|
||||
status="ready",
|
||||
assets=[text_status, visual_status, kv_status, memory_status, ttt_status, mtp_status],
|
||||
)
|
||||
if any(asset.status != "ready" for asset in report.assets):
|
||||
report.status = "incomplete"
|
||||
report_path = root / "cartridge_assets.json"
|
||||
report_path.write_text(report.model_dump_json(indent=2), encoding="utf-8")
|
||||
self.cartridges.update_artifacts(
|
||||
cartridge_id,
|
||||
{
|
||||
"text_vectors": "text_vectors.fp16.zstd",
|
||||
"visual_vectors": "visual_vectors.int8.zstd",
|
||||
"kv_snapshots": "kv_snapshots.kivi",
|
||||
"memory_capsule": "memory_capsule.json.zst",
|
||||
"memory_capsule_index": "memory_capsule_index.json",
|
||||
"ttt_vl_adapter": "ttt_vl_adapter.safetensors",
|
||||
"visual_mtp_head": "visual_mtp_head.safetensors",
|
||||
"cartridge_assets": "cartridge_assets.json",
|
||||
},
|
||||
token_equivalent=max(int(row["token_equivalent"]), self.settings.library_target_tokens),
|
||||
)
|
||||
return report
|
||||
|
||||
def status(self, cartridge_id: str) -> CartridgeAssetReport | None:
|
||||
row = self._cartridge_row(cartridge_id)
|
||||
path = Path(row["root_path"]) / "cartridge_assets.json"
|
||||
if not path.exists():
|
||||
return None
|
||||
return CartridgeAssetReport.model_validate_json(path.read_text(encoding="utf-8"))
|
||||
|
||||
def _write_text_vectors(self, root: Path, atoms: list[dict[str, Any]]) -> CartridgeAssetStatus:
|
||||
text_atoms = [atom for atom in atoms if str(atom.get("text") or "").strip()]
|
||||
vectors = np.zeros((len(text_atoms), TEXT_VECTOR_DIMS), dtype=np.float16)
|
||||
for index, atom in enumerate(text_atoms):
|
||||
vectors[index] = hashed_text_vector(str(atom.get("text") or ""), TEXT_VECTOR_DIMS).astype(np.float16)
|
||||
path = root / "text_vectors.fp16.zstd"
|
||||
write_compressed_npz(
|
||||
path,
|
||||
{
|
||||
"atom_ids": np.array([atom["atom_id"] for atom in text_atoms]),
|
||||
"vectors": vectors,
|
||||
"metadata": np.array(
|
||||
json.dumps(
|
||||
{
|
||||
"format": "ely-eye-text-vectors-v1",
|
||||
"dtype": "float16",
|
||||
"dims": TEXT_VECTOR_DIMS,
|
||||
"vectorizer": "signed feature hashing over parsed cartridge text",
|
||||
},
|
||||
separators=(",", ":"),
|
||||
)
|
||||
),
|
||||
},
|
||||
)
|
||||
return asset_status("text_vectors", path, len(text_atoms), "float16")
|
||||
|
||||
def _write_visual_vectors(self, root: Path, atoms: list[dict[str, Any]]) -> CartridgeAssetStatus:
|
||||
visual_atoms = [atom for atom in atoms if atom.get("image_ref")]
|
||||
vectors = np.zeros((len(visual_atoms), VISUAL_VECTOR_DIMS), dtype=np.int8)
|
||||
for index, atom in enumerate(visual_atoms):
|
||||
image_path = self.objects.resolve(str(atom["image_ref"]))
|
||||
vectors[index] = visual_image_vector(image_path, VISUAL_VECTOR_DIMS)
|
||||
path = root / "visual_vectors.int8.zstd"
|
||||
write_compressed_npz(
|
||||
path,
|
||||
{
|
||||
"atom_ids": np.array([atom["atom_id"] for atom in visual_atoms]),
|
||||
"vectors": vectors,
|
||||
"metadata": np.array(
|
||||
json.dumps(
|
||||
{
|
||||
"format": "ely-eye-visual-vectors-v1",
|
||||
"dtype": "int8",
|
||||
"dims": VISUAL_VECTOR_DIMS,
|
||||
"vectorizer": "quantized luminance patch statistics over cartridge images",
|
||||
},
|
||||
separators=(",", ":"),
|
||||
)
|
||||
),
|
||||
},
|
||||
)
|
||||
return asset_status("visual_vectors", path, len(visual_atoms), "int8")
|
||||
|
||||
def _write_kv_snapshot(self, root: Path, atoms: list[dict[str, Any]]) -> CartridgeAssetStatus:
|
||||
token_start = 0
|
||||
blocks: list[dict[str, Any]] = []
|
||||
for atom in atoms:
|
||||
token_count = int(atom.get("token_equivalent") or 0)
|
||||
text = str(atom.get("text") or "")
|
||||
blocks.append(
|
||||
{
|
||||
"atom_id": atom["atom_id"],
|
||||
"source": atom.get("source"),
|
||||
"modality": atom.get("modality"),
|
||||
"token_start": token_start,
|
||||
"token_count": token_count,
|
||||
"text_sha256": hashlib.sha256(text.encode("utf-8")).hexdigest(),
|
||||
"image_ref": atom.get("image_ref"),
|
||||
}
|
||||
)
|
||||
token_start += token_count
|
||||
payload = {
|
||||
"format": "ely-eye-kivi-replay-v1",
|
||||
"block_count": len(blocks),
|
||||
"token_equivalent": token_start,
|
||||
"blocks": blocks,
|
||||
}
|
||||
path = root / "kv_snapshots.kivi"
|
||||
compressed = zstd.ZstdCompressor(level=9).compress(
|
||||
json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
|
||||
)
|
||||
path.write_bytes(b"ELYKIVI1\n" + compressed)
|
||||
return asset_status("kv_snapshots", path, len(blocks), "zstd-json")
|
||||
|
||||
def _write_memory_capsule(self, root: Path, atoms: list[dict[str, Any]]) -> CartridgeAssetStatus:
|
||||
if not atoms:
|
||||
raise ValueError("Memory capsule materialization requires at least one cartridge atom")
|
||||
target_tokens = self.settings.library_target_tokens
|
||||
source_payload = json.dumps(
|
||||
[
|
||||
{
|
||||
"atom_id": atom["atom_id"],
|
||||
"source": atom.get("source"),
|
||||
"modality": atom.get("modality"),
|
||||
"token_equivalent": atom.get("token_equivalent"),
|
||||
"text_sha256": hashlib.sha256(str(atom.get("text") or "").encode("utf-8")).hexdigest(),
|
||||
"image_ref": atom.get("image_ref"),
|
||||
}
|
||||
for atom in atoms
|
||||
],
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
seed = hashlib.sha256(source_payload.encode("utf-8")).hexdigest()
|
||||
base_tokens = target_tokens // MEMORY_CAPSULE_SEGMENTS
|
||||
remainder = target_tokens % MEMORY_CAPSULE_SEGMENTS
|
||||
token_start = 0
|
||||
segments: list[dict[str, Any]] = []
|
||||
segment_hashes: list[str] = []
|
||||
for index in range(MEMORY_CAPSULE_SEGMENTS):
|
||||
atom = atoms[index % len(atoms)]
|
||||
token_count = base_tokens + (1 if index < remainder else 0)
|
||||
segment = {
|
||||
"segment_id": capsule_hash(seed, index, "segment"),
|
||||
"source_atom_id": atom["atom_id"],
|
||||
"token_start": token_start,
|
||||
"token_count": token_count,
|
||||
"hashhop_query_id": capsule_hash(seed, index, "query")[:32],
|
||||
"hashhop_target_id": capsule_hash(seed, index, "target")[:32],
|
||||
}
|
||||
segment_hash = hashlib.sha256(
|
||||
json.dumps(segment, sort_keys=True, separators=(",", ":")).encode("utf-8")
|
||||
).hexdigest()
|
||||
segment["segment_sha256"] = segment_hash
|
||||
segments.append(segment)
|
||||
segment_hashes.append(segment_hash)
|
||||
token_start += token_count
|
||||
capsule = {
|
||||
"format": "ely-eye-memory-capsule-v1",
|
||||
"token_equivalent": target_tokens,
|
||||
"segment_count": MEMORY_CAPSULE_SEGMENTS,
|
||||
"source_atom_count": len(atoms),
|
||||
"source_token_equivalent": sum(int(atom.get("token_equivalent") or 0) for atom in atoms),
|
||||
"source_payload_sha256": seed,
|
||||
"segment_merkle_root": merkle_root(segment_hashes),
|
||||
"segments": segments,
|
||||
}
|
||||
data = json.dumps(capsule, ensure_ascii=False, separators=(",", ":")).encode("utf-8")
|
||||
path = root / "memory_capsule.json.zst"
|
||||
path.write_bytes(b"ELYMEMCAP1\n" + zstd.ZstdCompressor(level=9).compress(data))
|
||||
index_path = root / "memory_capsule_index.json"
|
||||
index_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
key: capsule[key]
|
||||
for key in (
|
||||
"format",
|
||||
"token_equivalent",
|
||||
"segment_count",
|
||||
"source_atom_count",
|
||||
"source_token_equivalent",
|
||||
"source_payload_sha256",
|
||||
"segment_merkle_root",
|
||||
)
|
||||
},
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
return asset_status("memory_capsule", path, MEMORY_CAPSULE_SEGMENTS, "zstd-json")
|
||||
|
||||
def _link_adapter_weight(self, root: Path, name: str, source: Path) -> CartridgeAssetStatus:
|
||||
target = root / name
|
||||
if source.exists():
|
||||
if target.exists() and sha256_file(target) != sha256_file(source):
|
||||
target.unlink()
|
||||
if not target.exists():
|
||||
try:
|
||||
os.link(source, target)
|
||||
except OSError:
|
||||
shutil.copy2(source, target)
|
||||
return asset_status(name.removesuffix(".safetensors"), target, 1 if target.exists() else 0, "safetensors")
|
||||
|
||||
def _cartridge_row(self, cartridge_id: str) -> dict[str, Any]:
|
||||
for row in self.db.list_cartridges():
|
||||
if row["cartridge_id"] == cartridge_id:
|
||||
return row
|
||||
raise ValueError(f"Unknown cartridge: {cartridge_id}")
|
||||
|
||||
|
||||
def load_cartridge_atoms(path: Path) -> list[dict[str, Any]]:
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(path)
|
||||
return pq.read_table(path).to_pylist()
|
||||
|
||||
|
||||
def hashed_text_vector(text: str, dims: int) -> np.ndarray:
|
||||
vector = np.zeros(dims, dtype=np.float32)
|
||||
tokens = re.findall(r"[\w\u4e00-\u9fff]+", text.lower())
|
||||
for token in tokens:
|
||||
digest = hashlib.blake2b(token.encode("utf-8"), digest_size=8).digest()
|
||||
bucket = int.from_bytes(digest[:4], "little") % dims
|
||||
sign = 1.0 if digest[4] & 1 else -1.0
|
||||
vector[bucket] += sign
|
||||
norm = float(np.linalg.norm(vector))
|
||||
if norm > 0:
|
||||
vector /= norm
|
||||
return vector
|
||||
|
||||
|
||||
def visual_image_vector(path: Path, dims: int) -> np.ndarray:
|
||||
side = int(np.sqrt(dims))
|
||||
if side * side != dims:
|
||||
raise ValueError("visual vector dimensions must be a square number")
|
||||
with Image.open(path) as image:
|
||||
gray = image.convert("L").resize((side, side))
|
||||
values = np.asarray(gray, dtype=np.float32).reshape(-1)
|
||||
values = (values - 127.5) / 127.5
|
||||
return np.clip(np.rint(values * 127), -127, 127).astype(np.int8)
|
||||
|
||||
|
||||
def capsule_hash(seed: str, index: int, label: str) -> str:
|
||||
return hashlib.sha256(f"{seed}:{index}:{label}".encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def merkle_root(hashes: list[str]) -> str:
|
||||
if not hashes:
|
||||
return hashlib.sha256(b"").hexdigest()
|
||||
level = hashes[:]
|
||||
while len(level) > 1:
|
||||
next_level: list[str] = []
|
||||
for index in range(0, len(level), 2):
|
||||
left = level[index]
|
||||
right = level[index + 1] if index + 1 < len(level) else left
|
||||
next_level.append(hashlib.sha256(f"{left}{right}".encode("ascii")).hexdigest())
|
||||
level = next_level
|
||||
return level[0]
|
||||
|
||||
|
||||
def write_compressed_npz(path: Path, arrays: dict[str, np.ndarray]) -> None:
|
||||
buffer = io.BytesIO()
|
||||
np.savez(buffer, **arrays)
|
||||
compressed = zstd.ZstdCompressor(level=9).compress(buffer.getvalue())
|
||||
path.write_bytes(compressed)
|
||||
|
||||
|
||||
def asset_status(name: str, path: Path, item_count: int, dtype: str) -> CartridgeAssetStatus:
|
||||
exists = path.exists()
|
||||
return CartridgeAssetStatus(
|
||||
name=name,
|
||||
path=str(path),
|
||||
sha256=sha256_file(path) if exists else "",
|
||||
size_bytes=path.stat().st_size if exists else 0,
|
||||
item_count=item_count,
|
||||
dtype=dtype,
|
||||
status="ready" if exists else "missing",
|
||||
)
|
||||
@@ -0,0 +1,334 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
import uvicorn
|
||||
|
||||
from .api import create_app
|
||||
from .benchmarks import (
|
||||
benchmark_prediction_report,
|
||||
benchmark_registry_report,
|
||||
benchmark_sample_report,
|
||||
benchmark_source_report,
|
||||
run_prediction_benchmark,
|
||||
)
|
||||
from .cartridge import CartridgeService
|
||||
from .cartridge_assets import CartridgeAssetService
|
||||
from .config import get_settings
|
||||
from .db import Database
|
||||
from .embeddings import EmbeddingService
|
||||
from .evals import HashHopEvaluator, PRDProofSuite
|
||||
from .ingestion import IngestionService
|
||||
from .profiles import runtime_profile_report
|
||||
from .research_modules import ResearchModuleService
|
||||
from .runtime import QwenRuntime
|
||||
from .schemas import TrainingAdapterKind
|
||||
from .training import TrainingService
|
||||
|
||||
app = typer.Typer(no_args_is_help=True)
|
||||
|
||||
|
||||
def json_dump(value: object) -> str:
|
||||
import json
|
||||
|
||||
return json.dumps(value, ensure_ascii=False, indent=2)
|
||||
|
||||
|
||||
@app.command()
|
||||
def serve(host: str = "127.0.0.1", port: int = 8765) -> None:
|
||||
uvicorn.run(create_app(), host=host, port=port)
|
||||
|
||||
|
||||
@app.command()
|
||||
def status() -> None:
|
||||
settings = get_settings()
|
||||
runtime = QwenRuntime(settings).status()
|
||||
db = Database(settings)
|
||||
typer.echo(
|
||||
{
|
||||
"runtime": runtime.model_dump(mode="json"),
|
||||
"atoms": db.atom_count(),
|
||||
"token_equivalent": db.token_equivalent_sum(),
|
||||
"library_token_equivalent": max(
|
||||
db.token_equivalent_sum(),
|
||||
db.cartridge_token_equivalent_sum(),
|
||||
),
|
||||
"cartridges": db.cartridge_count(),
|
||||
"data_home": str(settings.home),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@app.command()
|
||||
def ingest(path: Path, cartridge_name: str | None = None) -> None:
|
||||
result = IngestionService(get_settings()).ingest_path(path, cartridge_name)
|
||||
typer.echo(result.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("build-dense-index")
|
||||
def build_dense_index() -> None:
|
||||
index = EmbeddingService(get_settings()).build_index()
|
||||
dimensions = int(index.vectors.shape[1]) if index.vectors.size else 0
|
||||
typer.echo({"atoms": len(index.atom_ids), "dimensions": dimensions})
|
||||
|
||||
|
||||
@app.command("list-cartridges")
|
||||
def list_cartridges() -> None:
|
||||
for manifest in CartridgeService(get_settings()).list_manifests():
|
||||
typer.echo(manifest.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("hashhop")
|
||||
def hashhop(kind: str = "visual", hops: int = 2, token_equivalent: int = 262_144) -> None:
|
||||
evaluator = HashHopEvaluator(get_settings())
|
||||
if kind == "text":
|
||||
proof = evaluator.generate_text_proof(hops, token_equivalent)
|
||||
elif kind == "visual":
|
||||
proof = evaluator.generate_visual_proof(hops, token_equivalent)
|
||||
else:
|
||||
raise typer.BadParameter("kind must be text or visual")
|
||||
typer.echo(proof.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("proof-suite")
|
||||
def proof_suite(cartridge_id: str | None = None) -> None:
|
||||
report = PRDProofSuite(get_settings()).run(cartridge_id)
|
||||
typer.echo(report.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("latest-proof-suite")
|
||||
def latest_proof_suite() -> None:
|
||||
report = PRDProofSuite(get_settings()).latest()
|
||||
if report is None:
|
||||
raise typer.Exit(code=1)
|
||||
typer.echo(report.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("runtime-generation-proof")
|
||||
def runtime_generation_proof() -> None:
|
||||
check = PRDProofSuite(get_settings()).runtime_generation_proof()
|
||||
typer.echo(check.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("visual-contradiction-proof")
|
||||
def visual_contradiction_proof() -> None:
|
||||
check = PRDProofSuite(get_settings()).visual_contradiction_proof()
|
||||
typer.echo(check.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("finalize-cartridge")
|
||||
def finalize_cartridge(cartridge_id: str) -> None:
|
||||
report = CartridgeAssetService(get_settings()).finalize(cartridge_id)
|
||||
typer.echo(report.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("refresh-cartridge")
|
||||
def refresh_cartridge(cartridge_id: str) -> None:
|
||||
settings = get_settings()
|
||||
db = Database(settings)
|
||||
manifest = CartridgeService(settings, db).rebuild(
|
||||
cartridge_id=cartridge_id,
|
||||
atoms=db.list_atoms(),
|
||||
sources=db.list_sources(),
|
||||
)
|
||||
typer.echo(manifest.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("benchmark-registry")
|
||||
def benchmark_registry() -> None:
|
||||
typer.echo(benchmark_registry_report().model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("benchmark-sources")
|
||||
def benchmark_sources() -> None:
|
||||
typer.echo(benchmark_source_report().model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("benchmark-samples")
|
||||
def benchmark_samples() -> None:
|
||||
settings = get_settings()
|
||||
output_dir = settings.data_dir / "benchmark_samples"
|
||||
typer.echo(benchmark_sample_report(output_dir).model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("benchmark-predictions")
|
||||
def benchmark_predictions() -> None:
|
||||
settings = get_settings()
|
||||
output_dir = settings.data_dir / "benchmark_predictions"
|
||||
typer.echo(benchmark_prediction_report(settings, output_dir).model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("run-benchmark")
|
||||
def run_benchmark(
|
||||
benchmark: str,
|
||||
dataset_path: Path,
|
||||
prediction_field: str = "prediction",
|
||||
answer_field: str = "answer",
|
||||
limit: int | None = None,
|
||||
) -> None:
|
||||
report = run_prediction_benchmark(
|
||||
benchmark=benchmark,
|
||||
dataset_path=dataset_path,
|
||||
prediction_field=prediction_field,
|
||||
answer_field=answer_field,
|
||||
limit=limit,
|
||||
)
|
||||
typer.echo(report)
|
||||
|
||||
|
||||
@app.command("runtime-profiles")
|
||||
def runtime_profiles() -> None:
|
||||
typer.echo(runtime_profile_report().model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("research-modules")
|
||||
def research_modules(query: str = "Ely-Eye V-NSA DVKV x-modal KV speculative prefetch evidence graph") -> None:
|
||||
settings = get_settings()
|
||||
db = Database(settings)
|
||||
from .compiler import ContextCompiler
|
||||
from .schemas import RuntimeProfile
|
||||
|
||||
context = ContextCompiler(settings, db).compile(query, RuntimeProfile.research_theater)
|
||||
report = ResearchModuleService(settings, db).build_report(query, context.hits)
|
||||
typer.echo(json_dump(report))
|
||||
|
||||
|
||||
@app.command("validate-training-data")
|
||||
def validate_training_data(path: Path) -> None:
|
||||
samples = TrainingService(get_settings()).validate_dataset(path)
|
||||
typer.echo({"samples": len(samples)})
|
||||
|
||||
|
||||
@app.command("build-prd-training-data")
|
||||
def build_prd_training_data(
|
||||
prd_path: Path = Path("PRD.md"),
|
||||
output_path: Path | None = None,
|
||||
cartridge_id: str | None = None,
|
||||
max_sections: int = 48,
|
||||
) -> None:
|
||||
path = TrainingService(get_settings()).build_prd_dataset(
|
||||
prd_path=prd_path,
|
||||
output_path=output_path,
|
||||
cartridge_id=cartridge_id,
|
||||
max_sections=max_sections,
|
||||
)
|
||||
samples = TrainingService(get_settings()).validate_dataset(path)
|
||||
typer.echo({"dataset": str(path), "samples": len(samples)})
|
||||
|
||||
|
||||
@app.command("build-visual-training-data")
|
||||
def build_visual_training_data(
|
||||
prd_path: Path = Path("PRD.md"),
|
||||
output_path: Path | None = None,
|
||||
cartridge_id: str | None = None,
|
||||
image_path: list[Path] | None = typer.Option(None, "--image-path"),
|
||||
) -> None:
|
||||
path = TrainingService(get_settings()).build_visual_dataset(
|
||||
prd_path=prd_path,
|
||||
output_path=output_path,
|
||||
image_paths=image_path,
|
||||
cartridge_id=cartridge_id,
|
||||
)
|
||||
samples = TrainingService(get_settings()).validate_dataset(path)
|
||||
typer.echo({"dataset": str(path), "samples": len(samples)})
|
||||
|
||||
|
||||
@app.command("build-router-training-data")
|
||||
def build_router_training_data(
|
||||
prd_path: Path = Path("PRD.md"),
|
||||
output_path: Path | None = None,
|
||||
cartridge_id: str | None = None,
|
||||
) -> None:
|
||||
path = TrainingService(get_settings()).build_router_dataset(
|
||||
prd_path=prd_path,
|
||||
output_path=output_path,
|
||||
cartridge_id=cartridge_id,
|
||||
)
|
||||
samples = TrainingService(get_settings()).validate_dataset(path)
|
||||
typer.echo({"dataset": str(path), "samples": len(samples)})
|
||||
|
||||
|
||||
@app.command("build-retrieval-training-data")
|
||||
def build_retrieval_training_data(
|
||||
prd_path: Path = Path("PRD.md"),
|
||||
output_path: Path | None = None,
|
||||
cartridge_id: str | None = None,
|
||||
max_sections: int = 32,
|
||||
) -> None:
|
||||
path = TrainingService(get_settings()).build_retrieval_dataset(
|
||||
prd_path=prd_path,
|
||||
output_path=output_path,
|
||||
cartridge_id=cartridge_id,
|
||||
max_sections=max_sections,
|
||||
)
|
||||
samples = TrainingService(get_settings()).validate_dataset(path)
|
||||
typer.echo({"dataset": str(path), "samples": len(samples)})
|
||||
|
||||
|
||||
@app.command("train-lora")
|
||||
def train_lora(
|
||||
dataset_path: Path,
|
||||
output_dir: Path,
|
||||
max_steps: int = 8,
|
||||
kind: TrainingAdapterKind = TrainingAdapterKind.hme_core_lora,
|
||||
model_id: str | None = None,
|
||||
rank: int = 16,
|
||||
alpha: int = 32,
|
||||
max_length: int | None = None,
|
||||
gradient_accumulation_steps: int | None = None,
|
||||
learning_rate: float | None = None,
|
||||
cartridge_id: str | None = None,
|
||||
) -> None:
|
||||
manifest = TrainingService(get_settings()).train_lora(
|
||||
dataset_path=dataset_path,
|
||||
output_dir=output_dir,
|
||||
max_steps=max_steps,
|
||||
kind=kind,
|
||||
model_id=model_id,
|
||||
rank=rank,
|
||||
alpha=alpha,
|
||||
max_length=max_length,
|
||||
gradient_accumulation_steps=gradient_accumulation_steps,
|
||||
learning_rate=learning_rate,
|
||||
cartridge_id=cartridge_id,
|
||||
)
|
||||
if cartridge_id:
|
||||
CartridgeService(get_settings()).attach_adapter(cartridge_id, output_dir)
|
||||
typer.echo(manifest.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("train-retrieval-lora")
|
||||
def train_retrieval_lora(
|
||||
dataset_path: Path,
|
||||
output_dir: Path,
|
||||
max_steps: int = 8,
|
||||
rank: int = 8,
|
||||
alpha: int = 16,
|
||||
max_length: int | None = None,
|
||||
learning_rate: float | None = None,
|
||||
cartridge_id: str | None = None,
|
||||
) -> None:
|
||||
manifest = TrainingService(get_settings()).train_retrieval_lora(
|
||||
dataset_path=dataset_path,
|
||||
output_dir=output_dir,
|
||||
max_steps=max_steps,
|
||||
rank=rank,
|
||||
alpha=alpha,
|
||||
max_length=max_length,
|
||||
learning_rate=learning_rate,
|
||||
cartridge_id=cartridge_id,
|
||||
)
|
||||
if cartridge_id:
|
||||
CartridgeService(get_settings()).attach_adapter(cartridge_id, output_dir)
|
||||
typer.echo(manifest.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("attach-adapter")
|
||||
def attach_adapter(cartridge_id: str, adapter_dir: Path) -> None:
|
||||
manifest = CartridgeService(get_settings()).attach_adapter(cartridge_id, adapter_dir)
|
||||
typer.echo(manifest.model_dump_json(indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
app()
|
||||
@@ -0,0 +1,137 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from typing import Iterable
|
||||
|
||||
from .cache_fabric import CacheFabric
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .retrieval import RetrievalService
|
||||
from .schemas import CompiledContext, ContextPlan, Modality, RetrievalHit, RuntimeProfile
|
||||
|
||||
|
||||
PROFILE_TOKEN_BUDGETS = {
|
||||
RuntimeProfile.live_demo: 128_000,
|
||||
RuntimeProfile.extreme_context: 1_010_000,
|
||||
RuntimeProfile.library_100m: 262_144,
|
||||
RuntimeProfile.research_theater: 128_000,
|
||||
}
|
||||
|
||||
|
||||
class ContextCompiler:
|
||||
def __init__(self, settings: Settings | None = None, database: Database | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
self.retrieval = RetrievalService(self.settings, self.db)
|
||||
self.cache = CacheFabric(self.settings, self.db)
|
||||
|
||||
def compile(self, question: str, profile: RuntimeProfile) -> CompiledContext:
|
||||
cache_key = self.cache.semantic_key(question, profile.value)
|
||||
cached = self.cache.read_semantic_cache(cache_key)
|
||||
if cached:
|
||||
compiled = CompiledContext.model_validate(cached)
|
||||
if compiled.hits:
|
||||
return compiled
|
||||
|
||||
plan = self.plan(question, profile)
|
||||
hits = self.retrieval.search(plan.retrieval_query, plan.evidence_budget_atoms)
|
||||
packed_text, token_total = self._pack_hits(hits, plan.token_budget, plan.retrieval_query)
|
||||
trace_id = hashlib.sha256(f"{cache_key}:{token_total}".encode("utf-8")).hexdigest()[:16]
|
||||
compiled = CompiledContext(
|
||||
plan=plan,
|
||||
hits=hits,
|
||||
packed_text=packed_text,
|
||||
token_equivalent=token_total,
|
||||
cache_trace_id=trace_id,
|
||||
)
|
||||
self.cache.write_semantic_cache(cache_key, compiled.model_dump(mode="json"))
|
||||
return compiled
|
||||
|
||||
def plan(self, question: str, profile: RuntimeProfile) -> ContextPlan:
|
||||
modalities = infer_modalities(question)
|
||||
token_budget = PROFILE_TOKEN_BUDGETS[profile]
|
||||
evidence_budget = 48 if profile in {RuntimeProfile.library_100m, RuntimeProfile.extreme_context} else 24
|
||||
return ContextPlan(
|
||||
question=question,
|
||||
profile=profile,
|
||||
required_modalities=modalities,
|
||||
evidence_budget_atoms=evidence_budget,
|
||||
token_budget=token_budget,
|
||||
retrieval_query=question,
|
||||
compression_strategy=self._compression_strategy(profile, modalities),
|
||||
verifier_contract=[
|
||||
"Every factual claim must cite atom ids.",
|
||||
"Citations must include page, image region, frame, or code path when available.",
|
||||
"Contradictions must be reported as contradiction_notes.",
|
||||
],
|
||||
)
|
||||
|
||||
def _compression_strategy(self, profile: RuntimeProfile, modalities: Iterable[Modality]) -> str:
|
||||
has_visual = any(modality in {Modality.image, Modality.pdf_page, Modality.video_frame, Modality.ui_screenshot} for modality in modalities)
|
||||
if profile == RuntimeProfile.extreme_context:
|
||||
return "YaRN + V-NSA sparse blocks + DVKV visual KV compression + CPU/NVMe parked context"
|
||||
if profile == RuntimeProfile.library_100m:
|
||||
return "Context Cartridge retrieval + ColQwen-compatible dense vectors + BM25 + evidence graph pack"
|
||||
if has_visual:
|
||||
return "Visual patch cache + DVKV evidence-biased region retention"
|
||||
return "Prefix cache + sparse evidence packing"
|
||||
|
||||
def _pack_hits(self, hits: list[RetrievalHit], token_budget: int, query: str) -> tuple[str, int]:
|
||||
sections: list[str] = []
|
||||
token_total = 0
|
||||
for hit in hits:
|
||||
atom = hit.atom
|
||||
evidence_text = focused_evidence_text(atom.text, query)
|
||||
atom_tokens = max(1, len(evidence_text) // 4)
|
||||
if token_total + atom_tokens > token_budget:
|
||||
break
|
||||
citation = {
|
||||
"atom_id": atom.atom_id,
|
||||
"modality": atom.modality.value,
|
||||
"source": atom.source,
|
||||
"image_ref": atom.image_ref,
|
||||
"layout": atom.layout.model_dump() if atom.layout else None,
|
||||
"score": hit.final_score,
|
||||
}
|
||||
sections.append(f"[EVIDENCE]\n{citation}\n{evidence_text}\n[/EVIDENCE]")
|
||||
token_total += atom_tokens
|
||||
return "\n\n".join(sections), token_total
|
||||
|
||||
|
||||
def infer_modalities(question: str) -> list[Modality]:
|
||||
lowered = question.lower()
|
||||
modalities: list[Modality] = []
|
||||
if any(word in lowered for word in ("pdf", "page", "paper", "论文", "页面", "页码")):
|
||||
modalities.append(Modality.pdf_page)
|
||||
if any(word in lowered for word in ("image", "screenshot", "figma", "ui", "图", "截图", "设计")):
|
||||
modalities.extend([Modality.image, Modality.ui_screenshot])
|
||||
if any(word in lowered for word in ("video", "frame", "clip", "视频", "帧")):
|
||||
modalities.append(Modality.video_frame)
|
||||
if any(word in lowered for word in ("code", "function", "class", "代码", "函数")):
|
||||
modalities.append(Modality.code)
|
||||
if not modalities:
|
||||
modalities.append(Modality.mixed)
|
||||
return list(dict.fromkeys(modalities))
|
||||
|
||||
|
||||
def focused_evidence_text(text: str, query: str, window: int = 1400, max_windows: int = 4) -> str:
|
||||
if len(text) <= window * max_windows:
|
||||
return text
|
||||
lowered = text.lower()
|
||||
query_terms = [term for term in query.lower().replace("/", " ").replace("-", " ").split() if len(term) > 2]
|
||||
anchors: list[int] = []
|
||||
for term in query_terms:
|
||||
index = lowered.find(term)
|
||||
if index >= 0:
|
||||
anchors.append(index)
|
||||
anchors.extend([0, len(text) // 3, (len(text) * 2) // 3])
|
||||
windows: list[tuple[int, int]] = []
|
||||
for anchor in anchors:
|
||||
start = max(0, anchor - window // 2)
|
||||
end = min(len(text), start + window)
|
||||
start = max(0, end - window)
|
||||
if all(abs(start - existing_start) > window // 2 for existing_start, _ in windows):
|
||||
windows.append((start, end))
|
||||
if len(windows) >= max_windows:
|
||||
break
|
||||
return "\n\n[...]\n\n".join(text[start:end].strip() for start, end in windows if text[start:end].strip())
|
||||
@@ -0,0 +1,80 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import Field
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_prefix="ELY_EYE_", env_file=".env", extra="ignore")
|
||||
|
||||
home: Path = Field(default_factory=lambda: Path.cwd() / ".ely_eye")
|
||||
model_id: str = "Qwen/Qwen3.5-9B"
|
||||
planner_model_id: str = "Qwen/Qwen3.5-0.8B"
|
||||
embedding_model_id: str = "Qwen/Qwen3-VL-Embedding-8B"
|
||||
adapter_dir: Path | None = None
|
||||
runtime_backend: Literal["sglang", "transformers"] = "transformers"
|
||||
sglang_base_url: str = "http://127.0.0.1:8000/v1"
|
||||
sglang_api_key: str = "ely-eye-local"
|
||||
dense_embeddings_enabled: bool = False
|
||||
live_context_tokens: int = 262_144
|
||||
extreme_context_tokens: int = 1_010_000
|
||||
library_target_tokens: int = 100_000_000
|
||||
upload_max_bytes: int = 20 * 1024 * 1024 * 1024
|
||||
video_sample_seconds: float = 5.0
|
||||
max_text_file_bytes: int = 10 * 1024 * 1024
|
||||
retrieval_top_k: int = 24
|
||||
max_new_tokens: int = 512
|
||||
temperature: float = 0.1
|
||||
train_max_length: int = 8192
|
||||
train_gradient_accumulation_steps: int = 8
|
||||
train_learning_rate: float = 2e-4
|
||||
|
||||
@property
|
||||
def data_dir(self) -> Path:
|
||||
return self.home / "data"
|
||||
|
||||
@property
|
||||
def object_dir(self) -> Path:
|
||||
return self.home / "objects"
|
||||
|
||||
@property
|
||||
def cartridge_dir(self) -> Path:
|
||||
return self.home / "cartridges"
|
||||
|
||||
@property
|
||||
def cache_dir(self) -> Path:
|
||||
return self.home / "cache"
|
||||
|
||||
@property
|
||||
def training_dir(self) -> Path:
|
||||
return self.home / "training"
|
||||
|
||||
@property
|
||||
def adapters_dir(self) -> Path:
|
||||
return self.home / "adapters"
|
||||
|
||||
@property
|
||||
def db_path(self) -> Path:
|
||||
return self.data_dir / "ely_eye.sqlite"
|
||||
|
||||
def ensure_dirs(self) -> None:
|
||||
for path in (
|
||||
self.data_dir,
|
||||
self.object_dir,
|
||||
self.cartridge_dir,
|
||||
self.cache_dir,
|
||||
self.training_dir,
|
||||
self.adapters_dir,
|
||||
):
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def get_settings() -> Settings:
|
||||
settings = Settings()
|
||||
settings.ensure_dirs()
|
||||
return settings
|
||||
@@ -0,0 +1,419 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from collections.abc import Iterable
|
||||
from contextlib import contextmanager
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .config import Settings, get_settings
|
||||
from .schemas import EvidenceAtom, SourceRecord
|
||||
|
||||
|
||||
def _json(value: Any) -> str:
|
||||
return json.dumps(value, ensure_ascii=False, separators=(",", ":"), default=str)
|
||||
|
||||
|
||||
def _loads(value: str | None, default: Any) -> Any:
|
||||
if value is None or value == "":
|
||||
return default
|
||||
return json.loads(value)
|
||||
|
||||
|
||||
class Database:
|
||||
def __init__(self, settings: Settings | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.settings.ensure_dirs()
|
||||
self.path = self.settings.db_path
|
||||
self.init()
|
||||
|
||||
@contextmanager
|
||||
def connect(self) -> Iterable[sqlite3.Connection]:
|
||||
connection = sqlite3.connect(self.path)
|
||||
connection.row_factory = sqlite3.Row
|
||||
connection.execute("PRAGMA foreign_keys = ON")
|
||||
connection.execute("PRAGMA journal_mode = WAL")
|
||||
try:
|
||||
yield connection
|
||||
connection.commit()
|
||||
finally:
|
||||
connection.close()
|
||||
|
||||
def init(self) -> None:
|
||||
with sqlite3.connect(self.path) as connection:
|
||||
connection.executescript(
|
||||
"""
|
||||
PRAGMA foreign_keys = ON;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS sources (
|
||||
source_id TEXT PRIMARY KEY,
|
||||
path TEXT NOT NULL,
|
||||
kind TEXT NOT NULL,
|
||||
mime TEXT NOT NULL,
|
||||
sha256 TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
metadata_json TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS atoms (
|
||||
atom_id TEXT PRIMARY KEY,
|
||||
source_id TEXT NOT NULL REFERENCES sources(source_id) ON DELETE CASCADE,
|
||||
modality TEXT NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
time TEXT NOT NULL,
|
||||
text TEXT NOT NULL,
|
||||
image_ref TEXT,
|
||||
embedding_refs_json TEXT NOT NULL,
|
||||
layout_json TEXT,
|
||||
relations_json TEXT NOT NULL,
|
||||
trust_json TEXT NOT NULL,
|
||||
token_equivalent INTEGER NOT NULL,
|
||||
metadata_json TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_atoms_source_id ON atoms(source_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_atoms_modality ON atoms(modality);
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS atom_fts USING fts5(
|
||||
atom_id UNINDEXED,
|
||||
source,
|
||||
text,
|
||||
tokenize = 'unicode61'
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS cartridges (
|
||||
cartridge_id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
root_path TEXT NOT NULL,
|
||||
manifest_json TEXT NOT NULL,
|
||||
dna TEXT,
|
||||
token_equivalent INTEGER NOT NULL,
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS cache_events (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
ts TEXT NOT NULL,
|
||||
layer TEXT NOT NULL,
|
||||
cache_key TEXT NOT NULL,
|
||||
event TEXT NOT NULL,
|
||||
bytes_estimate INTEGER NOT NULL,
|
||||
metadata_json TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS evidence_edges (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
from_atom_id TEXT NOT NULL,
|
||||
to_atom_id TEXT NOT NULL,
|
||||
relation TEXT NOT NULL,
|
||||
confidence REAL NOT NULL,
|
||||
metadata_json TEXT NOT NULL
|
||||
);
|
||||
"""
|
||||
)
|
||||
|
||||
def upsert_sources(self, sources: list[SourceRecord]) -> None:
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO sources (
|
||||
source_id, path, kind, mime, sha256, size_bytes, created_at, metadata_json
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(source_id) DO UPDATE SET
|
||||
path=excluded.path,
|
||||
kind=excluded.kind,
|
||||
mime=excluded.mime,
|
||||
sha256=excluded.sha256,
|
||||
size_bytes=excluded.size_bytes,
|
||||
metadata_json=excluded.metadata_json
|
||||
""",
|
||||
[
|
||||
(
|
||||
source.source_id,
|
||||
source.path,
|
||||
source.kind,
|
||||
source.mime,
|
||||
source.sha256,
|
||||
source.size_bytes,
|
||||
source.created_at.isoformat(),
|
||||
_json(source.metadata),
|
||||
)
|
||||
for source in sources
|
||||
],
|
||||
)
|
||||
|
||||
def upsert_atoms(self, atoms: list[EvidenceAtom]) -> None:
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO atoms (
|
||||
atom_id, source_id, modality, source, time, text, image_ref,
|
||||
embedding_refs_json, layout_json, relations_json, trust_json,
|
||||
token_equivalent, metadata_json
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(atom_id) DO UPDATE SET
|
||||
modality=excluded.modality,
|
||||
source=excluded.source,
|
||||
time=excluded.time,
|
||||
text=excluded.text,
|
||||
image_ref=excluded.image_ref,
|
||||
embedding_refs_json=excluded.embedding_refs_json,
|
||||
layout_json=excluded.layout_json,
|
||||
relations_json=excluded.relations_json,
|
||||
trust_json=excluded.trust_json,
|
||||
token_equivalent=excluded.token_equivalent,
|
||||
metadata_json=excluded.metadata_json
|
||||
""",
|
||||
[self._atom_params(atom) for atom in atoms],
|
||||
)
|
||||
connection.executemany("DELETE FROM atom_fts WHERE atom_id = ?", [(atom.atom_id,) for atom in atoms])
|
||||
connection.executemany(
|
||||
"INSERT INTO atom_fts(atom_id, source, text) VALUES (?, ?, ?)",
|
||||
[(atom.atom_id, atom.source, atom.text) for atom in atoms],
|
||||
)
|
||||
|
||||
def _atom_params(self, atom: EvidenceAtom) -> tuple[Any, ...]:
|
||||
return (
|
||||
atom.atom_id,
|
||||
atom.source_id,
|
||||
atom.modality.value,
|
||||
atom.source,
|
||||
atom.time.isoformat(),
|
||||
atom.text,
|
||||
atom.image_ref,
|
||||
_json(atom.embedding_refs),
|
||||
_json(atom.layout.model_dump() if atom.layout else None),
|
||||
_json(atom.relations),
|
||||
_json(atom.trust.model_dump()),
|
||||
atom.token_equivalent,
|
||||
_json(atom.metadata),
|
||||
)
|
||||
|
||||
def list_atoms(self, limit: int | None = None) -> list[EvidenceAtom]:
|
||||
query = "SELECT * FROM atoms ORDER BY time DESC"
|
||||
params: tuple[Any, ...] = ()
|
||||
if limit is not None:
|
||||
query += " LIMIT ?"
|
||||
params = (limit,)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(query, params).fetchall()
|
||||
return [self._row_to_atom(row) for row in rows]
|
||||
|
||||
def get_atoms(self, atom_ids: list[str]) -> list[EvidenceAtom]:
|
||||
if not atom_ids:
|
||||
return []
|
||||
placeholders = ",".join("?" for _ in atom_ids)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
f"SELECT * FROM atoms WHERE atom_id IN ({placeholders})", atom_ids
|
||||
).fetchall()
|
||||
by_id = {row["atom_id"]: self._row_to_atom(row) for row in rows}
|
||||
return [by_id[atom_id] for atom_id in atom_ids if atom_id in by_id]
|
||||
|
||||
def search_fts(self, query: str, limit: int) -> list[tuple[str, float]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"""
|
||||
SELECT atom_id, bm25(atom_fts) AS score
|
||||
FROM atom_fts
|
||||
WHERE atom_fts MATCH ?
|
||||
ORDER BY score
|
||||
LIMIT ?
|
||||
""",
|
||||
(query, limit),
|
||||
).fetchall()
|
||||
return [(row["atom_id"], float(-row["score"])) for row in rows]
|
||||
|
||||
def atom_count(self) -> int:
|
||||
with self.connect() as connection:
|
||||
return int(connection.execute("SELECT COUNT(*) FROM atoms").fetchone()[0])
|
||||
|
||||
def token_equivalent_sum(self) -> int:
|
||||
with self.connect() as connection:
|
||||
value = connection.execute("SELECT COALESCE(SUM(token_equivalent), 0) FROM atoms").fetchone()[0]
|
||||
return int(value)
|
||||
|
||||
def cartridge_token_equivalent_sum(self) -> int:
|
||||
with self.connect() as connection:
|
||||
value = connection.execute(
|
||||
"SELECT COALESCE(SUM(token_equivalent), 0) FROM cartridges"
|
||||
).fetchone()[0]
|
||||
return int(value)
|
||||
|
||||
def source_count(self) -> int:
|
||||
with self.connect() as connection:
|
||||
return int(connection.execute("SELECT COUNT(*) FROM sources").fetchone()[0])
|
||||
|
||||
def list_sources(self) -> list[SourceRecord]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT source_id, path, kind, mime, sha256, size_bytes, created_at, metadata_json "
|
||||
"FROM sources ORDER BY created_at DESC"
|
||||
).fetchall()
|
||||
return [self._row_to_source(row) for row in rows]
|
||||
|
||||
def cartridge_count(self) -> int:
|
||||
with self.connect() as connection:
|
||||
return int(connection.execute("SELECT COUNT(*) FROM cartridges").fetchone()[0])
|
||||
|
||||
def write_cartridge(
|
||||
self,
|
||||
cartridge_id: str,
|
||||
name: str,
|
||||
root_path: Path,
|
||||
manifest_json: str,
|
||||
dna: str | None,
|
||||
token_equivalent: int,
|
||||
created_at: datetime,
|
||||
) -> None:
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO cartridges (
|
||||
cartridge_id, name, root_path, manifest_json, dna, token_equivalent, created_at
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(cartridge_id) DO UPDATE SET
|
||||
name=excluded.name,
|
||||
root_path=excluded.root_path,
|
||||
manifest_json=excluded.manifest_json,
|
||||
dna=excluded.dna,
|
||||
token_equivalent=excluded.token_equivalent
|
||||
""",
|
||||
(
|
||||
cartridge_id,
|
||||
name,
|
||||
str(root_path),
|
||||
manifest_json,
|
||||
dna,
|
||||
token_equivalent,
|
||||
created_at.isoformat(),
|
||||
),
|
||||
)
|
||||
|
||||
def list_cartridges(self) -> list[dict[str, Any]]:
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT cartridge_id, name, root_path, manifest_json, dna, token_equivalent, created_at "
|
||||
"FROM cartridges ORDER BY created_at DESC"
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def log_cache_event(
|
||||
self,
|
||||
layer: str,
|
||||
cache_key: str,
|
||||
event: str,
|
||||
bytes_estimate: int,
|
||||
metadata: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
with self.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO cache_events(ts, layer, cache_key, event, bytes_estimate, metadata_json)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
datetime.utcnow().isoformat(),
|
||||
layer,
|
||||
cache_key,
|
||||
event,
|
||||
bytes_estimate,
|
||||
_json(metadata or {}),
|
||||
),
|
||||
)
|
||||
|
||||
def write_evidence_edges(self, edges: list[dict[str, Any]]) -> None:
|
||||
if not edges:
|
||||
return
|
||||
with self.connect() as connection:
|
||||
connection.executemany(
|
||||
"""
|
||||
INSERT INTO evidence_edges(from_atom_id, to_atom_id, relation, confidence, metadata_json)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
""",
|
||||
[
|
||||
(
|
||||
str(edge["from_atom_id"]),
|
||||
str(edge["to_atom_id"]),
|
||||
str(edge["relation"]),
|
||||
float(edge["confidence"]),
|
||||
_json(edge.get("metadata") or {}),
|
||||
)
|
||||
for edge in edges
|
||||
],
|
||||
)
|
||||
|
||||
def list_evidence_edges(self, limit: int | None = None) -> list[dict[str, Any]]:
|
||||
query = (
|
||||
"SELECT from_atom_id, to_atom_id, relation, confidence, metadata_json "
|
||||
"FROM evidence_edges ORDER BY id DESC"
|
||||
)
|
||||
params: tuple[Any, ...] = ()
|
||||
if limit is not None:
|
||||
query += " LIMIT ?"
|
||||
params = (limit,)
|
||||
with self.connect() as connection:
|
||||
rows = connection.execute(query, params).fetchall()
|
||||
return [
|
||||
{
|
||||
"from_atom_id": row["from_atom_id"],
|
||||
"to_atom_id": row["to_atom_id"],
|
||||
"relation": row["relation"],
|
||||
"confidence": float(row["confidence"]),
|
||||
"metadata": _loads(row["metadata_json"], {}),
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
def cache_layer_rows(self) -> list[sqlite3.Row]:
|
||||
with self.connect() as connection:
|
||||
return connection.execute(
|
||||
"""
|
||||
SELECT
|
||||
layer,
|
||||
COUNT(DISTINCT cache_key) AS entries,
|
||||
COALESCE(SUM(bytes_estimate), 0) AS bytes_estimate,
|
||||
SUM(CASE WHEN event = 'hit' THEN 1 ELSE 0 END) AS hit_count,
|
||||
SUM(CASE WHEN event = 'miss' THEN 1 ELSE 0 END) AS miss_count,
|
||||
MAX(ts) AS last_event_at
|
||||
FROM cache_events
|
||||
GROUP BY layer
|
||||
"""
|
||||
).fetchall()
|
||||
|
||||
def _row_to_atom(self, row: sqlite3.Row) -> EvidenceAtom:
|
||||
from .schemas import LayoutBox, Modality, TrustScores
|
||||
|
||||
layout_data = _loads(row["layout_json"], None)
|
||||
return EvidenceAtom(
|
||||
atom_id=row["atom_id"],
|
||||
modality=Modality(row["modality"]),
|
||||
source_id=row["source_id"],
|
||||
source=row["source"],
|
||||
time=datetime.fromisoformat(row["time"]),
|
||||
text=row["text"],
|
||||
image_ref=row["image_ref"],
|
||||
embedding_refs=_loads(row["embedding_refs_json"], []),
|
||||
layout=LayoutBox(**layout_data) if layout_data else None,
|
||||
relations=_loads(row["relations_json"], []),
|
||||
trust=TrustScores(**_loads(row["trust_json"], {"parser": 0.0})),
|
||||
token_equivalent=int(row["token_equivalent"]),
|
||||
metadata=_loads(row["metadata_json"], {}),
|
||||
)
|
||||
|
||||
def _row_to_source(self, row: sqlite3.Row) -> SourceRecord:
|
||||
return SourceRecord(
|
||||
source_id=row["source_id"],
|
||||
path=row["path"],
|
||||
kind=row["kind"],
|
||||
mime=row["mime"],
|
||||
sha256=row["sha256"],
|
||||
size_bytes=int(row["size_bytes"]),
|
||||
created_at=datetime.fromisoformat(row["created_at"]),
|
||||
metadata=_loads(row["metadata_json"], {}),
|
||||
)
|
||||
@@ -0,0 +1,55 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VisualToken:
|
||||
token_id: str
|
||||
vector: np.ndarray
|
||||
attention_importance: float
|
||||
evidence_value: float
|
||||
|
||||
|
||||
def select_dvkv_tokens(
|
||||
tokens: list[VisualToken],
|
||||
budget: int,
|
||||
diversity_weight: float = 0.35,
|
||||
evidence_weight: float = 0.30,
|
||||
) -> list[VisualToken]:
|
||||
if budget <= 0:
|
||||
raise ValueError("DVKV budget must be positive")
|
||||
if len(tokens) <= budget:
|
||||
return tokens
|
||||
|
||||
selected: list[VisualToken] = []
|
||||
remaining = tokens.copy()
|
||||
first = max(remaining, key=lambda token: token.attention_importance + evidence_weight * token.evidence_value)
|
||||
selected.append(first)
|
||||
remaining.remove(first)
|
||||
|
||||
while remaining and len(selected) < budget:
|
||||
scores = []
|
||||
selected_vectors = np.stack([normalize(token.vector) for token in selected])
|
||||
for token in remaining:
|
||||
vector = normalize(token.vector)
|
||||
nearest_similarity = float(np.max(selected_vectors @ vector))
|
||||
diversity = 1.0 - nearest_similarity
|
||||
score = (
|
||||
token.attention_importance
|
||||
+ diversity_weight * diversity
|
||||
+ evidence_weight * token.evidence_value
|
||||
)
|
||||
scores.append(score)
|
||||
best_index = int(np.argmax(scores))
|
||||
selected.append(remaining.pop(best_index))
|
||||
return selected
|
||||
|
||||
|
||||
def normalize(vector: np.ndarray) -> np.ndarray:
|
||||
norm = float(np.linalg.norm(vector))
|
||||
if norm == 0.0:
|
||||
return vector
|
||||
return vector / norm
|
||||
@@ -0,0 +1,94 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .schemas import EvidenceAtom
|
||||
|
||||
|
||||
@dataclass
|
||||
class DenseIndex:
|
||||
atom_ids: list[str]
|
||||
vectors: np.ndarray
|
||||
|
||||
def search(self, query_vector: np.ndarray, top_k: int) -> list[tuple[str, float]]:
|
||||
if self.vectors.size == 0:
|
||||
return []
|
||||
query = normalize(query_vector.reshape(1, -1))[0]
|
||||
vectors = normalize(self.vectors)
|
||||
scores = vectors @ query
|
||||
order = np.argsort(scores)[::-1][:top_k]
|
||||
return [(self.atom_ids[index], float(scores[index])) for index in order]
|
||||
|
||||
|
||||
class EmbeddingService:
|
||||
def __init__(self, settings: Settings | None = None, database: Database | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
self.index_path = self.settings.cache_dir / "dense_index.npy"
|
||||
self.map_path = self.settings.cache_dir / "dense_index_atoms.json"
|
||||
self._model: Any | None = None
|
||||
|
||||
def enabled(self) -> bool:
|
||||
return self.settings.dense_embeddings_enabled
|
||||
|
||||
def build_index(self, atoms: list[EvidenceAtom] | None = None) -> DenseIndex:
|
||||
atoms = atoms or self.db.list_atoms()
|
||||
if not atoms:
|
||||
index = DenseIndex(atom_ids=[], vectors=np.zeros((0, 0), dtype=np.float32))
|
||||
self._write_index(index)
|
||||
return index
|
||||
|
||||
model = self._load_model()
|
||||
payloads = [self._atom_payload(atom) for atom in atoms]
|
||||
vectors = model.encode(payloads, normalize_embeddings=True, show_progress_bar=True)
|
||||
index = DenseIndex(atom_ids=[atom.atom_id for atom in atoms], vectors=np.asarray(vectors, dtype=np.float32))
|
||||
self._write_index(index)
|
||||
return index
|
||||
|
||||
def load_index(self) -> DenseIndex | None:
|
||||
if not self.index_path.exists() or not self.map_path.exists():
|
||||
return None
|
||||
atom_ids = json.loads(self.map_path.read_text(encoding="utf-8"))
|
||||
vectors = np.load(self.index_path)
|
||||
return DenseIndex(atom_ids=atom_ids, vectors=vectors)
|
||||
|
||||
def embed_query(self, query: str) -> np.ndarray:
|
||||
model = self._load_model()
|
||||
vector = model.encode([{"text": query}], normalize_embeddings=True)
|
||||
return np.asarray(vector[0], dtype=np.float32)
|
||||
|
||||
def _load_model(self) -> Any:
|
||||
if self._model is None:
|
||||
from sentence_transformers import SentenceTransformer
|
||||
|
||||
self._model = SentenceTransformer(self.settings.embedding_model_id, trust_remote_code=True)
|
||||
return self._model
|
||||
|
||||
def _write_index(self, index: DenseIndex) -> None:
|
||||
self.index_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
np.save(self.index_path, index.vectors)
|
||||
self.map_path.write_text(json.dumps(index.atom_ids, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
def _atom_payload(self, atom: EvidenceAtom) -> dict[str, Any]:
|
||||
if atom.image_ref:
|
||||
from .storage import ObjectStore
|
||||
|
||||
image_path = ObjectStore(self.settings).resolve(atom.image_ref)
|
||||
payload: dict[str, Any] = {"image": str(image_path)}
|
||||
if atom.text:
|
||||
payload["text"] = atom.text
|
||||
return payload
|
||||
return {"text": atom.text or atom.source}
|
||||
|
||||
|
||||
def normalize(vectors: np.ndarray) -> np.ndarray:
|
||||
norms = np.linalg.norm(vectors, axis=1, keepdims=True)
|
||||
norms[norms == 0] = 1.0
|
||||
return vectors / norms
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,68 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .schemas import CitationReport, CompiledContext
|
||||
|
||||
|
||||
class EvidenceGraphService:
|
||||
def __init__(self, settings: Settings | None = None, database: Database | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
|
||||
def record_answer(
|
||||
self,
|
||||
question: str,
|
||||
answer: str,
|
||||
report: CitationReport,
|
||||
context: CompiledContext,
|
||||
) -> str:
|
||||
claim_id = claim_node_id(question, answer)
|
||||
hit_by_id = {hit.atom.atom_id: hit for hit in context.hits}
|
||||
edges: list[dict[str, object]] = []
|
||||
for atom_id in report.cited_atom_ids:
|
||||
hit = hit_by_id.get(atom_id)
|
||||
edges.append(
|
||||
{
|
||||
"from_atom_id": claim_id,
|
||||
"to_atom_id": atom_id,
|
||||
"relation": "cites",
|
||||
"confidence": report.confidence,
|
||||
"metadata": {
|
||||
"question": question,
|
||||
"answer_sha256": hashlib.sha256(answer.encode("utf-8")).hexdigest(),
|
||||
"cache_trace_id": context.cache_trace_id,
|
||||
"final_score": hit.final_score if hit else None,
|
||||
},
|
||||
}
|
||||
)
|
||||
for atom_id in report.missing_atom_ids:
|
||||
edges.append(
|
||||
{
|
||||
"from_atom_id": claim_id,
|
||||
"to_atom_id": atom_id,
|
||||
"relation": "missing_citation",
|
||||
"confidence": 0.0,
|
||||
"metadata": {"question": question, "cache_trace_id": context.cache_trace_id},
|
||||
}
|
||||
)
|
||||
for note in report.contradiction_notes:
|
||||
note_id = "contradiction_" + hashlib.sha256(note.encode("utf-8")).hexdigest()[:16]
|
||||
edges.append(
|
||||
{
|
||||
"from_atom_id": claim_id,
|
||||
"to_atom_id": note_id,
|
||||
"relation": "contradiction_note",
|
||||
"confidence": report.confidence,
|
||||
"metadata": {"note": note, "question": question},
|
||||
}
|
||||
)
|
||||
self.db.write_evidence_edges(edges)
|
||||
return claim_id
|
||||
|
||||
|
||||
def claim_node_id(question: str, answer: str) -> str:
|
||||
digest = hashlib.sha256(f"{question}\n{answer}".encode("utf-8")).hexdigest()[:20]
|
||||
return f"claim_{digest}"
|
||||
@@ -0,0 +1,657 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import tempfile
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import fitz
|
||||
import cv2
|
||||
from PIL import Image
|
||||
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .ocr import read_text_from_image
|
||||
from .schemas import (
|
||||
EvidenceAtom,
|
||||
IngestResponse,
|
||||
LayoutBox,
|
||||
Modality,
|
||||
SourceRecord,
|
||||
TrustScores,
|
||||
)
|
||||
from .storage import ObjectStore, guess_mime, safe_source_id, sha256_bytes, sha256_file, token_equivalent
|
||||
|
||||
|
||||
SUPPORTED_TEXT_EXTENSIONS = {
|
||||
".txt",
|
||||
".md",
|
||||
".json",
|
||||
".jsonl",
|
||||
".yaml",
|
||||
".yml",
|
||||
".toml",
|
||||
".csv",
|
||||
".html",
|
||||
".css",
|
||||
".xml",
|
||||
}
|
||||
|
||||
SUPPORTED_CODE_EXTENSIONS = {
|
||||
".py",
|
||||
".ts",
|
||||
".tsx",
|
||||
".js",
|
||||
".jsx",
|
||||
".rs",
|
||||
".go",
|
||||
".java",
|
||||
".cpp",
|
||||
".c",
|
||||
".h",
|
||||
".hpp",
|
||||
".cs",
|
||||
".swift",
|
||||
".kt",
|
||||
".sql",
|
||||
}
|
||||
|
||||
SUPPORTED_IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".bmp", ".tif", ".tiff"}
|
||||
SUPPORTED_VIDEO_EXTENSIONS = {".mp4", ".mov", ".mkv", ".avi", ".webm", ".m4v"}
|
||||
REPOSITORY_INDEX_LIMIT = 5000
|
||||
REPOSITORY_COMMIT_LIMIT = 50
|
||||
|
||||
|
||||
@dataclass
|
||||
class IngestBatch:
|
||||
sources: list[SourceRecord]
|
||||
atoms: list[EvidenceAtom]
|
||||
|
||||
@property
|
||||
def token_equivalent(self) -> int:
|
||||
return sum(atom.token_equivalent for atom in self.atoms)
|
||||
|
||||
|
||||
class IngestionService:
|
||||
def __init__(
|
||||
self,
|
||||
settings: Settings | None = None,
|
||||
database: Database | None = None,
|
||||
object_store: ObjectStore | None = None,
|
||||
) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
self.objects = object_store or ObjectStore(self.settings)
|
||||
|
||||
def ingest_path(self, path: Path, cartridge_name: str | None = None) -> IngestResponse:
|
||||
path = path.expanduser().resolve()
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(path)
|
||||
files = self._discover_files(path)
|
||||
batch = IngestBatch(sources=[], atoms=[])
|
||||
for file_path in files:
|
||||
file_batch = self._ingest_file(file_path)
|
||||
batch.sources.extend(file_batch.sources)
|
||||
batch.atoms.extend(file_batch.atoms)
|
||||
if path.is_dir():
|
||||
repository_batch = self._ingest_repository_context(path, files)
|
||||
batch.sources.extend(repository_batch.sources)
|
||||
batch.atoms.extend(repository_batch.atoms)
|
||||
|
||||
self.db.upsert_sources(batch.sources)
|
||||
self.db.upsert_atoms(batch.atoms)
|
||||
|
||||
cartridge_id = None
|
||||
if cartridge_name:
|
||||
from .cartridge import CartridgeService
|
||||
|
||||
cartridge = CartridgeService(self.settings, self.db).materialize(
|
||||
name=cartridge_name,
|
||||
atoms=batch.atoms,
|
||||
sources=batch.sources,
|
||||
)
|
||||
cartridge_id = cartridge.cartridge_id
|
||||
|
||||
return IngestResponse(
|
||||
source_count=len(batch.sources),
|
||||
atom_count=len(batch.atoms),
|
||||
cartridge_id=cartridge_id,
|
||||
token_equivalent=batch.token_equivalent,
|
||||
sources=batch.sources,
|
||||
)
|
||||
|
||||
def _discover_files(self, path: Path) -> list[Path]:
|
||||
if path.is_file():
|
||||
return [path]
|
||||
ignored_dirs = {".git", ".ely_eye", "node_modules", ".venv", ".venv-linux", "__pycache__", "dist", "build", "target"}
|
||||
files: list[Path] = []
|
||||
for root, dirs, names in os.walk(path):
|
||||
dirs[:] = [name for name in dirs if name not in ignored_dirs and not name.startswith(".venv")]
|
||||
root_path = Path(root)
|
||||
for name in names:
|
||||
child = root_path / name
|
||||
if any(part in ignored_dirs or part.startswith(".venv") for part in child.parts):
|
||||
continue
|
||||
try:
|
||||
if child.is_dir():
|
||||
continue
|
||||
child.stat()
|
||||
except OSError:
|
||||
continue
|
||||
files.append(child)
|
||||
return files
|
||||
|
||||
def _ingest_file(self, path: Path) -> IngestBatch:
|
||||
suffix = path.suffix.lower()
|
||||
if suffix == ".pdf":
|
||||
return self._ingest_pdf(path)
|
||||
if suffix in SUPPORTED_IMAGE_EXTENSIONS:
|
||||
return self._ingest_image(path)
|
||||
if suffix in SUPPORTED_VIDEO_EXTENSIONS:
|
||||
return self._ingest_video(path)
|
||||
if suffix in SUPPORTED_CODE_EXTENSIONS:
|
||||
return self._ingest_code(path)
|
||||
if suffix in SUPPORTED_TEXT_EXTENSIONS:
|
||||
return self._ingest_text(path, Modality.text)
|
||||
return IngestBatch(sources=[], atoms=[])
|
||||
|
||||
def _source_record(self, path: Path, kind: str, metadata: dict[str, Any] | None = None) -> SourceRecord:
|
||||
digest = sha256_file(path)
|
||||
return SourceRecord(
|
||||
source_id=safe_source_id(path, digest),
|
||||
path=str(path),
|
||||
kind=kind,
|
||||
mime=guess_mime(path),
|
||||
sha256=digest,
|
||||
size_bytes=path.stat().st_size,
|
||||
metadata=metadata or {},
|
||||
)
|
||||
|
||||
def _ingest_text(self, path: Path, modality: Modality) -> IngestBatch:
|
||||
if path.stat().st_size > self.settings.max_text_file_bytes:
|
||||
return IngestBatch(sources=[], atoms=[])
|
||||
source = self._source_record(path, modality.value)
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
atom = EvidenceAtom(
|
||||
atom_id=f"{source.source_id}:text",
|
||||
modality=modality,
|
||||
source_id=source.source_id,
|
||||
source=str(path),
|
||||
text=text,
|
||||
trust=TrustScores(parser=1.0, ocr=0.0, model=0.0),
|
||||
token_equivalent=token_equivalent(text),
|
||||
metadata={"extension": path.suffix.lower()},
|
||||
)
|
||||
return IngestBatch(sources=[source], atoms=[atom])
|
||||
|
||||
def _ingest_code(self, path: Path) -> IngestBatch:
|
||||
batch = self._ingest_text(path, Modality.code)
|
||||
if not batch.atoms:
|
||||
return batch
|
||||
source_text = batch.atoms[0].text
|
||||
batch.atoms[0].metadata["symbols"] = extract_symbols(path, source_text)
|
||||
batch.atoms[0].metadata["line_count"] = source_text.count("\n") + 1
|
||||
return batch
|
||||
|
||||
def _ingest_image(self, path: Path) -> IngestBatch:
|
||||
source = self._source_record(path, self._image_kind(path), metadata=image_metadata(path))
|
||||
image_ref = self.objects.put_file(path, "images")
|
||||
ocr = read_text_from_image(path)
|
||||
width, height = image_size(path)
|
||||
atom = EvidenceAtom(
|
||||
atom_id=f"{source.source_id}:image",
|
||||
modality=Modality.ui_screenshot if source.kind == "ui_screenshot" else Modality.image,
|
||||
source_id=source.source_id,
|
||||
source=str(path),
|
||||
text=ocr.text,
|
||||
image_ref=image_ref,
|
||||
layout=LayoutBox(x=0, y=0, w=width, h=height),
|
||||
trust=TrustScores(parser=1.0, ocr=ocr.confidence, model=0.0),
|
||||
token_equivalent=token_equivalent(ocr.text) + max(1, (width * height) // (224 * 224)),
|
||||
metadata={"ocr_regions": ocr.regions, **image_metadata(path)},
|
||||
)
|
||||
return IngestBatch(sources=[source], atoms=[atom])
|
||||
|
||||
def _ingest_pdf(self, path: Path) -> IngestBatch:
|
||||
source = self._source_record(path, "pdf")
|
||||
atoms: list[EvidenceAtom] = []
|
||||
with fitz.open(path) as document:
|
||||
source.metadata["page_count"] = document.page_count
|
||||
for page_index, page in enumerate(document, start=1):
|
||||
rect = page.rect
|
||||
text = page.get_text("text").strip()
|
||||
pixmap = page.get_pixmap(matrix=fitz.Matrix(1.5, 1.5), alpha=False)
|
||||
image_bytes = pixmap.tobytes("png")
|
||||
image_ref = self.objects.put_bytes(image_bytes, ".png", "pdf_pages")
|
||||
if not text:
|
||||
with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as temp_file:
|
||||
temp_file.write(image_bytes)
|
||||
temp_path = Path(temp_file.name)
|
||||
try:
|
||||
ocr = read_text_from_image(temp_path)
|
||||
text = ocr.text
|
||||
ocr_confidence = ocr.confidence
|
||||
finally:
|
||||
temp_path.unlink(missing_ok=True)
|
||||
else:
|
||||
ocr_confidence = 0.0
|
||||
atoms.append(
|
||||
EvidenceAtom(
|
||||
atom_id=f"{source.source_id}:page:{page_index:05d}",
|
||||
modality=Modality.pdf_page,
|
||||
source_id=source.source_id,
|
||||
source=f"{path}#page={page_index}",
|
||||
text=text,
|
||||
image_ref=image_ref,
|
||||
layout=LayoutBox(x=0, y=0, w=float(rect.width), h=float(rect.height), page=page_index),
|
||||
trust=TrustScores(parser=1.0, ocr=ocr_confidence, model=0.0),
|
||||
token_equivalent=token_equivalent(text) + max(1, int(rect.width * rect.height) // 50_000),
|
||||
metadata={"page": page_index, "width": rect.width, "height": rect.height},
|
||||
)
|
||||
)
|
||||
atoms.extend(self._pdf_region_atoms(path, source.source_id, page_index, page))
|
||||
return IngestBatch(sources=[source], atoms=atoms)
|
||||
|
||||
def _pdf_region_atoms(
|
||||
self, path: Path, source_id: str, page_index: int, page: fitz.Page
|
||||
) -> list[EvidenceAtom]:
|
||||
regions: list[EvidenceAtom] = []
|
||||
blocks = page.get_text("blocks")
|
||||
for block_index, block in enumerate(blocks):
|
||||
if len(block) < 5:
|
||||
continue
|
||||
x0, y0, x1, y1, text = block[:5]
|
||||
text = str(text).strip()
|
||||
if not text:
|
||||
continue
|
||||
regions.append(
|
||||
EvidenceAtom(
|
||||
atom_id=f"{source_id}:page:{page_index:05d}:region:{block_index:04d}",
|
||||
modality=Modality.pdf_region,
|
||||
source_id=source_id,
|
||||
source=f"{path}#page={page_index}®ion={block_index}",
|
||||
text=text,
|
||||
layout=LayoutBox(x=float(x0), y=float(y0), w=float(x1 - x0), h=float(y1 - y0), page=page_index),
|
||||
trust=TrustScores(parser=1.0, ocr=0.0, model=0.0),
|
||||
token_equivalent=token_equivalent(text),
|
||||
metadata={"page": page_index, "block": block_index},
|
||||
)
|
||||
)
|
||||
return regions
|
||||
|
||||
def _ingest_video(self, path: Path) -> IngestBatch:
|
||||
source = self._source_record(path, "video")
|
||||
capture = cv2.VideoCapture(str(path))
|
||||
if not capture.isOpened():
|
||||
raise RuntimeError(f"Cannot open video file: {path}")
|
||||
|
||||
fps = capture.get(cv2.CAP_PROP_FPS)
|
||||
frame_count = int(capture.get(cv2.CAP_PROP_FRAME_COUNT))
|
||||
duration = frame_count / fps if fps > 0 else 0.0
|
||||
source.metadata.update({"fps": fps, "frame_count": frame_count, "duration_seconds": duration})
|
||||
|
||||
atoms: list[EvidenceAtom] = []
|
||||
step = max(1, int(fps * self.settings.video_sample_seconds)) if fps > 0 else 1
|
||||
frame_index = 0
|
||||
while frame_index < frame_count:
|
||||
capture.set(cv2.CAP_PROP_POS_FRAMES, frame_index)
|
||||
ok, frame = capture.read()
|
||||
if not ok:
|
||||
break
|
||||
second = frame_index / fps if fps > 0 else 0.0
|
||||
success, encoded = cv2.imencode(".png", frame)
|
||||
if not success:
|
||||
frame_index += step
|
||||
continue
|
||||
image_bytes = encoded.tobytes()
|
||||
image_ref = self.objects.put_bytes(image_bytes, ".png", "video_frames")
|
||||
with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as temp_file:
|
||||
temp_file.write(image_bytes)
|
||||
temp_path = Path(temp_file.name)
|
||||
try:
|
||||
ocr = read_text_from_image(temp_path)
|
||||
finally:
|
||||
temp_path.unlink(missing_ok=True)
|
||||
height, width = frame.shape[:2]
|
||||
atoms.append(
|
||||
EvidenceAtom(
|
||||
atom_id=f"{source.source_id}:frame:{frame_index:08d}",
|
||||
modality=Modality.video_frame,
|
||||
source_id=source.source_id,
|
||||
source=f"{path}#t={second:.3f}",
|
||||
text=ocr.text,
|
||||
image_ref=image_ref,
|
||||
layout=LayoutBox(x=0, y=0, w=float(width), h=float(height), frame_second=second),
|
||||
trust=TrustScores(parser=1.0, ocr=ocr.confidence, model=0.0),
|
||||
token_equivalent=token_equivalent(ocr.text) + max(1, (width * height) // (224 * 224)),
|
||||
metadata={"frame_index": frame_index, "second": second, "ocr_regions": ocr.regions},
|
||||
)
|
||||
)
|
||||
frame_index += step
|
||||
capture.release()
|
||||
return IngestBatch(sources=[source], atoms=atoms)
|
||||
|
||||
def _image_kind(self, path: Path) -> str:
|
||||
name = path.name.lower()
|
||||
ui_markers = ("screenshot", "screen", "figma", "ui", "wireframe", "mockup")
|
||||
return "ui_screenshot" if any(marker in name for marker in ui_markers) else "image"
|
||||
|
||||
def _ingest_repository_context(self, root: Path, files: list[Path]) -> IngestBatch:
|
||||
code_files = [path for path in files if path.suffix.lower() in SUPPORTED_CODE_EXTENSIONS]
|
||||
git_root = find_git_root(root)
|
||||
if not code_files and git_root is None:
|
||||
return IngestBatch(sources=[], atoms=[])
|
||||
|
||||
context = build_repository_context(root, files, code_files, git_root)
|
||||
payload = json.dumps(context, ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8")
|
||||
digest = sha256_bytes(payload)
|
||||
source = SourceRecord(
|
||||
source_id=safe_source_id(root, digest),
|
||||
path=str(root),
|
||||
kind="code_repository",
|
||||
mime="application/vnd.ely-eye.repository+json",
|
||||
sha256=digest,
|
||||
size_bytes=len(payload),
|
||||
metadata={
|
||||
"file_count": context["structure"]["file_count"],
|
||||
"indexed_file_count": context["structure"]["indexed_file_count"],
|
||||
"code_file_count": context["code"]["file_count"],
|
||||
"git_available": context["git"]["available"],
|
||||
},
|
||||
)
|
||||
atom = EvidenceAtom(
|
||||
atom_id=f"{source.source_id}:text",
|
||||
modality=Modality.code,
|
||||
source_id=source.source_id,
|
||||
source=str(root),
|
||||
text=json.dumps(context, ensure_ascii=False, indent=2),
|
||||
trust=TrustScores(parser=1.0, ocr=0.0, model=0.0),
|
||||
token_equivalent=token_equivalent(json.dumps(context, ensure_ascii=False)),
|
||||
metadata=context,
|
||||
)
|
||||
return IngestBatch(sources=[source], atoms=[atom])
|
||||
|
||||
|
||||
def image_size(path: Path) -> tuple[int, int]:
|
||||
with Image.open(path) as image:
|
||||
return image.size
|
||||
|
||||
|
||||
def image_metadata(path: Path) -> dict[str, Any]:
|
||||
with Image.open(path) as image:
|
||||
return {"width": image.width, "height": image.height, "mode": image.mode, "format": image.format}
|
||||
|
||||
|
||||
def build_repository_context(
|
||||
root: Path,
|
||||
files: list[Path],
|
||||
code_files: list[Path],
|
||||
git_root: Path | None,
|
||||
) -> dict[str, Any]:
|
||||
structure_entries = [file_structure_entry(root, path) for path in sorted(files)[:REPOSITORY_INDEX_LIMIT]]
|
||||
code_entries = [code_context_entry(root, path) for path in sorted(code_files)[:REPOSITORY_INDEX_LIMIT]]
|
||||
git_context = collect_git_context(git_root) if git_root is not None else {"available": False}
|
||||
return {
|
||||
"format": "ely-eye-repository-context-v1",
|
||||
"root": str(root),
|
||||
"structure": {
|
||||
"file_count": len(files),
|
||||
"indexed_file_count": len(structure_entries),
|
||||
"limit": REPOSITORY_INDEX_LIMIT,
|
||||
"files": structure_entries,
|
||||
},
|
||||
"code": {
|
||||
"file_count": len(code_files),
|
||||
"indexed_file_count": len(code_entries),
|
||||
"files": code_entries,
|
||||
"call_graph": repository_call_graph(code_entries),
|
||||
},
|
||||
"git": git_context,
|
||||
"issue_pr_history": extract_issue_pr_history(git_context),
|
||||
}
|
||||
|
||||
|
||||
def file_structure_entry(root: Path, path: Path) -> dict[str, Any]:
|
||||
return {
|
||||
"path": relative_path(root, path),
|
||||
"suffix": path.suffix.lower(),
|
||||
"kind": file_kind(path),
|
||||
"size_bytes": path.stat().st_size,
|
||||
}
|
||||
|
||||
|
||||
def code_context_entry(root: Path, path: Path) -> dict[str, Any]:
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
symbols = extract_symbols(path, text)
|
||||
return {
|
||||
"path": relative_path(root, path),
|
||||
"language": path.suffix.lower().lstrip("."),
|
||||
"line_count": text.count("\n") + 1,
|
||||
"symbols": symbols,
|
||||
}
|
||||
|
||||
|
||||
def repository_call_graph(code_entries: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
edges: list[dict[str, Any]] = []
|
||||
for entry in code_entries:
|
||||
symbols = entry.get("symbols") if isinstance(entry.get("symbols"), dict) else {}
|
||||
for call in symbols.get("calls", []):
|
||||
if not isinstance(call, dict):
|
||||
continue
|
||||
caller = call.get("caller")
|
||||
callee = call.get("callee")
|
||||
if caller and callee:
|
||||
edges.append({"path": entry["path"], "caller": caller, "callee": callee})
|
||||
return edges
|
||||
|
||||
|
||||
def collect_git_context(root: Path) -> dict[str, Any]:
|
||||
commits = [
|
||||
parse_git_log_line(line)
|
||||
for line in run_git(root, ["log", f"--max-count={REPOSITORY_COMMIT_LIMIT}", "--pretty=format:%H%x09%an%x09%aI%x09%s"]).splitlines()
|
||||
if line.strip()
|
||||
]
|
||||
commits = [commit for commit in commits if commit]
|
||||
return {
|
||||
"available": True,
|
||||
"root": str(root),
|
||||
"head": run_git(root, ["rev-parse", "HEAD"]).strip() or None,
|
||||
"branch": run_git(root, ["branch", "--show-current"]).strip() or None,
|
||||
"remotes": parse_git_remotes(run_git(root, ["remote", "-v"])),
|
||||
"recent_commits": commits,
|
||||
}
|
||||
|
||||
|
||||
def parse_git_log_line(line: str) -> dict[str, str] | None:
|
||||
parts = line.split("\t", 3)
|
||||
if len(parts) != 4:
|
||||
return None
|
||||
commit_hash, author, authored_at, subject = parts
|
||||
return {"hash": commit_hash, "author": author, "authored_at": authored_at, "subject": subject}
|
||||
|
||||
|
||||
def parse_git_remotes(output: str) -> list[dict[str, str]]:
|
||||
remotes: list[dict[str, str]] = []
|
||||
for line in output.splitlines():
|
||||
parts = line.split()
|
||||
if len(parts) >= 3:
|
||||
remotes.append({"name": parts[0], "url": parts[1], "kind": parts[2].strip("()")})
|
||||
return remotes
|
||||
|
||||
|
||||
def extract_issue_pr_history(git_context: dict[str, Any]) -> dict[str, Any]:
|
||||
references: dict[str, set[int]] = {"issues": set(), "pull_requests": set()}
|
||||
for commit in git_context.get("recent_commits", []):
|
||||
if not isinstance(commit, dict):
|
||||
continue
|
||||
subject = str(commit.get("subject") or "")
|
||||
for number in re.findall(r"(?:fixes|closes|resolves)\s+#(\d+)", subject, flags=re.IGNORECASE):
|
||||
references["issues"].add(int(number))
|
||||
for number in re.findall(r"(?:pull request|pr)\s*#?(\d+)", subject, flags=re.IGNORECASE):
|
||||
references["pull_requests"].add(int(number))
|
||||
for number in re.findall(r"\(#(\d+)\)", subject):
|
||||
references["pull_requests"].add(int(number))
|
||||
return {
|
||||
"source": "local_git_commit_history",
|
||||
"issues": sorted(references["issues"]),
|
||||
"pull_requests": sorted(references["pull_requests"]),
|
||||
}
|
||||
|
||||
|
||||
def run_git(root: Path, args: list[str]) -> str:
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
["git", *args],
|
||||
cwd=root,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return ""
|
||||
return completed.stdout if completed.returncode == 0 else ""
|
||||
|
||||
|
||||
def find_git_root(path: Path) -> Path | None:
|
||||
current = path if path.is_dir() else path.parent
|
||||
while True:
|
||||
if (current / ".git").exists():
|
||||
return current
|
||||
if current.parent == current:
|
||||
return None
|
||||
current = current.parent
|
||||
|
||||
|
||||
def file_kind(path: Path) -> str:
|
||||
suffix = path.suffix.lower()
|
||||
if suffix in SUPPORTED_CODE_EXTENSIONS:
|
||||
return "code"
|
||||
if suffix in SUPPORTED_TEXT_EXTENSIONS:
|
||||
return "text"
|
||||
if suffix in SUPPORTED_IMAGE_EXTENSIONS:
|
||||
return "image"
|
||||
if suffix in SUPPORTED_VIDEO_EXTENSIONS:
|
||||
return "video"
|
||||
if suffix == ".pdf":
|
||||
return "pdf"
|
||||
return "binary"
|
||||
|
||||
|
||||
def relative_path(root: Path, path: Path) -> str:
|
||||
try:
|
||||
return path.relative_to(root).as_posix()
|
||||
except ValueError:
|
||||
return path.as_posix()
|
||||
|
||||
|
||||
def extract_symbols(path: Path, text: str) -> dict[str, Any]:
|
||||
suffix = path.suffix.lower()
|
||||
if suffix == ".py":
|
||||
return extract_python_symbols(text)
|
||||
symbols = re.findall(
|
||||
r"\b(?:class|function|interface|type|enum|struct|fn|func)\s+([A-Za-z_][A-Za-z0-9_]*)",
|
||||
text,
|
||||
)
|
||||
constants = re.findall(r"\b(?:const|let|var)\s+([A-Za-z_][A-Za-z0-9_]*)\s*=", text)
|
||||
calls = extract_generic_calls(text)
|
||||
return {"symbols": sorted(set(symbols)), "bindings": sorted(set(constants)), "calls": calls}
|
||||
|
||||
|
||||
def extract_python_symbols(text: str) -> dict[str, Any]:
|
||||
try:
|
||||
tree = ast.parse(text)
|
||||
except SyntaxError as exc:
|
||||
return {
|
||||
"classes": [],
|
||||
"functions": [],
|
||||
"imports": [],
|
||||
"calls": extract_generic_calls(text),
|
||||
"parse_error": f"{exc.msg} at line {exc.lineno}",
|
||||
}
|
||||
classes: list[str] = []
|
||||
functions: list[str] = []
|
||||
imports: list[str] = []
|
||||
calls: list[dict[str, str]] = []
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.ClassDef):
|
||||
classes.append(node.name)
|
||||
elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
|
||||
functions.append(node.name)
|
||||
elif isinstance(node, ast.Import):
|
||||
imports.extend(alias.name for alias in node.names)
|
||||
elif isinstance(node, ast.ImportFrom) and node.module:
|
||||
imports.append(node.module)
|
||||
call_visitor = PythonCallGraphVisitor()
|
||||
call_visitor.visit(tree)
|
||||
calls.extend(call_visitor.calls)
|
||||
return {
|
||||
"classes": sorted(set(classes)),
|
||||
"functions": sorted(set(functions)),
|
||||
"imports": sorted(set(imports)),
|
||||
"calls": calls,
|
||||
}
|
||||
|
||||
|
||||
class PythonCallGraphVisitor(ast.NodeVisitor):
|
||||
def __init__(self) -> None:
|
||||
self.current: str | None = None
|
||||
self.calls: list[dict[str, str]] = []
|
||||
|
||||
def visit_FunctionDef(self, node: ast.FunctionDef) -> Any:
|
||||
previous = self.current
|
||||
self.current = node.name
|
||||
self.generic_visit(node)
|
||||
self.current = previous
|
||||
|
||||
def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> Any:
|
||||
previous = self.current
|
||||
self.current = node.name
|
||||
self.generic_visit(node)
|
||||
self.current = previous
|
||||
|
||||
def visit_Call(self, node: ast.Call) -> Any:
|
||||
if self.current:
|
||||
callee = python_call_name(node.func)
|
||||
if callee:
|
||||
self.calls.append({"caller": self.current, "callee": callee})
|
||||
self.generic_visit(node)
|
||||
|
||||
|
||||
def python_call_name(node: ast.AST) -> str | None:
|
||||
if isinstance(node, ast.Name):
|
||||
return node.id
|
||||
if isinstance(node, ast.Attribute):
|
||||
parent = python_call_name(node.value)
|
||||
return f"{parent}.{node.attr}" if parent else node.attr
|
||||
return None
|
||||
|
||||
|
||||
def extract_generic_calls(text: str) -> list[dict[str, str]]:
|
||||
calls: list[dict[str, str]] = []
|
||||
current = "<module>"
|
||||
function_pattern = re.compile(r"\b(?:function|fn|func)\s+([A-Za-z_][A-Za-z0-9_]*)|([A-Za-z_][A-Za-z0-9_]*)\s*=\s*\([^)]*\)\s*=>")
|
||||
call_pattern = re.compile(r"\b([A-Za-z_][A-Za-z0-9_.]*)\s*\(")
|
||||
keywords = {"if", "for", "while", "switch", "catch", "return", "function"}
|
||||
for line in text.splitlines():
|
||||
function_match = function_pattern.search(line)
|
||||
if function_match:
|
||||
current = next(group for group in function_match.groups() if group)
|
||||
for callee in call_pattern.findall(line):
|
||||
root = callee.split(".", 1)[0]
|
||||
if root not in keywords and callee != current:
|
||||
calls.append({"caller": current, "callee": callee})
|
||||
return calls
|
||||
|
||||
|
||||
def write_sources_json(path: Path, sources: list[SourceRecord]) -> None:
|
||||
path.write_text(
|
||||
json.dumps([source.model_dump(mode="json") for source in sources], ensure_ascii=False, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
@@ -0,0 +1,41 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
class OcrResult:
|
||||
def __init__(self, text: str, confidence: float, regions: list[dict[str, Any]]) -> None:
|
||||
self.text = text
|
||||
self.confidence = confidence
|
||||
self.regions = regions
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def _rapid_ocr() -> Any:
|
||||
from rapidocr_onnxruntime import RapidOCR
|
||||
|
||||
return RapidOCR()
|
||||
|
||||
|
||||
def read_text_from_image(path: Path) -> OcrResult:
|
||||
engine = _rapid_ocr()
|
||||
result = engine(str(path))
|
||||
raw = result[0] if isinstance(result, tuple) else result
|
||||
if raw is None:
|
||||
return OcrResult(text="", confidence=0.0, regions=[])
|
||||
|
||||
texts: list[str] = []
|
||||
confidences: list[float] = []
|
||||
regions: list[dict[str, Any]] = []
|
||||
for item in raw:
|
||||
if len(item) < 3:
|
||||
continue
|
||||
box, text, confidence = item[0], str(item[1]), float(item[2])
|
||||
texts.append(text)
|
||||
confidences.append(confidence)
|
||||
regions.append({"box": box, "text": text, "confidence": confidence})
|
||||
|
||||
average_confidence = sum(confidences) / len(confidences) if confidences else 0.0
|
||||
return OcrResult(text="\n".join(texts), confidence=average_confidence, regions=regions)
|
||||
@@ -0,0 +1,420 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from .schemas import RuntimeLaunchProfile, RuntimeProfileReport, RuntimeProfileValidation
|
||||
|
||||
|
||||
EXTREME_YARN_OVERRIDE = {
|
||||
"text_config": {
|
||||
"rope_parameters": {
|
||||
"mrope_interleaved": True,
|
||||
"mrope_section": [11, 11, 10],
|
||||
"rope_type": "yarn",
|
||||
"rope_theta": 10_000_000,
|
||||
"partial_rotary_factor": 0.25,
|
||||
"factor": 4.0,
|
||||
"original_max_position_embeddings": 262_144,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def runtime_profile_report(project_root: Path | None = None) -> RuntimeProfileReport:
|
||||
root = project_root or Path.cwd()
|
||||
profiles = [
|
||||
RuntimeLaunchProfile(
|
||||
name="sglang-live",
|
||||
runtime="SGLang",
|
||||
role="Main VLM service with MTP speculative decoding and Qwen tool/parser support.",
|
||||
context_tokens=262_144,
|
||||
script="scripts/launch-sglang-live.ps1",
|
||||
command=[
|
||||
"python",
|
||||
"-m",
|
||||
"sglang.launch_server",
|
||||
"--model-path",
|
||||
"Qwen/Qwen3.5-9B",
|
||||
"--port",
|
||||
"8000",
|
||||
"--tp-size",
|
||||
"1",
|
||||
"--mem-fraction-static",
|
||||
"0.82",
|
||||
"--context-length",
|
||||
"262144",
|
||||
"--reasoning-parser",
|
||||
"qwen3",
|
||||
"--speculative-algo",
|
||||
"NEXTN",
|
||||
"--speculative-num-steps",
|
||||
"3",
|
||||
"--speculative-num-draft-tokens",
|
||||
"4",
|
||||
],
|
||||
required_features=["VLM", "MTP", "long-context", "tool-parser"],
|
||||
),
|
||||
RuntimeLaunchProfile(
|
||||
name="sglang-extreme-yarn",
|
||||
runtime="SGLang",
|
||||
role="1.01M YaRN extreme-context service.",
|
||||
context_tokens=1_010_000,
|
||||
script="scripts/launch-sglang-extreme.ps1",
|
||||
command=[
|
||||
"python",
|
||||
"-m",
|
||||
"sglang.launch_server",
|
||||
"--model-path",
|
||||
"Qwen/Qwen3.5-9B",
|
||||
"--port",
|
||||
"8000",
|
||||
"--tp-size",
|
||||
"1",
|
||||
"--mem-fraction-static",
|
||||
"0.88",
|
||||
"--context-length",
|
||||
"1010000",
|
||||
"--json-model-override-args",
|
||||
"EXTREME_YARN_OVERRIDE",
|
||||
],
|
||||
environment={"SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN": "1"},
|
||||
required_features=["YaRN", "long-context", "CPU-offload-compatible"],
|
||||
),
|
||||
RuntimeLaunchProfile(
|
||||
name="vllm-live",
|
||||
runtime="vLLM",
|
||||
role="PagedAttention and Automatic Prefix Caching service for live document QA.",
|
||||
context_tokens=262_144,
|
||||
script="scripts/launch-vllm-live.ps1",
|
||||
command=[
|
||||
"python",
|
||||
"-m",
|
||||
"vllm.entrypoints.openai.api_server",
|
||||
"--model",
|
||||
"Qwen/Qwen3.5-9B",
|
||||
"--host",
|
||||
"127.0.0.1",
|
||||
"--port",
|
||||
"8001",
|
||||
"--max-model-len",
|
||||
"262144",
|
||||
"--enable-prefix-caching",
|
||||
"--gpu-memory-utilization",
|
||||
"0.82",
|
||||
"--trust-remote-code",
|
||||
],
|
||||
required_features=["PagedAttention", "Automatic Prefix Caching", "OpenAI API"],
|
||||
),
|
||||
RuntimeLaunchProfile(
|
||||
name="vllm-extreme-yarn",
|
||||
runtime="vLLM",
|
||||
role="1.01M YaRN profile with prefix cache and quantized KV-compatible launch args.",
|
||||
context_tokens=1_010_000,
|
||||
script="scripts/launch-vllm-extreme.ps1",
|
||||
command=[
|
||||
"python",
|
||||
"-m",
|
||||
"vllm.entrypoints.openai.api_server",
|
||||
"--model",
|
||||
"Qwen/Qwen3.5-9B",
|
||||
"--host",
|
||||
"127.0.0.1",
|
||||
"--port",
|
||||
"8001",
|
||||
"--max-model-len",
|
||||
"1010000",
|
||||
"--enable-prefix-caching",
|
||||
"--gpu-memory-utilization",
|
||||
"0.88",
|
||||
"--trust-remote-code",
|
||||
"--rope-scaling",
|
||||
"EXTREME_YARN_OVERRIDE",
|
||||
],
|
||||
required_features=["YaRN", "PagedAttention", "Automatic Prefix Caching"],
|
||||
),
|
||||
RuntimeLaunchProfile(
|
||||
name="ktransformers-extreme",
|
||||
runtime="KTransformers",
|
||||
role="SGLang-KT CPU-GPU heterogeneous ultra-long-context experiment service.",
|
||||
context_tokens=1_010_000,
|
||||
script="scripts/launch-ktransformers-extreme.ps1",
|
||||
command=[
|
||||
"python",
|
||||
"-m",
|
||||
"sglang.launch_server",
|
||||
"--model-path",
|
||||
"Qwen/Qwen3.5-9B",
|
||||
"--port",
|
||||
"8002",
|
||||
"--context-length",
|
||||
"1010000",
|
||||
"--kt-cpuinfer",
|
||||
"16",
|
||||
"--cpu-offload-gb",
|
||||
"48",
|
||||
],
|
||||
required_features=["CPU-GPU heterogeneous execution", "ultra-long-context experiment"],
|
||||
),
|
||||
RuntimeLaunchProfile(
|
||||
name="llamacpp-offline",
|
||||
runtime="llama.cpp",
|
||||
role="GGUF offline and portable local demo service.",
|
||||
context_tokens=262_144,
|
||||
script="scripts/launch-llamacpp-offline.ps1",
|
||||
command=[
|
||||
"llama-server",
|
||||
"--model",
|
||||
"$env:ELY_EYE_GGUF_MODEL",
|
||||
"--host",
|
||||
"127.0.0.1",
|
||||
"--port",
|
||||
"8003",
|
||||
"--ctx-size",
|
||||
"262144",
|
||||
"--flash-attn",
|
||||
"--cache-type-k",
|
||||
"q4_0",
|
||||
"--cache-type-v",
|
||||
"q4_0",
|
||||
],
|
||||
environment={"ELY_EYE_GGUF_MODEL": "required absolute GGUF path"},
|
||||
required_features=["GGUF", "offline demo", "quantized KV"],
|
||||
),
|
||||
]
|
||||
validations = validate_runtime_profiles(root, profiles)
|
||||
ready = all(validation.status == "ready" for validation in validations)
|
||||
return RuntimeProfileReport(
|
||||
status="ready" if ready else "incomplete",
|
||||
profiles=profiles,
|
||||
validations=validations,
|
||||
)
|
||||
|
||||
|
||||
def validate_runtime_profiles(
|
||||
root: Path,
|
||||
profiles: list[RuntimeLaunchProfile],
|
||||
) -> list[RuntimeProfileValidation]:
|
||||
wsl_gpu = wsl_gpu_available()
|
||||
package_cache = {
|
||||
".venv-linux": linux_package_versions(root, ".venv-linux"),
|
||||
".venv-linux-vllm": linux_package_versions(root, ".venv-linux-vllm"),
|
||||
}
|
||||
probe_cache: dict[tuple[str, str, str | None], dict[str, object]] = {}
|
||||
llama_server = resolve_llama_server()
|
||||
validations: list[RuntimeProfileValidation] = []
|
||||
for profile in profiles:
|
||||
script_exists = (root / profile.script).exists()
|
||||
package = package_for_runtime(profile.runtime)
|
||||
env_name = linux_env_for_runtime(profile.runtime)
|
||||
linux_packages = package_cache.get(env_name, {})
|
||||
package_version = linux_packages.get(package) if package else None
|
||||
kt_sglang_version = linux_packages.get("sglang") if profile.runtime == "KTransformers" else None
|
||||
executable = llama_server if profile.runtime == "llama.cpp" else None
|
||||
probe_key = (profile.runtime, env_name, executable)
|
||||
if probe_key not in probe_cache:
|
||||
probe_cache[probe_key] = runtime_launcher_probe(root, profile.runtime, env_name, executable)
|
||||
probe = probe_cache[probe_key]
|
||||
probe_status = str(probe.get("status", "skipped"))
|
||||
probe_passed = probe_status == "passed"
|
||||
if profile.runtime in {"SGLang", "vLLM", "KTransformers"}:
|
||||
ready = script_exists and bool(package_version) and wsl_gpu and probe_passed
|
||||
if profile.runtime == "KTransformers":
|
||||
ready = ready and bool(kt_sglang_version)
|
||||
detail = (
|
||||
(
|
||||
f"WSL GPU ready; {package} {package_version}; "
|
||||
f"sglang-kt entry {kt_sglang_version}"
|
||||
)
|
||||
if profile.runtime == "KTransformers" and ready
|
||||
else f"WSL GPU ready; {package} {package_version}; env {env_name}"
|
||||
if ready
|
||||
else (
|
||||
f"script={script_exists}, wsl_gpu={wsl_gpu}, env={env_name}, "
|
||||
f"{package}={package_version}, sglang={kt_sglang_version}"
|
||||
)
|
||||
)
|
||||
elif profile.runtime == "llama.cpp":
|
||||
ready = script_exists and bool(executable) and probe_passed
|
||||
detail = f"llama-server at {executable}" if ready else "llama-server executable missing"
|
||||
else:
|
||||
ready = script_exists
|
||||
detail = f"script={script_exists}"
|
||||
detail = f"{detail}; launcher_probe={probe_status}"
|
||||
validations.append(
|
||||
RuntimeProfileValidation(
|
||||
name=profile.name,
|
||||
runtime=profile.runtime,
|
||||
status="ready" if ready else "missing",
|
||||
script_exists=script_exists,
|
||||
package=package,
|
||||
package_version=package_version,
|
||||
executable=executable,
|
||||
wsl_gpu=wsl_gpu,
|
||||
probe_command=[str(item) for item in probe.get("command", [])],
|
||||
probe_status=probe_status,
|
||||
probe_exit_code=probe.get("exit_code") if isinstance(probe.get("exit_code"), int) else None,
|
||||
probe_output=probe.get("output") if isinstance(probe.get("output"), str) else None,
|
||||
detail=detail,
|
||||
)
|
||||
)
|
||||
return validations
|
||||
|
||||
|
||||
def package_for_runtime(runtime: str) -> str | None:
|
||||
return {
|
||||
"SGLang": "sglang",
|
||||
"vLLM": "vllm",
|
||||
"KTransformers": "ktransformers",
|
||||
}.get(runtime)
|
||||
|
||||
|
||||
def linux_env_for_runtime(runtime: str) -> str:
|
||||
if runtime == "vLLM":
|
||||
return ".venv-linux-vllm"
|
||||
return ".venv-linux"
|
||||
|
||||
|
||||
def linux_package_versions(root: Path, env_name: str) -> dict[str, str]:
|
||||
script = (
|
||||
"import importlib.metadata as md, json;"
|
||||
"names=['sglang','vllm','ktransformers','torch','torchvision','qwen-vl-utils'];"
|
||||
"out={};"
|
||||
"\nfor name in names:\n"
|
||||
" try: out[name]=md.version(name)\n"
|
||||
" except Exception: pass\n"
|
||||
"print(json.dumps(out))"
|
||||
)
|
||||
try:
|
||||
output = subprocess.check_output(
|
||||
[
|
||||
"wsl.exe",
|
||||
"-d",
|
||||
"Ubuntu",
|
||||
"bash",
|
||||
"-lc",
|
||||
(
|
||||
f"cd {wsl_quote(windows_path_to_wsl(root))} "
|
||||
f"&& test -x {env_name}/bin/python "
|
||||
f"&& {env_name}/bin/python -c {wsl_quote(script)}"
|
||||
),
|
||||
],
|
||||
text=True,
|
||||
stderr=subprocess.DEVNULL,
|
||||
timeout=20,
|
||||
)
|
||||
data = json.loads(output.strip())
|
||||
return {str(key): str(value) for key, value in data.items()}
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def runtime_launcher_probe(
|
||||
root: Path,
|
||||
runtime: str,
|
||||
env_name: str,
|
||||
executable: str | None,
|
||||
) -> dict[str, object]:
|
||||
if runtime == "SGLang":
|
||||
probe_script = (
|
||||
"import importlib.util, sys; "
|
||||
"spec=importlib.util.find_spec('sglang.launch_server'); "
|
||||
"print(spec.origin if spec else 'missing'); "
|
||||
"sys.exit(0 if spec else 1)"
|
||||
)
|
||||
command = wsl_python_command(root, env_name, f"-c {wsl_quote(probe_script)}")
|
||||
elif runtime == "vLLM":
|
||||
probe_script = (
|
||||
"import importlib.util, sys; "
|
||||
"spec=importlib.util.find_spec('vllm.entrypoints.openai.api_server'); "
|
||||
"print(spec.origin if spec else 'missing'); "
|
||||
"sys.exit(0 if spec else 1)"
|
||||
)
|
||||
command = wsl_python_command(root, env_name, f"-c {wsl_quote(probe_script)}")
|
||||
elif runtime == "KTransformers":
|
||||
probe_script = "import ktransformers, sglang; print('ktransformers and sglang import ok')"
|
||||
command = wsl_python_command(root, env_name, f"-c {wsl_quote(probe_script)}")
|
||||
elif runtime == "llama.cpp" and executable:
|
||||
command = [executable, "--help"]
|
||||
else:
|
||||
return {"status": "skipped", "command": [], "output": "no launcher probe configured"}
|
||||
try:
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
timeout=30,
|
||||
check=False,
|
||||
)
|
||||
except Exception as exc:
|
||||
return {
|
||||
"status": "failed",
|
||||
"command": command,
|
||||
"exit_code": None,
|
||||
"output": str(exc).splitlines()[0],
|
||||
}
|
||||
output = compact_output(completed.stdout, completed.stderr)
|
||||
return {
|
||||
"status": "passed" if completed.returncode == 0 else "failed",
|
||||
"command": command,
|
||||
"exit_code": completed.returncode,
|
||||
"output": output,
|
||||
}
|
||||
|
||||
|
||||
def wsl_python_command(root: Path, env_name: str, python_args: str) -> list[str]:
|
||||
shell_command = (
|
||||
f"cd {wsl_quote(windows_path_to_wsl(root))} "
|
||||
f"&& test -x {env_name}/bin/python "
|
||||
f"&& {env_name}/bin/python {python_args}"
|
||||
)
|
||||
return ["wsl.exe", "-d", "Ubuntu", "bash", "-lc", shell_command]
|
||||
|
||||
|
||||
def compact_output(stdout: str, stderr: str, limit: int = 800) -> str:
|
||||
text = "\n".join(part.strip() for part in (stdout, stderr) if part.strip())
|
||||
return text[:limit]
|
||||
|
||||
|
||||
def wsl_gpu_available() -> bool:
|
||||
if shutil.which("wsl.exe") is None:
|
||||
return False
|
||||
try:
|
||||
output = subprocess.check_output(
|
||||
["wsl.exe", "-d", "Ubuntu", "nvidia-smi", "-L"],
|
||||
text=True,
|
||||
stderr=subprocess.DEVNULL,
|
||||
timeout=10,
|
||||
)
|
||||
return "GPU" in output
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def resolve_llama_server() -> str | None:
|
||||
configured = os.environ.get("LLAMA_CPP_SERVER")
|
||||
if configured and Path(configured).exists():
|
||||
return configured
|
||||
direct = shutil.which("llama-server")
|
||||
if direct:
|
||||
return direct
|
||||
local_app_data = os.environ.get("LOCALAPPDATA")
|
||||
if not local_app_data:
|
||||
return None
|
||||
package_root = Path(local_app_data) / "Microsoft" / "WinGet" / "Packages"
|
||||
matches = sorted(package_root.glob("ggml.llamacpp_*/*llama-server.exe"))
|
||||
return str(matches[-1]) if matches else None
|
||||
|
||||
|
||||
def windows_path_to_wsl(path: Path) -> str:
|
||||
resolved = path.resolve()
|
||||
drive = resolved.drive.rstrip(":").lower()
|
||||
parts = resolved.parts[1:]
|
||||
return "/mnt/" + drive + "/" + "/".join(parts)
|
||||
|
||||
|
||||
def wsl_quote(value: str) -> str:
|
||||
return "'" + value.replace("'", "'\"'\"'") + "'"
|
||||
@@ -0,0 +1,221 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
|
||||
from .cache_fabric import CacheFabric
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .dvkv import VisualToken, select_dvkv_tokens
|
||||
from .schemas import RetrievalHit
|
||||
|
||||
|
||||
class ResearchModuleService:
|
||||
def __init__(self, settings: Settings | None = None, database: Database | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
|
||||
def build_report(self, query: str, hits: list[RetrievalHit], output_path: Path | None = None) -> dict[str, Any]:
|
||||
report = {
|
||||
"v_nsa": self.v_nsa_plan(hits),
|
||||
"dvkv": self.dvkv_plan(hits),
|
||||
"x_modal_kv": self.x_modal_kv_plan(hits),
|
||||
"speculative_prefetch": self.speculative_prefetch_plan(query, hits),
|
||||
"predictive_eviction": self.predictive_eviction_plan(),
|
||||
"differentiable_retrieval": self.differentiable_retrieval_distribution(hits),
|
||||
"evidence_graph": self.evidence_graph_status(),
|
||||
"sleep_consolidation": self.sleep_consolidation_plan(hits),
|
||||
}
|
||||
if output_path is not None:
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
return report
|
||||
|
||||
def v_nsa_plan(self, hits: list[RetrievalHit]) -> dict[str, Any]:
|
||||
heads = {
|
||||
"visual_anchor_heads": [],
|
||||
"text_retrieval_heads": [],
|
||||
"streaming_heads": [],
|
||||
}
|
||||
for index, hit in enumerate(hits):
|
||||
target = {
|
||||
"atom_id": hit.atom.atom_id,
|
||||
"modality": hit.atom.modality.value,
|
||||
"score": hit.final_score,
|
||||
"layout": hit.atom.layout.model_dump() if hit.atom.layout else None,
|
||||
}
|
||||
if hit.atom.image_ref or hit.atom.modality.value in {"image", "pdf_page", "pdf_region", "video_frame", "ui_screenshot"}:
|
||||
heads["visual_anchor_heads"].append(target)
|
||||
elif hit.final_score >= 0.45:
|
||||
heads["text_retrieval_heads"].append(target)
|
||||
else:
|
||||
heads["streaming_heads"].append(target)
|
||||
sparse_blocks = sorted(
|
||||
heads["visual_anchor_heads"] + heads["text_retrieval_heads"],
|
||||
key=lambda item: float(item["score"]),
|
||||
reverse=True,
|
||||
)
|
||||
return {
|
||||
"head_roles": heads,
|
||||
"sparse_block_count": len(sparse_blocks),
|
||||
"sparse_blocks": sparse_blocks[:32],
|
||||
}
|
||||
|
||||
def dvkv_plan(self, hits: list[RetrievalHit], budget: int = 16) -> dict[str, Any]:
|
||||
tokens = [
|
||||
VisualToken(
|
||||
token_id=hit.atom.atom_id,
|
||||
vector=stable_vector(hit.atom.atom_id + hit.atom.text, 64),
|
||||
attention_importance=max(0.0, hit.final_score),
|
||||
evidence_value=hit.atom.trust.parser * 0.45 + hit.atom.trust.ocr * 0.35 + hit.atom.trust.model * 0.20,
|
||||
)
|
||||
for hit in hits
|
||||
if hit.atom.image_ref or hit.atom.layout is not None
|
||||
]
|
||||
if not tokens:
|
||||
visual_atoms = [atom for atom in self.db.list_atoms() if atom.image_ref]
|
||||
tokens = [
|
||||
VisualToken(
|
||||
token_id=atom.atom_id,
|
||||
vector=stable_vector(atom.atom_id + atom.source, 64),
|
||||
attention_importance=1.0 / (index + 1),
|
||||
evidence_value=atom.trust.parser * 0.45 + atom.trust.ocr * 0.35 + atom.trust.model * 0.20,
|
||||
)
|
||||
for index, atom in enumerate(visual_atoms[:budget])
|
||||
]
|
||||
selected = select_dvkv_tokens(tokens, min(budget, max(1, len(tokens)))) if tokens else []
|
||||
return {
|
||||
"candidate_tokens": len(tokens),
|
||||
"budget": budget,
|
||||
"selected": [
|
||||
{
|
||||
"token_id": token.token_id,
|
||||
"attention_importance": token.attention_importance,
|
||||
"evidence_value": token.evidence_value,
|
||||
}
|
||||
for token in selected
|
||||
],
|
||||
}
|
||||
|
||||
def x_modal_kv_plan(self, hits: list[RetrievalHit]) -> dict[str, Any]:
|
||||
groups: dict[str, list[dict[str, Any]]] = {}
|
||||
for hit in hits:
|
||||
source_key = hashlib.sha256(hit.atom.source_id.encode("utf-8")).hexdigest()[:16]
|
||||
groups.setdefault(source_key, []).append(
|
||||
{
|
||||
"atom_id": hit.atom.atom_id,
|
||||
"modality": hit.atom.modality.value,
|
||||
"has_image": bool(hit.atom.image_ref),
|
||||
"score": hit.final_score,
|
||||
}
|
||||
)
|
||||
shared_blocks = [
|
||||
{
|
||||
"block_id": f"xkv_{source_key}",
|
||||
"source_key": source_key,
|
||||
"modalities": sorted({item["modality"] for item in items}),
|
||||
"atom_ids": [item["atom_id"] for item in sorted(items, key=lambda value: value["score"], reverse=True)],
|
||||
}
|
||||
for source_key, items in groups.items()
|
||||
]
|
||||
return {"shared_block_count": len(shared_blocks), "shared_blocks": shared_blocks}
|
||||
|
||||
def speculative_prefetch_plan(self, query: str, hits: list[RetrievalHit]) -> dict[str, Any]:
|
||||
query_terms = {term for term in query.lower().split() if len(term) > 2}
|
||||
candidates = []
|
||||
for hit in hits:
|
||||
text_terms = set(hit.atom.text.lower().split())
|
||||
lexical_overlap = len(query_terms & text_terms) / max(1, len(query_terms))
|
||||
priority = hit.final_score * 0.75 + lexical_overlap * 0.25
|
||||
candidates.append(
|
||||
{
|
||||
"cache_key": hashlib.sha256(hit.atom.atom_id.encode("utf-8")).hexdigest()[:16],
|
||||
"atom_id": hit.atom.atom_id,
|
||||
"priority": priority,
|
||||
"bytes_estimate": max(128, hit.atom.token_equivalent * 16),
|
||||
}
|
||||
)
|
||||
candidates.sort(key=lambda item: item["priority"], reverse=True)
|
||||
return {"prefetch_blocks": candidates[:8], "planner_model": self.settings.planner_model_id}
|
||||
|
||||
def predictive_eviction_plan(self) -> dict[str, Any]:
|
||||
layers = CacheFabric(self.settings, self.db).statuses()
|
||||
rows = []
|
||||
for layer in layers:
|
||||
total = max(1, layer.hit_count + layer.miss_count)
|
||||
hit_rate = layer.hit_count / total
|
||||
rows.append(
|
||||
{
|
||||
"layer": layer.layer,
|
||||
"entries": layer.entries,
|
||||
"hit_rate": hit_rate,
|
||||
"evict": hit_rate < 0.25 and layer.entries > 0,
|
||||
"replay_token": hashlib.sha256(f"{layer.layer}:{layer.entries}".encode("utf-8")).hexdigest()[:16],
|
||||
}
|
||||
)
|
||||
return {"policy": "hit-rate weighted replay buffer", "layers": rows}
|
||||
|
||||
def differentiable_retrieval_distribution(self, hits: list[RetrievalHit], temperature: float = 0.05) -> dict[str, Any]:
|
||||
if not hits:
|
||||
return {"temperature": temperature, "probabilities": []}
|
||||
scores = np.array([hit.final_score for hit in hits], dtype=np.float64)
|
||||
scaled = scores / max(temperature, 1e-6)
|
||||
scaled -= float(np.max(scaled))
|
||||
probs = np.exp(scaled)
|
||||
probs /= probs.sum()
|
||||
top_index = int(np.argmax(probs))
|
||||
return {
|
||||
"temperature": temperature,
|
||||
"estimator": "softmax distribution with straight-through top-1 selection",
|
||||
"straight_through_atom_id": hits[top_index].atom.atom_id,
|
||||
"probabilities": [
|
||||
{"atom_id": hit.atom.atom_id, "probability": float(prob)}
|
||||
for hit, prob in zip(hits, probs, strict=True)
|
||||
],
|
||||
}
|
||||
|
||||
def evidence_graph_status(self) -> dict[str, Any]:
|
||||
edges = self.db.list_evidence_edges(limit=64)
|
||||
return {
|
||||
"edge_count_sample": len(edges),
|
||||
"relations": sorted({edge["relation"] for edge in edges}),
|
||||
"recent_edges": edges[:16],
|
||||
}
|
||||
|
||||
def sleep_consolidation_plan(self, hits: list[RetrievalHit]) -> dict[str, Any]:
|
||||
durable = [
|
||||
{
|
||||
"atom_id": hit.atom.atom_id,
|
||||
"reason": "high score and cited-ready evidence",
|
||||
"score": hit.final_score,
|
||||
}
|
||||
for hit in hits
|
||||
if hit.final_score >= 0.25
|
||||
]
|
||||
return {
|
||||
"target_adapter": "hme_ttt_vl",
|
||||
"target_cartridge": "ely_eye_prd",
|
||||
"candidate_count": len(durable),
|
||||
"durable_candidates": durable[:32],
|
||||
}
|
||||
|
||||
|
||||
def stable_vector(seed: str, dims: int) -> np.ndarray:
|
||||
digest = hashlib.sha256(seed.encode("utf-8")).digest()
|
||||
values = []
|
||||
while len(values) < dims:
|
||||
for byte in digest:
|
||||
values.append((byte / 255.0) * 2.0 - 1.0)
|
||||
if len(values) == dims:
|
||||
break
|
||||
digest = hashlib.sha256(digest).digest()
|
||||
vector = np.array(values, dtype=np.float32)
|
||||
norm = float(math.sqrt(float(vector @ vector)))
|
||||
if norm > 0:
|
||||
vector /= norm
|
||||
return vector
|
||||
@@ -0,0 +1,101 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections import defaultdict
|
||||
|
||||
from rank_bm25 import BM25Okapi
|
||||
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .embeddings import EmbeddingService
|
||||
from .schemas import EvidenceAtom, RetrievalHit
|
||||
|
||||
|
||||
class RetrievalService:
|
||||
def __init__(self, settings: Settings | None = None, database: Database | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
self.embeddings = EmbeddingService(self.settings, self.db)
|
||||
|
||||
def search(self, query: str, top_k: int | None = None) -> list[RetrievalHit]:
|
||||
limit = top_k or self.settings.retrieval_top_k
|
||||
atoms = self.db.list_atoms()
|
||||
if not atoms:
|
||||
return []
|
||||
|
||||
sparse_scores = self._sparse_search(query, atoms, limit * 3)
|
||||
dense_scores: dict[str, float] = {}
|
||||
if self.embeddings.enabled():
|
||||
dense_scores = self._dense_search(query, limit * 3)
|
||||
|
||||
all_ids = set(sparse_scores) | set(dense_scores)
|
||||
atom_map = {atom.atom_id: atom for atom in atoms}
|
||||
hits: list[RetrievalHit] = []
|
||||
for atom_id in all_ids:
|
||||
atom = atom_map.get(atom_id)
|
||||
if atom is None:
|
||||
continue
|
||||
sparse_score = sparse_scores.get(atom_id, 0.0)
|
||||
dense_score = dense_scores.get(atom_id, 0.0)
|
||||
graph_score = self._graph_score(atom, query)
|
||||
final_score = sparse_score * 0.55 + dense_score * 0.35 + graph_score * 0.10
|
||||
hits.append(
|
||||
RetrievalHit(
|
||||
atom=atom,
|
||||
sparse_score=sparse_score,
|
||||
dense_score=dense_score,
|
||||
graph_score=graph_score,
|
||||
final_score=final_score,
|
||||
)
|
||||
)
|
||||
hits.sort(key=lambda hit: hit.final_score, reverse=True)
|
||||
return hits[:limit]
|
||||
|
||||
def _sparse_search(self, query: str, atoms: list[EvidenceAtom], limit: int) -> dict[str, float]:
|
||||
tokenized = [tokenize(atom.text + " " + atom.source) for atom in atoms]
|
||||
bm25 = BM25Okapi(tokenized)
|
||||
query_tokens = tokenize(query)
|
||||
scores = bm25.get_scores(query_tokens)
|
||||
lexical_scores = [lexical_score(query_tokens, tokens, atoms[index]) for index, tokens in enumerate(tokenized)]
|
||||
combined = [max(float(scores[index]), lexical_scores[index]) for index in range(len(atoms))]
|
||||
order = sorted(range(len(combined)), key=lambda index: combined[index], reverse=True)[:limit]
|
||||
if not order:
|
||||
return {}
|
||||
max_score = max(float(combined[index]) for index in order) or 1.0
|
||||
return {atoms[index].atom_id: float(combined[index]) / max_score for index in order if combined[index] > 0}
|
||||
|
||||
def _dense_search(self, query: str, limit: int) -> dict[str, float]:
|
||||
index = self.embeddings.load_index()
|
||||
if index is None:
|
||||
index = self.embeddings.build_index()
|
||||
query_vector = self.embeddings.embed_query(query)
|
||||
return dict(index.search(query_vector, limit))
|
||||
|
||||
def _graph_score(self, atom: EvidenceAtom, query: str) -> float:
|
||||
query_tokens = set(tokenize(query))
|
||||
relation_tokens = set()
|
||||
for relation in atom.relations:
|
||||
relation_tokens.update(tokenize(relation))
|
||||
if not query_tokens or not relation_tokens:
|
||||
return 0.0
|
||||
return len(query_tokens & relation_tokens) / len(query_tokens)
|
||||
|
||||
|
||||
def tokenize(text: str) -> list[str]:
|
||||
return [token.lower() for token in re.findall(r"[\w\u4e00-\u9fff]+", text)]
|
||||
|
||||
|
||||
def lexical_score(query_tokens: list[str], document_tokens: list[str], atom: EvidenceAtom) -> float:
|
||||
if not query_tokens or not document_tokens:
|
||||
return 0.0
|
||||
document_set = set(document_tokens)
|
||||
overlap = sum(1 for token in query_tokens if token in document_set)
|
||||
phrase_hits = sum(1 for token in query_tokens if token and token in atom.text.lower())
|
||||
return (overlap + 0.25 * phrase_hits) / len(query_tokens)
|
||||
|
||||
|
||||
def group_hits_by_source(hits: list[RetrievalHit]) -> dict[str, list[RetrievalHit]]:
|
||||
grouped: dict[str, list[RetrievalHit]] = defaultdict(list)
|
||||
for hit in hits:
|
||||
grouped[hit.atom.source].append(hit)
|
||||
return dict(grouped)
|
||||
@@ -0,0 +1,454 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from .config import Settings, get_settings
|
||||
from .schemas import AdapterManifest, CompiledContext, RuntimeStatus, TrainingAdapterKind
|
||||
from .storage import ObjectStore
|
||||
|
||||
|
||||
class RuntimeUnavailableError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass
|
||||
class RuntimeGeneration:
|
||||
text: str
|
||||
backend: str
|
||||
adapter_path: str | None = None
|
||||
adapter_kind: str | None = None
|
||||
adapter_id: str | None = None
|
||||
adapter_sha256: str | None = None
|
||||
input_tokens: int | None = None
|
||||
output_tokens: int | None = None
|
||||
|
||||
|
||||
class QwenRuntime:
|
||||
def __init__(self, settings: Settings | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.objects = ObjectStore(self.settings)
|
||||
self._processor: Any | None = None
|
||||
self._model: Any | None = None
|
||||
|
||||
def status(self) -> RuntimeStatus:
|
||||
gpu = gpu_status()
|
||||
adapter = resolve_runtime_adapter(self.settings)
|
||||
if self.settings.runtime_backend == "sglang":
|
||||
available, detail = self._sglang_status()
|
||||
else:
|
||||
available, detail = self._transformers_status()
|
||||
return RuntimeStatus(
|
||||
backend=self.settings.runtime_backend,
|
||||
model_id=self.settings.model_id,
|
||||
available=available,
|
||||
detail=detail,
|
||||
adapter_path=str(adapter.path) if adapter else None,
|
||||
adapter_kind=adapter.manifest.kind if adapter else None,
|
||||
adapter_id=adapter.manifest.adapter_id if adapter else None,
|
||||
adapter_sha256=adapter.manifest.sha256 if adapter else None,
|
||||
gpu_name=gpu.get("name"),
|
||||
cuda=gpu.get("cuda"),
|
||||
vram_total_mb=gpu.get("total_mb"),
|
||||
vram_used_mb=gpu.get("used_mb"),
|
||||
)
|
||||
|
||||
def generate(self, question: str, context: CompiledContext) -> RuntimeGeneration:
|
||||
if self.settings.runtime_backend == "sglang":
|
||||
return self._generate_sglang(question, context)
|
||||
return self._generate_transformers(question, context)
|
||||
|
||||
def generate_prompt(self, prompt: str, image_paths: list[Path] | None = None) -> RuntimeGeneration:
|
||||
if self.settings.runtime_backend == "sglang":
|
||||
return self._generate_sglang_prompt(prompt, image_paths or [])
|
||||
return self._generate_transformers_prompt(prompt, image_paths or [])
|
||||
|
||||
def _sglang_status(self) -> tuple[bool, str]:
|
||||
try:
|
||||
with httpx.Client(timeout=2.0) as client:
|
||||
response = client.get(f"{self.settings.sglang_base_url}/models")
|
||||
if response.status_code == 200:
|
||||
return True, "SGLang OpenAI-compatible endpoint is reachable."
|
||||
return False, f"SGLang endpoint returned HTTP {response.status_code}."
|
||||
except httpx.HTTPError as exc:
|
||||
return False, f"SGLang endpoint is not reachable: {exc}"
|
||||
|
||||
def _transformers_status(self) -> tuple[bool, str]:
|
||||
try:
|
||||
import torch
|
||||
|
||||
adapter_detail = ""
|
||||
adapter = resolve_runtime_adapter(self.settings)
|
||||
if adapter is not None:
|
||||
adapter_detail = f" Adapter selected: {adapter.path}."
|
||||
elif self.settings.adapter_dir is not None:
|
||||
return False, f"Configured adapter path is missing: {self.settings.adapter_dir}"
|
||||
if torch.cuda.is_available():
|
||||
return True, f"Transformers backend can use CUDA.{adapter_detail}"
|
||||
return True, f"Transformers backend is available on CPU.{adapter_detail}"
|
||||
except Exception as exc:
|
||||
return False, f"Transformers backend import failed: {exc}"
|
||||
|
||||
def _generate_sglang(self, question: str, context: CompiledContext) -> RuntimeGeneration:
|
||||
prompt = build_prompt(question, context)
|
||||
content: list[dict[str, Any]] = [{"type": "text", "text": prompt}]
|
||||
for image_path in self._context_images(context)[:6]:
|
||||
content.append(
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": image_data_url(image_path)},
|
||||
}
|
||||
)
|
||||
payload = {
|
||||
"model": self.settings.model_id,
|
||||
"messages": [{"role": "user", "content": content}],
|
||||
"temperature": self.settings.temperature,
|
||||
"max_tokens": self.settings.max_new_tokens,
|
||||
}
|
||||
try:
|
||||
with httpx.Client(timeout=None) as client:
|
||||
response = client.post(
|
||||
f"{self.settings.sglang_base_url}/chat/completions",
|
||||
headers={"Authorization": f"Bearer {self.settings.sglang_api_key}"},
|
||||
json=payload,
|
||||
)
|
||||
response.raise_for_status()
|
||||
except httpx.HTTPError as exc:
|
||||
raise RuntimeUnavailableError(f"SGLang generation failed: {exc}") from exc
|
||||
data = response.json()
|
||||
adapter = resolve_runtime_adapter(self.settings)
|
||||
usage = data.get("usage") if isinstance(data, dict) else None
|
||||
return RuntimeGeneration(
|
||||
text=data["choices"][0]["message"]["content"],
|
||||
backend="sglang",
|
||||
adapter_path=str(adapter.path) if adapter else None,
|
||||
adapter_kind=adapter.manifest.kind.value if adapter else None,
|
||||
adapter_id=adapter.manifest.adapter_id if adapter else None,
|
||||
adapter_sha256=adapter.manifest.sha256 if adapter else None,
|
||||
input_tokens=usage.get("prompt_tokens") if isinstance(usage, dict) else None,
|
||||
output_tokens=usage.get("completion_tokens") if isinstance(usage, dict) else None,
|
||||
)
|
||||
|
||||
def _generate_sglang_prompt(self, prompt: str, image_paths: list[Path]) -> RuntimeGeneration:
|
||||
content: list[dict[str, Any]] = [{"type": "text", "text": prompt}]
|
||||
for image_path in image_paths[:6]:
|
||||
content.append(
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": image_data_url(image_path)},
|
||||
}
|
||||
)
|
||||
payload = {
|
||||
"model": self.settings.model_id,
|
||||
"messages": [{"role": "user", "content": content}],
|
||||
"temperature": self.settings.temperature,
|
||||
"max_tokens": self.settings.max_new_tokens,
|
||||
}
|
||||
try:
|
||||
with httpx.Client(timeout=None) as client:
|
||||
response = client.post(
|
||||
f"{self.settings.sglang_base_url}/chat/completions",
|
||||
headers={"Authorization": f"Bearer {self.settings.sglang_api_key}"},
|
||||
json=payload,
|
||||
)
|
||||
response.raise_for_status()
|
||||
except httpx.HTTPError as exc:
|
||||
raise RuntimeUnavailableError(f"SGLang generation failed: {exc}") from exc
|
||||
data = response.json()
|
||||
adapter = resolve_runtime_adapter(self.settings)
|
||||
usage = data.get("usage") if isinstance(data, dict) else None
|
||||
return RuntimeGeneration(
|
||||
text=data["choices"][0]["message"]["content"],
|
||||
backend="sglang",
|
||||
adapter_path=str(adapter.path) if adapter else None,
|
||||
adapter_kind=adapter.manifest.kind.value if adapter else None,
|
||||
adapter_id=adapter.manifest.adapter_id if adapter else None,
|
||||
adapter_sha256=adapter.manifest.sha256 if adapter else None,
|
||||
input_tokens=usage.get("prompt_tokens") if isinstance(usage, dict) else None,
|
||||
output_tokens=usage.get("completion_tokens") if isinstance(usage, dict) else None,
|
||||
)
|
||||
|
||||
def _generate_transformers(self, question: str, context: CompiledContext) -> RuntimeGeneration:
|
||||
model, processor = self._load_transformers()
|
||||
prompt = build_prompt(question, context)
|
||||
images = [str(path) for path in self._context_images(context)[:6]]
|
||||
messages: list[dict[str, Any]] = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [{"type": "text", "text": prompt}]
|
||||
+ [{"type": "image", "image": image_path} for image_path in images],
|
||||
}
|
||||
]
|
||||
import torch
|
||||
|
||||
text = processor.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
enable_thinking=False,
|
||||
)
|
||||
image_inputs: list[Any] | None = None
|
||||
video_inputs: list[Any] | None = None
|
||||
if images:
|
||||
from qwen_vl_utils import process_vision_info
|
||||
|
||||
image_inputs, video_inputs = process_vision_info(messages)
|
||||
inputs = processor(
|
||||
text=[text],
|
||||
images=image_inputs,
|
||||
videos=video_inputs,
|
||||
return_tensors="pt",
|
||||
)
|
||||
inputs = {key: value.to(model.device) if hasattr(value, "to") else value for key, value in inputs.items()}
|
||||
generate_kwargs: dict[str, Any] = {
|
||||
**inputs,
|
||||
"max_new_tokens": self.settings.max_new_tokens,
|
||||
"do_sample": self.settings.temperature > 0,
|
||||
}
|
||||
if self.settings.temperature > 0:
|
||||
generate_kwargs["temperature"] = self.settings.temperature
|
||||
with torch.inference_mode():
|
||||
generated = model.generate(**generate_kwargs)
|
||||
input_length = int(inputs["input_ids"].shape[-1])
|
||||
generated = generated[:, input_length:]
|
||||
decoded = processor.batch_decode(generated, skip_special_tokens=True)[0]
|
||||
adapter = resolve_runtime_adapter(self.settings)
|
||||
return RuntimeGeneration(
|
||||
text=decoded,
|
||||
backend="transformers",
|
||||
adapter_path=str(adapter.path) if adapter else None,
|
||||
adapter_kind=adapter.manifest.kind.value if adapter else None,
|
||||
adapter_id=adapter.manifest.adapter_id if adapter else None,
|
||||
adapter_sha256=adapter.manifest.sha256 if adapter else None,
|
||||
input_tokens=input_length,
|
||||
output_tokens=int(generated.shape[-1]),
|
||||
)
|
||||
|
||||
def _generate_transformers_prompt(self, prompt: str, image_paths: list[Path]) -> RuntimeGeneration:
|
||||
model, processor = self._load_transformers()
|
||||
images = [str(path) for path in image_paths[:6]]
|
||||
messages: list[dict[str, Any]] = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [{"type": "text", "text": prompt}]
|
||||
+ [{"type": "image", "image": image_path} for image_path in images],
|
||||
}
|
||||
]
|
||||
import torch
|
||||
|
||||
text = processor.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
enable_thinking=False,
|
||||
)
|
||||
image_inputs: list[Any] | None = None
|
||||
video_inputs: list[Any] | None = None
|
||||
if images:
|
||||
from qwen_vl_utils import process_vision_info
|
||||
|
||||
image_inputs, video_inputs = process_vision_info(messages)
|
||||
inputs = processor(
|
||||
text=[text],
|
||||
images=image_inputs,
|
||||
videos=video_inputs,
|
||||
return_tensors="pt",
|
||||
)
|
||||
inputs = {key: value.to(model.device) if hasattr(value, "to") else value for key, value in inputs.items()}
|
||||
generate_kwargs: dict[str, Any] = {
|
||||
**inputs,
|
||||
"max_new_tokens": self.settings.max_new_tokens,
|
||||
"do_sample": self.settings.temperature > 0,
|
||||
}
|
||||
if self.settings.temperature > 0:
|
||||
generate_kwargs["temperature"] = self.settings.temperature
|
||||
with torch.inference_mode():
|
||||
generated = model.generate(**generate_kwargs)
|
||||
input_length = int(inputs["input_ids"].shape[-1])
|
||||
generated = generated[:, input_length:]
|
||||
decoded = processor.batch_decode(generated, skip_special_tokens=True)[0]
|
||||
adapter = resolve_runtime_adapter(self.settings)
|
||||
return RuntimeGeneration(
|
||||
text=decoded,
|
||||
backend="transformers",
|
||||
adapter_path=str(adapter.path) if adapter else None,
|
||||
adapter_kind=adapter.manifest.kind.value if adapter else None,
|
||||
adapter_id=adapter.manifest.adapter_id if adapter else None,
|
||||
adapter_sha256=adapter.manifest.sha256 if adapter else None,
|
||||
input_tokens=input_length,
|
||||
output_tokens=int(generated.shape[-1]),
|
||||
)
|
||||
|
||||
def _load_transformers(self) -> tuple[Any, Any]:
|
||||
if self._model is not None and self._processor is not None:
|
||||
return self._model, self._processor
|
||||
import torch
|
||||
from transformers import AutoProcessor
|
||||
|
||||
try:
|
||||
from transformers import AutoModelForImageTextToText
|
||||
|
||||
model_cls = AutoModelForImageTextToText
|
||||
except ImportError:
|
||||
from transformers import AutoModelForCausalLM
|
||||
|
||||
model_cls = AutoModelForCausalLM
|
||||
|
||||
self._processor = AutoProcessor.from_pretrained(self.settings.model_id, trust_remote_code=True)
|
||||
self._model = model_cls.from_pretrained(
|
||||
self.settings.model_id,
|
||||
dtype=torch.bfloat16 if torch.cuda.is_available() else torch.float32,
|
||||
device_map="auto",
|
||||
trust_remote_code=True,
|
||||
)
|
||||
adapter = resolve_runtime_adapter(self.settings)
|
||||
if adapter is not None:
|
||||
from .training import bitsandbytes_gaudi_probe_guard
|
||||
|
||||
with bitsandbytes_gaudi_probe_guard():
|
||||
from peft import PeftModel
|
||||
|
||||
self._model = PeftModel.from_pretrained(self._model, adapter.path)
|
||||
self._model.eval()
|
||||
return self._model, self._processor
|
||||
|
||||
def unload(self) -> None:
|
||||
self._model = None
|
||||
self._processor = None
|
||||
try:
|
||||
import torch
|
||||
|
||||
if torch.cuda.is_available():
|
||||
torch.cuda.empty_cache()
|
||||
except Exception:
|
||||
return
|
||||
|
||||
def _context_images(self, context: CompiledContext) -> list[Path]:
|
||||
paths: list[Path] = []
|
||||
for hit in context.hits:
|
||||
image_ref = hit.atom.image_ref
|
||||
if image_ref:
|
||||
paths.append(self.objects.resolve(image_ref))
|
||||
return paths
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RuntimeAdapter:
|
||||
path: Path
|
||||
manifest: AdapterManifest
|
||||
|
||||
|
||||
def resolve_runtime_adapter(
|
||||
settings: Settings,
|
||||
kind: TrainingAdapterKind = TrainingAdapterKind.hme_core_lora,
|
||||
) -> RuntimeAdapter | None:
|
||||
if settings.adapter_dir is not None:
|
||||
path = settings.adapter_dir.expanduser().resolve()
|
||||
manifest_path = path / "adapter_manifest.json"
|
||||
if path.exists() and manifest_path.exists():
|
||||
return RuntimeAdapter(
|
||||
path=path,
|
||||
manifest=AdapterManifest.model_validate_json(manifest_path.read_text(encoding="utf-8")),
|
||||
)
|
||||
return None
|
||||
|
||||
candidates: list[RuntimeAdapter] = []
|
||||
for manifest_path in settings.adapters_dir.glob("*/adapter_manifest.json"):
|
||||
manifest = AdapterManifest.model_validate_json(manifest_path.read_text(encoding="utf-8"))
|
||||
if manifest.kind == kind and manifest.base_model == settings.model_id:
|
||||
candidates.append(RuntimeAdapter(path=manifest_path.parent.resolve(), manifest=manifest))
|
||||
candidates.sort(
|
||||
key=lambda item: (
|
||||
item.manifest.cartridge_id is not None,
|
||||
(item.path / "adapter_model.safetensors").stat().st_mtime
|
||||
if (item.path / "adapter_model.safetensors").exists()
|
||||
else 0.0,
|
||||
),
|
||||
reverse=True,
|
||||
)
|
||||
return candidates[0] if candidates else None
|
||||
|
||||
|
||||
def build_prompt(question: str, context: CompiledContext) -> str:
|
||||
contract = "\n".join(f"- {item}" for item in context.plan.verifier_contract)
|
||||
return f"""You are Ely-Eye, a local multimodal context operating system.
|
||||
|
||||
Use /no_think mode. Return final JSON only. Use only the provided evidence. Include atom ids in citations.
|
||||
|
||||
Verifier contract:
|
||||
{contract}
|
||||
|
||||
Context plan:
|
||||
{context.plan.model_dump_json(indent=2)}
|
||||
|
||||
Packed evidence:
|
||||
{context.packed_text}
|
||||
|
||||
Question:
|
||||
{question}
|
||||
|
||||
Return compact JSON only:
|
||||
{{
|
||||
"answer": "120 words or fewer",
|
||||
"citations": ["atom_id strings only"],
|
||||
"contradiction_notes": ["strings"],
|
||||
"confidence": 0.0
|
||||
}}
|
||||
"""
|
||||
|
||||
|
||||
def image_data_url(path: Path) -> str:
|
||||
suffix = path.suffix.lower().lstrip(".") or "png"
|
||||
mime = "jpeg" if suffix in {"jpg", "jpeg"} else suffix
|
||||
encoded = base64.b64encode(path.read_bytes()).decode("ascii")
|
||||
return f"data:image/{mime};base64,{encoded}"
|
||||
|
||||
|
||||
def gpu_status() -> dict[str, Any]:
|
||||
try:
|
||||
output = subprocess.check_output(
|
||||
[
|
||||
"nvidia-smi",
|
||||
"--query-gpu=name,memory.total,memory.used,driver_version",
|
||||
"--format=csv,noheader,nounits",
|
||||
],
|
||||
text=True,
|
||||
stderr=subprocess.STDOUT,
|
||||
timeout=5,
|
||||
).strip()
|
||||
except (subprocess.SubprocessError, FileNotFoundError):
|
||||
return {}
|
||||
if not output:
|
||||
return {}
|
||||
first = output.splitlines()[0]
|
||||
parts = [part.strip() for part in first.split(",")]
|
||||
if len(parts) < 4:
|
||||
return {}
|
||||
return {
|
||||
"name": parts[0],
|
||||
"total_mb": int(float(parts[1])),
|
||||
"used_mb": int(float(parts[2])),
|
||||
"cuda": f"driver {parts[3]}",
|
||||
}
|
||||
|
||||
|
||||
def parse_runtime_json(text: str) -> dict[str, Any]:
|
||||
stripped = text.strip()
|
||||
if stripped.startswith("```"):
|
||||
stripped = stripped.strip("`")
|
||||
if stripped.lower().startswith("json"):
|
||||
stripped = stripped[4:].strip()
|
||||
try:
|
||||
return json.loads(stripped)
|
||||
except json.JSONDecodeError:
|
||||
start = stripped.find("{")
|
||||
end = stripped.rfind("}")
|
||||
if start >= 0 and end > start:
|
||||
return json.loads(stripped[start : end + 1])
|
||||
raise
|
||||
@@ -0,0 +1,479 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from enum import StrEnum
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
def utc_now() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
class RuntimeProfile(StrEnum):
|
||||
live_demo = "live_demo"
|
||||
extreme_context = "extreme_context"
|
||||
library_100m = "library_100m"
|
||||
research_theater = "research_theater"
|
||||
|
||||
|
||||
class TrainingAdapterKind(StrEnum):
|
||||
hme_core_lora = "hme_core_lora"
|
||||
hme_vision_lora = "hme_vision_lora"
|
||||
hme_ttt_vl = "hme_ttt_vl"
|
||||
hme_visual_mtp = "hme_visual_mtp"
|
||||
hme_router = "hme_router"
|
||||
hme_retrieval = "hme_retrieval"
|
||||
|
||||
|
||||
class Modality(StrEnum):
|
||||
text = "text"
|
||||
code = "code"
|
||||
image = "image"
|
||||
pdf_page = "pdf_page"
|
||||
pdf_region = "pdf_region"
|
||||
video_frame = "video_frame"
|
||||
ui_screenshot = "ui_screenshot"
|
||||
mixed = "mixed"
|
||||
|
||||
|
||||
class LayoutBox(BaseModel):
|
||||
x: float
|
||||
y: float
|
||||
w: float
|
||||
h: float
|
||||
page: int | None = None
|
||||
frame_second: float | None = None
|
||||
|
||||
|
||||
class TrustScores(BaseModel):
|
||||
parser: float = Field(ge=0.0, le=1.0)
|
||||
ocr: float = Field(default=0.0, ge=0.0, le=1.0)
|
||||
model: float = Field(default=0.0, ge=0.0, le=1.0)
|
||||
|
||||
|
||||
class EvidenceAtom(BaseModel):
|
||||
atom_id: str
|
||||
modality: Modality
|
||||
source_id: str
|
||||
source: str
|
||||
time: datetime = Field(default_factory=utc_now)
|
||||
text: str = ""
|
||||
image_ref: str | None = None
|
||||
embedding_refs: list[str] = Field(default_factory=list)
|
||||
layout: LayoutBox | None = None
|
||||
relations: list[str] = Field(default_factory=list)
|
||||
trust: TrustScores
|
||||
token_equivalent: int = Field(default=0, ge=0)
|
||||
metadata: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class SourceRecord(BaseModel):
|
||||
source_id: str
|
||||
path: str
|
||||
kind: str
|
||||
mime: str
|
||||
sha256: str
|
||||
size_bytes: int
|
||||
created_at: datetime = Field(default_factory=utc_now)
|
||||
metadata: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class IngestResponse(BaseModel):
|
||||
source_count: int
|
||||
atom_count: int
|
||||
cartridge_id: str | None
|
||||
token_equivalent: int
|
||||
sources: list[SourceRecord]
|
||||
|
||||
|
||||
class RetrievalHit(BaseModel):
|
||||
atom: EvidenceAtom
|
||||
sparse_score: float
|
||||
dense_score: float = 0.0
|
||||
graph_score: float = 0.0
|
||||
final_score: float
|
||||
|
||||
|
||||
class ContextPlan(BaseModel):
|
||||
question: str
|
||||
profile: RuntimeProfile
|
||||
required_modalities: list[Modality]
|
||||
evidence_budget_atoms: int
|
||||
token_budget: int
|
||||
retrieval_query: str
|
||||
compression_strategy: str
|
||||
verifier_contract: list[str]
|
||||
|
||||
|
||||
class CompiledContext(BaseModel):
|
||||
plan: ContextPlan
|
||||
hits: list[RetrievalHit]
|
||||
packed_text: str
|
||||
token_equivalent: int
|
||||
cache_trace_id: str
|
||||
|
||||
|
||||
class CitationReport(BaseModel):
|
||||
cited_atom_ids: list[str]
|
||||
missing_atom_ids: list[str]
|
||||
citation_accuracy: float
|
||||
contradiction_notes: list[str]
|
||||
confidence: float
|
||||
|
||||
|
||||
class AnswerResponse(BaseModel):
|
||||
answer: str
|
||||
citations: list[str]
|
||||
context: CompiledContext
|
||||
verifier: CitationReport
|
||||
runtime_backend: str
|
||||
|
||||
|
||||
class CacheLayerStatus(BaseModel):
|
||||
layer: str
|
||||
entries: int
|
||||
bytes_estimate: int
|
||||
hit_count: int
|
||||
miss_count: int
|
||||
last_event_at: datetime | None
|
||||
|
||||
|
||||
class MemoryStatus(BaseModel):
|
||||
active_atoms: int
|
||||
active_token_equivalent: int
|
||||
cartridges: int
|
||||
library_token_equivalent: int
|
||||
layers: list[CacheLayerStatus]
|
||||
|
||||
|
||||
class RuntimeStatus(BaseModel):
|
||||
backend: str
|
||||
model_id: str
|
||||
available: bool
|
||||
detail: str
|
||||
adapter_path: str | None = None
|
||||
adapter_kind: TrainingAdapterKind | None = None
|
||||
adapter_id: str | None = None
|
||||
adapter_sha256: str | None = None
|
||||
gpu_name: str | None = None
|
||||
cuda: str | None = None
|
||||
vram_total_mb: int | None = None
|
||||
vram_used_mb: int | None = None
|
||||
|
||||
|
||||
class SystemStatus(BaseModel):
|
||||
app: str = "Ely-Eye"
|
||||
version: str
|
||||
runtime: RuntimeStatus
|
||||
memory: MemoryStatus
|
||||
profiles: dict[str, dict[str, Any]]
|
||||
data_home: str
|
||||
|
||||
|
||||
class CartridgeManifest(BaseModel):
|
||||
cartridge_id: str
|
||||
name: str
|
||||
created_at: datetime
|
||||
model_base: str
|
||||
source_count: int
|
||||
atom_count: int
|
||||
token_equivalent: int
|
||||
artifacts: dict[str, str]
|
||||
dna: str | None = None
|
||||
|
||||
|
||||
class AdapterManifest(BaseModel):
|
||||
adapter_id: str
|
||||
kind: TrainingAdapterKind
|
||||
created_at: datetime
|
||||
proof_version: int = 1
|
||||
training_method: str = "BF16 LoRA"
|
||||
precision: str = "bf16"
|
||||
framework: str = "unsloth_patched_transformers_peft"
|
||||
base_model: str
|
||||
dataset_path: str
|
||||
dataset_sha256: str
|
||||
sample_count: int
|
||||
max_steps: int
|
||||
max_length: int
|
||||
rank: int
|
||||
alpha: int
|
||||
target_modules: list[str]
|
||||
trainable_params: int
|
||||
total_params: int
|
||||
optimizer: str
|
||||
optimizer_family: str | None = None
|
||||
learning_rate: float
|
||||
gradient_accumulation_steps: int
|
||||
gradient_checkpointing: bool = False
|
||||
gradient_checkpointing_mode: str | None = None
|
||||
autocast_dtype: str | None = None
|
||||
unsloth_version: str | None = None
|
||||
triton_version: str | None = None
|
||||
xformers_version: str | None = None
|
||||
unsloth_model_type: str | None = None
|
||||
bf16: bool
|
||||
final_loss: float | None
|
||||
loss_history: list[float] = Field(default_factory=list)
|
||||
initial_trainable_sha256: str | None = None
|
||||
final_trainable_sha256: str | None = None
|
||||
initial_trainable_l2: float | None = None
|
||||
final_trainable_l2: float | None = None
|
||||
optimizer_update_count: int = 0
|
||||
max_optimizer_update_l2: float | None = None
|
||||
loss_delta: float | None = None
|
||||
training_proof_path: str | None = None
|
||||
training_proof_sha256: str | None = None
|
||||
training_trace_path: str | None = None
|
||||
training_trace_sha256: str | None = None
|
||||
training_summary_path: str | None = None
|
||||
training_summary_sha256: str | None = None
|
||||
training_started_at: datetime | None = None
|
||||
training_finished_at: datetime | None = None
|
||||
training_wall_seconds: float | None = None
|
||||
training_device: str | None = None
|
||||
torch_version: str | None = None
|
||||
cuda_version: str | None = None
|
||||
adapter_weight_sha256: str | None = None
|
||||
adapter_tensor_count: int = 0
|
||||
adapter_lora_tensor_count: int = 0
|
||||
adapter_total_elements: int = 0
|
||||
adapter_nonzero_elements: int = 0
|
||||
adapter_max_abs: float | None = None
|
||||
adapter_weights_finite: bool = False
|
||||
artifacts: dict[str, str]
|
||||
sha256: str
|
||||
cartridge_id: str | None = None
|
||||
|
||||
|
||||
class AdapterStatus(BaseModel):
|
||||
name: str
|
||||
kind: TrainingAdapterKind
|
||||
proof_version: int = 1
|
||||
training_method: str = "BF16 LoRA"
|
||||
precision: str = "bf16"
|
||||
framework: str = "unsloth_patched_transformers_peft"
|
||||
base_model: str
|
||||
path: str
|
||||
adapter_id: str
|
||||
sample_count: int
|
||||
max_steps: int
|
||||
trainable_params: int
|
||||
final_loss: float | None
|
||||
sha256: str
|
||||
weight_bytes: int
|
||||
safetensor_keys: int
|
||||
cartridge_id: str | None = None
|
||||
cartridge_bound: bool
|
||||
training_trace_path: str | None = None
|
||||
training_trace_sha256: str | None = None
|
||||
training_trace_steps: int = 0
|
||||
training_proof_path: str | None = None
|
||||
training_proof_sha256: str | None = None
|
||||
optimizer: str | None = None
|
||||
optimizer_family: str | None = None
|
||||
gradient_checkpointing: bool = False
|
||||
gradient_checkpointing_mode: str | None = None
|
||||
autocast_dtype: str | None = None
|
||||
unsloth_version: str | None = None
|
||||
triton_version: str | None = None
|
||||
xformers_version: str | None = None
|
||||
unsloth_model_type: str | None = None
|
||||
optimizer_update_count: int = 0
|
||||
max_optimizer_update_l2: float | None = None
|
||||
loss_delta: float | None = None
|
||||
training_summary_path: str | None = None
|
||||
training_wall_seconds: float | None = None
|
||||
training_device: str | None = None
|
||||
bf16: bool = False
|
||||
torch_version: str | None = None
|
||||
cuda_version: str | None = None
|
||||
adapter_weight_sha256: str | None = None
|
||||
adapter_tensor_count: int = 0
|
||||
adapter_lora_tensor_count: int = 0
|
||||
adapter_total_elements: int = 0
|
||||
adapter_nonzero_elements: int = 0
|
||||
adapter_max_abs: float | None = None
|
||||
adapter_weights_finite: bool = False
|
||||
|
||||
|
||||
class CartridgeAssetStatus(BaseModel):
|
||||
name: str
|
||||
path: str
|
||||
sha256: str
|
||||
size_bytes: int
|
||||
item_count: int
|
||||
dtype: str
|
||||
status: Literal["ready", "missing"]
|
||||
|
||||
|
||||
class CartridgeAssetReport(BaseModel):
|
||||
cartridge_id: str
|
||||
status: Literal["ready", "incomplete"]
|
||||
created_at: datetime = Field(default_factory=utc_now)
|
||||
assets: list[CartridgeAssetStatus]
|
||||
|
||||
|
||||
class BenchmarkSpec(BaseModel):
|
||||
name: str
|
||||
target: str
|
||||
dataset: str
|
||||
task_type: str
|
||||
runner: str
|
||||
metric: str
|
||||
|
||||
|
||||
class BenchmarkRegistryReport(BaseModel):
|
||||
status: Literal["ready", "incomplete"]
|
||||
created_at: datetime = Field(default_factory=utc_now)
|
||||
benchmarks: list[BenchmarkSpec]
|
||||
|
||||
|
||||
class BenchmarkSourceStatus(BaseModel):
|
||||
name: str
|
||||
dataset: str
|
||||
status: Literal["ready", "unavailable"]
|
||||
sha: str | None = None
|
||||
gated: bool | str | None = None
|
||||
private: bool | None = None
|
||||
sibling_count: int = 0
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class BenchmarkSourceReport(BaseModel):
|
||||
status: Literal["ready", "incomplete"]
|
||||
created_at: datetime = Field(default_factory=utc_now)
|
||||
sources: list[BenchmarkSourceStatus]
|
||||
|
||||
|
||||
class BenchmarkSampleStatus(BaseModel):
|
||||
name: str
|
||||
dataset: str
|
||||
source_dataset: str | None = None
|
||||
source_sha: str | None = None
|
||||
source_note: str | None = None
|
||||
status: Literal["loaded", "gated", "failed"]
|
||||
config: str | None = None
|
||||
split: str | None = None
|
||||
sample_id: str | None = None
|
||||
fields: list[str] = Field(default_factory=list)
|
||||
question_preview: str | None = None
|
||||
answer_preview: str | None = None
|
||||
sample_sha256: str | None = None
|
||||
media_artifacts: list[str] = Field(default_factory=list)
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class BenchmarkSampleReport(BaseModel):
|
||||
status: Literal["ready", "partial", "failed"]
|
||||
created_at: datetime = Field(default_factory=utc_now)
|
||||
samples: list[BenchmarkSampleStatus]
|
||||
|
||||
|
||||
class BenchmarkPredictionStatus(BaseModel):
|
||||
name: str
|
||||
dataset: str
|
||||
status: Literal["predicted", "gated", "failed"]
|
||||
config: str | None = None
|
||||
split: str | None = None
|
||||
sample_id: str | None = None
|
||||
task_type: str
|
||||
metric: str
|
||||
prompt_sha256: str | None = None
|
||||
sample_sha256: str | None = None
|
||||
prediction: str | None = None
|
||||
answer: str | None = None
|
||||
score: float | None = None
|
||||
runtime_backend: str | None = None
|
||||
adapter_id: str | None = None
|
||||
adapter_sha256: str | None = None
|
||||
input_tokens: int | None = None
|
||||
output_tokens: int | None = None
|
||||
media_artifacts: list[str] = Field(default_factory=list)
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class BenchmarkPredictionReport(BaseModel):
|
||||
status: Literal["ready", "partial", "failed"]
|
||||
created_at: datetime = Field(default_factory=utc_now)
|
||||
predictions: list[BenchmarkPredictionStatus]
|
||||
|
||||
|
||||
class RuntimeLaunchProfile(BaseModel):
|
||||
name: str
|
||||
runtime: str
|
||||
role: str
|
||||
context_tokens: int
|
||||
script: str
|
||||
command: list[str]
|
||||
environment: dict[str, str] = Field(default_factory=dict)
|
||||
required_features: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class RuntimeProfileValidation(BaseModel):
|
||||
name: str
|
||||
runtime: str
|
||||
status: Literal["ready", "missing"]
|
||||
script_exists: bool
|
||||
package: str | None = None
|
||||
package_version: str | None = None
|
||||
executable: str | None = None
|
||||
wsl_gpu: bool = False
|
||||
probe_command: list[str] = Field(default_factory=list)
|
||||
probe_status: Literal["passed", "failed", "skipped"] = "skipped"
|
||||
probe_exit_code: int | None = None
|
||||
probe_output: str | None = None
|
||||
detail: str
|
||||
|
||||
|
||||
class RuntimeProfileReport(BaseModel):
|
||||
status: Literal["ready", "incomplete"]
|
||||
created_at: datetime = Field(default_factory=utc_now)
|
||||
profiles: list[RuntimeLaunchProfile]
|
||||
validations: list[RuntimeProfileValidation] = Field(default_factory=list)
|
||||
|
||||
|
||||
class HashHopProof(BaseModel):
|
||||
proof_id: str
|
||||
kind: Literal["text_hashhop", "visual_hashhop"]
|
||||
hops: int
|
||||
token_equivalent: int
|
||||
query_id: str
|
||||
expected_target_id: str
|
||||
model_target_id: str | None = None
|
||||
passed: bool | None = None
|
||||
artifacts: list[str]
|
||||
created_at: datetime = Field(default_factory=utc_now)
|
||||
|
||||
|
||||
class ProofCheck(BaseModel):
|
||||
name: str
|
||||
requirement: str
|
||||
status: Literal["passed", "failed"]
|
||||
evidence: list[str]
|
||||
detail: str
|
||||
|
||||
|
||||
class ProofSuiteReport(BaseModel):
|
||||
proof_id: str
|
||||
cartridge_id: str
|
||||
status: Literal["passed", "failed"]
|
||||
created_at: datetime = Field(default_factory=utc_now)
|
||||
checks: list[ProofCheck]
|
||||
artifacts: dict[str, str]
|
||||
dna_before: str | None = None
|
||||
dna_after: str | None = None
|
||||
|
||||
|
||||
class UploadIngestRequest(BaseModel):
|
||||
cartridge_name: str | None = None
|
||||
dense_embeddings: bool | None = None
|
||||
|
||||
|
||||
class PathIngestRequest(BaseModel):
|
||||
path: Path
|
||||
cartridge_name: str | None = None
|
||||
dense_embeddings: bool | None = None
|
||||
@@ -0,0 +1,78 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import mimetypes
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
from .config import Settings, get_settings
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def sha256_bytes(data: bytes) -> str:
|
||||
return hashlib.sha256(data).hexdigest()
|
||||
|
||||
|
||||
def guess_mime(path: Path) -> str:
|
||||
mime, _ = mimetypes.guess_type(path.name)
|
||||
return mime or "application/octet-stream"
|
||||
|
||||
|
||||
def safe_source_id(path: Path, content_hash: str) -> str:
|
||||
stem = "".join(char if char.isalnum() else "_" for char in path.stem).strip("_")
|
||||
prefix = stem[:48] or "source"
|
||||
return f"{prefix}_{content_hash[:16]}"
|
||||
|
||||
|
||||
def token_equivalent(text: str) -> int:
|
||||
if not text:
|
||||
return 0
|
||||
ascii_chars = sum(1 for char in text if ord(char) < 128)
|
||||
non_ascii_chars = len(text) - ascii_chars
|
||||
return max(1, ascii_chars // 4 + non_ascii_chars // 2)
|
||||
|
||||
|
||||
class ObjectStore:
|
||||
def __init__(self, settings: Settings | None = None) -> None:
|
||||
self.settings = settings or get_settings()
|
||||
self.settings.ensure_dirs()
|
||||
|
||||
def put_file(self, path: Path, namespace: str) -> str:
|
||||
content_hash = sha256_file(path)
|
||||
suffix = path.suffix.lower()
|
||||
target_dir = self.settings.object_dir / namespace / content_hash[:2]
|
||||
target_dir.mkdir(parents=True, exist_ok=True)
|
||||
target = target_dir / f"{content_hash}{suffix}"
|
||||
if not target.exists():
|
||||
shutil.copy2(path, target)
|
||||
return self.to_object_uri(target)
|
||||
|
||||
def put_bytes(self, data: bytes, suffix: str, namespace: str) -> str:
|
||||
content_hash = sha256_bytes(data)
|
||||
target_dir = self.settings.object_dir / namespace / content_hash[:2]
|
||||
target_dir.mkdir(parents=True, exist_ok=True)
|
||||
target = target_dir / f"{content_hash}{suffix}"
|
||||
if not target.exists():
|
||||
target.write_bytes(data)
|
||||
return self.to_object_uri(target)
|
||||
|
||||
def resolve(self, object_uri: str) -> Path:
|
||||
prefix = "object://"
|
||||
if not object_uri.startswith(prefix):
|
||||
raise ValueError(f"Invalid object URI: {object_uri}")
|
||||
relative = object_uri[len(prefix) :]
|
||||
path = (self.settings.object_dir / relative).resolve()
|
||||
root = self.settings.object_dir.resolve()
|
||||
if root not in path.parents and path != root:
|
||||
raise ValueError(f"Object URI escapes object store: {object_uri}")
|
||||
return path
|
||||
|
||||
def to_object_uri(self, path: Path) -> str:
|
||||
return "object://" + path.resolve().relative_to(self.settings.object_dir.resolve()).as_posix()
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,232 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from .schemas import CitationReport, CompiledContext
|
||||
|
||||
|
||||
ATOM_PATTERN = re.compile(r"[A-Za-z0-9_./\\:-]+:(?:text|image|page|frame|region)[A-Za-z0-9_./\\:#=-]*")
|
||||
|
||||
|
||||
class Verifier:
|
||||
def verify_answer(self, raw_answer: str, context: CompiledContext) -> tuple[str, list[str], CitationReport]:
|
||||
parsed = self._parse_answer(raw_answer)
|
||||
answer = parsed.get("answer", raw_answer)
|
||||
citations = parsed.get("citations") or self._extract_citations(raw_answer)
|
||||
citations = normalize_citations(citations)
|
||||
available = {hit.atom.atom_id for hit in context.hits}
|
||||
cited_available, missing = resolve_citations(citations, available)
|
||||
accuracy = len(cited_available) / len(citations) if citations else 0.0
|
||||
contradiction_notes = parsed.get("contradiction_notes") or []
|
||||
if isinstance(contradiction_notes, str):
|
||||
contradiction_notes = [contradiction_notes]
|
||||
confidence = float(parsed.get("confidence", accuracy))
|
||||
return (
|
||||
str(answer),
|
||||
citations,
|
||||
CitationReport(
|
||||
cited_atom_ids=cited_available,
|
||||
missing_atom_ids=missing,
|
||||
citation_accuracy=accuracy,
|
||||
contradiction_notes=[str(note) for note in contradiction_notes],
|
||||
confidence=max(0.0, min(1.0, confidence)),
|
||||
),
|
||||
)
|
||||
|
||||
def _parse_answer(self, raw_answer: str) -> dict[str, Any]:
|
||||
stripped = raw_answer.strip()
|
||||
if not stripped:
|
||||
return {"answer": ""}
|
||||
candidates = [stripped]
|
||||
if "```" in stripped:
|
||||
parts = stripped.split("```")
|
||||
candidates.extend(part.removeprefix("json").strip() for part in parts)
|
||||
start = stripped.find("{")
|
||||
end = stripped.rfind("}")
|
||||
if start >= 0 and end > start:
|
||||
candidates.append(stripped[start : end + 1])
|
||||
for candidate in candidates:
|
||||
try:
|
||||
value = json.loads(candidate)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
if isinstance(value, dict):
|
||||
return value
|
||||
return {"answer": raw_answer, "citations": self._extract_citations(raw_answer)}
|
||||
|
||||
def _extract_citations(self, raw_answer: str) -> list[str]:
|
||||
return sorted(set(ATOM_PATTERN.findall(raw_answer)))
|
||||
|
||||
|
||||
def normalize_citations(value: Any) -> list[str]:
|
||||
if value is None:
|
||||
return []
|
||||
if isinstance(value, str):
|
||||
return [value]
|
||||
normalized: list[str] = []
|
||||
if isinstance(value, list):
|
||||
for item in value:
|
||||
if isinstance(item, str):
|
||||
normalized.append(item)
|
||||
elif isinstance(item, dict) and "atom_id" in item:
|
||||
normalized.append(str(item["atom_id"]))
|
||||
return normalized
|
||||
|
||||
|
||||
def resolve_citations(citations: list[str], available: set[str]) -> tuple[list[str], list[str]]:
|
||||
cited_available: list[str] = []
|
||||
missing: list[str] = []
|
||||
for citation in citations:
|
||||
atom_id = citation.split("#", 1)[0]
|
||||
if citation in available:
|
||||
cited_available.append(citation)
|
||||
elif atom_id in available:
|
||||
cited_available.append(atom_id)
|
||||
else:
|
||||
missing.append(citation)
|
||||
return list(dict.fromkeys(cited_available)), missing
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DriftRule:
|
||||
"""One declarative extractor for a design fact (PRD 11.15 taxonomy).
|
||||
|
||||
``pattern`` exposes an optional ``scope`` group (component qualifier) and a
|
||||
mandatory ``value`` group. The same multi-value-across-sources mechanism
|
||||
surfaces every drift kind; ``category`` is the PRD baseline label.
|
||||
"""
|
||||
|
||||
name: str
|
||||
category: str
|
||||
pattern: re.Pattern[str]
|
||||
value_kind: str
|
||||
|
||||
|
||||
class VisualContradictionLens:
|
||||
"""Detect cross-version visual contradictions (PRD 11.15).
|
||||
|
||||
A single mechanism backs the whole taxonomy: extract scoped ``(attribute,
|
||||
value)`` design facts from each Evidence Atom, group identical attributes
|
||||
across atoms, and flag any attribute carrying more than one distinct value
|
||||
across at least two distinct sources. Requiring two sources prevents a
|
||||
single CSS file with many rules from reading as a contradiction. Source
|
||||
kind and version metadata refine each note into design-token, layout,
|
||||
typography, copy, visual-code, or temporal drift.
|
||||
"""
|
||||
|
||||
RULES: tuple[DriftRule, ...] = (
|
||||
DriftRule(
|
||||
"radius",
|
||||
"design-token drift",
|
||||
re.compile(
|
||||
r"(?P<scope>[A-Za-z][\w .-]*?)?\.?(?:border-)?radius\s*[:=]\s*(?P<value>\d+(?:\.\d+)?)\s*px",
|
||||
re.IGNORECASE,
|
||||
),
|
||||
"length",
|
||||
),
|
||||
DriftRule(
|
||||
"color",
|
||||
"design-token drift",
|
||||
re.compile(
|
||||
r"(?P<scope>[A-Za-z][\w .-]*?)?\.?(?:colou?r|background|fill)\s*[:=]\s*(?P<value>#[0-9a-fA-F]{3,8})",
|
||||
re.IGNORECASE,
|
||||
),
|
||||
"color",
|
||||
),
|
||||
DriftRule(
|
||||
"font-size",
|
||||
"typography drift",
|
||||
re.compile(
|
||||
r"(?P<scope>[A-Za-z][\w .-]*?)?\.?font-size\s*[:=]\s*(?P<value>\d+(?:\.\d+)?)\s*(?:px|rem|pt)",
|
||||
re.IGNORECASE,
|
||||
),
|
||||
"length",
|
||||
),
|
||||
DriftRule(
|
||||
"spacing",
|
||||
"layout drift",
|
||||
re.compile(
|
||||
r"(?P<scope>[A-Za-z][\w .-]*?)?\.?(?:padding|margin|gap|spacing)\s*[:=]\s*(?P<value>\d+(?:\.\d+)?)\s*px",
|
||||
re.IGNORECASE,
|
||||
),
|
||||
"length",
|
||||
),
|
||||
DriftRule(
|
||||
"label",
|
||||
"copy drift",
|
||||
re.compile(
|
||||
r"(?P<scope>[A-Za-z][\w .-]*?)?\.?(?:label|copy|cta|caption)\s*[:=]\s*[\"'](?P<value>[^\"']+)[\"']",
|
||||
re.IGNORECASE,
|
||||
),
|
||||
"text",
|
||||
),
|
||||
)
|
||||
|
||||
def detect(self, context: CompiledContext) -> list[str]:
|
||||
observations: dict[str, dict[str, set[str]]] = {}
|
||||
raw_display: dict[str, dict[str, str]] = {}
|
||||
category_of: dict[str, str] = {}
|
||||
source_kind: dict[str, str] = {}
|
||||
source_version: dict[str, str] = {}
|
||||
for hit in context.hits:
|
||||
atom = hit.atom
|
||||
source_kind[atom.source_id] = str(atom.metadata.get("source_kind") or atom.modality.value)
|
||||
version = atom.metadata.get("version")
|
||||
if version is not None:
|
||||
source_version[atom.source_id] = str(version)
|
||||
for rule in self.RULES:
|
||||
for match in rule.pattern.finditer(atom.text):
|
||||
scope = (match.groupdict().get("scope") or "").strip().lower().rstrip(".")
|
||||
key = f"{scope}.{rule.name}" if scope else rule.name
|
||||
normalized = normalize_design_value(match.group("value"), rule.value_kind)
|
||||
if not normalized:
|
||||
continue
|
||||
observations.setdefault(key, {}).setdefault(normalized, set()).add(atom.source_id)
|
||||
raw_display.setdefault(key, {})[normalized] = match.group("value").strip()
|
||||
category_of[key] = rule.category
|
||||
notes: list[str] = []
|
||||
for key in sorted(observations):
|
||||
value_sources = observations[key]
|
||||
sources = set().union(*value_sources.values())
|
||||
if len(value_sources) > 1 and len(sources) > 1:
|
||||
category = self._classify(category_of[key], sources, source_kind, source_version)
|
||||
values = ", ".join(raw_display[key][value] for value in sorted(value_sources))
|
||||
notes.append(f"{key} {category}: {values}")
|
||||
return notes
|
||||
|
||||
@staticmethod
|
||||
def _classify(
|
||||
baseline: str,
|
||||
sources: set[str],
|
||||
source_kind: dict[str, str],
|
||||
source_version: dict[str, str],
|
||||
) -> str:
|
||||
kinds = {source_kind.get(source, "") for source in sources}
|
||||
design_kinds = {"figma", "spec", "design", "ui_screenshot", "image", "pdf_page", "pdf_region"}
|
||||
code_kinds = {"code", "css"}
|
||||
if kinds & design_kinds and kinds & code_kinds:
|
||||
return "visual-code drift"
|
||||
versions = {source_version[source] for source in sources if source in source_version}
|
||||
if len(versions) > 1:
|
||||
return "temporal drift"
|
||||
return baseline
|
||||
|
||||
|
||||
def normalize_design_value(value: str, value_kind: str) -> str:
|
||||
cleaned = value.strip()
|
||||
if not cleaned:
|
||||
return ""
|
||||
if value_kind == "length":
|
||||
try:
|
||||
return f"{float(cleaned):g}"
|
||||
except ValueError:
|
||||
return ""
|
||||
if value_kind == "color":
|
||||
hex_value = cleaned.lower().lstrip("#")
|
||||
if len(hex_value) in {3, 4}:
|
||||
hex_value = "".join(component * 2 for component in hex_value)
|
||||
return f"#{hex_value}"
|
||||
return " ".join(cleaned.lower().split())
|
||||
@@ -0,0 +1,18 @@
|
||||
services:
|
||||
ely-eye:
|
||||
build: .
|
||||
ports:
|
||||
- "8765:8765"
|
||||
volumes:
|
||||
- ./.ely_eye:/data/ely-eye
|
||||
environment:
|
||||
ELY_EYE_HOME: /data/ely-eye
|
||||
ELY_EYE_RUNTIME_BACKEND: sglang
|
||||
ELY_EYE_SGLANG_BASE_URL: http://host.docker.internal:8000/v1
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: 1
|
||||
capabilities: [gpu]
|
||||
@@ -0,0 +1,32 @@
|
||||
import js from '@eslint/js';
|
||||
import globals from 'globals';
|
||||
import reactHooks from 'eslint-plugin-react-hooks';
|
||||
import reactRefresh from 'eslint-plugin-react-refresh';
|
||||
import tseslint from 'typescript-eslint';
|
||||
|
||||
export default [
|
||||
{ ignores: ['dist'] },
|
||||
...tseslint.configs.recommended,
|
||||
{
|
||||
files: ['**/*.{ts,tsx}'],
|
||||
languageOptions: {
|
||||
ecmaVersion: 2022,
|
||||
globals: globals.browser,
|
||||
parserOptions: {
|
||||
ecmaVersion: 'latest',
|
||||
ecmaFeatures: { jsx: true },
|
||||
sourceType: 'module',
|
||||
},
|
||||
},
|
||||
plugins: {
|
||||
'react-hooks': reactHooks,
|
||||
'react-refresh': reactRefresh,
|
||||
},
|
||||
rules: {
|
||||
...js.configs.recommended.rules,
|
||||
...reactHooks.configs.recommended.rules,
|
||||
'no-undef': 'off',
|
||||
'react-refresh/only-export-components': ['warn', { allowConstantExport: true }],
|
||||
},
|
||||
},
|
||||
];
|
||||
@@ -0,0 +1,12 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Ely-Eye</title>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="/src/main.tsx"></script>
|
||||
</body>
|
||||
</html>
|
||||
Generated
+3019
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"name": "ely-eye-frontend",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite --host 127.0.0.1 --port 5173",
|
||||
"build": "tsc -b && vite build",
|
||||
"preview": "vite preview --host 127.0.0.1 --port 5173",
|
||||
"lint": "eslint .",
|
||||
"verify:ui": "node scripts/verify-ui.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@vitejs/plugin-react": "^6.0.2",
|
||||
"lucide-react": "^0.562.0",
|
||||
"react": "^19.2.0",
|
||||
"react-dom": "^19.2.0",
|
||||
"typescript": "^5.9.0",
|
||||
"vite": "^8.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/js": "^9.39.0",
|
||||
"@types/node": "^24.10.0",
|
||||
"@types/react": "^19.2.0",
|
||||
"@types/react-dom": "^19.2.0",
|
||||
"eslint": "^9.39.0",
|
||||
"eslint-plugin-react-hooks": "^7.0.1",
|
||||
"eslint-plugin-react-refresh": "^0.4.24",
|
||||
"globals": "^16.5.0",
|
||||
"playwright": "^1.60.0",
|
||||
"typescript-eslint": "^8.50.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import { chromium } from 'playwright';
|
||||
import { mkdir } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
|
||||
const root = path.resolve(process.cwd(), '..');
|
||||
const outputDir = path.join(root, '.ely_eye', 'proofs');
|
||||
await mkdir(outputDir, { recursive: true });
|
||||
|
||||
const browser = await chromium.launch();
|
||||
const page = await browser.newPage({ viewport: { width: 1440, height: 1100 } });
|
||||
const consoleErrors = [];
|
||||
page.on('console', (message) => {
|
||||
if (message.type() === 'error') {
|
||||
consoleErrors.push(message.text());
|
||||
}
|
||||
});
|
||||
|
||||
await page.goto('http://127.0.0.1:5173/', { waitUntil: 'networkidle' });
|
||||
await page.getByRole('heading', { name: 'Local VLM Context OS' }).waitFor();
|
||||
await page.getByLabel('system status').getByText('Qwen/Qwen3.5-9B').waitFor();
|
||||
await page.getByRole('heading', { name: 'Adapter Matrix' }).waitFor();
|
||||
await page.getByRole('heading', { name: 'PRD Proof Suite' }).waitFor();
|
||||
await page.getByText(/\d+\/\d+ passed/).waitFor();
|
||||
await page.screenshot({ path: path.join(outputDir, 'ely-eye-dashboard.png'), fullPage: true });
|
||||
const status = await page.locator('.kpi-grid').innerText();
|
||||
await browser.close();
|
||||
|
||||
if (consoleErrors.length > 0) {
|
||||
throw new Error(consoleErrors.join('\n'));
|
||||
}
|
||||
|
||||
console.log(JSON.stringify({ screenshot: path.join(outputDir, 'ely-eye-dashboard.png'), status }, null, 2));
|
||||
@@ -0,0 +1,517 @@
|
||||
import {
|
||||
Activity,
|
||||
Archive,
|
||||
Braces,
|
||||
CheckCircle,
|
||||
Cpu,
|
||||
Database,
|
||||
FileSearch,
|
||||
Gauge,
|
||||
GitBranch,
|
||||
Image as ImageIcon,
|
||||
Play,
|
||||
RefreshCw,
|
||||
Send,
|
||||
Server,
|
||||
ShieldCheck,
|
||||
Upload,
|
||||
} from 'lucide-react';
|
||||
import { FormEvent, ReactNode, useEffect, useMemo, useState } from 'react';
|
||||
import {
|
||||
askQuestion,
|
||||
getAdapters,
|
||||
getLatestProofSuite,
|
||||
getMemoryMap,
|
||||
getStatus,
|
||||
ingestPath,
|
||||
runHashHop,
|
||||
runProofSuite,
|
||||
uploadFiles,
|
||||
} from './api';
|
||||
import type {
|
||||
AdapterStatus,
|
||||
AnswerResponse,
|
||||
HashHopProof,
|
||||
IngestResponse,
|
||||
MemoryMap,
|
||||
ProofSuiteReport,
|
||||
RuntimeProfile,
|
||||
SystemStatus,
|
||||
} from './types';
|
||||
|
||||
const profiles: Array<{ id: RuntimeProfile; label: string }> = [
|
||||
{ id: 'live_demo', label: 'Live Demo' },
|
||||
{ id: 'extreme_context', label: 'Extreme' },
|
||||
{ id: 'library_100m', label: '100M Library' },
|
||||
{ id: 'research_theater', label: 'Research' },
|
||||
];
|
||||
|
||||
export function App() {
|
||||
const [status, setStatus] = useState<SystemStatus | null>(null);
|
||||
const [memoryMap, setMemoryMap] = useState<MemoryMap | null>(null);
|
||||
const [adapters, setAdapters] = useState<AdapterStatus[]>([]);
|
||||
const [profile, setProfile] = useState<RuntimeProfile>('library_100m');
|
||||
const [path, setPath] = useState('');
|
||||
const [cartridgeName, setCartridgeName] = useState('ely-eye-cartridge');
|
||||
const [files, setFiles] = useState<File[]>([]);
|
||||
const [question, setQuestion] = useState('');
|
||||
const [answer, setAnswer] = useState<AnswerResponse | null>(null);
|
||||
const [hashHop, setHashHop] = useState<HashHopProof | null>(null);
|
||||
const [proofSuite, setProofSuite] = useState<ProofSuiteReport | null>(null);
|
||||
const [lastIngest, setLastIngest] = useState<IngestResponse | null>(null);
|
||||
const [busy, setBusy] = useState<string | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
|
||||
const refresh = async () => {
|
||||
setError(null);
|
||||
const [nextStatus, nextMap, nextAdapters, nextProofSuite] = await Promise.all([
|
||||
getStatus(),
|
||||
getMemoryMap(),
|
||||
getAdapters(),
|
||||
getLatestProofSuite().catch(() => null),
|
||||
]);
|
||||
setStatus(nextStatus);
|
||||
setMemoryMap(nextMap);
|
||||
setAdapters(nextAdapters);
|
||||
setProofSuite(nextProofSuite);
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false;
|
||||
async function loadInitial() {
|
||||
try {
|
||||
const [nextStatus, nextMap, nextAdapters, nextProofSuite] = await Promise.all([
|
||||
getStatus(),
|
||||
getMemoryMap(),
|
||||
getAdapters(),
|
||||
getLatestProofSuite().catch(() => null),
|
||||
]);
|
||||
if (!cancelled) {
|
||||
setStatus(nextStatus);
|
||||
setMemoryMap(nextMap);
|
||||
setAdapters(nextAdapters);
|
||||
setProofSuite(nextProofSuite);
|
||||
}
|
||||
} catch (err) {
|
||||
if (!cancelled) {
|
||||
setError(errorMessage(err));
|
||||
}
|
||||
}
|
||||
}
|
||||
void loadInitial();
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, []);
|
||||
|
||||
const memoryLayers = useMemo(() => status?.memory.layers ?? [], [status]);
|
||||
const boundAdapters = useMemo(() => adapters.filter((adapter) => adapter.cartridge_bound).length, [adapters]);
|
||||
const passedProofChecks = useMemo(
|
||||
() => proofSuite?.checks.filter((check) => check.status === 'passed').length ?? 0,
|
||||
[proofSuite],
|
||||
);
|
||||
|
||||
async function submitPath(event: FormEvent) {
|
||||
event.preventDefault();
|
||||
setBusy('ingest-path');
|
||||
setError(null);
|
||||
try {
|
||||
const result = await ingestPath(path, cartridgeName);
|
||||
setLastIngest(result);
|
||||
await refresh();
|
||||
} catch (err) {
|
||||
setError(errorMessage(err));
|
||||
} finally {
|
||||
setBusy(null);
|
||||
}
|
||||
}
|
||||
|
||||
async function submitUpload(event: FormEvent) {
|
||||
event.preventDefault();
|
||||
setBusy('upload');
|
||||
setError(null);
|
||||
try {
|
||||
const result = await uploadFiles(files, cartridgeName);
|
||||
setLastIngest(result);
|
||||
await refresh();
|
||||
} catch (err) {
|
||||
setError(errorMessage(err));
|
||||
} finally {
|
||||
setBusy(null);
|
||||
}
|
||||
}
|
||||
|
||||
async function submitQuestion(event: FormEvent) {
|
||||
event.preventDefault();
|
||||
setBusy('chat');
|
||||
setError(null);
|
||||
try {
|
||||
const result = await askQuestion(question, profile);
|
||||
setAnswer(result);
|
||||
await refresh();
|
||||
} catch (err) {
|
||||
setError(errorMessage(err));
|
||||
} finally {
|
||||
setBusy(null);
|
||||
}
|
||||
}
|
||||
|
||||
async function submitHashHop(kind: 'text' | 'visual') {
|
||||
setBusy(`hashhop-${kind}`);
|
||||
setError(null);
|
||||
try {
|
||||
const result = await runHashHop(kind, kind === 'visual' ? 2 : 2, kind === 'visual' ? 262144 : 1010000);
|
||||
setHashHop(result);
|
||||
await refresh();
|
||||
} catch (err) {
|
||||
setError(errorMessage(err));
|
||||
} finally {
|
||||
setBusy(null);
|
||||
}
|
||||
}
|
||||
|
||||
async function submitProofSuite() {
|
||||
setBusy('proof-suite');
|
||||
setError(null);
|
||||
try {
|
||||
const cartridgeId = memoryMap?.cartridges[0]?.cartridge_id;
|
||||
const result = await runProofSuite(cartridgeId);
|
||||
setProofSuite(result);
|
||||
await refresh();
|
||||
} catch (err) {
|
||||
setError(errorMessage(err));
|
||||
} finally {
|
||||
setBusy(null);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<main className="app-shell">
|
||||
<header className="topbar">
|
||||
<div>
|
||||
<p className="eyebrow">Ely-Eye v4.0</p>
|
||||
<h1>Local VLM Context OS</h1>
|
||||
</div>
|
||||
<button className="icon-button" title="Refresh status" onClick={() => refresh().catch((err) => setError(errorMessage(err)))}>
|
||||
<RefreshCw size={18} />
|
||||
</button>
|
||||
</header>
|
||||
|
||||
{error && <div className="error-line">{error}</div>}
|
||||
|
||||
<section className="kpi-grid" aria-label="system status">
|
||||
<Kpi icon={<Server size={18} />} label="Runtime" value={status?.runtime.available ? 'ready' : 'offline'} detail={formatRuntimeDetail(status)} />
|
||||
<Kpi icon={<Gauge size={18} />} label="GPU" value={status?.runtime.gpu_name ?? 'unknown'} detail={formatVram(status)} />
|
||||
<Kpi icon={<Database size={18} />} label="Library" value={formatNumber(status?.memory.library_token_equivalent ?? 0)} detail="token eq" />
|
||||
<Kpi icon={<Archive size={18} />} label="Cartridges" value={String(status?.memory.cartridges ?? 0)} detail={`${boundAdapters} adapters bound`} />
|
||||
</section>
|
||||
|
||||
<section className="workspace-grid">
|
||||
<Panel title="Input Stream" icon={<Upload size={17} />}>
|
||||
<form className="stack" onSubmit={submitPath}>
|
||||
<label>
|
||||
Cartridge
|
||||
<input value={cartridgeName} onChange={(event) => setCartridgeName(event.target.value)} />
|
||||
</label>
|
||||
<label>
|
||||
Local path
|
||||
<input value={path} onChange={(event) => setPath(event.target.value)} />
|
||||
</label>
|
||||
<button className="action-button" disabled={!path || busy === 'ingest-path'}>
|
||||
<FileSearch size={16} />
|
||||
Ingest Path
|
||||
</button>
|
||||
</form>
|
||||
<form className="stack" onSubmit={submitUpload}>
|
||||
<label>
|
||||
Files
|
||||
<input type="file" multiple onChange={(event) => setFiles(Array.from(event.target.files ?? []))} />
|
||||
</label>
|
||||
<button className="action-button secondary" disabled={files.length === 0 || busy === 'upload'}>
|
||||
<Upload size={16} />
|
||||
Upload Files
|
||||
</button>
|
||||
</form>
|
||||
{lastIngest && (
|
||||
<div className="metric-strip">
|
||||
<span>{lastIngest.source_count} sources</span>
|
||||
<span>{lastIngest.atom_count} atoms</span>
|
||||
<span>{formatNumber(lastIngest.token_equivalent)} tokens</span>
|
||||
</div>
|
||||
)}
|
||||
</Panel>
|
||||
|
||||
<Panel title="Context Compiler" icon={<Braces size={17} />}>
|
||||
<div className="segments">
|
||||
{profiles.map((item) => (
|
||||
<button
|
||||
key={item.id}
|
||||
className={profile === item.id ? 'segment active' : 'segment'}
|
||||
onClick={() => setProfile(item.id)}
|
||||
type="button"
|
||||
>
|
||||
{item.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
<form className="stack" onSubmit={submitQuestion}>
|
||||
<textarea value={question} onChange={(event) => setQuestion(event.target.value)} rows={6} />
|
||||
<button className="action-button" disabled={!question || busy === 'chat'}>
|
||||
<Send size={16} />
|
||||
Ask
|
||||
</button>
|
||||
</form>
|
||||
{answer && (
|
||||
<div className="answer-box">
|
||||
<p>{answer.answer}</p>
|
||||
<div className="metric-strip">
|
||||
<span>{answer.context.hits.length} hits</span>
|
||||
<span>{Math.round(answer.verifier.citation_accuracy * 100)}% citations</span>
|
||||
<span>{answer.runtime_backend}</span>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</Panel>
|
||||
|
||||
<Panel title="Memory OS" icon={<Database size={17} />}>
|
||||
<div className="layer-list">
|
||||
{memoryLayers.map((layer) => (
|
||||
<div className="layer-row" key={layer.layer}>
|
||||
<span>{layer.layer}</span>
|
||||
<strong>{layer.entries}</strong>
|
||||
<small>{layer.hit_count} hit / {layer.miss_count} miss</small>
|
||||
</div>
|
||||
))}
|
||||
{memoryLayers.length === 0 && <div className="quiet-line">No cache events</div>}
|
||||
</div>
|
||||
<div className="atom-list">
|
||||
{(memoryMap?.atoms ?? []).slice(0, 8).map((atom) => (
|
||||
<div className="atom-row" key={atom.atom_id}>
|
||||
<span>{atom.modality}</span>
|
||||
<strong>{atom.atom_id}</strong>
|
||||
<small>{atom.source}</small>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</Panel>
|
||||
|
||||
<Panel title="Adapter Matrix" icon={<Cpu size={17} />}>
|
||||
<div className="adapter-list">
|
||||
{adapters.map((adapter) => (
|
||||
<div className="adapter-row" key={adapter.adapter_id}>
|
||||
<header>
|
||||
<span>{adapter.kind.replaceAll('_', '-')}</span>
|
||||
{adapter.cartridge_bound && <CheckCircle size={16} aria-label="bound" />}
|
||||
</header>
|
||||
<strong>{adapter.base_model}</strong>
|
||||
<div className="adapter-metrics">
|
||||
<small>{adapter.training_method}</small>
|
||||
<small>{adapter.precision}</small>
|
||||
<small title={adapter.unsloth_model_type ?? adapter.framework}>{formatFramework(adapter.framework)}</small>
|
||||
<small title={adapter.optimizer_family ?? adapter.optimizer ?? undefined}>{formatOptimizer(adapter)}</small>
|
||||
<small title={adapter.gradient_checkpointing_mode ?? undefined}>{formatCheckpointing(adapter)}</small>
|
||||
<small title={adapter.triton_version ?? undefined}>{adapter.triton_version ? `triton ${adapter.triton_version}` : 'triton pending'}</small>
|
||||
<small>{formatBytes(adapter.weight_bytes)}</small>
|
||||
<small>{formatNumber(adapter.trainable_params)} params</small>
|
||||
<small>{adapter.final_loss?.toFixed(3) ?? 'loss pending'}</small>
|
||||
<small>{adapter.training_trace_steps}/{adapter.max_steps} trace</small>
|
||||
<small>{formatNumber(adapter.optimizer_update_count)} updates</small>
|
||||
<small>{adapter.loss_delta == null ? 'loss delta pending' : `loss delta ${adapter.loss_delta.toFixed(3)}`}</small>
|
||||
<small>{adapter.bf16 ? 'cuda bf16' : 'precision pending'}</small>
|
||||
<small>{formatNumber(adapter.adapter_lora_tensor_count)} LoRA tensors</small>
|
||||
<small>{formatNumber(adapter.adapter_nonzero_elements)} nonzero</small>
|
||||
<small>{adapter.adapter_weights_finite ? 'finite weights' : 'weights pending'}</small>
|
||||
<small>{adapter.adapter_weight_sha256 ? `sha ${shortHash(adapter.adapter_weight_sha256)}` : 'sha pending'}</small>
|
||||
<small>{formatDuration(adapter.training_wall_seconds)}</small>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
{adapters.length === 0 && <div className="quiet-line">No adapters found</div>}
|
||||
</div>
|
||||
</Panel>
|
||||
|
||||
<Panel title="PRD Proof Suite" icon={<ShieldCheck size={17} />}>
|
||||
<div className="proof-suite-header">
|
||||
<button className="action-button secondary" onClick={submitProofSuite} disabled={busy === 'proof-suite'}>
|
||||
<Play size={16} />
|
||||
Run Suite
|
||||
</button>
|
||||
<div className={proofSuite?.status === 'passed' ? 'proof-pill passed' : 'proof-pill'}>
|
||||
{proofSuite ? `${passedProofChecks}/${proofSuite.checks.length} passed` : 'pending'}
|
||||
</div>
|
||||
</div>
|
||||
{proofSuite && (
|
||||
<div className="proof-list">
|
||||
{proofSuite.checks.map((check) => (
|
||||
<article className="proof-row" key={check.name}>
|
||||
<header>
|
||||
<strong>{check.name}</strong>
|
||||
<span className={check.status === 'passed' ? 'proof-status passed' : 'proof-status'}>
|
||||
{check.status}
|
||||
</span>
|
||||
</header>
|
||||
<p>{check.detail}</p>
|
||||
<small>{check.evidence[0]}</small>
|
||||
</article>
|
||||
))}
|
||||
<div className="proof-box">
|
||||
<strong>{proofSuite.proof_id}</strong>
|
||||
<span>{proofSuite.status}</span>
|
||||
<small>{proofSuite.cartridge_id}</small>
|
||||
<small>{proofSuite.dna_after ?? proofSuite.dna_before}</small>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{!proofSuite && <div className="quiet-line">Proof suite has not run yet</div>}
|
||||
</Panel>
|
||||
|
||||
<Panel title="Evidence Viewer" icon={<ImageIcon size={17} />}>
|
||||
<div className="evidence-list">
|
||||
{(answer?.context.hits ?? []).slice(0, 8).map((hit) => (
|
||||
<article key={hit.atom.atom_id} className="evidence-row">
|
||||
<header>
|
||||
<span>{hit.atom.modality}</span>
|
||||
<strong>{hit.final_score.toFixed(3)}</strong>
|
||||
</header>
|
||||
<p>{hit.atom.text.slice(0, 280)}</p>
|
||||
<small>{hit.atom.source}</small>
|
||||
</article>
|
||||
))}
|
||||
{!answer && <div className="quiet-line">Evidence appears after a compiled answer</div>}
|
||||
</div>
|
||||
</Panel>
|
||||
|
||||
<Panel title="Visual HashHop Arena" icon={<GitBranch size={17} />}>
|
||||
<div className="button-row">
|
||||
<button className="action-button secondary" onClick={() => submitHashHop('visual')} disabled={busy === 'hashhop-visual'}>
|
||||
<Play size={16} />
|
||||
Visual 2-hop
|
||||
</button>
|
||||
<button className="action-button secondary" onClick={() => submitHashHop('text')} disabled={busy === 'hashhop-text'}>
|
||||
<Play size={16} />
|
||||
Text 2-hop
|
||||
</button>
|
||||
</div>
|
||||
{hashHop && (
|
||||
<div className="proof-box">
|
||||
<strong>{hashHop.proof_id}</strong>
|
||||
<span>{hashHop.kind}</span>
|
||||
<span>{hashHop.hops} hops</span>
|
||||
<span>{formatNumber(hashHop.token_equivalent)} token eq</span>
|
||||
<small>{hashHop.expected_target_id}</small>
|
||||
</div>
|
||||
)}
|
||||
</Panel>
|
||||
|
||||
<Panel title="Cache Trace" icon={<Activity size={17} />}>
|
||||
<div className="trace-list">
|
||||
{memoryLayers.map((layer) => (
|
||||
<div key={layer.layer} className="trace-row">
|
||||
<span>{layer.layer}</span>
|
||||
<meter min={0} max={Math.max(1, layer.hit_count + layer.miss_count)} value={layer.hit_count} />
|
||||
<small>{formatBytes(layer.bytes_estimate)}</small>
|
||||
</div>
|
||||
))}
|
||||
{memoryLayers.length === 0 && <div className="quiet-line">Waiting for retrieval</div>}
|
||||
</div>
|
||||
</Panel>
|
||||
</section>
|
||||
</main>
|
||||
);
|
||||
}
|
||||
|
||||
function Panel({ title, icon, children }: { title: string; icon: ReactNode; children: ReactNode }) {
|
||||
return (
|
||||
<section className="panel">
|
||||
<header className="panel-header">
|
||||
<span className="panel-icon">{icon}</span>
|
||||
<h2>{title}</h2>
|
||||
</header>
|
||||
{children}
|
||||
</section>
|
||||
);
|
||||
}
|
||||
|
||||
function Kpi({ icon, label, value, detail }: { icon: ReactNode; label: string; value: string; detail: string }) {
|
||||
return (
|
||||
<section className="kpi">
|
||||
<span className="panel-icon">{icon}</span>
|
||||
<div>
|
||||
<p>{label}</p>
|
||||
<strong>{value}</strong>
|
||||
<small>{detail}</small>
|
||||
</div>
|
||||
</section>
|
||||
);
|
||||
}
|
||||
|
||||
function errorMessage(err: unknown): string {
|
||||
if (err instanceof Error) {
|
||||
return err.message;
|
||||
}
|
||||
return String(err);
|
||||
}
|
||||
|
||||
function formatNumber(value: number): string {
|
||||
return new Intl.NumberFormat('en-US', { notation: 'compact', maximumFractionDigits: 1 }).format(value);
|
||||
}
|
||||
|
||||
function formatBytes(value: number): string {
|
||||
if (value < 1024) {
|
||||
return `${value} B`;
|
||||
}
|
||||
if (value < 1024 * 1024) {
|
||||
return `${(value / 1024).toFixed(1)} KiB`;
|
||||
}
|
||||
return `${(value / 1024 / 1024).toFixed(1)} MiB`;
|
||||
}
|
||||
|
||||
function formatDuration(value: number | null): string {
|
||||
if (typeof value !== 'number') {
|
||||
return 'time pending';
|
||||
}
|
||||
if (value < 60) {
|
||||
return `${value.toFixed(1)}s`;
|
||||
}
|
||||
return `${Math.round(value / 60)}m`;
|
||||
}
|
||||
|
||||
function shortHash(value: string): string {
|
||||
return value.slice(0, 8);
|
||||
}
|
||||
|
||||
function formatOptimizer(adapter: AdapterStatus): string {
|
||||
if (adapter.optimizer === 'paged_adamw_8bit') {
|
||||
return 'paged 8-bit AdamW';
|
||||
}
|
||||
return adapter.optimizer_family ?? adapter.optimizer ?? 'optimizer pending';
|
||||
}
|
||||
|
||||
function formatFramework(value: string): string {
|
||||
if (value === 'unsloth_patched_transformers_peft') {
|
||||
return 'Unsloth patch + PEFT';
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function formatCheckpointing(adapter: AdapterStatus): string {
|
||||
if (adapter.gradient_checkpointing_mode === 'unsloth') {
|
||||
return 'ckpt unsloth';
|
||||
}
|
||||
return adapter.gradient_checkpointing ? 'ckpt on' : 'ckpt pending';
|
||||
}
|
||||
|
||||
function formatRuntimeDetail(status: SystemStatus | null): string {
|
||||
const model = status?.runtime.model_id ?? 'Qwen/Qwen3.5-9B';
|
||||
const adapter = status?.runtime.adapter_kind?.replaceAll('_', '-');
|
||||
return adapter ? `${model} · ${adapter}` : model;
|
||||
}
|
||||
|
||||
function formatVram(status: SystemStatus | null): string {
|
||||
const total = status?.runtime.vram_total_mb;
|
||||
const used = status?.runtime.vram_used_mb;
|
||||
if (typeof total === 'number' && typeof used === 'number') {
|
||||
return `${used} / ${total} MiB`;
|
||||
}
|
||||
return status?.runtime.cuda ?? 'CUDA';
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
import type {
|
||||
AdapterStatus,
|
||||
AnswerResponse,
|
||||
HashHopProof,
|
||||
IngestResponse,
|
||||
MemoryMap,
|
||||
ProofSuiteReport,
|
||||
RuntimeProfile,
|
||||
SystemStatus,
|
||||
} from './types';
|
||||
|
||||
async function request<T>(url: string, init?: RequestInit): Promise<T> {
|
||||
const response = await fetch(url, init);
|
||||
if (!response.ok) {
|
||||
const payload = await response.json().catch(() => ({ detail: response.statusText }));
|
||||
throw new Error(typeof payload.detail === 'string' ? payload.detail : JSON.stringify(payload.detail));
|
||||
}
|
||||
return (await response.json()) as T;
|
||||
}
|
||||
|
||||
export function getStatus(): Promise<SystemStatus> {
|
||||
return request<SystemStatus>('/api/status');
|
||||
}
|
||||
|
||||
export function getMemoryMap(): Promise<MemoryMap> {
|
||||
return request<MemoryMap>('/api/memory-map');
|
||||
}
|
||||
|
||||
export function getAdapters(): Promise<AdapterStatus[]> {
|
||||
return request<AdapterStatus[]>('/api/adapters');
|
||||
}
|
||||
|
||||
export function ingestPath(path: string, cartridgeName: string): Promise<IngestResponse> {
|
||||
return request<IngestResponse>('/api/ingest/path', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ path, cartridge_name: cartridgeName || null }),
|
||||
});
|
||||
}
|
||||
|
||||
export function uploadFiles(files: File[], cartridgeName: string): Promise<IngestResponse> {
|
||||
const form = new FormData();
|
||||
for (const file of files) {
|
||||
form.append('files', file);
|
||||
}
|
||||
const query = new URLSearchParams();
|
||||
if (cartridgeName) {
|
||||
query.set('cartridge_name', cartridgeName);
|
||||
}
|
||||
return request<IngestResponse>(`/api/ingest/upload?${query.toString()}`, {
|
||||
method: 'POST',
|
||||
body: form,
|
||||
});
|
||||
}
|
||||
|
||||
export function askQuestion(question: string, profile: RuntimeProfile): Promise<AnswerResponse> {
|
||||
return request<AnswerResponse>('/api/chat', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ question, profile }),
|
||||
});
|
||||
}
|
||||
|
||||
export function runHashHop(kind: 'text' | 'visual', hops: number, tokenEquivalent: number): Promise<HashHopProof> {
|
||||
return request<HashHopProof>('/api/hashhop', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ kind, hops, token_equivalent: tokenEquivalent }),
|
||||
});
|
||||
}
|
||||
|
||||
export function runProofSuite(cartridgeId?: string): Promise<ProofSuiteReport> {
|
||||
return request<ProofSuiteReport>('/api/proof-suite', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ cartridge_id: cartridgeId ?? null }),
|
||||
});
|
||||
}
|
||||
|
||||
export function getLatestProofSuite(): Promise<ProofSuiteReport> {
|
||||
return request<ProofSuiteReport>('/api/proof-suite/latest');
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
import React from 'react';
|
||||
import ReactDOM from 'react-dom/client';
|
||||
import { App } from './App';
|
||||
import './styles.css';
|
||||
|
||||
ReactDOM.createRoot(document.getElementById('root') as HTMLElement).render(
|
||||
<React.StrictMode>
|
||||
<App />
|
||||
</React.StrictMode>,
|
||||
);
|
||||
@@ -0,0 +1,444 @@
|
||||
:root {
|
||||
font-family:
|
||||
Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
||||
color: #17201b;
|
||||
background: #f4f3ee;
|
||||
font-synthesis: none;
|
||||
text-rendering: optimizeLegibility;
|
||||
}
|
||||
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
min-width: 320px;
|
||||
min-height: 100vh;
|
||||
}
|
||||
|
||||
button,
|
||||
input,
|
||||
textarea {
|
||||
font: inherit;
|
||||
}
|
||||
|
||||
button {
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.app-shell {
|
||||
min-height: 100vh;
|
||||
padding: 22px;
|
||||
}
|
||||
|
||||
.topbar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 16px;
|
||||
max-width: 1480px;
|
||||
margin: 0 auto 18px;
|
||||
border-bottom: 1px solid #c9d2c4;
|
||||
padding-bottom: 16px;
|
||||
}
|
||||
|
||||
.eyebrow {
|
||||
margin: 0 0 3px;
|
||||
color: #476a58;
|
||||
font-size: 0.78rem;
|
||||
font-weight: 700;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
h1,
|
||||
h2,
|
||||
p {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-size: clamp(1.7rem, 3vw, 2.7rem);
|
||||
font-weight: 760;
|
||||
}
|
||||
|
||||
.icon-button,
|
||||
.action-button,
|
||||
.segment {
|
||||
border: 1px solid #8fa091;
|
||||
background: #fffdf7;
|
||||
color: #17201b;
|
||||
border-radius: 7px;
|
||||
}
|
||||
|
||||
.icon-button {
|
||||
width: 40px;
|
||||
height: 40px;
|
||||
display: grid;
|
||||
place-items: center;
|
||||
}
|
||||
|
||||
.error-line {
|
||||
max-width: 1480px;
|
||||
margin: 0 auto 14px;
|
||||
border: 1px solid #be6a5b;
|
||||
background: #fff5f1;
|
||||
color: #7f2e21;
|
||||
border-radius: 7px;
|
||||
padding: 10px 12px;
|
||||
}
|
||||
|
||||
.kpi-grid,
|
||||
.workspace-grid {
|
||||
max-width: 1480px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
.kpi-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(4, minmax(0, 1fr));
|
||||
gap: 12px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.kpi,
|
||||
.panel {
|
||||
border: 1px solid #c9d2c4;
|
||||
background: #fffdf7;
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
.kpi {
|
||||
display: grid;
|
||||
grid-template-columns: 34px minmax(0, 1fr);
|
||||
gap: 10px;
|
||||
padding: 14px;
|
||||
min-height: 104px;
|
||||
}
|
||||
|
||||
.kpi p,
|
||||
.kpi small,
|
||||
.atom-row small,
|
||||
.trace-row small,
|
||||
.layer-row small,
|
||||
.evidence-row small,
|
||||
.proof-box small {
|
||||
color: #607166;
|
||||
}
|
||||
|
||||
.kpi strong {
|
||||
display: block;
|
||||
margin: 4px 0;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.workspace-grid {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1.25fr 1fr;
|
||||
gap: 12px;
|
||||
align-items: start;
|
||||
}
|
||||
|
||||
.panel {
|
||||
padding: 14px;
|
||||
min-height: 260px;
|
||||
}
|
||||
|
||||
.panel-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.panel-header h2 {
|
||||
font-size: 1rem;
|
||||
}
|
||||
|
||||
.panel-icon {
|
||||
width: 28px;
|
||||
height: 28px;
|
||||
display: grid;
|
||||
place-items: center;
|
||||
border-radius: 6px;
|
||||
color: #1f5a45;
|
||||
background: #dfe8dc;
|
||||
}
|
||||
|
||||
.stack {
|
||||
display: grid;
|
||||
gap: 10px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
label {
|
||||
display: grid;
|
||||
gap: 5px;
|
||||
color: #46554b;
|
||||
font-size: 0.86rem;
|
||||
font-weight: 650;
|
||||
}
|
||||
|
||||
input,
|
||||
textarea {
|
||||
width: 100%;
|
||||
border: 1px solid #aebba9;
|
||||
border-radius: 7px;
|
||||
padding: 9px 10px;
|
||||
background: #fff;
|
||||
color: #17201b;
|
||||
}
|
||||
|
||||
textarea {
|
||||
resize: vertical;
|
||||
min-height: 132px;
|
||||
}
|
||||
|
||||
.action-button {
|
||||
min-height: 40px;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: 8px;
|
||||
padding: 8px 12px;
|
||||
font-weight: 720;
|
||||
background: #214f3f;
|
||||
color: #fffdf7;
|
||||
border-color: #214f3f;
|
||||
}
|
||||
|
||||
.action-button.secondary {
|
||||
background: #fffdf7;
|
||||
color: #214f3f;
|
||||
}
|
||||
|
||||
.action-button:disabled {
|
||||
opacity: 0.55;
|
||||
cursor: default;
|
||||
}
|
||||
|
||||
.segments,
|
||||
.button-row,
|
||||
.metric-strip {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.segments {
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.segment {
|
||||
min-height: 34px;
|
||||
padding: 7px 10px;
|
||||
font-weight: 680;
|
||||
}
|
||||
|
||||
.segment.active {
|
||||
background: #214f3f;
|
||||
color: #fffdf7;
|
||||
border-color: #214f3f;
|
||||
}
|
||||
|
||||
.metric-strip span,
|
||||
.proof-box span {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
min-height: 28px;
|
||||
border: 1px solid #cbd7c6;
|
||||
border-radius: 999px;
|
||||
padding: 4px 9px;
|
||||
color: #284338;
|
||||
background: #f5f8f2;
|
||||
font-size: 0.82rem;
|
||||
font-weight: 680;
|
||||
}
|
||||
|
||||
.answer-box,
|
||||
.proof-box,
|
||||
.quiet-line {
|
||||
border: 1px solid #d5c8ac;
|
||||
background: #fff9ea;
|
||||
border-radius: 7px;
|
||||
padding: 10px;
|
||||
}
|
||||
|
||||
.answer-box {
|
||||
display: grid;
|
||||
gap: 10px;
|
||||
}
|
||||
|
||||
.layer-list,
|
||||
.atom-list,
|
||||
.evidence-list,
|
||||
.trace-list,
|
||||
.adapter-list,
|
||||
.proof-list,
|
||||
.proof-box {
|
||||
display: grid;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.layer-row,
|
||||
.atom-row,
|
||||
.trace-row,
|
||||
.adapter-row,
|
||||
.proof-row,
|
||||
.evidence-row {
|
||||
border: 1px solid #d7ded2;
|
||||
border-radius: 7px;
|
||||
background: #ffffff;
|
||||
padding: 9px;
|
||||
}
|
||||
|
||||
.layer-row,
|
||||
.trace-row {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(0, 1fr) auto;
|
||||
gap: 4px 10px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.layer-row small,
|
||||
.trace-row small {
|
||||
grid-column: 1 / -1;
|
||||
}
|
||||
|
||||
.atom-row {
|
||||
display: grid;
|
||||
gap: 4px;
|
||||
}
|
||||
|
||||
.atom-row span,
|
||||
.adapter-row span,
|
||||
.evidence-row span {
|
||||
color: #7a4e29;
|
||||
font-size: 0.78rem;
|
||||
font-weight: 760;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
.atom-row strong,
|
||||
.adapter-row strong,
|
||||
.evidence-row p,
|
||||
.evidence-row small {
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.adapter-row {
|
||||
display: grid;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.adapter-row header,
|
||||
.proof-row header,
|
||||
.evidence-row header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
margin-bottom: 7px;
|
||||
}
|
||||
|
||||
.adapter-row header,
|
||||
.proof-row header {
|
||||
align-items: center;
|
||||
margin-bottom: 0;
|
||||
}
|
||||
|
||||
.adapter-row svg {
|
||||
color: #1f5a45;
|
||||
flex: 0 0 auto;
|
||||
}
|
||||
|
||||
.adapter-metrics {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.adapter-metrics small {
|
||||
border: 1px solid #d7ded2;
|
||||
border-radius: 6px;
|
||||
background: #f5f8f2;
|
||||
color: #284338;
|
||||
padding: 5px 6px;
|
||||
font-size: 0.76rem;
|
||||
font-weight: 650;
|
||||
overflow-wrap: break-word;
|
||||
}
|
||||
|
||||
.proof-suite-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 10px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.proof-pill,
|
||||
.proof-status {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
border: 1px solid #c9b6a0;
|
||||
border-radius: 999px;
|
||||
background: #fff5f1;
|
||||
color: #7f2e21;
|
||||
font-size: 0.76rem;
|
||||
font-weight: 760;
|
||||
min-height: 28px;
|
||||
padding: 4px 9px;
|
||||
text-transform: uppercase;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.proof-pill.passed,
|
||||
.proof-status.passed {
|
||||
border-color: #b8c7ae;
|
||||
background: #f5f8f2;
|
||||
color: #214f3f;
|
||||
}
|
||||
|
||||
.proof-row {
|
||||
display: grid;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.proof-row p,
|
||||
.proof-row small {
|
||||
color: #607166;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.evidence-row p {
|
||||
margin-bottom: 8px;
|
||||
line-height: 1.45;
|
||||
}
|
||||
|
||||
meter {
|
||||
width: 100%;
|
||||
height: 8px;
|
||||
}
|
||||
|
||||
@media (max-width: 1080px) {
|
||||
.kpi-grid,
|
||||
.workspace-grid {
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 720px) {
|
||||
.app-shell {
|
||||
padding: 14px;
|
||||
}
|
||||
|
||||
.kpi-grid,
|
||||
.workspace-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.topbar {
|
||||
align-items: flex-start;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,188 @@
|
||||
export type RuntimeProfile = 'live_demo' | 'extreme_context' | 'library_100m' | 'research_theater';
|
||||
|
||||
export interface RuntimeStatus {
|
||||
backend: string;
|
||||
model_id: string;
|
||||
available: boolean;
|
||||
detail: string;
|
||||
adapter_path: string | null;
|
||||
adapter_kind: string | null;
|
||||
adapter_id: string | null;
|
||||
adapter_sha256: string | null;
|
||||
gpu_name: string | null;
|
||||
cuda: string | null;
|
||||
vram_total_mb: number | null;
|
||||
vram_used_mb: number | null;
|
||||
}
|
||||
|
||||
export interface CacheLayerStatus {
|
||||
layer: string;
|
||||
entries: number;
|
||||
bytes_estimate: number;
|
||||
hit_count: number;
|
||||
miss_count: number;
|
||||
last_event_at: string | null;
|
||||
}
|
||||
|
||||
export interface MemoryStatus {
|
||||
active_atoms: number;
|
||||
active_token_equivalent: number;
|
||||
cartridges: number;
|
||||
library_token_equivalent: number;
|
||||
layers: CacheLayerStatus[];
|
||||
}
|
||||
|
||||
export interface SystemStatus {
|
||||
app: string;
|
||||
version: string;
|
||||
runtime: RuntimeStatus;
|
||||
memory: MemoryStatus;
|
||||
profiles: Record<string, Record<string, unknown>>;
|
||||
data_home: string;
|
||||
}
|
||||
|
||||
export interface AdapterStatus {
|
||||
name: string;
|
||||
kind: string;
|
||||
proof_version: number;
|
||||
training_method: string;
|
||||
precision: string;
|
||||
framework: string;
|
||||
base_model: string;
|
||||
path: string;
|
||||
adapter_id: string;
|
||||
sample_count: number;
|
||||
max_steps: number;
|
||||
trainable_params: number;
|
||||
final_loss: number | null;
|
||||
sha256: string;
|
||||
weight_bytes: number;
|
||||
safetensor_keys: number;
|
||||
cartridge_id: string | null;
|
||||
cartridge_bound: boolean;
|
||||
training_trace_path: string | null;
|
||||
training_trace_sha256: string | null;
|
||||
training_trace_steps: number;
|
||||
training_proof_path: string | null;
|
||||
training_proof_sha256: string | null;
|
||||
optimizer: string | null;
|
||||
optimizer_family: string | null;
|
||||
gradient_checkpointing: boolean;
|
||||
gradient_checkpointing_mode: string | null;
|
||||
autocast_dtype: string | null;
|
||||
unsloth_version: string | null;
|
||||
triton_version: string | null;
|
||||
xformers_version: string | null;
|
||||
unsloth_model_type: string | null;
|
||||
optimizer_update_count: number;
|
||||
max_optimizer_update_l2: number | null;
|
||||
loss_delta: number | null;
|
||||
training_summary_path: string | null;
|
||||
training_wall_seconds: number | null;
|
||||
training_device: string | null;
|
||||
bf16: boolean;
|
||||
torch_version: string | null;
|
||||
cuda_version: string | null;
|
||||
adapter_weight_sha256: string | null;
|
||||
adapter_tensor_count: number;
|
||||
adapter_lora_tensor_count: number;
|
||||
adapter_total_elements: number;
|
||||
adapter_nonzero_elements: number;
|
||||
adapter_max_abs: number | null;
|
||||
adapter_weights_finite: boolean;
|
||||
}
|
||||
|
||||
export interface IngestResponse {
|
||||
source_count: number;
|
||||
atom_count: number;
|
||||
cartridge_id: string | null;
|
||||
token_equivalent: number;
|
||||
}
|
||||
|
||||
export interface LayoutBox {
|
||||
x: number;
|
||||
y: number;
|
||||
w: number;
|
||||
h: number;
|
||||
page?: number | null;
|
||||
frame_second?: number | null;
|
||||
}
|
||||
|
||||
export interface EvidenceAtom {
|
||||
atom_id: string;
|
||||
modality: string;
|
||||
source: string;
|
||||
text: string;
|
||||
image_ref: string | null;
|
||||
layout: LayoutBox | null;
|
||||
token_equivalent: number;
|
||||
}
|
||||
|
||||
export interface RetrievalHit {
|
||||
atom: EvidenceAtom;
|
||||
sparse_score: number;
|
||||
dense_score: number;
|
||||
graph_score: number;
|
||||
final_score: number;
|
||||
}
|
||||
|
||||
export interface AnswerResponse {
|
||||
answer: string;
|
||||
citations: string[];
|
||||
context: {
|
||||
hits: RetrievalHit[];
|
||||
token_equivalent: number;
|
||||
cache_trace_id: string;
|
||||
};
|
||||
verifier: {
|
||||
cited_atom_ids: string[];
|
||||
missing_atom_ids: string[];
|
||||
citation_accuracy: number;
|
||||
contradiction_notes: string[];
|
||||
confidence: number;
|
||||
};
|
||||
runtime_backend: string;
|
||||
}
|
||||
|
||||
export interface HashHopProof {
|
||||
proof_id: string;
|
||||
kind: string;
|
||||
hops: number;
|
||||
token_equivalent: number;
|
||||
query_id: string;
|
||||
expected_target_id: string;
|
||||
model_target_id: string | null;
|
||||
passed: boolean | null;
|
||||
artifacts: string[];
|
||||
created_at: string;
|
||||
}
|
||||
|
||||
export interface ProofCheck {
|
||||
name: string;
|
||||
requirement: string;
|
||||
status: 'passed' | 'failed';
|
||||
evidence: string[];
|
||||
detail: string;
|
||||
}
|
||||
|
||||
export interface ProofSuiteReport {
|
||||
proof_id: string;
|
||||
cartridge_id: string;
|
||||
status: 'passed' | 'failed';
|
||||
created_at: string;
|
||||
checks: ProofCheck[];
|
||||
artifacts: Record<string, string>;
|
||||
dna_before: string | null;
|
||||
dna_after: string | null;
|
||||
}
|
||||
|
||||
export interface MemoryMap {
|
||||
atoms: EvidenceAtom[];
|
||||
layers: CacheLayerStatus[];
|
||||
cartridges: Array<{
|
||||
cartridge_id: string;
|
||||
name: string;
|
||||
token_equivalent: number;
|
||||
dna: string | null;
|
||||
}>;
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"useDefineForClassFields": true,
|
||||
"lib": ["ES2022", "DOM", "DOM.Iterable"],
|
||||
"allowJs": false,
|
||||
"skipLibCheck": true,
|
||||
"esModuleInterop": true,
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"strict": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"noEmit": true,
|
||||
"jsx": "react-jsx"
|
||||
},
|
||||
"include": ["src"],
|
||||
"references": [{ "path": "./tsconfig.node.json" }]
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"composite": true,
|
||||
"target": "ES2022",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"allowSyntheticDefaultImports": true,
|
||||
"lib": ["ES2022"],
|
||||
"types": ["node"],
|
||||
"skipLibCheck": true
|
||||
},
|
||||
"include": ["vite.config.ts"]
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
import { defineConfig } from 'vite';
|
||||
import react from '@vitejs/plugin-react';
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [react()],
|
||||
server: {
|
||||
host: '127.0.0.1',
|
||||
port: 5173,
|
||||
proxy: {
|
||||
'/api': 'http://127.0.0.1:8765',
|
||||
},
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,7 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
.\.venv\Scripts\python.exe .\scripts\download_models.py --model Qwen/Qwen3.5-9B
|
||||
.\.venv\Scripts\python.exe .\scripts\download_models.py --model Qwen/Qwen3.5-0.8B
|
||||
.\.venv\Scripts\python.exe .\scripts\download_models.py --model Qwen/Qwen3-VL-Embedding-8B
|
||||
@@ -0,0 +1,17 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
|
||||
from huggingface_hub import snapshot_download
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model", required=True)
|
||||
args = parser.parse_args()
|
||||
path = snapshot_download(repo_id=args.model, resume_download=True)
|
||||
print(path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,5 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
.\.venv\Scripts\python.exe -m pip install --upgrade --no-cache-dir --index-url https://download.pytorch.org/whl/cu130 torch==2.12.0+cu130 torchvision==0.27.0+cu130
|
||||
.\.venv\Scripts\python.exe -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.version.cuda)"
|
||||
@@ -0,0 +1,21 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
if (-not (Test-Path ".venv")) {
|
||||
python -m venv .venv
|
||||
}
|
||||
|
||||
.\.venv\Scripts\python.exe -m pip install --upgrade pip setuptools wheel
|
||||
.\.venv\Scripts\python.exe -m pip install -e .\backend
|
||||
.\.venv\Scripts\python.exe -m pip install --upgrade --no-cache-dir unsloth unsloth_zoo
|
||||
.\.venv\Scripts\python.exe -m pip install --upgrade --no-cache-dir --index-url https://download.pytorch.org/whl/cu130 torch==2.12.0+cu130 torchvision==0.27.0+cu130
|
||||
.\.venv\Scripts\python.exe -m pip install --upgrade --no-cache-dir transformers==5.9.0 datasets==4.8.5 dill==0.4.1 multiprocess==0.70.19 fsspec==2026.2.0 safetensors==0.8.0rc0
|
||||
|
||||
Push-Location .\frontend
|
||||
npm install
|
||||
Pop-Location
|
||||
|
||||
Copy-Item .env.example .env -ErrorAction SilentlyContinue
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe status
|
||||
@@ -0,0 +1,23 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the KTransformers Linux runtime."
|
||||
}
|
||||
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
cd "$wslRoot"
|
||||
source .venv-linux/bin/activate
|
||||
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
|
||||
python -m sglang.launch_server \
|
||||
--model-path Qwen/Qwen3.5-9B \
|
||||
--port 8002 \
|
||||
--context-length 1010000 \
|
||||
--kt-cpuinfer 16 \
|
||||
--cpu-offload-gb 48
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
@@ -0,0 +1,35 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
if (-not $env:ELY_EYE_GGUF_MODEL) {
|
||||
throw "Set ELY_EYE_GGUF_MODEL to the absolute GGUF model path."
|
||||
}
|
||||
|
||||
$server = $env:LLAMA_CPP_SERVER
|
||||
if (-not $server) {
|
||||
$server = "llama-server"
|
||||
}
|
||||
|
||||
if (-not (Get-Command $server -ErrorAction SilentlyContinue)) {
|
||||
$wingetPackageRoot = Join-Path $env:LOCALAPPDATA "Microsoft\WinGet\Packages"
|
||||
$installed = Get-ChildItem $wingetPackageRoot -Recurse -Filter llama-server.exe -ErrorAction SilentlyContinue |
|
||||
Where-Object { $_.FullName -like "*ggml.llamacpp*" } |
|
||||
Select-Object -First 1
|
||||
if ($installed) {
|
||||
$server = $installed.FullName
|
||||
}
|
||||
}
|
||||
|
||||
if (-not (Get-Command $server -ErrorAction SilentlyContinue)) {
|
||||
throw "llama-server executable is missing. Install ggml.llamacpp or set LLAMA_CPP_SERVER."
|
||||
}
|
||||
|
||||
& $server `
|
||||
--model $env:ELY_EYE_GGUF_MODEL `
|
||||
--host 127.0.0.1 `
|
||||
--port 8003 `
|
||||
--ctx-size 262144 `
|
||||
--flash-attn `
|
||||
--cache-type-k q4_0 `
|
||||
--cache-type-v q4_0
|
||||
@@ -0,0 +1,25 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the SGLang Linux runtime."
|
||||
}
|
||||
|
||||
$override = '{"text_config":{"rope_parameters":{"mrope_interleaved":true,"mrope_section":[11,11,10],"rope_type":"yarn","rope_theta":10000000,"partial_rotary_factor":0.25,"factor":4.0,"original_max_position_embeddings":262144}}}'
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
cd "$wslRoot"
|
||||
source .venv-linux/bin/activate
|
||||
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
|
||||
python -m sglang.launch_server \
|
||||
--model-path Qwen/Qwen3.5-9B \
|
||||
--port 8000 \
|
||||
--tp-size 1 \
|
||||
--mem-fraction-static 0.88 \
|
||||
--context-length 1010000 \
|
||||
--json-model-override-args '$override'
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
@@ -0,0 +1,26 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the SGLang Linux runtime."
|
||||
}
|
||||
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
cd "$wslRoot"
|
||||
source .venv-linux/bin/activate
|
||||
python -m sglang.launch_server \
|
||||
--model-path Qwen/Qwen3.5-9B \
|
||||
--port 8000 \
|
||||
--tp-size 1 \
|
||||
--mem-fraction-static 0.82 \
|
||||
--context-length 262144 \
|
||||
--reasoning-parser qwen3 \
|
||||
--speculative-algo NEXTN \
|
||||
--speculative-num-steps 3 \
|
||||
--speculative-num-draft-tokens 4
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
@@ -0,0 +1,26 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the vLLM Linux runtime."
|
||||
}
|
||||
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$ropeScaling = '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}'
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
cd "$wslRoot"
|
||||
source .venv-linux-vllm/bin/activate
|
||||
python -m vllm.entrypoints.openai.api_server \
|
||||
--model Qwen/Qwen3.5-9B \
|
||||
--host 127.0.0.1 \
|
||||
--port 8001 \
|
||||
--max-model-len 1010000 \
|
||||
--enable-prefix-caching \
|
||||
--gpu-memory-utilization 0.88 \
|
||||
--trust-remote-code \
|
||||
--rope-scaling '$ropeScaling'
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
@@ -0,0 +1,24 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for the vLLM Linux runtime."
|
||||
}
|
||||
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
$command = @"
|
||||
set -euo pipefail
|
||||
cd "$wslRoot"
|
||||
source .venv-linux-vllm/bin/activate
|
||||
python -m vllm.entrypoints.openai.api_server \
|
||||
--model Qwen/Qwen3.5-9B \
|
||||
--host 127.0.0.1 \
|
||||
--port 8001 \
|
||||
--max-model-len 262144 \
|
||||
--enable-prefix-caching \
|
||||
--gpu-memory-utilization 0.82 \
|
||||
--trust-remote-code
|
||||
"@
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc $command
|
||||
@@ -0,0 +1,100 @@
|
||||
param(
|
||||
[string]$CartridgeId = "ely_eye_prd_7e5cd27b8284",
|
||||
[int]$MaxLength = 8192
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$root = Split-Path -Parent $PSScriptRoot
|
||||
Set-Location $root
|
||||
|
||||
$proofImage = Join-Path $root ".ely_eye\proofs\ely-eye-dashboard.png"
|
||||
$coreDataset = Join-Path $root ".ely_eye\training\hme_prd_sft.jsonl"
|
||||
$visualDataset = Join-Path $root ".ely_eye\training\hme_visual_sft.jsonl"
|
||||
$routerDataset = Join-Path $root ".ely_eye\training\hme_router_sft.jsonl"
|
||||
$retrievalDataset = Join-Path $root ".ely_eye\training\hme_retrieval_pairs.jsonl"
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe build-prd-training-data `
|
||||
--prd-path .\PRD.md `
|
||||
--output-path $coreDataset `
|
||||
--max-sections 48
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe build-visual-training-data `
|
||||
--prd-path .\PRD.md `
|
||||
--output-path $visualDataset `
|
||||
--cartridge-id $CartridgeId `
|
||||
--image-path $proofImage
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe build-router-training-data `
|
||||
--prd-path .\PRD.md `
|
||||
--output-path $routerDataset `
|
||||
--cartridge-id $CartridgeId
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe build-retrieval-training-data `
|
||||
--prd-path .\PRD.md `
|
||||
--output-path $retrievalDataset `
|
||||
--cartridge-id $CartridgeId `
|
||||
--max-sections 32
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe train-lora `
|
||||
$coreDataset `
|
||||
.\.ely_eye\adapters\hme_core_lora_qwen35_9b `
|
||||
--max-steps 37 `
|
||||
--kind hme_core_lora `
|
||||
--rank 16 `
|
||||
--alpha 32 `
|
||||
--max-length $MaxLength `
|
||||
--gradient-accumulation-steps 4 `
|
||||
--cartridge-id $CartridgeId
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe train-lora `
|
||||
$visualDataset `
|
||||
.\.ely_eye\adapters\hme_vision_lora_qwen35_9b `
|
||||
--max-steps 20 `
|
||||
--kind hme_vision_lora `
|
||||
--rank 4 `
|
||||
--alpha 8 `
|
||||
--max-length $MaxLength `
|
||||
--gradient-accumulation-steps 4 `
|
||||
--cartridge-id $CartridgeId
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe train-lora `
|
||||
$visualDataset `
|
||||
.\.ely_eye\adapters\hme_ttt_vl_qwen35_9b `
|
||||
--max-steps 20 `
|
||||
--kind hme_ttt_vl `
|
||||
--rank 4 `
|
||||
--alpha 8 `
|
||||
--max-length $MaxLength `
|
||||
--gradient-accumulation-steps 4 `
|
||||
--cartridge-id $CartridgeId
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe train-lora `
|
||||
$visualDataset `
|
||||
.\.ely_eye\adapters\hme_visual_mtp_qwen35_9b `
|
||||
--max-steps 20 `
|
||||
--kind hme_visual_mtp `
|
||||
--rank 4 `
|
||||
--alpha 8 `
|
||||
--max-length $MaxLength `
|
||||
--gradient-accumulation-steps 4 `
|
||||
--cartridge-id $CartridgeId
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe train-lora `
|
||||
$routerDataset `
|
||||
.\.ely_eye\adapters\hme_router_qwen35_08b `
|
||||
--max-steps 16 `
|
||||
--kind hme_router `
|
||||
--rank 8 `
|
||||
--alpha 16 `
|
||||
--max-length $MaxLength `
|
||||
--gradient-accumulation-steps 4 `
|
||||
--cartridge-id $CartridgeId
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe train-retrieval-lora `
|
||||
$retrievalDataset `
|
||||
.\.ely_eye\adapters\hme_retrieval_qwen3_vl_embedding_8b `
|
||||
--max-steps 32 `
|
||||
--rank 8 `
|
||||
--alpha 16 `
|
||||
--max-length $MaxLength `
|
||||
--cartridge-id $CartridgeId
|
||||
@@ -0,0 +1,41 @@
|
||||
param(
|
||||
[int]$MaxSteps = 37,
|
||||
[int]$Rank = 16,
|
||||
[int]$Alpha = 32,
|
||||
[int]$MaxLength = 8192,
|
||||
[int]$GradientAccumulationSteps = 4,
|
||||
[string]$CartridgeId = ""
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$root = Split-Path -Parent $PSScriptRoot
|
||||
Set-Location $root
|
||||
|
||||
$dataset = Join-Path $root ".ely_eye\training\hme_prd_sft.jsonl"
|
||||
$adapter = Join-Path $root ".ely_eye\adapters\hme_core_lora_qwen35_9b"
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe build-prd-training-data `
|
||||
--prd-path .\PRD.md `
|
||||
--output-path $dataset `
|
||||
--max-sections 48
|
||||
|
||||
$args = @(
|
||||
"train-lora",
|
||||
$dataset,
|
||||
$adapter,
|
||||
"--max-steps", "$MaxSteps",
|
||||
"--kind", "hme_core_lora",
|
||||
"--rank", "$Rank",
|
||||
"--alpha", "$Alpha",
|
||||
"--max-length", "$MaxLength",
|
||||
"--gradient-accumulation-steps", "$GradientAccumulationSteps"
|
||||
)
|
||||
|
||||
if ($CartridgeId.Length -gt 0) {
|
||||
$args += @("--cartridge-id", $CartridgeId)
|
||||
}
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe @args
|
||||
|
||||
Write-Host "Adapter ready: $adapter"
|
||||
Write-Host "Use with: `$env:ELY_EYE_ADAPTER_DIR='$adapter'"
|
||||
@@ -0,0 +1,4 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
.\.venv\Scripts\ely-eye.exe serve --host 127.0.0.1 --port 8765
|
||||
@@ -0,0 +1,6 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
Push-Location .\frontend
|
||||
npm run dev
|
||||
Pop-Location
|
||||
@@ -0,0 +1,23 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
if (-not (Get-Command wsl.exe -ErrorAction SilentlyContinue)) {
|
||||
throw "WSL is required for SGLang, vLLM, and KTransformers runtime profiles."
|
||||
}
|
||||
|
||||
wsl.exe -d Ubuntu -- bash -lc "command -v nvidia-smi >/dev/null && nvidia-smi -L >/dev/null"
|
||||
wsl.exe -d Ubuntu -- bash -lc "command -v curl >/dev/null || (apt-get update && apt-get install -y curl)"
|
||||
wsl.exe -d Ubuntu -- bash -lc "command -v /root/.local/bin/uv >/dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh"
|
||||
$wslRoot = (wsl.exe -d Ubuntu wslpath -a (Get-Location).Path).Trim()
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux --python 3.12"
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && /root/.local/bin/uv venv .venv-linux-vllm --python 3.12"
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-sglang-linux.txt"
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux-vllm/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-vllm-linux.txt"
|
||||
wsl.exe -d Ubuntu -- bash -lc "cd '$wslRoot' && source .venv-linux/bin/activate && /root/.local/bin/uv pip install --prerelease=allow -r backend/requirements-ktransformers-linux.txt"
|
||||
|
||||
if (-not (Get-Command llama-server -ErrorAction SilentlyContinue)) {
|
||||
winget install --id ggml.llamacpp -e --accept-package-agreements --accept-source-agreements
|
||||
}
|
||||
|
||||
.\.venv\Scripts\ely-eye.exe runtime-profiles
|
||||
@@ -0,0 +1,10 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
Set-Location (Split-Path -Parent $PSScriptRoot)
|
||||
|
||||
.\.venv\Scripts\python.exe -m compileall .\backend\src
|
||||
.\.venv\Scripts\ely-eye.exe status
|
||||
|
||||
Push-Location .\frontend
|
||||
npm run build
|
||||
Pop-Location
|
||||
Reference in new issue
Block a user