ZacharyZhang-NY
cd9851a12c
The dense leg used to be a pair of files next to the database: a full .npy re-encoded from every atom, re-normalized on every query, and stale the moment anything was ingested. It is now a vec0 table inside the same SQLite file, written in the same transaction as the atom it describes, so a memory is queryable the instant it is remembered. - vector_index.py maintains the table; the writer inserts, the reader never rebuilds. A repair scan (ely-eye build-dense-index) catches what a crash hid, and it is not on the search path. - Ranking stays in the engine; every boundary (closed, retired, not yet true, another holder's private memory, the mounted cartridge) is checked once against the atoms table afterwards. A visibility subquery inside the KNN cost 48.9s on 153,857 vectors; the same check over 200 candidates costs 0.2ms. When the boundaries reject enough candidates the ask widens once, never twice. - The Go mirror and the web panel carry the new status fields. - bench/dense_scale.py and bench/quant_probe.py measure the leg at corpus size; bench/scale_check.py now works on a copy so the frozen baseline stays frozen. Measured on the frozen 165,973-atom corpus. bench/reports/DENSE.md, QUANT.md and SCALE.md carry the numbers and the machine they were taken on. Gates: 285 pytest, ruff clean, pyright 0 on the touched files, go build/vet/test and gofmt clean, frontend build and eslint clean. mypy is 25 findings against the parent's 24: the new one is "Skipping analyzing sqlite_vec: missing library stubs", the same class as the two existing qwen_vl_utils lines, and the repo keeps no type: ignore anywhere.
Ely-Eye
Ely-Eye is a local multimodal context operating system built on Qwen3.5-9B. It runs on a single consumer GPU and gives a small model three tiers of context: a 262K native window, a 1.01M extended window, and a 100M-token memory capsule it can retrieve from, cite, and recall across sessions.
Under the hood it ingests PDFs, screenshots, video, and code into Evidence Atoms, packs them into a Context Cartridge, retrieves with sparse and Qwen embedding search, runs Qwen3.5-9B with trained LoRA adapters, and checks every answer with a verifier. A fifteen-check proof suite exercises the whole thing against the live model, so a real run can be told apart from a fabricated one.
The landing site lives at https://model.elydora.com/
(its source is under ely-eye-web/, which this repository does not track).
Requirements
- Windows with an NVIDIA GPU and CUDA. Training and the proof suite were built on an RTX 4090; deployment targets a 16GB card such as an RTX 4080.
- The extended-context runtimes (SGLang, vLLM, KTransformers, llama.cpp) are optional and run under WSL. The default Windows runtime uses Transformers and needs none of them.
Run it locally
.\scripts\install.ps1 # virtual env, PyTorch + CUDA, Transformers, PEFT, bitsandbytes
.\scripts\download-models.ps1 # Qwen3.5-9B, the 0.8B planner, Qwen3-VL-Embedding-8B
.\scripts\serve-backend.ps1 # API on http://127.0.0.1:8765
.\scripts\serve-frontend.ps1 # dashboard on http://127.0.0.1:5173
When ELY_EYE_ADAPTER_DIR is empty, the runtime picks the latest trained
HME-Core-LoRA adapter for Qwen/Qwen3.5-9B on its own; ely-eye status prints
which adapter it resolved.
For the 262K and 1.01M extended-context profiles, set up the Linux runtimes once and launch one:
.\scripts\setup-linux-runtimes.ps1
.\scripts\launch-sglang-live.ps1 # 262K
.\scripts\launch-sglang-extreme.ps1 # 1.01M via YaRN
Core commands
.\.venv\Scripts\ely-eye.exe status
.\.venv\Scripts\ely-eye.exe ingest C:\path\to\package --cartridge-name project
.\.venv\Scripts\ely-eye.exe hashhop --kind visual --hops 2 --token-equivalent 262144
.\.venv\Scripts\ely-eye.exe proof-suite
The console is served by the API itself once ELY_EYE_UI_DIR points at
frontend/dist. It shows the state the CLI prints — runtime, GPU, library,
cartridges, senses, adapters, proof suite — and it is the only place to search
the store as somebody, page through atoms, forget one, and mount a cartridge.
scripts/ui_smoke.sh drives that console in a real browser against a fresh
store and fails if any of it stops working.
Fine-tuning
Training runs real BF16 LoRA updates on the GPU through Unsloth-patched Transformers, PEFT, and bitsandbytes with paged 8-bit AdamW. Each run writes the PEFT adapter alongside a manifest, a per-step training trace, a summary, and a proof file that records dataset and weight hashes, per-step loss, optimizer-update norms, and the CUDA/BF16 environment.
The adapter set:
HME-Core-LoRA Qwen3.5-9B attention/MLP
HME-Vision-LoRA Qwen3.5-9B visual projector and selected vision layers
HME-TTT-VL Qwen3.5-9B mutable MLP and visual adapter
HME-Visual-MTP Qwen3.5-9B draft-head LoRA
HME-Router Qwen3.5-0.8B context planner
HME-Retrieval Qwen3-VL-Embedding-8B contrastive retrieval adapter
Proof suite
ely-eye proof-suite runs fifteen checks against the live model and writes the
evidence under .ely_eye/data/eval_proofs/. Among them:
- real CUDA BF16 LoRA training — changed weight hashes, nonzero optimizer updates, per-step loss
- the runtime loading the model and adapter and generating a cited answer
- Visual HashHop: multi-hop visual addressing the model has to actually solve, with no OCR shortcut
- the 100M-token memory capsule committing to a reproducible Memory DNA
- the Visual Contradiction Lens localizing cross-version drift (design-token, layout, typography, copy, visual-code, temporal) at >=80% recall with no false positives
- standard benchmark samples (MMMU-Pro, OCRBench, and others) scored against ground truth
.\.venv\Scripts\ely-eye.exe proof-suite
.\.venv\Scripts\ely-eye.exe latest-proof-suite
.\.venv\Scripts\ely-eye.exe runtime-generation-proof
.\.venv\Scripts\ely-eye.exe visual-contradiction-proof
MCP server for coding agents
ely-eye-mcp/ is a local Model Context Protocol server that exposes the
.ely_eye memory store to coding agents such as Codex and Claude Code. One
command builds a native Go binary (or downloads a prebuilt one), installs it
under .ely_eye/bin/, and registers it with both clients. No Docker.
Windows:
powershell -ExecutionPolicy Bypass -File .\ely-eye-mcp\install.ps1 -Client both
Linux and macOS:
bash ./ely-eye-mcp/install.sh --client both
The agent then has nine read-only tools over the live memory — status,
cartridge listing and detail, Evidence Atom search and fetch, evidence-pack
compilation, proof suites, and HashHop proofs. The server opens the database
read-only and never writes to it. The same installer works against any
.ely_eye store, not only this repository. See ely-eye-mcp/README.md for the
full tool list and the release pipeline.
Data layout
Runtime state lives under .ely_eye/ (git-ignored):
.ely_eye/
├─ adapters/ trained LoRA adapters
├─ data/ sqlite database and eval proofs
├─ training/ datasets and training traces
├─ objects/ ingested source files
├─ cache/ cache fabric
└─ cartridges/ materialized Context Cartridges
A Context Cartridge is a self-contained memory package: the Evidence Atoms, a sparse index and vector indexes, a temporal graph, KV snapshots, the memory capsule, any attached adapters, and the eval proofs.
Project layout
backend/ Python: ingestion, retrieval, runtime, training, proof suite, FastAPI
frontend/ the local dashboard (Vite + React)
scripts/ install, model download, serving, runtimes, fine-tuning (.ps1 on Windows, .sh on Linux)
ely-eye-mcp/ Go: local MCP server exposing the memory store to coding agents
License
AGPL-3.0-only. Copyright © 2026 雷电芽衣 zhangyanghaha0407@outlook.com.
The PDF reader (PyMuPDF) is dual-licensed under AGPL-3.0 or an Artifex Commercial license, so this project ships under AGPL while that dependency is in use. Buying the Artifex license is what would allow a permissive one.