ZacharyZhang-NY
8bed319848
A quantity conflict compared labels, and a label can carry a number that
identifies the thing rather than measuring it. 「Room 3 holds 22 degrees」 and
「Room 4 holds 24 degrees」 shared the subject 「room」, so the store recorded a
conflict over the room numbers and dropped Room 3 from the answer page: asking
about degrees returned one room's reading and hid the other room entirely.
「The router 1/2 forwards 220/240 packets」 and 「2026 saw 3 / 2027 saw 5
releases」 failed the same way.
A bare number standing before any predicate is part of the subject's name, so it
joins the subject key and is never recorded as a value. A number that stands past
a word still measures: 「The rent is 2200 / 2400 dollars」 conflicts over rent,
and a number fused inside a CJK run stays a value, because 「恒温器调到」 states its
predicate inside the subject token and the motivating pair must keep conflicting.
The same year counting twice is now a conflict (「2026 saw 3 / 5 releases」), which
the old rule missed.
Only a number renames a subject: 「Room A」 and 「Room B」 still read as one room
whose reading moved, and an English qualifier does not split a subject the way a
Chinese one does. Both boundaries are disclosed and pinned by tests rather than
smoothed over.
Also collapses six two-line comment blocks back to one line each, which is the
rule this repo already enforces, and corrects the record where it claimed the
nearest label was 「书房」 (it is 「设定」) and that the subject rule only ever
missed conflicts.
Gates: pytest -q 379 passed (conflicts 17 -> 22, six p2-3 files 78, three tier
files 35); ruff clean; bench.filesize_check clean; mypy cold cache 24 errors in 9
files, byte-identical to the recorded 9cb46c1 baseline; pyright 0 errors on the
touched files; go build/vet/test and gofmt clean; npm run build unchanged at
index-CSDHCFNo.js; mcp_write_smoke.sh PASS; whole-line comment runs of length >=2:
0 across tracked Python files.
Ely-Eye
Ely-Eye is a local multimodal context operating system built on Qwen3.5-9B. It runs on a single consumer GPU and gives a small model three tiers of context: a 262K native window, a 1.01M extended window, and a 100M-token memory capsule it can retrieve from, cite, and recall across sessions.
Under the hood it ingests PDFs, screenshots, video, and code into Evidence Atoms, packs them into a Context Cartridge, retrieves with sparse and Qwen embedding search, runs Qwen3.5-9B with trained LoRA adapters, and checks every answer with a verifier. A fifteen-check proof suite exercises the whole thing against the live model, so a real run can be told apart from a fabricated one.
The landing site lives at https://model.elydora.com/
(its source is under ely-eye-web/, which this repository does not track).
Requirements
- Windows with an NVIDIA GPU and CUDA. Training and the proof suite were built on an RTX 4090; deployment targets a 16GB card such as an RTX 4080.
- The extended-context runtimes (SGLang, vLLM, KTransformers, llama.cpp) are optional and run under WSL. The default Windows runtime uses Transformers and needs none of them.
Run it locally
.\scripts\install.ps1 # virtual env, PyTorch + CUDA, Transformers, PEFT, bitsandbytes
.\scripts\download-models.ps1 # Qwen3.5-9B, the 0.8B planner, Qwen3-VL-Embedding-8B
.\scripts\serve-backend.ps1 # API on http://127.0.0.1:8765
.\scripts\serve-frontend.ps1 # dashboard on http://127.0.0.1:5173
When ELY_EYE_ADAPTER_DIR is empty, the runtime picks the latest trained
HME-Core-LoRA adapter for Qwen/Qwen3.5-9B on its own; ely-eye status prints
which adapter it resolved.
For the 262K and 1.01M extended-context profiles, set up the Linux runtimes once and launch one:
.\scripts\setup-linux-runtimes.ps1
.\scripts\launch-sglang-live.ps1 # 262K
.\scripts\launch-sglang-extreme.ps1 # 1.01M via YaRN
Core commands
.\.venv\Scripts\ely-eye.exe status
.\.venv\Scripts\ely-eye.exe ingest C:\path\to\package --cartridge-name project
.\.venv\Scripts\ely-eye.exe hashhop --kind visual --hops 2 --token-equivalent 262144
.\.venv\Scripts\ely-eye.exe proof-suite
The console is served by the API itself once ELY_EYE_UI_DIR points at
frontend/dist. It shows the state the CLI prints — runtime, GPU, library,
cartridges, senses, adapters, proof suite — and it is the only place to search
the store as somebody, page through atoms, forget one, and mount a cartridge.
scripts/ui_smoke.sh drives that console in a real browser against a fresh
store and fails if any of it stops working.
Fine-tuning
Training runs real BF16 LoRA updates on the GPU through Unsloth-patched Transformers, PEFT, and bitsandbytes with paged 8-bit AdamW. Each run writes the PEFT adapter alongside a manifest, a per-step training trace, a summary, and a proof file that records dataset and weight hashes, per-step loss, optimizer-update norms, and the CUDA/BF16 environment.
The adapter set:
HME-Core-LoRA Qwen3.5-9B attention/MLP
HME-Vision-LoRA Qwen3.5-9B visual projector and selected vision layers
HME-TTT-VL Qwen3.5-9B mutable MLP and visual adapter
HME-Visual-MTP Qwen3.5-9B draft-head LoRA
HME-Router Qwen3.5-0.8B context planner
HME-Retrieval Qwen3-VL-Embedding-8B contrastive retrieval adapter
Proof suite
ely-eye proof-suite runs fifteen checks against the live model and writes the
evidence under .ely_eye/data/eval_proofs/. Among them:
- real CUDA BF16 LoRA training — changed weight hashes, nonzero optimizer updates, per-step loss
- the runtime loading the model and adapter and generating a cited answer
- Visual HashHop: multi-hop visual addressing the model has to actually solve, with no OCR shortcut
- the 100M-token memory capsule committing to a reproducible Memory DNA
- the Visual Contradiction Lens localizing cross-version drift (design-token, layout, typography, copy, visual-code, temporal) at >=80% recall with no false positives
- standard benchmark samples (MMMU-Pro, OCRBench, and others) scored against ground truth
.\.venv\Scripts\ely-eye.exe proof-suite
.\.venv\Scripts\ely-eye.exe latest-proof-suite
.\.venv\Scripts\ely-eye.exe runtime-generation-proof
.\.venv\Scripts\ely-eye.exe visual-contradiction-proof
MCP server for coding agents
ely-eye-mcp/ is a local Model Context Protocol server that exposes the
.ely_eye memory store to coding agents such as Codex and Claude Code. One
command builds a native Go binary (or downloads a prebuilt one), installs it
under .ely_eye/bin/, and registers it with both clients. No Docker.
Windows:
powershell -ExecutionPolicy Bypass -File .\ely-eye-mcp\install.ps1 -Client both
Linux and macOS:
bash ./ely-eye-mcp/install.sh --client both
The agent then has nine read-only tools over the live memory — status,
cartridge listing and detail, Evidence Atom search and fetch, evidence-pack
compilation, proof suites, and HashHop proofs. The server opens the database
read-only and never writes to it. The same installer works against any
.ely_eye store, not only this repository. See ely-eye-mcp/README.md for the
full tool list and the release pipeline.
Data layout
Runtime state lives under .ely_eye/ (git-ignored):
.ely_eye/
├─ adapters/ trained LoRA adapters
├─ data/ sqlite database and eval proofs
├─ training/ datasets and training traces
├─ objects/ ingested source files
├─ cache/ cache fabric
└─ cartridges/ materialized Context Cartridges
A Context Cartridge is a self-contained memory package: the Evidence Atoms, a sparse index and vector indexes, a temporal graph, KV snapshots, the memory capsule, any attached adapters, and the eval proofs.
Project layout
backend/ Python: ingestion, retrieval, runtime, training, proof suite, FastAPI
frontend/ the local dashboard (Vite + React)
scripts/ install, model download, serving, runtimes, fine-tuning (.ps1 on Windows, .sh on Linux)
ely-eye-mcp/ Go: local MCP server exposing the memory store to coding agents
License
AGPL-3.0-only. Copyright © 2026 雷电芽衣 zhangyanghaha0407@outlook.com.
The PDF reader (PyMuPDF) is dual-licensed under AGPL-3.0 or an Artifex Commercial license, so this project ships under AGPL while that dependency is in use. Buying the Artifex license is what would allow a permissive one.