ZacharyZhang-NY
334071199e
An independent reviewer planted two witnesses that contradict their claims and both were believed: 「恒温器很贵」 was corroborated by 「恒温器很便宜」, 「这房子值得买」 by 「这房子不值得买」. The Chinese pairs were spread inside each run on its own, and equal-length pairs kept insertion order, so the three demanded pairs of a multi-clause claim all came from its first clause — and for a four-pair claim the evenly spaced index picked 0, 1, 2, clustering them at the start anyway. An unrelated memory that merely opened the same way pushed a quarantined sentence into the packed context. Pairs are now pooled across the whole claim and the demanded positions are its start, its middle and its end. The last pair is the assertion, so a witness has to agree with the claim rather than merely introduce it. The term-level boundary is pinned in both languages: a restatement that keeps the compared positions counts, one that replaces them does not, a denial that differs by one word still counts, and a changed number does not. Also corrected the record where it had drifted: the gate cell was labelled with a commit that never produced those numbers, the acceptance cell pinned a total its own command could not reach, and the reader-path count stated a conclusion without its measurement rule. Gates: 320 passed; ruff clean; mypy cold cache 24 errors in 9 files, net zero against the recorded baseline; pyright 0 errors on the touched files; filesize, product audit, go suite and the two-principal MCP smoke all pass.
Ely-Eye
Ely-Eye is a local multimodal context operating system built on Qwen3.5-9B. It runs on a single consumer GPU and gives a small model three tiers of context: a 262K native window, a 1.01M extended window, and a 100M-token memory capsule it can retrieve from, cite, and recall across sessions.
Under the hood it ingests PDFs, screenshots, video, and code into Evidence Atoms, packs them into a Context Cartridge, retrieves with sparse and Qwen embedding search, runs Qwen3.5-9B with trained LoRA adapters, and checks every answer with a verifier. A fifteen-check proof suite exercises the whole thing against the live model, so a real run can be told apart from a fabricated one.
The landing site lives at https://model.elydora.com/
(its source is under ely-eye-web/, which this repository does not track).
Requirements
- Windows with an NVIDIA GPU and CUDA. Training and the proof suite were built on an RTX 4090; deployment targets a 16GB card such as an RTX 4080.
- The extended-context runtimes (SGLang, vLLM, KTransformers, llama.cpp) are optional and run under WSL. The default Windows runtime uses Transformers and needs none of them.
Run it locally
.\scripts\install.ps1 # virtual env, PyTorch + CUDA, Transformers, PEFT, bitsandbytes
.\scripts\download-models.ps1 # Qwen3.5-9B, the 0.8B planner, Qwen3-VL-Embedding-8B
.\scripts\serve-backend.ps1 # API on http://127.0.0.1:8765
.\scripts\serve-frontend.ps1 # dashboard on http://127.0.0.1:5173
When ELY_EYE_ADAPTER_DIR is empty, the runtime picks the latest trained
HME-Core-LoRA adapter for Qwen/Qwen3.5-9B on its own; ely-eye status prints
which adapter it resolved.
For the 262K and 1.01M extended-context profiles, set up the Linux runtimes once and launch one:
.\scripts\setup-linux-runtimes.ps1
.\scripts\launch-sglang-live.ps1 # 262K
.\scripts\launch-sglang-extreme.ps1 # 1.01M via YaRN
Core commands
.\.venv\Scripts\ely-eye.exe status
.\.venv\Scripts\ely-eye.exe ingest C:\path\to\package --cartridge-name project
.\.venv\Scripts\ely-eye.exe hashhop --kind visual --hops 2 --token-equivalent 262144
.\.venv\Scripts\ely-eye.exe proof-suite
The console is served by the API itself once ELY_EYE_UI_DIR points at
frontend/dist. It shows the state the CLI prints — runtime, GPU, library,
cartridges, senses, adapters, proof suite — and it is the only place to search
the store as somebody, page through atoms, forget one, and mount a cartridge.
scripts/ui_smoke.sh drives that console in a real browser against a fresh
store and fails if any of it stops working.
Fine-tuning
Training runs real BF16 LoRA updates on the GPU through Unsloth-patched Transformers, PEFT, and bitsandbytes with paged 8-bit AdamW. Each run writes the PEFT adapter alongside a manifest, a per-step training trace, a summary, and a proof file that records dataset and weight hashes, per-step loss, optimizer-update norms, and the CUDA/BF16 environment.
The adapter set:
HME-Core-LoRA Qwen3.5-9B attention/MLP
HME-Vision-LoRA Qwen3.5-9B visual projector and selected vision layers
HME-TTT-VL Qwen3.5-9B mutable MLP and visual adapter
HME-Visual-MTP Qwen3.5-9B draft-head LoRA
HME-Router Qwen3.5-0.8B context planner
HME-Retrieval Qwen3-VL-Embedding-8B contrastive retrieval adapter
Proof suite
ely-eye proof-suite runs fifteen checks against the live model and writes the
evidence under .ely_eye/data/eval_proofs/. Among them:
- real CUDA BF16 LoRA training — changed weight hashes, nonzero optimizer updates, per-step loss
- the runtime loading the model and adapter and generating a cited answer
- Visual HashHop: multi-hop visual addressing the model has to actually solve, with no OCR shortcut
- the 100M-token memory capsule committing to a reproducible Memory DNA
- the Visual Contradiction Lens localizing cross-version drift (design-token, layout, typography, copy, visual-code, temporal) at >=80% recall with no false positives
- standard benchmark samples (MMMU-Pro, OCRBench, and others) scored against ground truth
.\.venv\Scripts\ely-eye.exe proof-suite
.\.venv\Scripts\ely-eye.exe latest-proof-suite
.\.venv\Scripts\ely-eye.exe runtime-generation-proof
.\.venv\Scripts\ely-eye.exe visual-contradiction-proof
MCP server for coding agents
ely-eye-mcp/ is a local Model Context Protocol server that exposes the
.ely_eye memory store to coding agents such as Codex and Claude Code. One
command builds a native Go binary (or downloads a prebuilt one), installs it
under .ely_eye/bin/, and registers it with both clients. No Docker.
Windows:
powershell -ExecutionPolicy Bypass -File .\ely-eye-mcp\install.ps1 -Client both
Linux and macOS:
bash ./ely-eye-mcp/install.sh --client both
The agent then has nine read-only tools over the live memory — status,
cartridge listing and detail, Evidence Atom search and fetch, evidence-pack
compilation, proof suites, and HashHop proofs. The server opens the database
read-only and never writes to it. The same installer works against any
.ely_eye store, not only this repository. See ely-eye-mcp/README.md for the
full tool list and the release pipeline.
Data layout
Runtime state lives under .ely_eye/ (git-ignored):
.ely_eye/
├─ adapters/ trained LoRA adapters
├─ data/ sqlite database and eval proofs
├─ training/ datasets and training traces
├─ objects/ ingested source files
├─ cache/ cache fabric
└─ cartridges/ materialized Context Cartridges
A Context Cartridge is a self-contained memory package: the Evidence Atoms, a sparse index and vector indexes, a temporal graph, KV snapshots, the memory capsule, any attached adapters, and the eval proofs.
Project layout
backend/ Python: ingestion, retrieval, runtime, training, proof suite, FastAPI
frontend/ the local dashboard (Vite + React)
scripts/ install, model download, serving, runtimes, fine-tuning (.ps1 on Windows, .sh on Linux)
ely-eye-mcp/ Go: local MCP server exposing the memory store to coding agents
License
AGPL-3.0-only. Copyright © 2026 雷电芽衣 zhangyanghaha0407@outlook.com.
The PDF reader (PyMuPDF) is dual-licensed under AGPL-3.0 or an Artifex Commercial license, so this project ships under AGPL while that dependency is in use. Buying the Artifex license is what would allow a permissive one.