ZacharyZhang-NY
89c880fd36
The README described a project that no longer exists: a Windows-only Qwen runtime with nine read-only tools, a console as the only way to operate the store, and a 100M-token capsule as if it were measured. It is now a description of what is here, with the report and the command that re-measures each number, and a section that names what has not been run instead of leaving the reader to discover it. TRAINING_ARTIFACTS.md keeps its artifact tables and loses the scores that were never run: MMMU-Pro and OCRBench each read one sample, and the HashHop gate collected proofs until it hit its target. What stays is annotated with what it actually measures — citation accuracy there means the cited id existed in the context that was handed over, not that the passage supports the answer. Gaps.md gains a re-audit section: ten of the twelve gaps closed, two partial, each row naming the command or report behind it, and the remaining work stated plainly — the 0.90 recall bar is unmet and the deficit is entirely in the visual and audio legs. AGENTS.md stops being ignored. It records the constraints this repository paid for: the measurement rules, the environment traps, and the review protocol, so the next agent does not pay them again. The CLI could filter by holder but could not look through a holder's eyes, so search gains --as-holder: three states of eyes on every face, not two.
9.3 KiB
Ely-Eye
Ely-Eye is a local memory store for AI. Every memory it keeps carries who it belongs to, where it came from, when it was true, in what situation, and who may read it. Retrieval answers as the person asking, so a store shared by several people and several agents does not blend them into one voice. A Context Cartridge is a scope you mount, not only a bundle you export. Coding agents reach the same store over MCP.
The difference between this and a vector database is the referral: "I don't know where the keys are, I know my wife knows, and I can find my wife." When the store cannot answer, it says nobody knows and names who does — instead of inventing a plausible answer.
Everything below is either measured in this repository, with the report and the command that re-measures it, or it is marked as not measured.
What is measured
| question | answer | report |
|---|---|---|
| can it find the memory that answers a question? | recall@10 0.863, MRR 0.682, over 300 answerable probes drawn from six third-party corpora, each asked as the holder that corpus assigns it | bench/reports/LEGS.md |
| is that any good? | a Mem0-style counterpart on the same corpus scores 0.866 inside one holder's memory and 0.412 across all of them; chance is 0.0002 whole-corpus and 0.369 inside one holder | bench/reports/BASELINE_EXTERNAL.md |
| does the holder boundary matter? | +0.454 recall@10 with an identical ranker and identical items — the gain is scoping, not ranking | same report |
| how fast at corpus size? | p50 0.072 s, p95 0.218 s over 165,973 atoms; the search this replaced took 20.16 s on the same corpus | bench/reports/SCALE.md |
| how much does it hold? | 165,973 atoms and 153,857 vectors in one SQLite file; the vector build runs 106 memories/s. One million is an extrapolation, not a measurement | bench/reports/DENSE.md |
| can a wordless picture be found? | yes for icon crops, weakly for whole screenshots — the report shows every arm, including the ones that hurt | bench/reports/VISUAL.md |
| what does speech cost? | WER 0.242 over 533 reference words; 258.18 s of audio transcribed in 3.12 s (68.4–111.2× realtime per video, CPU int8), byte-identical across two passes | bench/reports/ASR.md |
| is the acceptance bar met? | no. The bar is recall@10 ≥ 0.90; shipped is 0.863 | bench/reports/LEGS.md |
Where the deficit lives, by modality (recall@10, same run): code 0.945,
conversation 1.000, pdf 0.885, ui_screenshot 0.818, image 0.720, video 0.455.
Video is the worst because the frozen stores hold no audio atoms — a video is
sampled frames plus OCR, so a question answered by speech has nothing to match.
Images and screenshots are limited because CLIP cannot localize an icon inside a
screenshot. bench/reports/LEGS.md names the arms; nothing here rounds those
numbers up.
Re-measure any of them:
python -m bench.validate_sets # the frozen corpus and its gold
python -m bench.leg_check --probes 300 # the shipped retrieval path
python -m bench.run_external # the counterparts and the chance floor
python -m bench.dense_scale # the vector leg at corpus size
What is not measured here
The generator side is written and runnable but has not been run in this repository's reports, so no score from it is quoted anywhere in this README:
- generation with Qwen3.5-9B and the trained LoRA adapters —
ely-eye runtime-generation-proof - Visual HashHop multi-hop visual addressing —
ely-eye hashhop --kind visual - the Visual Contradiction Lens —
ely-eye visual-contradiction-proof - standard benchmarks (MMMU-Pro, OCRBench, and the rest) —
ely-eye run-benchmark - the fifteen-check proof suite as a whole —
ely-eye proof-suite
Each needs a CUDA machine and a model download. They are marked DEFERRED in
CHECKLIST.md, and a run that has not happened is not allowed to look like a
result: ely-eye proof-suite writes what it observed, and the checks that
depend on a model report what they could not do rather than passing quietly.
TRAINING_ARTIFACTS.md keeps the six-adapter training table, which is
artifact-backed on the machine that trained them, and no longer quotes benchmark
scores that were never run.
What a memory carries
holder_id, origin (self / observed / artifact / third_party), scenario,
event_time with its precision and how it was learned, valid_from /
valid_to, supersedes, visibility (holder / visible_to / everyone),
retention_until, plus ingestion time and trust scores. Ingestion stamps
provenance; a client cannot claim to be somebody else, because the server
decides who a caller is from the token it presents.
Four verbs, one writer: remember, supersede, forget, search. They exist
over HTTP, the CLI, and MCP, and they all go through the same Python writer, so
the rules cannot drift apart. forget cascades to everything derived — vectors,
FTS rows, edges, cache entries — and reports which cartridges contained the
memory rather than silently rewriting an export whose digest somebody already
quoted.
On top of that: a working tier (what one session recorded, in order), consolidation into a standing summary, an archive tier for memories that are neither used nor valued, and a conflict list for dated memories that disagree and that nobody has settled. A third-party memory that no other memory vouches for is quarantined: it stays in the store, and it is not citable.
Run it
Linux and macOS:
bash scripts/install.sh
bash scripts/serve-backend.sh # API on http://127.0.0.1:8765
bash scripts/serve-frontend.sh # dashboard on http://127.0.0.1:5173
Windows: the same names as .ps1. There is also a container image
(docker build -t ely-eye .), and CI runs the whole suite on Linux.
ely-eye ingest ~/some/package --cartridge-name project
ely-eye search "where did we put the archive key" --as-holder wife
ely-eye remember "the archive key is in the second drawer" --holder wife --scenario home
ely-eye who-knows "archive key"
ely-eye ask "where is the archive key" --as-holder husband
ely-eye card
ely-eye mounts && ely-eye mount project && ely-eye unmount
ely-eye backup ./backup && ely-eye doctor
bash scripts/mcp_write_smoke.sh # two processes, one store, one SQLite
The dashboard is served by the API itself once ELY_EYE_UI_DIR points at
frontend/dist. scripts/ui_smoke.sh drives it in a real browser against a
fresh store and fails if any of it stops working. The CLI does everything the
console does, so nothing requires a browser.
Retrieval
Three legs, fused by rank. The words leg always runs. The dense leg runs only
when ELY_EYE_DENSE_EMBEDDINGS_ENABLED=1 and an embedding model is configured.
On the shipped page that leg moves recall from 0.863 to 0.860 while p50 goes
from 122 ms to 713 ms; on a wider page both reach 0.873 while p50 goes from
84 ms to 579 ms. It buys the memory questions (multi-session 0.857 → 0.920,
preference 0.800 → 0.960) and costs the picture ones (image 0.720 → 0.680),
which is why it is opt-in rather than the default. The picture leg runs only
when the words page comes back thin and ELY_EYE_CLIP_MODEL_DIR is set.
Vectors live in the database beside the atoms, written in the same transaction
as the memory itself, so a memory is findable the moment it is stored and there
is no index to rebuild.
MCP server for coding agents
ely-eye-mcp/ is a Go MCP server. One command builds a native binary and
registers it with Codex and Claude Code. Reads mirror the store; writes go
through the API, so the write rules live in one place. A read with no
ELY_EYE_AUTH_TOKEN sees only memories nobody owns and memories marked for
everyone; ely_eye_card answers "can this store answer my kind of question"
before anything sensitive is sent, and it carries no memory text.
bash ./ely-eye-mcp/install.sh --client both
See ely-eye-mcp/README.md for the full tool list.
Data layout
Runtime state lives under .ely_eye/ (git-ignored):
.ely_eye/
├─ data/ the SQLite store: atoms, FTS, vectors (vec0), edges, mounts
├─ objects/ ingested source files
├─ cache/ compiled-context cache and the picture index
├─ cartridges/ materialized Context Cartridges
├─ adapters/ trained LoRA adapters
└─ training/ datasets and training traces
A Context Cartridge is a self-contained memory package: the Evidence Atoms, the sparse and vector indexes, the temporal graph, and a DNA digest of the files. Mounting one scopes every subsequent search to it; switching is one command.
Project layout
backend/ Python: ingestion, retrieval, memory, runtime, training, FastAPI
bench/ the benchmark suite: frozen third-party corpora, harnesses, reports
frontend/ the local dashboard (Vite + React)
scripts/ install, serving, runtimes, smoke (.sh on Linux/macOS, .ps1 on Windows)
ely-eye-mcp/ Go: the MCP server
CHECKLIST.md is the ordered work list with the acceptance line for every item;
Gaps.md is the audit this work started from.
License
AGPL-3.0-only. Copyright © 2026 雷电芽衣 zhangyanghaha0407@outlook.com.
The PDF reader (PyMuPDF) is dual-licensed under AGPL-3.0 or an Artifex Commercial license, so this project ships under AGPL while that dependency is in use. Buying the Artifex license is what would allow a permissive one.