The product path pulled every atom into Python and rebuilt a BM25Okapi
index per query: p50 20.16s over 165,973 atoms. It now runs one SQLite
statement with LIMIT/OFFSET.
- text_index.py: one normalizer for both sides of the index. CJK runs are
split per character (unicode61 cannot segment them), camelCase and
separators are split so "search atoms" finds SearchAtoms, latin tokens
get a prefix match. Rows written before this existed hold raw text in
atom_fts, so schema v3 rebuilds the index on upgrade.
- retrieval.py: filters (holder / scenario / source / modality / since /
until / min_trust), a base64 cursor, and reciprocal rank fusion in place
of the 0.55/0.35/0.10 blend whose graph term was always zero.
- An empty result is {status: "no_match"}, not a bare array.
- Go: search_atoms and compile_context rank through POST /api/search. The
second Go tokenizer is deleted; two implementations could drift, which
is what the MCP/HTTP parity requirement forbids.
- db.py passed 500 lines, so the row codec moved to row_codec.py.
- Tooling: installed the repo's own pyright. pyrightconfig.json had no
exclude and swept the downloaded corpus (1171 errors); with excludes the
count is 62, all pre-existing (evals 48, profiles 4, training 4,
ingestion 3, runtime 2, cartridge_assets 1). backend/pyproject.toml
carried a second [tool.pyright] that shadowed the root config; deleted.
- bench/scale_check.py measures the product search at corpus size. The
100k-atom p50 number is not written until it is measured on an idle
machine.
Verification: 112 pytest passed; go build/vet/test clean, gofmt clean;
ruff clean on every touched file; mypy clean on the 14 touched Python
files; repo pyright 62 errors, unchanged per-file ledger.
remember / supersede / forget now reach the one Python writer from a
terminal, from MCP, and from HTTP. The server, not the caller, decides
who owns a memory.
- cli: remember, supersede, forget. --holder falls back to
settings.default_holder until the holder registry lands. Timestamps are
parsed once with one clear message.
- ingestion: import pymupdf as fitz. The deprecated fitz import printed a
warning to stdout, so every CLI command's JSON answer was unparseable.
- mcp: remember, supersede_memory and forget_memory forward to the Python
writer over HTTP (ELY_EYE_API, default http://127.0.0.1:8765). SQLite
stays read-only to the Go side, so the write rules live in one place.
- mcp reads follow schema v2: atoms.time became ingested_at, and reads go
through v_atoms_current, so a superseded memory stops being findable.
holder, origin, scenario and event_time now show up in fetch_atom and
search_atoms.
- scripts/mcp_write_smoke.sh proves the loop end to end: remember, search,
supersede, forget, then not found.
Evidence: 92 pytest passed; go build, go vet and go test clean; mypy clean
on the 10 touched source files with a fresh cache; ruff clean except a
pre-existing B008 at cli.py:292, identical at HEAD.
An EvidenceAtom could say what was remembered and nothing else: not whose memory it
is, not where it came from, not when it was true, not in what situation it may be
used. The baseline measured the cost of that: the same ranker scores recall@10 0.412
over the whole corpus and 0.866 inside one holder's memory, so the missing holder
column is worth more than any ranking change in the ablation.
- schemas.py: MemoryOrigin, TimePrecision, Visibility; EvidenceAtom gains holder_id,
origin, scenario, event_time / event_time_end / event_time_precision /
event_time_source, valid_from / valid_to, supersedes, visibility, retention_until.
`time` is renamed `ingested_at` — ingestion time gets one name, not two.
A validator refuses valid_to before valid_from, event_time_end before event_time,
and an atom that supersedes itself.
- migrations.py: schema v2 with PRAGMA user_version. A first-version database is
upgraded in place (ALTER + RENAME COLUMN), rows intact, and a database newer than
this build refuses to open instead of being silently downgraded.
- v_atoms_current hides memories that are closed, lapsed or not yet begun; unknown
bounds read as visible. Time travel is db.atoms_as_of(moment, holder) because a
SQLite view cannot take a parameter.
- evaluation_schemas.py: the proof and benchmark report shapes move out of schemas.py
to keep it under the file-size limit; importers follow.
- backend/tests/test_memory_schema.py: 8 tests, including the v1 -> v2 upgrade and the
validity timeline.
Verification of record: pytest backend/tests bench/tests -q -> 70 passed; ruff check on
every touched file -> all checks passed; mypy (fresh cache) on db.py, migrations.py,
schemas.py, evaluation_schemas.py and backend/tests -> no issues. evals.py carries 40
pre-existing mypy errors at HEAD and 40 after this change (measured on both copies);
its pyright findings are the same debt, and p0-6 rewrites that file. `peft` and
`pyarrow.parquet` "could not be resolved" is the documented pyright artifact: pi-lens
runs it with 0 config files, and both import at runtime.
rankings.jsonl stores the seconds each query took, so the report inherited whatever
the machine was doing at the time: the frozen baseline's dense latency included a
per-query model load, and the 50,000-source row moved from 1.315s to 0.809s between
two runs of identical code.
- bench/run.py: --remeasure-latency N re-times a stride sample with warm models and
replaces each arm's latency section, recording the sample size and a note; recall
stays a full-pass number. BASELINE.md prints that note under the table.
- bench/tests/test_joint.py: 62 tests total.
Verification: pytest bench/tests -q -> 62 passed; ruff check bench/ -> all checks
passed; mypy (fresh cache) -> no issues in 25 source files.
Three defects made the external baseline numbers and labels wrong.
- bench/systems.py: _sentence_model rebuilt the model on every query, so dense arm
latency was mostly model loading and the latency-vs-size curve was flat because a
constant dominated it. Models are now cached per process.
- bench/counterpart.py: visible() and search() walked 404k Python objects per query.
Per-item holder/source/time columns are built once and the query path is numpy;
12.5 queries/s instead of 0.7.
- bench/metrics.py: the aggregate key was named wrong_holder_confidence_unanswerable
while it counts top hits that ARE from the asking holder. BASELINE.md read that
backwards. Renamed to top_hit_from_holder_unanswerable and the sentence rewritten.
- bench/run_external.py: the by-modality table read row['answerable'] but aggregate
emits n_answerable, so the writer died after scoring finished. Each store is now
scored twice, [holder scope] and [whole corpus], so the gain from the holder
boundary is separable from the gain from ranking; progress lines every 500 queries.
Verification: pytest bench/tests -q -> 61 passed; ruff check bench/ -> all checks
passed; mypy (fresh cache) -> no issues in 25 source files.
Latency measured while another job hogs the cores is not comparable: the scale curve
moved from 1.315s to 0.809s at 50,000 sources between two runs of the same code.
- bench/systems.py: machine_facts() reports cpus plus the 1/5/15 minute load average.
- bench/run.py, bench/run_external.py: both reports carry it, and BASELINE.md labels
the latency table a ceiling when the machine was busy.
Verification: pytest bench/tests -q -> 60 passed; ruff check bench/ -> all checks
passed; mypy (fresh cache) -> no issues in 25 source files.
The frozen baseline showed the unmodified RetrievalService agreeing with an equivalent
BM25 index on 0% of its top ten results. Reproduced by hand: its top ten atoms came
from one document's pages, so a source-level ranking cannot overlap it.
- bench/systems.py: live_product_rankings also counts distinct sources behind the
product's top-k atoms.
- bench/run.py: scale probes no longer filtered by gold (a source prefix keeps almost
no gold, so the 500/2,000/10,000 rows were silently skipped); scale_arm_names reads
only latency dicts, which is what crashed write_markdown on the note field; the
unanswerable share is computed instead of hard-coded.
- bench/tests/test_joint.py: 60 tests total, including the writer regression.
Verification: pytest bench/tests -q -> 60 passed; ruff check bench/ -> all checks
passed; mypy (fresh cache) -> no issues in 25 source files.
The per-arm runner recomputed BM25 and the dense matmul for every arm, which put the
full ablation at roughly 16 hours and left no way to survive a kill. JointRunner
computes each signal once per query, attributes latency to the components each arm
uses, and checkpoints every query to runs/baseline/rankings.jsonl.
- bench/systems.py: JointRunner + ARM_COMPONENTS signal attribution.
- bench/run.py: joint pass, rankings cache with a run fingerprint, per-query resume,
latency summary, resident-memory progress lines.
- bench/tests/test_joint.py: joint rankings equal the per-arm rankings for all five
arms; the sparse corpus is scored once even with three arms that need it; cache
fingerprint mismatch and malformed rows fail fast.
- bench/reports/EXTERNAL_APPENDIX.md: what a real Mem0/Zep run would cost, and which
published numbers are comparable at all.
Verification: pytest bench/tests -q -> 57 passed; ruff check bench/ -> all checks
passed; mypy (fresh cache) -> no issues in 25 source files.
The merged database names a source by content hash (`11089_4f2a...`) while the
frozen sets name the same file by its corpus id, so every arm scored 0.000 until
load_atoms joined atoms through sources.path. It now returns corpus ids and
refuses an atom it cannot trace to a corpus file.
Also fixes found by running the harness:
- merge rebuilt atom_fts from the deduped atoms table; two attempts on one file
left 288,266 fts rows for 165,973 atoms.
- merge accepts the two honest reasons a corpus file has no product source
(500 byte-identical LongMemEval sessions collapse under content hashing, and
12 photos that ONNX Runtime cannot decode) and reports them.
- --arms used action=extend against a non-empty default, so naming two arms ran
seven.
- the dense cache is keyed by the atom set, so a scale-curve subset cannot be
scored with full-corpus vectors.
- write_markdown crashed on a run that covered only some modalities.
- bench/ingest_baseline.py drives the unmodified product IngestionService over
all 50,789 corpus files in sharded SQLite DBs, then merges them. Thread and
memory ceilings are set explicitly: ONNX Runtime defaulted to every core
(95 threads/worker, load 490) and its CPU arena kept freed pages mapped,
which OOM-killed workers.
- bench/metrics.py: recall@k, MRR, nDCG@10, Wilson lower bound, chance
baseline. No model, no product import, so the scorer cannot grade itself.
- bench/systems.py + bench/run.py: the five ablation arms over the merged DB,
latency percentiles, scale curve, and a top-10 equivalence check against the
live RetrievalService.
- bench/counterpart.py + bench/run_external.py: in-repo Mem0-style flat store
and Zep-style temporal store as the external reference point.
- bench/audit.py: enforces single-attempt scoring, frozen n, a chance row
beside every score, and producer != verifier. It already flags the product's
retry-until-target loop at backend/src/ely_eye/evals.py:2351.
- backend/pyproject.toml: rapidocr-onnxruntime>=1.4.4 was unsatisfiable
(PyPI maxes at 1.2.3); the successor is the renamed rapidocr package.
Ground truth comes only from third-party annotations (MMLongBench-Doc
evidence_page_ids, VisRAG qrels, ScreenSpot-v2 bboxes, Video-MME QA,
LongMemEval answer_session_ids, CodeSearchNet func_path+func_name) and
from document structure; no model judgement is used anywhere.
- bench/corpus/manifest.jsonl: 28 sources with url, retrieved_at, sha256,
annotation_source, license, files[]; verified by bench.manifest_check
- bench/sets/{pdf,image,ui_screenshot,video,code,conversation}.{jsonl} plus
corpus indexes and a manifest that freezes every file's sha256
- 2,928 answerable + 2,380 unanswerable queries over 854 holders; every
query carries scope {holder, scenario, as_of} for transactive routing
- bench/validate_sets.py fails on missing gold, holder mismatch between
gold and query, gold observed after as_of, and manifest digest drift
Local read-only Model Context Protocol server (Go) exposing the .ely_eye memory
store to coding agents such as Codex and Claude Code: nine tools over status,
cartridges, Evidence Atoms, compiled evidence packs, proof suites, and HashHop
proofs. One-line cross-platform install (no Docker), with a prebuilt-binary
release workflow.
Real-models implementation: multimodal ingestion, Context Cartridge memory,
sparse + Qwen embedding retrieval, Cache Fabric, Qwen runtime adapters, verifier
contracts, Visual HashHop proofs, BF16 LoRA fine-tuning, and a local dashboard.
Verified by a 15-check PRD proof suite on the live model.
ZacharyZhang-NY