The dense leg used to be a pair of files next to the database: a full .npy
re-encoded from every atom, re-normalized on every query, and stale the moment
anything was ingested. It is now a vec0 table inside the same SQLite file,
written in the same transaction as the atom it describes, so a memory is
queryable the instant it is remembered.
- vector_index.py maintains the table; the writer inserts, the reader never
rebuilds. A repair scan (ely-eye build-dense-index) catches what a crash hid,
and it is not on the search path.
- Ranking stays in the engine; every boundary (closed, retired, not yet true,
another holder's private memory, the mounted cartridge) is checked once against
the atoms table afterwards. A visibility subquery inside the KNN cost 48.9s on
153,857 vectors; the same check over 200 candidates costs 0.2ms. When the
boundaries reject enough candidates the ask widens once, never twice.
- The Go mirror and the web panel carry the new status fields.
- bench/dense_scale.py and bench/quant_probe.py measure the leg at corpus size;
bench/scale_check.py now works on a copy so the frozen baseline stays frozen.
Measured on the frozen 165,973-atom corpus. bench/reports/DENSE.md, QUANT.md and
SCALE.md carry the numbers and the machine they were taken on.
Gates: 285 pytest, ruff clean, pyright 0 on the touched files, go build/vet/test
and gofmt clean, frontend build and eslint clean. mypy is 25 findings against the
parent's 24: the new one is "Skipping analyzing sqlite_vec: missing library stubs",
the same class as the two existing qwen_vl_utils lines, and the repo keeps no
type: ignore anywhere.
9f91ff7 pushed schemas.py from 498 to 534 lines and broke the 500-line rule
the repo enforces on every file. The status family (CacheLayerStatus,
MemoryStatus, RuntimeStatus, SenseStatus, SystemStatus) moves to
status_schemas.py, where it is self-contained: it needs only
TrainingAdapterKind from schemas.py, and schemas.py no longer looks back.
The same commit added three multi-line docstrings (browse.picture,
senses.sense_facts, SenseStatus). Each is now one line; the absent-vs-missing
distinction they carried survives in SenseStatus's own line, and the producer
senses.py still states why the two failures need different sentences.
Measured, not asserted: 263 tests pass; ruff clean on the eight touched
files; repo-wide mypy still 24 errors in 6 files (runtime.py's two are in
that baseline and reproduce on the HEAD copy of the file); pyright reports
the same two pre-existing runtime.py tuple findings on both trees;
multi-line docstrings repo-wide are back to 114, the parent's count.
ZacharyZhang-NY