Commit Graph
2 Commits
Author SHA1 Message Date
ZacharyZhang-NY 9145a23f18 Require a token for writes and quarantine what nobody vouches for
Sixteen mutating routes now refuse a caller who cannot present the token the
server started with; queries stay open, because a question is not a write. The
environment variable is ELY_EYE_AUTH_TOKEN, following the repo's env_prefix
convention — the shorter name I reached for first returned 403 where the smoke
expected 401, and the server printed the reason.

Words from a stranger are quarantined on the way in: an HTTP upload is
third_party by definition, and a sentence carrying an instruction-shaped phrase
is quarantined whoever wrote it. A quarantined atom stays findable and stays out
of the compiled context until a second, live, unquarantined source agrees with
its rarest words. Corroboration is computed at read time because it is a
property of the corpus now, not of the moment a memory was written; it is
term-level, and the docstring says so.

forget now erases the source row and the compiled contexts beside the atoms, so
a forgotten memory leaves nothing that can still speak. The semantic cache key
carries a store revision, which fixes a real staleness: a context compiled
before new evidence arrived kept its old quarantine decision forever.

bench.audit --product still pointed at evals.py, which the p1-6 split moved into
backend/src/ely_eye/proofs/. It failed fast, but the anti-self-justification
rule had quietly stopped auditing the product; it now walks the package, and a
test pins that the package exists and holds no re-roll.

Gates: pytest 306 passed; ruff clean; mypy cold cache 24 errors in 9 files, the
recorded baseline, net zero; pyright 0 errors on the 23 backend files and the 2
bench files; go build/vet/test and gofmt clean; npm run build 221.39 kB with
eslint exit 0; bench/filesize_check clean; bench.audit --product clean over 31
harness files; scripts/mcp_write_smoke.sh PASS with the gate live across two
servers and the MCP child.
2026-10-08 23:05:15 -04:00
ZacharyZhang-NY 674c88e84e Rank memory search in SQL instead of rebuilding BM25 in Python
The product path pulled every atom into Python and rebuilt a BM25Okapi
index per query: p50 20.16s over 165,973 atoms. It now runs one SQLite
statement with LIMIT/OFFSET.

- text_index.py: one normalizer for both sides of the index. CJK runs are
  split per character (unicode61 cannot segment them), camelCase and
  separators are split so "search atoms" finds SearchAtoms, latin tokens
  get a prefix match. Rows written before this existed hold raw text in
  atom_fts, so schema v3 rebuilds the index on upgrade.
- retrieval.py: filters (holder / scenario / source / modality / since /
  until / min_trust), a base64 cursor, and reciprocal rank fusion in place
  of the 0.55/0.35/0.10 blend whose graph term was always zero.
- An empty result is {status: "no_match"}, not a bare array.
- Go: search_atoms and compile_context rank through POST /api/search. The
  second Go tokenizer is deleted; two implementations could drift, which
  is what the MCP/HTTP parity requirement forbids.
- db.py passed 500 lines, so the row codec moved to row_codec.py.
- Tooling: installed the repo's own pyright. pyrightconfig.json had no
  exclude and swept the downloaded corpus (1171 errors); with excludes the
  count is 62, all pre-existing (evals 48, profiles 4, training 4,
  ingestion 3, runtime 2, cartridge_assets 1). backend/pyproject.toml
  carried a second [tool.pyright] that shadowed the root config; deleted.
- bench/scale_check.py measures the product search at corpus size. The
  100k-atom p50 number is not written until it is measured on an idle
  machine.

Verification: 112 pytest passed; go build/vet/test clean, gofmt clean;
ruff clean on every touched file; mypy clean on the 14 touched Python
files; repo pyright 62 errors, unchanged per-file ledger.
2026-10-07 16:02:40 -04:00