Commit Graph
2 Commits
Author SHA1 Message Date
ZacharyZhang-NY 9145a23f18 Require a token for writes and quarantine what nobody vouches for
Sixteen mutating routes now refuse a caller who cannot present the token the
server started with; queries stay open, because a question is not a write. The
environment variable is ELY_EYE_AUTH_TOKEN, following the repo's env_prefix
convention — the shorter name I reached for first returned 403 where the smoke
expected 401, and the server printed the reason.

Words from a stranger are quarantined on the way in: an HTTP upload is
third_party by definition, and a sentence carrying an instruction-shaped phrase
is quarantined whoever wrote it. A quarantined atom stays findable and stays out
of the compiled context until a second, live, unquarantined source agrees with
its rarest words. Corroboration is computed at read time because it is a
property of the corpus now, not of the moment a memory was written; it is
term-level, and the docstring says so.

forget now erases the source row and the compiled contexts beside the atoms, so
a forgotten memory leaves nothing that can still speak. The semantic cache key
carries a store revision, which fixes a real staleness: a context compiled
before new evidence arrived kept its old quarantine decision forever.

bench.audit --product still pointed at evals.py, which the p1-6 split moved into
backend/src/ely_eye/proofs/. It failed fast, but the anti-self-justification
rule had quietly stopped auditing the product; it now walks the package, and a
test pins that the package exists and holds no re-roll.

Gates: pytest 306 passed; ruff clean; mypy cold cache 24 errors in 9 files, the
recorded baseline, net zero; pyright 0 errors on the 23 backend files and the 2
bench files; go build/vet/test and gofmt clean; npm run build 221.39 kB with
eslint exit 0; bench/filesize_check clean; bench.audit --product clean over 31
harness files; scripts/mcp_write_smoke.sh PASS with the gate live across two
servers and the MCP child.
2026-10-08 23:05:15 -04:00
ZacharyZhang-NY fc0698c242 Add baseline harness: ingest, metrics, ablation arms, external counterpart, audit
- bench/ingest_baseline.py drives the unmodified product IngestionService over
  all 50,789 corpus files in sharded SQLite DBs, then merges them. Thread and
  memory ceilings are set explicitly: ONNX Runtime defaulted to every core
  (95 threads/worker, load 490) and its CPU arena kept freed pages mapped,
  which OOM-killed workers.
- bench/metrics.py: recall@k, MRR, nDCG@10, Wilson lower bound, chance
  baseline. No model, no product import, so the scorer cannot grade itself.
- bench/systems.py + bench/run.py: the five ablation arms over the merged DB,
  latency percentiles, scale curve, and a top-10 equivalence check against the
  live RetrievalService.
- bench/counterpart.py + bench/run_external.py: in-repo Mem0-style flat store
  and Zep-style temporal store as the external reference point.
- bench/audit.py: enforces single-attempt scoring, frozen n, a chance row
  beside every score, and producer != verifier. It already flags the product's
  retry-until-target loop at backend/src/ely_eye/evals.py:2351.
- backend/pyproject.toml: rapidocr-onnxruntime>=1.4.4 was unsatisfiable
  (PyPI maxes at 1.2.3); the successor is the renamed rapidocr package.
2026-10-06 14:36:02 -04:00