Commit Graph
2 Commits
Author SHA1 Message Date
ZacharyZhang-NY 3b87b77b7f Measure the last key-test scenario instead of writing it off
ci / backend (push) Failing after 3m19s
ci / mcp (push) Successful in 1m1s
ci / interface (push) Successful in 20s
ci / image (push) Failing after 5m54s
Scenario 7 claimed nothing was observable. It is: compiling the same
question twice hands back the same pack and the second compile is served
from the semantic cache (cache_events layer=semantic hit=1 miss=1), so the
row is now PARTLY MEASURED with those numbers and only the time-to-first-token
half stays unmeasured. Scenario 4 gains the settling step: an unresolved
disagreement shows the newer side, and after resolve-conflicts one page shows
both readings (measured 2 hits) — settlement is a retrieval behavior, not a
deletion, which is also why my first draft of that scenario was wrong.

Gaps.md §6 now carries a verdict table with what each scenario actually
measured, and §0.5 quotes whole-suite recall (0.850 words-only over 2,928
answerable queries) instead of the 300-probe sample's 0.863, naming why the
sample reads high.

Gates: 422 passed, ruff clean, filesize OK, bash -n clean, key test 6 PASS +
1 PARTLY MEASURED with exit 0.
2026-10-10 03:35:39 -04:00
ZacharyZhang-NY eab6bd7c8c Answer from the moment a question asks about
The key test in Gaps.md §6 asks what was true last week. It could not be
answered: search_atoms joined v_atoms_current, so a superseded memory is
unreachable no matter what since/until says — those two fields bound when a
memory was true, not which moment the store answers from.

SearchFilters.as_of moves that moment. Without it the words leg repeats the
view's currency predicate verbatim; with it a memory answers when
valid_from <= as_of < valid_to, its event time has arrived, and it is inside
its retention. archived_at is ignored on purpose: archival is a storage tier,
not a claim about the world, so an archived memory still answers for the time
before it was archived. The dense leg reads the same window through
vector_index._allowed, so the two legs cannot disagree about what existed.

scripts/key_test.sh runs the seven §6 scenarios end to end against one live
store — HTTP, the Go MCP binary over stdio, and the CLI mounting cartridges.
Six pass. The seventh, "a cartridge saves compute", is reported NOT MEASURED
rather than passed: it claims time-to-first-token and a cache-hit rate, no
generator has ever run here, and there is no compile or cache route to observe
even the second half.

Gates: pytest -q 406 passed; ruff clean; filesize OK; pyright 0 on the seven
touched files; mypy cold cache Success on the same seven; key test exit 0.
2026-10-09 18:25:30 -04:00