Scenario 7 claimed nothing was observable. It is: compiling the same
question twice hands back the same pack and the second compile is served
from the semantic cache (cache_events layer=semantic hit=1 miss=1), so the
row is now PARTLY MEASURED with those numbers and only the time-to-first-token
half stays unmeasured. Scenario 4 gains the settling step: an unresolved
disagreement shows the newer side, and after resolve-conflicts one page shows
both readings (measured 2 hits) — settlement is a retrieval behavior, not a
deletion, which is also why my first draft of that scenario was wrong.
Gaps.md §6 now carries a verdict table with what each scenario actually
measured, and §0.5 quotes whole-suite recall (0.850 words-only over 2,928
answerable queries) instead of the 300-probe sample's 0.863, naming why the
sample reads high.
Gates: 422 passed, ruff clean, filesize OK, bash -n clean, key test 6 PASS +
1 PARTLY MEASURED with exit 0.
Round four of the independent review found seven record and coverage
defects. The functional side it verified by running; these are the fixes.
- `_tracked` asked the current checkout's index, so a reviewer in a worktree
pointing at the main checkout's committed report was told it was untracked.
It now asks the repository that owns the path, and the guard no longer
compares directories: any tracked file is committed evidence.
- `retrieval.py` claimed the dense index "catches up inside search". It does
not: `_stale` is only used by `sync()` and `stats()`, so a store upgraded
from v11 ranks an empty dense leg until `ely-eye build-dense-index` runs.
The docstring now states that, and `doctor` already names it.
- Four wirings had no discriminating test, each proven by mutation:
a stubbed `_tracked` (now a test uses the real git index), `copy_store`
claiming the scratch (now a live holder blocks the copy before a byte is
copied), `_stale` reading the atoms table (now a closed memory that loses
its vector is owed one), and the v10/v11 to v12 `DROP VIEW` gate (now a
hand-made v11 store is upgraded and its view body re-checked).
- The record corrected against the reviewer's measurements: the five
mutations did not each turn one test red (M1 turned two, M3 turned two,
`_stale` alone turned none), the pyright/mypy file set was wrong, and the
round added seven tests rather than three.
Gates: 422 passed; ruff clean; filesize clean; audit clean; mypy 24 errors in
9 files repo-wide, equal to the 9cb46c1 baseline; pyright 0 on the nine
touched files; zero whole-line comment blocks over 157 tracked files.
The 0.90 bar is 0.035 away, and weighting the measured per-modality recall by
the answerable query counts shows the whole deficit sits in the visual legs:
ui_screenshot 0.818 is worth 0.0144, video 0.493 even at +0.256 from audio is
worth 0.0189, image 0.720 to 0.85 is worth 0.0107. All three together clear
the bar; any one of them alone does not. So the remaining work is how a
picture is represented, not how the ranking is tuned.
p2-6 records that arithmetic, the audio run already measuring it, and the one
cheap idea left for screenshots - tile the screen so the picture leg encodes
regions instead of one whole-screen vector, which is what made the picture leg
cost ui_screenshot points in VISUAL.md. If tiling does not pay, the arm gets
deleted and the report says so.
A scratch copy is one run's working store. The video harness defaults to
bench/runs/video_audio and copy_store() copies straight over it, so a smoke
run started while the full run was mid-ingest replaced the database under it:
the full run's after-arm would have scored a store whose speech atoms had been
swapped out, and the smoke itself died on "database is locked" inside
text_index. Six harnesses share that copy path, so the claim lives in
bench/scratch.py: a live pid in held_by refuses the path, a stale one is taken
over, and the same process may re-copy per arm.
The reviewer's smoke knob also measured nothing instead of saying so. With
--limit the probe sample narrows to video only, and a small --probes strides
past that modality entirely, so the run crashed in aggregate() after the
ingest. probe_sample() now switches the arm off at --probes 0 and fails in
half a second otherwise.
The frozen-report guard had two holes found by running it: it compared
cwd-relative parents, so an absolute path to the frozen file passed, and it
refused the harness's own default destination once that file existed, which
would have stopped the shipped run. It now compares resolved paths and refuses
only committed evidence.
Record fixes from review round 3: the round-2 line credited a reproduction to
the wrong round and quoted a git log -S hit for a string that never existed in
code; the suite counts now name the commit each was measured on (401 at
83dfc14, 406 at eab6bd7, 410 at cf2de31, 411 at 92237c8).
Gates: pytest -q 418 passed; ruff clean; bench.filesize_check OK; mypy cold
cache and pyright clean on the four touched files.
The second review round of p2-4 measured three behaviour defects behind the
as_of feature it had just accepted, and all three came from one mistake: the
code asked "is this memory current?" in three places and answered differently.
- A memory that has closed kept no vector at all, so the dense leg could only
exclude an old fact, never recall it. Vectors now follow existence, not
currency; `_allowed` decides who may see one.
- "Open right now" meant `valid_to IS NULL` in the view and in the words leg
but `valid_to IS NULL OR valid_to > now` in the vector leg, so a supersede
booked for next year hid a fact that is still true today. All three now read
the same rule; schema v12 records the change and the upgrade drops the old
view, since CREATE VIEW IF NOT EXISTS would have kept the old body.
- The dense as_of test was名不副实: dense is off by default, so it passed while
the vector side had no coverage. It now turns dense on with a stand-in model
and pins both `VectorIndex.search(as_of=…)` and the plumbing in retrieval.
Five mutations, each caught by exactly one test: the dense as_of branch, the
retrieval hand-off, the three vector reads, the words-leg currency, the view.
Record side: the round-2 line quoted a D-list I had misread out of the
reviewer's PASS tables (cf2de31 fixed those, and they were its precision notes,
not its findings), the stale line-number parenthetical, an acceptance grep that
scanned CHECKLIST's own quotations, and a Gaps.md tally that disagreed with its
own table (8 closed, 3 closed with reservations, 1 partial).
Gates: 411 passed; ruff, filesize, audit 33/2/6 clean; pyright 0 on the six
touched files; mypy cold cache Success on the same six; scripts/key_test.sh
exit 0 (6 PASS, 1 NOT MEASURED).
Round 2 found the functional side true and six record/discipline defects.
- D9: the write-path line said "comparison uses secrets.compare_digest" as if
it covered both tokens. It does not: the operator's plaintext is compared with
compare_digest (api_auth.py:54, :110), a holder token is SQL equality on its
sha256 digest (holder_store.verify_token) because no plaintext is stored. The
boundary is now stated in verify_token's docstring as well as the record.
- D10/D13: three historical numbers were written as if they were current. The
four-file 59 is pinned to 2ea48d8 (test_trust_quarantine.py was 25 tests there,
26 from d56996b on), and both 401 gate lines are pinned to the commit they were
measured on (83dfc14, 89c880f) with the current 406 named next to them.
- D12: the compiler cache-key line overstated one direction. The old key had no
identity part at all, so whoever compiled first got served to the next caller;
an empty pack reaching the operator is one direction, not the mechanism.
- D11: the p2-5 acceptance cell's suite count is pinned the same way.
- D14: bench/video_audio_check.py wrote straight into bench/reports/, so a
reviewer running it in its own worktree would have overwritten frozen
evidence. It now takes --report/--payload and refuses a destination that
already holds a frozen report; refuse_frozen_overwrite is tested, along with
the guards that stop the harness from measuring nothing.
Gates: pytest -q 410 passed; ruff clean; bench.filesize_check OK;
bench.audit --product 33 harness files / 2 reports / 6 set files, clean;
pyright 0 errors on the three touched files; mypy cold cache Success on the same
three.
Round one of the p2-4 review found the behaviour sound and the record wrong
in eight places. All eight are fixed here, each against a measurement:
D1 "compare_digest" was written as if it covered holder tokens. It does not:
the store keeps only a sha256 digest, so verify_token is SQL equality;
compare_digest guards the operator's plaintext (api_auth.py:54, :110).
D2 register_minimal is called at memory.py:94, not :120.
D3 the compiler cache key never had an "as_holder or ''" step — the old key
had no identity component at all, which is a stronger defect than the one
I described.
D4 browse.py's own comment said four copies, two here. There are five, three
here (:52, :66, :93), which is what test_visibility_rule asserts.
D5 the store_search / vector_index line pointers moved with the as_of work.
D6 my review prompt said 33 routes; measured 34 decorators / 31 distinct
paths at the parent, 38 / 35 at the reviewed commit.
D7 the semgrep claim cannot be re-run in this container (neither semgrep nor
opengrep is installed), so it is recorded as an observation from the p0-1
gate run, and the anti-drift job is given to the test that does not need
that tool.
D8 "a holder can only write as itself, now guaranteed by the token" is the
opposite of what runs: only the operator writes, attribution comes from
default_holder, and the real guarantee is that RememberRequest has no
holder field.
Four precision notes are recorded too: a revoked token is a hard 401 rather
than a downgrade to anonymous, an anonymous caller passing an empty ?holder=
gets 403, the capability card also carries newest_memory timestamps and
token_issued, and leg_check's commit line is the cwd's HEAD.
The p2-3 entry gains the as_of gap the key test exposed, and the final-accept
entry now carries the seven measured scenarios with the two reasons it stays
open: scenario 7 unmeasured, and recall@10 0.863 against a 0.90 bar.
The README described a project that no longer exists: a Windows-only Qwen
runtime with nine read-only tools, a console as the only way to operate the
store, and a 100M-token capsule as if it were measured. It is now a description
of what is here, with the report and the command that re-measures each number,
and a section that names what has not been run instead of leaving the reader to
discover it.
TRAINING_ARTIFACTS.md keeps its artifact tables and loses the scores that were
never run: MMMU-Pro and OCRBench each read one sample, and the HashHop gate
collected proofs until it hit its target. What stays is annotated with what it
actually measures — citation accuracy there means the cited id existed in the
context that was handed over, not that the passage supports the answer.
Gaps.md gains a re-audit section: ten of the twelve gaps closed, two partial,
each row naming the command or report behind it, and the remaining work stated
plainly — the 0.90 recall bar is unmet and the deficit is entirely in the
visual and audio legs.
AGENTS.md stops being ignored. It records the constraints this repository paid
for: the measurement rules, the environment traps, and the review protocol, so
the next agent does not pay them again.
The CLI could filter by holder but could not look through a holder's eyes, so
search gains --as-holder: three states of eyes on every face, not two.
A holder is now somebody who can prove it. Schema v11 stores a sha256 of a
per-holder token beside the directory row, so a copied database cannot replay
it and the plaintext is shown exactly once. The operator token mints and
revokes those tokens; a holder token reads that holder's own memories and
nothing else; naming somebody else without the operator token is refused with
both ids in the message.
The visibility rule was leaking before this: browse had none at all, and the
search rules short-circuited to the whole library whenever no holder was
named. The rule is now written out in five copies with one parameter carried
four times, and a test compares the copies so they cannot drift — semgrep
rejects an interpolated constant, so the copies are not a choice.
Eyes carry three states on purpose. NULL means the whole library, which is
what a process that owns the store is entitled to; a holder id means that
holder; the empty string means nobody, which is what a tokenless HTTP caller
is. Making NULL mean nobody instead would have forced every internal caller —
CLI, proofs, the compiler, the bench harnesses — to name eyes, contradicting
the rule that a local process reads as its own trusted principal.
The capability card answers "can this store answer my kind of question" before
anything is sent. It is derived from what the store holds, so it cannot
promise a modality the store lacks, and it carries no memory text, so it is
safe to read before trusting the store. HTTP, CLI and one MCP tool all serve
it without a token.
The smoke script's build fingerprint asserted on the MCP reply, but the Go
layer decodes the HTTP answer into its own struct and re-emits a missing field
as zero, so a server from before the memory tiers still passed. Point the
fingerprint at the Python answer instead: /api/memory/working answers 404 on
4c6af87 and 200 here, both measured by starting each build myself.
Record the miss side of the rename rule. Whether a bare number renames its
subject depends on that memory's own sentence, so 「温度」 and 「温度24」 never
compare and an update goes unnoticed; both memories stay on the answer page,
so the miss costs a notice rather than a wrong number. Pin it with a test and
keep the mechanism, since comparing an unrenamed subject as a fallback would
put room numbers back in the ring as readings.
Correct the direction of the rule in the record: the rename slot is the token
after the subject word, not before it. A leading number becomes the subject
rather than fusing forward, which 「3 room holds 22 degrees」 shows by taking
「3」 as its subject.
门禁：pytest 384 passed；六文件 83（conflicts 27）；三文件 35；ruff clean；
filesize clean；audit 32/2/6 clean；mypy 冷缓存 24 errors / 9 files（净 0）；
pyright 0；go 全套 + gofmt 空；npm 产物未变；smoke PASS。
A two-character Chinese subject — 温度, 室温, 设定, 预算 — is the common
way to state a reading, and the rename rule was swallowing its number:
「温度 22 度」 became a thing called 温度22 with nothing left to report, so
the lane recorded no conflict at all and no later memory could disagree
with it. A bare number written onto the subject is a name only while the
memory still has another number waiting behind it; with nothing after it,
it is the reading. Measured across 33 pairs: 6 change — five missed
conflicts now recorded, plus one new disclosed boundary (「Room 3 is big」
and 「Room 4 is big」 read as one quantity that moved, because a label
cannot tell an identifier from a reading). Everything else is byte-equal.
Also corrected: a memory that is only a number does have a subject — the
number itself — so 「22 度」 and 「24 度」 record nothing because they name
two different things, not because the store found nothing to compare.
The smoke script now refuses an answer from a server that predates the
memory tiers: it requires recall_count in the remember response, a field
the pre-p2-3 Go build never emits.
A quantity conflict compared labels, and a label can carry a number that
identifies the thing rather than measuring it. 「Room 3 holds 22 degrees」 and
「Room 4 holds 24 degrees」 shared the subject 「room」, so the store recorded a
conflict over the room numbers and dropped Room 3 from the answer page: asking
about degrees returned one room's reading and hid the other room entirely.
「The router 1/2 forwards 220/240 packets」 and 「2026 saw 3 / 2027 saw 5
releases」 failed the same way.
A bare number standing before any predicate is part of the subject's name, so it
joins the subject key and is never recorded as a value. A number that stands past
a word still measures: 「The rent is 2200 / 2400 dollars」 conflicts over rent,
and a number fused inside a CJK run stays a value, because 「恒温器调到」 states its
predicate inside the subject token and the motivating pair must keep conflicting.
The same year counting twice is now a conflict (「2026 saw 3 / 5 releases」), which
the old rule missed.
Only a number renames a subject: 「Room A」 and 「Room B」 still read as one room
whose reading moved, and an English qualifier does not split a subject the way a
Chinese one does. Both boundaries are disclosed and pinned by tests rather than
smoothed over.
Also collapses six two-line comment blocks back to one line each, which is the
rule this repo already enforces, and corrects the record where it claimed the
nearest label was 「书房」 (it is 「设定」) and that the subject rule only ever
missed conflicts.
Gates: pytest -q 379 passed (conflicts 17 -> 22, six p2-3 files 78, three tier
files 35); ruff clean; bench.filesize_check clean; mypy cold cache 24 errors in 9
files, byte-identical to the recorded 9cb46c1 baseline; pyright 0 errors on the
touched files; go build/vet/test and gofmt clean; npm run build unchanged at
index-CSDHCFNo.js; mcp_write_smoke.sh PASS; whole-line comment runs of length >=2:
0 across tracked Python files.
A shared verb is not a shared fact. 「客厅的恒温器调到 22 度」 and
「卧室的恒温器调到 24 度」 agree with each other; the store had been
recording them as one thermostat moving, and the older side then left the
answer page — asking about the living room returned the bedroom's reading
as the only evidence. A conflict now requires the two memories to lead
with the same subject, and the subject is the first content word rather
than the first word, so 「The router forwards 220 packets」 still leads
with its router behind the article.
The rule trades one error for another, and both directions are measured
and pinned: a qualifier on the subject reads as a different thing, and
the first number under a label is the value that gets compared. Neither
hides a memory — each leaves a disagreement unrecorded for a person to
supersede by hand.
Six review findings, all measured before being changed:
- the archive rule string hid that an unset importance archives too
- a never-archived id reported {"restored": 1} while nothing moved
- the suppression condition (both sides in the page) was undocumented
- a determinism claim had no reproducible path, two latency numbers were
never committed, and one cited test name did not exist
Gates: 374 passed, six acceptance files 73 passed, ruff clean, mypy cold
cache 24 errors in 9 files (the recorded 9cb46c1 baseline, net 0),
pyright 0 on the touched files, go build/vet/test + gofmt clean, npm
bundle unchanged, product audit clean, MCP smoke passing across processes.
The tier report is regenerated against a52c104, which carries the harness
that produced it, and now names the schema the product actually ran on: the
copy arrives at v9, the product migrates it on open, every arm ran on v10.
Every recall cell is identical across three runs on three different machine
loads (0.847 / 0.863 / 0.873 / 0.860 / 0.873 / 0.817 / 0.848), which is the
regression proof p2-3 needs: the tier code adds state and rules, not a
ranker, and this corpus holds no conflicts and no archived rows. Latency is
not comparable across those runs (p50 101.6 / 87.5 / 122.5 ms at load 27.4 /
21.4 / 17.2) and the report says so.
The memory subset is now measured whole — all 150 answerable conversation
queries instead of 15 sampled probes — because a 15-probe sample leaves one
to three queries per LongMemEval question type. It gives the two categories
the contract asks for: knowledge-update 1.000 and temporal-reasoning 0.879
as shipped, 0.950 with the dense leg. The newest-first column is the sharper
question beside them: 0.52 / 0.48 / 0.37 as shipped, so a perfect recall can
still lead with a value the speaker has since replaced.
CHECKLIST p2-3 is marked with the measured cells, the two real defects the
tests caught (a consolidation watermark that double-counted under a lagging
clock, and typer calling a variadic command callback by keyword), and the
deviation that decay is not a stored field because an unmeasured decay
formula would be a story.
Eight rounds, 29 findings, the last two rounds measuring only the
record. The reviewer also walked a third smoke path I had not tested:
a non-Ely-Eye HTTP server on the smoke port slips past the port guard
because it answers /api/status with 404, and is caught by the process
liveness check instead.
Two precision notes from that round are folded into the limitation
line: the demonstrative and interrogative words are on the stop-word
list too ("This is important" and "What caused the outage" are never
citable), and the sentence that dies through "over" also dies through
"she".
The limitation line listed "her" as a word the fixed list removes. It
is not on the list, and that matters: "her" is still counted as a
demanded word, so "I can meet her tomorrow" survives with
tomorrow / meet / her, while "She looked over the report" dies through
"over". The line now names the measured membership instead.
The same row described "ai" still occupying a slot as behaviour from
before the fix. Nothing ever changed "ai" — it was never a function
word — so the observation is current, and the row said both things at
once.
Also pinned the D15 example to its full string: the ellipsis made it
non-reproducible, and lengthening the same claim moves the middle
position onto filler and flips the verdict, which is the boundary the
next row already discloses.
Gates after the edit: 325 passed, ruff clean. Code untouched since
b696fc7.
The function-word list is fixed on purpose, and a fixed list cannot tell
"the" from "May": both are on it, so a claim built only from words that
double as function words offers nothing to compare and is never citable.
Measured — The meeting is in May, I will do it tomorrow and It can do
that all yield no claim terms, while Will Smith signed the lease still
yields signed / smith / lease. That is fail-closed, but the docstring
claimed the opposite ("only counted if it carries content") and the
checklist never said it. Both now state it, and a test pins it.
The checklist also carried three wrong pins for that list (line, size,
one-line comment) and named "ai" as already excluded when it is not: a
mixed claim still demands ['档案','温器','八度','ai']. The mechanism line
said "the three longest words", which is true of Latin and false of
CJK, where the demanded pairs are spread across the claim.
The smoke script trusted any answer on its port. With a server left over
from another run it measured that one and reported a false failure; it
now refuses a busy port and checks that its own process is still alive
before calling the server ready.
Gates: 325 passed, 60 in the four acceptance files, ruff clean, mypy
cold cache 24 errors in 9 files (the recorded baseline, net zero),
pyright 0, filesize clean, product audit clean, zero multi-line comment
blocks across 141 tracked files, smoke passes and exits 1 on a collision.
A claim too short to compare in its own script was rescued by a Latin
filler: 恒温器 the offered two Chinese pairs, the rule asked for three
terms, and "the" completed the set — so the claim was corroborated by
anything that happened to carry the same filler. English was isomorphic:
Pineapple answer stayed uncited, Pineapple answer the did not.
The shortness test is now per script, and function words are out of the
demanded set. A mixed claim still asks for its three Chinese positions
plus its longest real Latin word, so dropping "the" from the demand no
longer blocks a legitimate witness: 档案室的恒温器保持十八度 the is now
corroborated by 恒温器一直放在档案室，读数十八度, which the last review
round measured as False.
A witness that translates the demanded Latin word is still read as
disagreement, and a witness that inflects a demanded verb
(holds / hold) is too: the rule is term-level, and the checklist now
says so with both measurements instead of the earlier blanket claim.
Gates: 324 passed, 59 in the four acceptance files, ruff clean, mypy
cold cache 24 errors in 9 files (the recorded baseline, net zero),
pyright 0 on both touched files, filesize clean, product audit clean,
MCP write smoke passes.
A claim written in Chinese and English was checked only in English: the
longest Latin words took every slot the rule demands, so one planted
English word moved the closing Chinese pair out of the comparison and a
witness that contradicted the number became a second source. The attack
costs nothing — append "the" to a Chinese sentence — and it fails open,
which is the wrong direction for a rule whose whole job is to keep an
unvouched claim out of the prompt.
Each script is now checked in its own terms: a claim with CJK pairs is
demanded at its opening, middle and tail, plus its single most
distinctive Latin word. A Latin-only claim keeps its three longest
words. The demanded set is what claim_terms returns, all of it.
Measured: the planted-word witnesses are no longer believed, a verbatim
repeat still is, and a mixed claim that agrees in only one script is not
a witness. The term-level boundaries are listed in CHECKLIST for both
scripts, including the two the reviewer found that this rule does not
catch: a demanded pair inside a longer word, and a long claim whose
middle position lands on filler.
Also corrects the record: the boundaries were not the same in both
scripts as claimed, a historical line said "now" instead of naming a
commit, and a limitation line I deleted in the last round is restored.
Gates: 322 passed, ruff clean, mypy 24 errors / 9 files (baseline, net
zero), pyright 0 on the two touched files, filesize clean, product audit
clean.
An independent reviewer planted two witnesses that contradict their claims
and both were believed: 「恒温器很贵」 was corroborated by 「恒温器很便宜」,
「这房子值得买」 by 「这房子不值得买」. The Chinese pairs were spread inside
each run on its own, and equal-length pairs kept insertion order, so the
three demanded pairs of a multi-clause claim all came from its first
clause — and for a four-pair claim the evenly spaced index picked 0, 1, 2,
clustering them at the start anyway. An unrelated memory that merely
opened the same way pushed a quarantined sentence into the packed context.
Pairs are now pooled across the whole claim and the demanded positions are
its start, its middle and its end. The last pair is the assertion, so a
witness has to agree with the claim rather than merely introduce it. The
term-level boundary is pinned in both languages: a restatement that keeps
the compared positions counts, one that replaces them does not, a denial
that differs by one word still counts, and a changed number does not.
Also corrected the record where it had drifted: the gate cell was labelled
with a commit that never produced those numbers, the acceptance cell pinned
a total its own command could not reach, and the reader-path count stated a
conclusion without its measurement rule.
Gates: 320 passed; ruff clean; mypy cold cache 24 errors in 9 files, net
zero against the recorded baseline; pyright 0 errors on the touched files;
filesize, product audit, go suite and the two-principal MCP smoke all pass.
An independent reviewer repeated the reviewer's own counterexample in
Chinese and found the corroboration channel was dead for it: fts_terms
counts one CJK run as one term, so a Chinese claim could never reach the
three words the rule demands, however many independent sources repeated
it. A run is now broken into its adjacent two-character pairs and three
are taken from across the sentence, so a second source has to agree in
three separate places rather than copy one stretch of characters. The
pairs belonging to an injection phrase are stripped too, so an
instruction cannot vouch for its own characters.
Corroboration stays term-level in both languages: a paraphrase that
keeps the meaning and changes the wording is not a second witness, and a
claim with fewer usable words than the rule demands is never believed.
The record was wrong in four places the same review measured: the token
rename breakdown summed to 11 while claiming 13 across 6 files (it is 13
across 7), the acceptance cell counted tests that no longer exist, the
gate cell quoted the previous round's totals, and a cited line had
drifted onto a blank. The section also stops calling the injection work
"sanitization": 19 literal phrases plus quarantine plus one prompt
sentence is detection, and a reworded directive walks straight past it.
An independent reviewer broke the quarantine rule twice by running it. In a
two-atom library, a second source repeating the claim word for word could not
vouch for it; 40 unrelated memories could. The old rule handed the choice of
words to rare_terms, which drops words common to the corpus, so the corpus size
and the attack's own words decided whether a stranger was believed.
Corroboration now demands the claim's three longest words, with the instruction
phrases stripped out, ANDed by the new text_index.fts_all_expression. Neither
the library's size nor the attack can flip it, and both cases are tests.
Also from that review:
- A question is a read. The DF cache no longer remembers a word the corpus has
never used, so an unauthenticated search cannot grow the store one row per
novel word. "A question is not a write" was false for /api/search and /api/ask.
- The prompt now states that packed evidence is quoted material, not an
instruction: once a quarantined sentence is corroborated it does reach the model.
- ForgetReport's semantic_cache_cleared reached the frontend as well as Go.
- runtime.py: process_vision_info returns a third element only when
return_video_kwargs is set, and this call does not set it, so the unpack takes
the first two. Two pre-existing pyright errors go, behaviour is unchanged.
CHECKLIST corrected against measurement: 13 renamed lines rather than 11, and
10 English / 9 Chinese injection signals rather than 11 / 8.
Gates: pytest 312 passed, ruff clean, mypy 24 errors in 9 files (net zero),
pyright 0 errors on the five touched files, npm build and eslint clean,
scripts/mcp_write_smoke.sh PASS.
Sixteen mutating routes now refuse a caller who cannot present the token the
server started with; queries stay open, because a question is not a write. The
environment variable is ELY_EYE_AUTH_TOKEN, following the repo's env_prefix
convention — the shorter name I reached for first returned 403 where the smoke
expected 401, and the server printed the reason.
Words from a stranger are quarantined on the way in: an HTTP upload is
third_party by definition, and a sentence carrying an instruction-shaped phrase
is quarantined whoever wrote it. A quarantined atom stays findable and stays out
of the compiled context until a second, live, unquarantined source agrees with
its rarest words. Corroboration is computed at read time because it is a
property of the corpus now, not of the moment a memory was written; it is
term-level, and the docstring says so.
forget now erases the source row and the compiled contexts beside the atoms, so
a forgotten memory leaves nothing that can still speak. The semantic cache key
carries a store revision, which fixes a real staleness: a context compiled
before new evidence arrived kept its old quarantine decision forever.
bench.audit --product still pointed at evals.py, which the p1-6 split moved into
backend/src/ely_eye/proofs/. It failed fast, but the anti-self-justification
rule had quietly stopped auditing the product; it now walks the package, and a
test pins that the package exists and holds no re-roll.
Gates: pytest 306 passed; ruff clean; mypy cold cache 24 errors in 9 files, the
recorded baseline, net zero; pyright 0 errors on the 23 backend files and the 2
bench files; go build/vet/test and gofmt clean; npm run build 221.39 kB with
eslint exit 0; bench/filesize_check clean; bench.audit --product clean over 31
harness files; scripts/mcp_write_smoke.sh PASS with the gate live across two
servers and the MCP child.
Seven sentences in the log were either one run deeper than the committed
reports, a delta computed across two different configurations, or a claim
about a path that was never measured. Each now quotes the number the report
carries, names the sample behind it, or says plainly that the figure was
never captured and is therefore not evidence.
The truncation rows were measured on query-time slices, so the record says
so instead of describing a storage layout nobody tested. The gate line names
the file set it covers: 29 Python files for pyright, and the three frontend
and one Go files that moved in the same range are covered by the build and
lint gates that actually apply to them.
A gate record is a measurement of one tree. Three lines in the log read as
if they described today's checkout, and one of them quoted a count taken
before a later commit fixed the thing it counted.
Each line now names the commit it was measured against, the command that
produces the number, and what is in the file set — the pyright figure covers
46 Python files because that commit touched no frontend file.
Both reports were produced by a harness that had just changed, so their
stamps named a commit without that code in it. Re-running them on an idle
machine puts the stamp and the source back in sync.
Recall did not move: all seven leg configurations reproduce to the third
decimal, which is the part of the number anyone should act on. Latency moved
within its usual load band, and the encoder probe came in slower this time,
so the picture index's build cost is now quoted as a band rather than a
single remembered figure.
A review of the p2-1 record found seven places where the prose had drifted
away from the reports it cites. None were product defects; all were record
keeping, and record keeping is what a memory system is judged on.
The quantization row mis-transcribed nine latencies, and the two worst made
dimension truncation look twice as fast as the full scan when the report
measures six percent. The extrapolation table scaled latency by atoms and
bytes by vectors under one heading that named neither switch. The picture
index stats existed only in prose, so the harness now reads the index shape
off the artifact and times the encoder on a sample, which reproduces the
hand-timed build within seven percent. The dense arms named a store that
holds no vectors, so the report names the store the vectors came from and
refuses to run where a dense leg would rank an empty index.
The frozen baseline was migrated in place by the product CLI while the
picture index was built, so its pinned mtime, digest and schema version are
re-pinned and the step that moved it is named. The sqlite_vec import added
two mypy errors; the fix is the missing stub declaration, not an inline
ignore, and the count is back to the recorded baseline of twenty-four.
Every number in CHECKLIST p2-1 and p0-3.7 came out of a scratch script: the
kind of artifact a reader cannot re-run, and the kind that quietly drifts
away from the code it describes. These reports are generated by committed
harnesses and each one stamps the commit that carries the code it measures.
DENSE.md 165,973 atoms / 153,857 vectors: build 1421.0s, search p50 75.6ms
p95 93.0ms, 0.998 agreement with a brute-force scan, one write into
a full index p50 518ms, 1,729 bytes per stored vector.
QUANT.md twelve encodings measured on the same vectors: int8 keeps 0.977 of
the ranking for a quarter of the bytes and no speed, bit is 14x
faster and keeps 0.728, so bit is a coarse filter and nothing more.
TERMS.md the stop-word bound swept, not guessed: 0.05 is the measured peak
(recall@10 0.803 at p50 69.5ms) and also the fastest of the rows
that keep recall.
SCALE.md the product's own search at 165,973 atoms: p50 71.7ms against the
frozen baseline's 20.16s.
LEGS.md what each retrieval leg is worth, per modality, with the page size
the product actually asks for in the table.
The acceptance bar the operator set is recall@10 >= 0.90 at n>=300. It is
not met. The best measured row is 0.858 (dense leg on, page widened to 30,
p50 414ms); the shipped configuration gives 0.848 at p50 72.5ms; the
in-repo counterpart at holder scope gives 0.866. The shortfall is
concentrated in picture golds (video 0.583-0.708, ui_screenshot 0.500-0.583,
image 0.750) while the word-shaped ones already pass (code 0.953, pdf 0.919).
Latency passes by three hundred fold; recall does not, and the gap is four
points of cross-modal retrieval, not index scale.
The product fuses a sparse page, a dense vector leg, and a picture leg that
only speaks when the words come back thin. Two of the three are off by
default, and until now no committed artifact said why.
bench/leg_check.py asks the frozen suite's 300 answerable queries as their
own holder, once per configuration, and prints recall, first-hit quality,
latency and per-modality recall in one table. The picture index and the
CLIP snapshot are 48 minutes of encoding, so they are carried in rather
than rebuilt per row; the harness refuses to run a leg whose snapshot is
missing instead of quietly scoring nothing.
The numbers that came back:
- words only, 10 atoms: recall@10 0.832, MRR 0.683, p50 64ms
- words only, the configured 24: 0.848, 0.688, 69ms
- words only, page 30: 0.855, 0.690, 67ms
- dense leg on: 0.845 / 0.858, MRR down, p50 469ms / 411ms
- picture leg available: 0.815 / 0.825, MRR 0.553 / 0.551, p50 148ms / 164ms
So both defaults stay off, and the acceptance bar (recall@10 >= 0.90) is
not met by the product path yet: 0.858 is the best measured row, the
in-repo counterpart at holder scope is 0.866. The loss concentrates in
picture golds, and the current CLIP leg pays more in dilution than it
returns there.
CHECKLIST p2-1 and p0-3.7 now quote the committed reports (DENSE.md,
QUANT.md, SCALE.md, TERMS.md) instead of scratch scripts.
Round 1 found a real defect: moving code between files dropped three
multi-line docstrings, and what dropped was contract — the rule that
every search filter is one `? IS NULL OR` predicate in a literal
statement, the as_holder sharing rule, the five arms of the ablation.
Round 2 found the line numbers had been read off a tree pi-lens reflowed.
The previous fix quoted migrations.py:248,250, which are a PRAGMA and a
blank line in the committed file. Those two numbers came from a working
tree pi-lens had already reflowed, so the ladder looked nine lines lower
than it is.
Cited from the commit: 181 and 186 define the v2 steps, 239 and 241 call
them, 243 rebuilds FTS for v3. The other six line citations on that
CHECKLIST line were re-checked the same way and hold.
A split moves code; it does not get to rewrite what the code promises. The
first pass collapsed three multi-line docstrings into one-liners and the
line counts in the checklist were measured before that happened.
- store_search.py: search_atoms keeps its filter-shape rule and the
as_holder sharing rule, FtsPage keeps why the cursor is an offset
- bench/run.py: the module docstring says what the five arms are again
- CHECKLIST: 116/341/83 lines, 11 call sites over 7 methods, and the
migration ladder now cites 181/186/248/250 instead of the parent's lines
Repo-wide multi-line docstrings are back to 114, the parent's count.
The 500-line rule and the one-line-comment rule were enforced only from
memory, so they drifted. bench/filesize_check.py now checks the budget on
every tracked source file, and it passes.
Split, with behaviour proven unchanged by comparing each moved function body
against HEAD byte for byte (53 training bodies, 21 benchmark bodies, 0
differences):
db.py 503 -> 396 store_search.py 105
bench/run.py 609 -> 351 query_set.py 80, report.py 211
benchmarks.py 619 -> 424 benchmark_scoring.py 262
training.py 1630 -> 94 training_{common,encode,metrics,datasets,lora,retrieval}
TrainingService stays as a thin facade for the CLI; the facade's parameter
lists are pinned against the module functions by inspect, which is what
catches a leaked self.
Deleted the eight top-level functions no file called, after fixing the scan
that had reported 201 false positives by skipping the defining file. The
v1/v2/v3 sweep found nothing to delete and says so: every remaining version
tag is an on-disk format marker, a migration step, a fixture name, or a
third-party model name.
Collapsed 39 multi-line comment blocks (86 lines) to one line each, keeping
the measured facts and error strings that justify the code.
269 tests pass; ruff clean; mypy 24 errors in 9 files against a 24-in-6
baseline, net zero; pyright 4, the same four HEAD measures.
p1-6's acceptance names bench/filesize_check, and the file did not exist:
the 500-line rule was enforced by memory. It now runs, and it fails on
training.py 1639, benchmarks.py 619, bench/run.py 609 and db.py 503. That
is the honest state of the tree, not a passing gate invented after the work.
The console work passed independent review on round three: the reviewer
re-ran the smoke, then broke the pager three ways and watched each break go
red - a pager that does nothing, a pager that appends, and a Previous button
that stops restoring.
The E2E waited for the number of `.atom-row` elements to change, but the
search list and the browse list share that class, so the count it watched
was 12 browse rows plus one search hit. A pager that did nothing still
passed. Rows now carry `data-atom` and the two lists carry `data-testid`,
and the test asserts the ordered id list: page one holds exactly PAGE_SIZE
rows, page two shares none of them, page two holds total - PAGE_SIZE rows,
and Previous restores the first list.
Both mutations the reviewer suggested now fail: dropping setCursor and
loadBrowse from nextPage gives "Next page did not swap the page: the pager
is not paging", and appending instead of replacing gives "the next page
repeated 12 of 12 atoms from the first page".
The record also said 130 lines for App.tsx when it is 131, and quoted a
mount refusal the harness had cut off mid-sentence; the refusal's own
remediation ("set ELY_EYE_MODEL_DIR to the model snapshot") is the useful
half, so the slice is 200 now and the quote is the product's full sentence.
The store had a write path, a mount path and a search path, and the console
still showed a dashboard of counters. Nothing in the UI could ask what someone
knows, page through atoms, look at a wordless picture, forget one memory, or
mount a cartridge — so the mechanisms built in P0 were invisible.
Backend:
- senses.py: one function decides what this machine can perceive. /api/status
and doctor each computed it before, which is two truths. Picture atoms are now
counted by image_ref rather than by modality, so a screenshot crop from a PDF
counts and a picture whose bytes were deleted stops claiming it can be encoded.
- browse.py: GET /api/atoms pages newest-first through v_atoms_current with
modality/holder/source filters; GET /api/atoms/{id}/image serves the stored
bytes so a picture with no OCR text can be looked at.
- Deleted 8 HTTP routes nobody calls (4 benchmark, research modules, dense index,
visual-contradiction-proof, memory-map). Their functions stay: the CLI and the
proof checks use them. The surface is 30 routes, pinned as a set by a test so a
new unused route fails the suite.
Frontend: App.tsx (517 lines) became a 131-line shell plus seven panels that own
their own fetches, and lib/format.ts. types.ts now carries the memory dimensions
the backend has been sending since schema v2.
Two invented fields were rendering lies: AdapterPanel showed gpu_available and
probe_status, which AdapterLoadReport does not have — every machine looked GPU-less
including this one. The panel now shows only what the server says.
Proof: scripts/ui_smoke.sh starts the API on a fresh temp home, serves the built
UI from the same origin, and drives five interactions in a real browser — search
as somebody in a scenario, page and view a wordless picture, forget, mount, read
adapter states with reasons. It prints the mount refusal verbatim rather than
pretending the mount succeeded.
Gates: 263 pytest passed; ruff and mypy clean on the 8 touched backend files
(repo-wide mypy unchanged at 24 errors / 6 files); pyright 0 errors there;
go build/vet/test/gofmt clean; npm build + eslint clean; ui_smoke and
mcp_write_smoke both PASS.
The reviewer ran things rather than reading my claims: gh run view, pytest,
ruff, cold-cache mypy diffed against the pre-p1-4 tree, pyright, the full go
suite, bash -n on all 15 .sh, pip wheel, sha256sum of LICENSE against gnu.org,
the 22.4GB container answering three URLs, and three mutation tests on the
harness stubs. Round 1 failed 9 checklist numbers, round 2 failed one two-line
docstring, round 3 passed and measured that p1-4 now adds zero multi-line
docstrings repo-wide (114 -> 114).
The reviewer's round 2 verified all nine round-1 corrections by re-measuring
them, then failed the commit on the one thing it added that the repo's own bar
forbids: a two-line docstring on _TinyDense. p1-6.4 still lists "comments to one
line" as open work, so a new commit should not widen it. The 470MB rationale
already lives in CHECKLIST.md.
Also cites the CI run for the commit under review (37744547992 @ 20c0416, also
success) instead of stopping at the previous one.
Nine claims in CHECKLIST.md did not reproduce. All nine were prose, not
product, and all nine are now the measured value:
- test files: 19 (18 test_*.py + conftest.py), not 24
- suite size: 253 passed (239 at 922d012), not 252
- build context: 81G on this machine (bench/runs 60.2GB, bench/corpus 20.7GB,
.venv 2.5GB), not 19GB
- profiles.py probes: linux_package_versions catches the 3-tuple because
json.loads needs ValueError; runtime_launcher_probe and gpu_available catch
the 2-tuple. The old sentence claimed one tuple for all three.
- training.py: the CUDA raise is at :1271, not :1272
- cli.py is 490 lines at this commit, not 488
- the bf16 falsifier said "a 4090 that does not support bf16". A 4090 is Ada
(compute 8.9) and does support BF16; the test fabricates the flag. Naming the
card was exactly the error this commit removes, so the sentence now names the
flag.
- the backup digest is a fingerprint of one store snapshot: hashing is
deterministic per state, but the store was rewritten after the commit, so the
current digest differs (498b0cd9...). The record now pins mtime, size and row
counts next to the digest.
- wall times moved between runs (backup 4.0s vs 2.24s, restore 2.8s vs 5.65s),
so they are reported as a range with the reason instead of a single number.
CI was a workflow that could not run its own tests: pytest is not a
dependency of anything, so a fresh clone fails with "No module named
pytest". Test tooling now lives in the backend's dev extra, so CI and a
new contributor install the same tools, and ruff is pinned there instead
of drifting as a bare version in a workflow step.
backup, restore and doctor move the store. Backup uses SQLite's own
online API so a running server is safe, and the manifest records the
schema version, row counts, sha256 and size read back from the copy.
Restore verifies the digest, integrity_check, required tables and schema
age before it touches the live file, and keeps what it replaces as
.before-restore.
Doctor separates broken from degraded. On the 906 MB baseline store
(165,973 atoms) it reports no problems and one true warning: 21,362
picture atoms with no CLIP snapshot configured, so those atoms are
findable only by the text already on them. The first version called that
store healthy because it only complained about paths that were set and
missing, never about a sense the store needs and never had.
AGPL-3.0-only, because PyMuPDF is dual-licensed AGPL-or-commercial and
the repository is public. A permissive license needs the Artifex
commercial license bought first; the README says so rather than hiding it.
Gates: 253 tests pass; ruff clean repo-wide (17 errors at HEAD); mypy
unchanged from HEAD on touched files; pyright 0 errors on the new files;
backup 4.0s and restore 2.8s on real data with counts verified equal.
The reviewer ran its own probes and mutation tests rather than reading the
prose, and confirmed the repo rules. One of its observations, that mount
state is per-connection, I disproved by measurement: mounts live in a table
and a second Database object on the same home reads them back, so the test
asserts an empty mount list only because that test mounts nothing.
Round three of the adapter review found two leftovers from the previous fix.
With three or more live runtimes the reason read "another live model still
holds adapter-a and adapter-b", a singular subject attributing several
adapters to one model, when each runtime attaches one adapter per kind. The
subject now agrees in number with what is held.
LoadRecord still stored its own copy of the manifest sha256, which is the
same second source of truth this work removed for kind, adapter_id and path:
the only constructor copied it, and nothing stopped a caller from passing a
hash that disagreed with the source handed in. It is a property now, so
runtime.py keeps reading record.sha256 and there is one value.
Also recorded, measured by the reviewer and deliberately not treated as
defects: a row for a kind with nothing on disk now reports base_model as null
rather than the runtime's model id, since that row describes no adapter at
all; the same adapter id can appear in both a loaded row and a shadowed row
when it is attached while another copy wins the mount, and both rows are
true; and when the attached adapter is deleted from disk the row says missing
with null identity instead of claiming a gone model.
Three tests carry this: the three-runtime case asserts the plural wording, and
both fixes were mutation-checked (dropping the plural branch fails two tests,
restoring the stored hash fails the generation test).
pytest -q: 228 passed. ruff clean. mypy cold cache on the three touched source
files: 2 errors, identical to the stashed HEAD tree, net zero. pyright: 2
errors, both the known qwen_vl_utils tuple artifact.
The report row contradicted itself. The reason came from the attachment
record while path, adapter_id, base_model, origin and cartridge_id came from
whatever resolution preferred right now. Reproduced without any stubbing: a
live model holds adapter X, training writes a newer adapter Y, and the
endpoint then answers state=loaded adapter_id=adapter-Y while explaining
"attached to the live model from .../x-attached". A JSON consumer reads the
identity columns as the loaded adapter and gets the wrong one.
The same root cause had a second symptom. Attachments were keyed by kind with
last-writer-wins, so with two live runtimes one unload made the row claim
"that model is gone" while the other runtime's model was still alive holding
its adapter.
LoadRecord now carries the AdapterSource it came from, so a row built from a
record has exactly one identity, and the registry is a list of attachments
each holding its own weak reference: loaded means some live model still holds
an adapter, and when more than one does the reason names the other.
clear_loads had no production caller once unload stopped erasing the process
record, so it is gone rather than kept as a test-only entry point.
Two tests pin both symptoms, and both were mutation-checked: restoring the
identity to the resolved source fails the first, and letting
live_attachments ignore liveness fails the second and the unload test.
pytest -q: 227 passed. ruff clean. mypy cold cache on the three touched
source files: 2 errors, byte-identical to the same command on a stashed HEAD
tree, net zero. pyright: 2 errors, both the known qwen_vl_utils tuple
artifact, unchanged from HEAD.
Independent review of the adapter resolution work found six defects, all
real and all fixed here.
A kind whose only candidates were rejected had no winning path, so its row
carried path=None and the rejected candidate fell into the shadow loop: the
report said "another adapter of this kind wins for the current mount" while
nothing won, and the real deferred reason was buried. A shadow row now
requires a winner and names it.
"loaded" was backed by a process-global record, so a fresh runtime with no
model still claimed a live adapter. The record is now held by the instance
that attached it, and the report keeps a weak reference: while the model
lives the row says loaded, once it is gone the row says available and
explains that the model is gone.
The sort key had lost the manifest's cartridge tag, which changed which
adapter loads on any machine holding a tagged library copy, and neither the
commit message nor the checklist said so. The tag is back as the middle key.
Proof checks resolved through the library while generation resolved through
the mount, so one proof payload could name two different copies. Both now
resolve the way generation does.
loaded_at was written and never read, so it is gone. The checklist claim that
peft is lazily imported in one place was true only for generation and is now
scoped that way.
The four generation paths and status() had no test coverage, so "no function
was lost" rested on reading. One test now walks the transformers prompt path
end to end with stub weights and asserts the generation answer carries the
adapter it attached.
pytest -q: 225 passed. ruff clean. mypy on the three touched source files:
2 errors, the identical 2 on a stashed HEAD tree, net zero. pyright: 2
errors, both the known qwen_vl_utils tuple artifact, unchanged from HEAD.
runtime.py resolved adapters at 7 call sites and only ever looked in the
adapter library, so an adapter attached to a cartridge (cartridge.py:230
copies it into <cartridge>/adapters/{kind}/) was never loaded by anything.
Resolution now lives in adapters.py: it reads the mounted cartridge set,
prefers a mounted cartridge's copy over the library, and answers every one
of the six kinds with a state the facts support — loaded, available,
deferred (base_model mismatch or missing weights, both models named in the
reason), missing — plus a shadowed row for each adapter that lost the
choice. A mounted cartridge is the scenario selection; neither manifest
carries a scenario field, so none was invented.
Two bugs the new tests caught: the preference sort lost reverse=True, so a
library adapter beat a mounted cartridge's copy, and the load branch had
never been executed by a test. Stub weights now prove peft is asked to
attach, that the returned model is put in eval mode, and that the endpoint
reports loaded until unload clears it.
Faces: GET /api/runtime/adapters, `ely-eye adapters`, and a log line naming
the reason whenever a generation runs without an adapter.
Gates: 222 passed; ruff clean; mypy cold cache on the 7 touched files gives
the same 2 errors as HEAD's runtime.py, and pyright gives the same 2 as
HEAD — both are qwen_vl_utils typing artifacts (process_vision_info returns
2 or 3 values depending on return_video_kwargs; the product takes 2).
The reviewer ran the harness and the gates itself rather than trusting the
prose: round one failed on 11 defects, round two on 4, round three passed
after re-running bench.asr_check in a fresh process, re-checking the stamp
chain and matching 18 checklist assertions.
The report is regenerated after 10a39c5 so its stamp names the code it
describes. Speed is quoted as a range in the checklist because transcribe
seconds move with machine load (3.26s then 3.12s on this box) while WER,
denominators and word counts do not move at all.
Numbers: 258.18s of audio, WER 0.242 over 533 reference words, 80 audio atoms
and 563 word timestamps read back from the store, 0 out of order and 0 before
their own segment measured on the stored words.
ZacharyZhang-NY