f8a9428f5ad1573cfd38d816d9beec110f690f82
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ZacharyZhang-NY
|
f8a9428f5a
|
Ask the repository that owns the path, and pin the wirings that had no test
Round four of the independent review found seven record and coverage
defects. The functional side it verified by running; these are the fixes.
- `_tracked` asked the current checkout's index, so a reviewer in a worktree
pointing at the main checkout's committed report was told it was untracked.
It now asks the repository that owns the path, and the guard no longer
compares directories: any tracked file is committed evidence.
- `retrieval.py` claimed the dense index "catches up inside search". It does
not: `_stale` is only used by `sync()` and `stats()`, so a store upgraded
from v11 ranks an empty dense leg until `ely-eye build-dense-index` runs.
The docstring now states that, and `doctor` already names it.
- Four wirings had no discriminating test, each proven by mutation:
a stubbed `_tracked` (now a test uses the real git index), `copy_store`
claiming the scratch (now a live holder blocks the copy before a byte is
copied), `_stale` reading the atoms table (now a closed memory that loses
its vector is owed one), and the v10/v11 to v12 `DROP VIEW` gate (now a
hand-made v11 store is upgraded and its view body re-checked).
- The record corrected against the reviewer's measurements: the five
mutations did not each turn one test red (M1 turned two, M3 turned two,
`_stale` alone turned none), the pyright/mypy file set was wrong, and the
round added seven tests rather than three.
Gates: 422 passed; ruff clean; filesize clean; audit clean; mypy 24 errors in
9 files repo-wide, equal to the
|
||
|
ZacharyZhang-NY
|
8bed319848 |
Stop a room number from being read as a room's reading
A quantity conflict compared labels, and a label can carry a number that
identifies the thing rather than measuring it. 「Room 3 holds 22 degrees」 and
「Room 4 holds 24 degrees」 shared the subject 「room」, so the store recorded a
conflict over the room numbers and dropped Room 3 from the answer page: asking
about degrees returned one room's reading and hid the other room entirely.
「The router 1/2 forwards 220/240 packets」 and 「2026 saw 3 / 2027 saw 5
releases」 failed the same way.
A bare number standing before any predicate is part of the subject's name, so it
joins the subject key and is never recorded as a value. A number that stands past
a word still measures: 「The rent is 2200 / 2400 dollars」 conflicts over rent,
and a number fused inside a CJK run stays a value, because 「恒温器调到」 states its
predicate inside the subject token and the motivating pair must keep conflicting.
The same year counting twice is now a conflict (「2026 saw 3 / 5 releases」), which
the old rule missed.
Only a number renames a subject: 「Room A」 and 「Room B」 still read as one room
whose reading moved, and an English qualifier does not split a subject the way a
Chinese one does. Both boundaries are disclosed and pinned by tests rather than
smoothed over.
Also collapses six two-line comment blocks back to one line each, which is the
rule this repo already enforces, and corrects the record where it claimed the
nearest label was 「书房」 (it is 「设定」) and that the subject rule only ever
missed conflicts.
Gates: pytest -q 379 passed (conflicts 17 -> 22, six p2-3 files 78, three tier
files 35); ruff clean; bench.filesize_check clean; mypy cold cache 24 errors in 9
files, byte-identical to the recorded
|
||
|
ZacharyZhang-NY
|
dc9cfb5a30 |
Give memories a working tier, a long-term tier, and a record of what disagrees
A memory used to have one state: present. A session could not be reloaded
without searching, nothing separated what a person said today from what the
store ingested a year ago, and two statements that disagree about a number
were both handed to the reader as if the store believed both.
Schema v10. Atoms gain session_id, importance, recall_count, last_recalled_at
and archived_at; two tables arrive beside them — consolidations (one watermark
per holder+scenario lane) and conflicts (a pair, the term they disagree on,
both values, and whether anyone has settled it). v_atoms_current now also
hides archived rows, and apply() drops the view before rebuilding it because
CREATE VIEW IF NOT EXISTS would keep the old body and ignore the archive.
Working memory is a read, not a store: `working <holder> <session>` returns
what that session recorded, in order, with no search involved. Consolidation
rolls a lane's new memories into one standing summary and moves the watermark
to the newest memory actually rolled up — not to the clock of the pass, which
made a second pass count the same memories twice. The archive rule is stated
in the command that runs it (recorded before N days, never returned by a
search, importance at or under a floor) rather than hidden in a decay formula
nobody measured; --dry-run names the victims first.
A conflict is recorded when a new memory repeats a label with a different
number in the same holder's same scenario. Retrieval drops the older side
while the conflict is open and both sides answer again once it is settled:
settling is not superseding, and a word-only disagreement is left alone
because a reworded sentence would fire on every paraphrase. Numbers are keyed
to both the memory's subject and the nearest preceding label, measured: the
two thermostat sentences share only their subject, so a nearest-word-only rule
found nothing.
Search now counts the memories it handed back, which is the only use estimate
the store has, and the archive rule reads it. The dense index applies the same
currency rules as the view, so an archived memory cannot stay findable through
one leg while the other hides it.
Two reads over HTTP (working, conflicts) and two tools over MCP
(working_set, memory_conflicts); curation stays with the operator because an
agent should not decide what its own past is worth. remember and supersede
carry a session id; importance does not cross MCP, because recall_count is
observed and importance is stated.
Gates: 366 passed; ruff clean; mypy 24 errors in 9 files, the recorded
|
||
|
ZacharyZhang-NY
|
9cb46c18d3 |
Keep every file inside the rules the repo already wrote down
The 500-line rule and the one-line-comment rule were enforced only from
memory, so they drifted. bench/filesize_check.py now checks the budget on
every tracked source file, and it passes.
Split, with behaviour proven unchanged by comparing each moved function body
against HEAD byte for byte (53 training bodies, 21 benchmark bodies, 0
differences):
db.py 503 -> 396 store_search.py 105
bench/run.py 609 -> 351 query_set.py 80, report.py 211
benchmarks.py 619 -> 424 benchmark_scoring.py 262
training.py 1630 -> 94 training_{common,encode,metrics,datasets,lora,retrieval}
TrainingService stays as a thin facade for the CLI; the facade's parameter
lists are pinned against the module functions by inspect, which is what
catches a leaked self.
Deleted the eight top-level functions no file called, after fixing the scan
that had reported 201 false positives by skipping the defining file. The
v1/v2/v3 sweep found nothing to delete and says so: every remaining version
tag is an on-disk format marker, a migration step, a fixture name, or a
third-party model name.
Collapsed 39 multi-line comment blocks (86 lines) to one line each, keeping
the measured facts and error strings that justify the code.
269 tests pass; ruff clean; mypy 24 errors in 9 files against a 24-in-6
baseline, net zero; pyright 4, the same four HEAD measures.
|
||
|
ZacharyZhang-NY
|
674c88e84e |
Rank memory search in SQL instead of rebuilding BM25 in Python
The product path pulled every atom into Python and rebuilt a BM25Okapi
index per query: p50 20.16s over 165,973 atoms. It now runs one SQLite
statement with LIMIT/OFFSET.
- text_index.py: one normalizer for both sides of the index. CJK runs are
split per character (unicode61 cannot segment them), camelCase and
separators are split so "search atoms" finds SearchAtoms, latin tokens
get a prefix match. Rows written before this existed hold raw text in
atom_fts, so schema v3 rebuilds the index on upgrade.
- retrieval.py: filters (holder / scenario / source / modality / since /
until / min_trust), a base64 cursor, and reciprocal rank fusion in place
of the 0.55/0.35/0.10 blend whose graph term was always zero.
- An empty result is {status: "no_match"}, not a bare array.
- Go: search_atoms and compile_context rank through POST /api/search. The
second Go tokenizer is deleted; two implementations could drift, which
is what the MCP/HTTP parity requirement forbids.
- db.py passed 500 lines, so the row codec moved to row_codec.py.
- Tooling: installed the repo's own pyright. pyrightconfig.json had no
exclude and swept the downloaded corpus (1171 errors); with excludes the
count is 62, all pre-existing (evals 48, profiles 4, training 4,
ingestion 3, runtime 2, cartridge_assets 1). backend/pyproject.toml
carried a second [tool.pyright] that shadowed the root config; deleted.
- bench/scale_check.py measures the product search at corpus size. The
100k-atom p50 number is not written until it is measured on an idle
machine.
Verification: 112 pytest passed; go build/vet/test clean, gofmt clean;
ruff clean on every touched file; mypy clean on the 14 touched Python
files; repo pyright 62 errors, unchanged per-file ledger.
|
||
|
ZacharyZhang-NY
|
181f6bf57e |
Give every memory a holder, an origin, a time and a scenario
An EvidenceAtom could say what was remembered and nothing else: not whose memory it is, not where it came from, not when it was true, not in what situation it may be used. The baseline measured the cost of that: the same ranker scores recall@10 0.412 over the whole corpus and 0.866 inside one holder's memory, so the missing holder column is worth more than any ranking change in the ablation. - schemas.py: MemoryOrigin, TimePrecision, Visibility; EvidenceAtom gains holder_id, origin, scenario, event_time / event_time_end / event_time_precision / event_time_source, valid_from / valid_to, supersedes, visibility, retention_until. `time` is renamed `ingested_at` — ingestion time gets one name, not two. A validator refuses valid_to before valid_from, event_time_end before event_time, and an atom that supersedes itself. - migrations.py: schema v2 with PRAGMA user_version. A first-version database is upgraded in place (ALTER + RENAME COLUMN), rows intact, and a database newer than this build refuses to open instead of being silently downgraded. - v_atoms_current hides memories that are closed, lapsed or not yet begun; unknown bounds read as visible. Time travel is db.atoms_as_of(moment, holder) because a SQLite view cannot take a parameter. - evaluation_schemas.py: the proof and benchmark report shapes move out of schemas.py to keep it under the file-size limit; importers follow. - backend/tests/test_memory_schema.py: 8 tests, including the v1 -> v2 upgrade and the validity timeline. Verification of record: pytest backend/tests bench/tests -q -> 70 passed; ruff check on every touched file -> all checks passed; mypy (fresh cache) on db.py, migrations.py, schemas.py, evaluation_schemas.py and backend/tests -> no issues. evals.py carries 40 pre-existing mypy errors at HEAD and 40 after this change (measured on both copies); its pyright findings are the same debt, and p0-6 rewrites that file. `peft` and `pyarrow.parquet` "could not be resolved" is the documented pyright artifact: pi-lens runs it with 0 config files, and both import at runtime. |