Stop the checklist from quoting numbers no artifact carries
A review of the p2-1 record found seven places where the prose had drifted away from the reports it cites. None were product defects; all were record keeping, and record keeping is what a memory system is judged on. The quantization row mis-transcribed nine latencies, and the two worst made dimension truncation look twice as fast as the full scan when the report measures six percent. The extrapolation table scaled latency by atoms and bytes by vectors under one heading that named neither switch. The picture index stats existed only in prose, so the harness now reads the index shape off the artifact and times the encoder on a sample, which reproduces the hand-timed build within seven percent. The dense arms named a store that holds no vectors, so the report names the store the vectors came from and refuses to run where a dense leg would rank an empty index. The frozen baseline was migrated in place by the product CLI while the picture index was built, so its pinned mtime, digest and schema version are re-pinned and the step that moved it is named. The sqlite_vec import added two mypy errors; the fix is the missing stub declaration, not an inline ignore, and the count is back to the recorded baseline of twenty-four.
This commit is contained in:
1 parent
6cc5640856
commit
71733d86d6
7 files changed
+153
-31
No files matched your search
@@ -37,6 +37,10 @@ def test_the_report_prints_every_arm_with_its_numbers(tmp_path: Path) -> None:
|
||||
"commit": "abcdef123456",
|
||||
"schema_version": 8,
|
||||
"store": "bench/runs/leg_check/data/ely_eye.sqlite",
|
||||
"source_store": "bench/runs/dense_scale",
|
||||
"dense_vectors": 153857,
|
||||
"picture_index": {"path": "bench/runs/leg_check_index", "atoms": 53668, "dim": 512, "bytes": 109912192},
|
||||
"encode_rate": {"pictures": 200, "seconds": 10.0, "per_second": 20.0},
|
||||
"probes": 300,
|
||||
"arms": [arm],
|
||||
"machine": {
|
||||
@@ -53,4 +57,10 @@ def test_the_report_prints_every_arm_with_its_numbers(tmp_path: Path) -> None:
|
||||
assert "words only" in text
|
||||
assert "0.832" in text and "63.7" in text, "recall and latency must sit in the same row"
|
||||
assert "code" in text and "0.953" in text, "per-modality recall is where the picture legs show up"
|
||||
assert (
|
||||
"bench/runs/dense_scale" in text and "153,857 dense vectors" in text
|
||||
), "a dense arm must name the store its vectors came from, not imply the frozen baseline"
|
||||
assert (
|
||||
"53,668 atoms at 512 dimensions" in text and "44.7" in text
|
||||
), "the picture index needs its shape in the report, and its build cost derived from a measured rate"
|
||||
assert "abcdef123456" in text and "load_average" not in text
|
||||
Reference in new issue
Block a user