The product fuses a sparse page, a dense vector leg, and a picture leg that
only speaks when the words come back thin. Two of the three are off by
default, and until now no committed artifact said why.
bench/leg_check.py asks the frozen suite's 300 answerable queries as their
own holder, once per configuration, and prints recall, first-hit quality,
latency and per-modality recall in one table. The picture index and the
CLIP snapshot are 48 minutes of encoding, so they are carried in rather
than rebuilt per row; the harness refuses to run a leg whose snapshot is
missing instead of quietly scoring nothing.
The numbers that came back:
- words only, 10 atoms: recall@10 0.832, MRR 0.683, p50 64ms
- words only, the configured 24: 0.848, 0.688, 69ms
- words only, page 30: 0.855, 0.690, 67ms
- dense leg on: 0.845 / 0.858, MRR down, p50 469ms / 411ms
- picture leg available: 0.815 / 0.825, MRR 0.553 / 0.551, p50 148ms / 164ms
So both defaults stay off, and the acceptance bar (recall@10 >= 0.90) is
not met by the product path yet: 0.858 is the best measured row, the
in-repo counterpart at holder scope is 0.866. The loss concentrates in
picture golds, and the current CLIP leg pays more in dilution than it
returns there.
CHECKLIST p2-1 and p0-3.7 now quote the committed reports (DENSE.md,
QUANT.md, SCALE.md, TERMS.md) instead of scratch scripts.
ZacharyZhang-NY
dff5977ab6