The frozen baseline showed the unmodified RetrievalService agreeing with an equivalent
BM25 index on 0% of its top ten results. Reproduced by hand: its top ten atoms came
from one document's pages, so a source-level ranking cannot overlap it.
- bench/systems.py: live_product_rankings also counts distinct sources behind the
product's top-k atoms.
- bench/run.py: scale probes no longer filtered by gold (a source prefix keeps almost
no gold, so the 500/2,000/10,000 rows were silently skipped); scale_arm_names reads
only latency dicts, which is what crashed write_markdown on the note field; the
unanswerable share is computed instead of hard-coded.
- bench/tests/test_joint.py: 60 tests total, including the writer regression.
Verification: pytest bench/tests -q -> 60 passed; ruff check bench/ -> all checks
passed; mypy (fresh cache) -> no issues in 25 source files.
ZacharyZhang-NY
61f6506104