Record machine load with every latency number

Latency measured while another job hogs the cores is not comparable: the scale curve
moved from 1.315s to 0.809s at 50,000 sources between two runs of the same code.

- bench/systems.py: machine_facts() reports cpus plus the 1/5/15 minute load average.
- bench/run.py, bench/run_external.py: both reports carry it, and BASELINE.md labels
  the latency table a ceiling when the machine was busy.

Verification: pytest bench/tests -q -> 60 passed; ruff check bench/ -> all checks
passed; mypy (fresh cache) -> no issues in 25 source files.
This commit is contained in:
ZacharyZhang-NY committed 2026-10-07 12:03:44 -04:00
1 parent 61f6506104
commit c4e2982506
6 files changed
+62 -27

No files matched your search

+2
View File
@@ -158,6 +158,7 @@ def _report() -> dict:
"corpus_sources": 10,
"queries": 16,
"answerable": 8,
"machine": {"system": "Linux", "release": "x", "cpus": 32, "load_average_1_5_15": [1.0, 2.0, 3.0]},
"models": {"dense": "d", "rerank": "r"},
},
"chance": {"recall_at_1": 0.0, "recall_at_5": 0.0, "recall_at_10": 0.0},
@@ -191,6 +192,7 @@ def test_the_report_writer_survives_a_scale_note(tmp_path: Path) -> None:
text = path.read_text(encoding="utf-8")
assert "| 500 | 900 | 0.001/0.002 |" in text
assert "_latency only_" in text
assert "load average 1.0 on 32 cores" in text
assert "## unanswerable queries (50.0% of the suite)" in text