Record machine load with every latency number

Latency measured while another job hogs the cores is not comparable: the scale curve
moved from 1.315s to 0.809s at 50,000 sources between two runs of the same code.

- bench/systems.py: machine_facts() reports cpus plus the 1/5/15 minute load average.
- bench/run.py, bench/run_external.py: both reports carry it, and BASELINE.md labels
  the latency table a ceiling when the machine was busy.

Verification: pytest bench/tests -q -> 60 passed; ruff check bench/ -> all checks
passed; mypy (fresh cache) -> no issues in 25 source files.
This commit is contained in:
ZacharyZhang-NY committed 2026-10-07 12:03:44 -04:00
1 parent 61f6506104
commit c4e2982506
6 files changed
+62 -27

No files matched your search

+12
View File
@@ -10,6 +10,8 @@ from __future__ import annotations
import hashlib
import json
import os
import platform
import re
import sqlite3
import time
@@ -30,6 +32,16 @@ DENSE_MODEL = "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
RERANK_MODEL = "BAAI/bge-reranker-base"
def machine_facts() -> dict[str, Any]:
"""Hardware plus the load at report time: latency numbers mean nothing without it."""
return {
"system": platform.system(),
"release": platform.release(),
"cpus": os.cpu_count(),
"load_average_1_5_15": [round(value, 2) for value in os.getloadavg()],
}
def local_model_dir(models: Path, repo_id: str) -> Path:
"""Where a fetched model lives: one flat directory per repo, named by its repo name only."""
return models / repo_id.split("/")[-1]