Stop two harnesses from measuring each other's store

A scratch copy is one run's working store. The video harness defaults to
bench/runs/video_audio and copy_store() copies straight over it, so a smoke
run started while the full run was mid-ingest replaced the database under it:
the full run's after-arm would have scored a store whose speech atoms had been
swapped out, and the smoke itself died on "database is locked" inside
text_index. Six harnesses share that copy path, so the claim lives in
bench/scratch.py: a live pid in held_by refuses the path, a stale one is taken
over, and the same process may re-copy per arm.

The reviewer's smoke knob also measured nothing instead of saying so. With
--limit the probe sample narrows to video only, and a small --probes strides
past that modality entirely, so the run crashed in aggregate() after the
ingest. probe_sample() now switches the arm off at --probes 0 and fails in
half a second otherwise.

The frozen-report guard had two holes found by running it: it compared
cwd-relative parents, so an absolute path to the frozen file passed, and it
refused the harness's own default destination once that file existed, which
would have stopped the shipped run. It now compares resolved paths and refuses
only committed evidence.

Record fixes from review round 3: the round-2 line credited a reproduction to
the wrong round and quoted a git log -S hit for a string that never existed in
code; the suite counts now name the commit each was measured on (401 at
83dfc14, 406 at eab6bd7, 410 at cf2de31, 411 at 92237c8).

Gates: pytest -q 418 passed; ruff clean; bench.filesize_check OK; mypy cold
cache and pyright clean on the four touched files.

CHECKLIST.md

@@ -439,11 +439,13 @@
   - 验收：`pytest backend/tests/test_holder_acl.py backend/tests/test_visibility_rule.py backend/tests/test_transactive.py` → **13 + 2 + 19** 全绿（两 holder 互不可见、无 token 读不到私钥、`everyone` 无 token 可搜、`visible_to` 只放行被指名的人、声称别人 403 且点名两人、token 只出现一次且库里只有 sha256、吊销后旧 token 401、卡无 token 可读且不含正文、陌生人的图片 404 而无主的 200、`compile(as_holder="")` 返回 0 hits 且 `cache_trace_id` 与 `as_holder="wife"` / `None` 都不同）。跨进程：`bash scripts/mcp_write_smoke.sh` → 两行 PASS，第二行是「a caller with no token sees only public memory, a holder's token sees her own, naming somebody else is refused, and the capability card answers without a token」——它先证明无 token 搜不到私钥，再用 operator token 铸出妻子的 token、证明她能搜到自己的、搜不到 `local` 写的 `the oven needs descaling`，最后**把 MCP 进程不带 `ELY_EYE_AUTH_TOKEN` 重启一遍**，证明 Go 那条路守同一条规则（不是假设）。
   - 顺手修的两条自己写的断言缺陷（都是跑出来的）：smoke 里 `if secret in found(None, token=token)` 是同义反复（搜的还是那句话，当然命中），改成拿**别人写的**记忆当探针；指纹探测 `GET /api/memory/working?holder=fingerprint` 现在返回 **403**（路由在、但匿名不能声称 holder），所以探测必须带 operator token——旧树上没有这条路由仍是 404，判别力保住。
   - **评审第 1 轮 FAIL**（同一独立 pi，`209f2d0..83dfc14`，736 工具轮）：**功能面它全部实测成立**，记录面 8 条对不上，已全部按实测改掉——D1 「比对用 `compare_digest`」对 holder token 是假的（库里只有摘要，`verify_token` 是 SQL 等值，`compare_digest` 只用在 operator 的明文上）；D2 指针错（`register_minimal` 在 `memory.py:94`，不是 `:120`）；D3 「先前 `as_holder or ""` 两边哈希 `""`」描述的机制从来没存在过（旧 key 根本没有身份成分）；D4 `browse.py:14` 注释说「four — two here」而实际五份、这里三份（`:52`/`:66`/`:93`）；D5 指针被 as_of 改动挪了行（`vector_index.py:288`、`store_search.py:103`）；D6 我 prompt 里写「33 条路由」而实测父提交 34 个装饰器 / 31 条不同路径、HEAD 38 / 35；D7 semgrep 那条在它的环境里无法复现（`which semgrep` / `which opengrep` 都空），已改成「当时观察到的」并把防漂移的责任交给不依赖那个工具的测试；D8 「holder 也只能是 caller 自己…由 token 保证」与实测相反（只有 operator 能写，归属来自 `default_holder`，真正的保护是 `RememberRequest` 没有 holder 字段）。它同时实测确认：三条漏在父提交上是真的、在 HEAD 关上；38 条路由里会吐正文/字节的 7 条全带眼睛；11 个攻击全关（含吊销 token 401、holder token 不能写）；三态没被照瞎（CLI 命中、`bench.leg_check --probes 20` recall@10 0.900 / p50 72.3 ms、smoke PASS）；五份副本它自己数过，把 `? <> ''` 改掉立刻让 `test_visibility_rule.py` 红；缓存 key 两种顺序四个 trace id 全不同；smoke 新增那段是真判别子（父提交树会把私钥搜出来）。另采纳四条精度说明（吊销硬拒、匿名空串 403、能力卡还带时间戳与 `token_issued`、`leg_check` 的 commit 行是 cwd 的 HEAD）。**D7 后续补上的证据**：那个门禁在本机确实跑得起来，只是不是独立 CLI——pi-lens 的 semgrep runner 在 `migrations.py:245`（`f"PRAGMA table_info({table})"`，`git show HEAD` 同位置同样存在）上 fired `python.sqlalchemy.security.sqlalchemy-execute-raw-query`，所以「插值变量会被报」不是口述，可重跑；那个站点是只拿字面量调的私有 helper。
-  - **评审第 2 轮 FAIL**（同一独立 pi，`df812f3`，7 项全跑）：D1–D8 全部实测确认已修（它在 `209f2d0` worktree 里重现了旧 key 无身份成分、`git log -S` 重现了 `as_holder or ""` 只存在于 `83dfc14`）。as_of 的行为面它自己写脚本测通：退役的记忆在 `as_of` 下应答、下周的事今天不应答、归档不挡 as_of、不跨 holder 借看。`scripts/key_test.sh` 在 `df812f3` 上 exit 0（6 PASS + 1 NOT MEASURED），把 `store_search.py` 换回父提交就红在场景 3。新缺陷六条，三条行为三条记录：**D9** 那条 dense 测试名字不副实、as_of 的向量侧零覆盖（包一层计数实测 0 次调用；改 `_allowed` 的 as_of 分支、删 `retrieval.py:119` 都全绿）；**D10** 真开 dense 时两条腿不一致，词腿回旧事实、dense 腿回空，因为退役记忆根本没向量（`_expected`/追赶都读 `v_atoms_current`），commit message 里「cannot disagree」写过头；**D11** 排期到未来的 supersede 不披露且不对称：不带 as_of 的搜索 0 命中、`GET /api/atoms` total 0、dense 腿 1 行；**D12** `:430` 那个旧行号括号仍是错的；**D13** p2-5 验收 grep 实测 **2 命中**不是 0（它扫到了 CHECKLIST 自己引用的被删字符串）；**D14** `:455` 写「10 项闭合、2 项部分闭合」与 Gaps §0.5 自己那张表对不上（实为 8 / 3 / 1）。六条已全部按机制修完，见上面 p2-3 那条 as_of 与下面本轮门禁格。**读错评审答案的教训（真实代价）**：我第一次读的是它 session JSONL 里最后一个文本块的 PASS 表格，据此拆出一份错的 D-list（compare_digest / 历史行没钉提交 / 401 vs 406 / 缓存 key 后果 / video harness 写死报告），`cf2de31` 就是按那份错的清单修的——那些修法本身都成立（它们是评审的精度说明与上一轮残留），但不是本轮的 D9–D14，而且第 3 轮 prompt 已按错清单发出去了（作废重发）。以后读 verdict：先按 index 数文本块、只读它自己的「缺陷清单」那一节，不读 PASS 表。另采纳它的四条精度说明（不算缺陷）：本轮范围是 3 个 commit 不是 2；naive `--as-of` 被接受（与 `--since` 同策略，awareness 校验只在 `schemas.py:128-145` 的 `EvidenceAtom` 上，`SearchFilters` 没有）；smoke 的端口守卫只在启动前查（它曾拿到一个正在死的 server：「FAIL: the session did not reload in the order it was recorded: []」）；`browse.py` 那条子句只被 pytest 卡住（删掉它 key_test 与 smoke 都绿，因为场景 5 测的是隔离不是 ACL）。
+  - **评审第 2 轮 FAIL**（同一独立 pi，`df812f3`，7 项全跑）：D1–D8 全部实测确认已修。as_of 的行为面它自己写脚本测通：退役的记忆在 `as_of` 下应答、下周的事今天不应答、归档不挡 as_of、不跨 holder 借看。`scripts/key_test.sh` 在 `df812f3` 上 exit 0（6 PASS + 1 NOT MEASURED），把 `store_search.py` 换回父提交就红在场景 3。新缺陷六条，三条行为三条记录：**D9** 那条 dense 测试名字不副实、as_of 的向量侧零覆盖（包一层计数实测 0 次调用；改 `_allowed` 的 as_of 分支、删 `retrieval.py:119` 都全绿）；**D10** 真开 dense 时两条腿不一致，词腿回旧事实、dense 腿回空，因为退役记忆根本没向量（`_expected`/追赶都读 `v_atoms_current`），commit message 里「cannot disagree」写过头；**D11** 排期到未来的 supersede 不披露且不对称：不带 as_of 的搜索 0 命中、`GET /api/atoms` total 0、dense 腿 1 行；**D12** `:430` 那个旧行号括号仍是错的；**D13** p2-5 验收 grep 实测 **2 命中**不是 0（它扫到了 CHECKLIST 自己引用的被删字符串）；**D14** `:455` 写「10 项闭合、2 项部分闭合」与 Gaps §0.5 自己那张表对不上（实为 8 / 3 / 1）。六条已全部按机制修完，见上面 p2-3 那条 as_of 与下面本轮门禁格。**读错评审答案的教训（真实代价）**：我第一次读的是它 session JSONL 里最后一个文本块的 PASS 表格，据此拆出一份错的 D-list（compare_digest / 历史行没钉提交 / 401 vs 406 / 缓存 key 后果 / video harness 写死报告），`cf2de31` 就是按那份错的清单修的——那些修法本身都成立（它们是评审的精度说明与上一轮残留），但不是本轮的 D9–D14，而且第 3 轮 prompt 已按错清单发出去了（作废重发）。以后读 verdict：先按 index 数文本块、只读它自己的「缺陷清单」那一节，不读 PASS 表。另采纳它的四条精度说明（不算缺陷）：本轮范围是 3 个 commit 不是 2；naive `--as-of` 被接受（与 `--since` 同策略，awareness 校验只在 `schemas.py:128-145` 的 `EvidenceAtom` 上，`SearchFilters` 没有）；smoke 的端口守卫只在启动前查（它曾拿到一个正在死的 server：「FAIL: the session did not reload in the order it was recorded: []」）；`browse.py` 那条子句只被 pytest 卡住（删掉它 key_test 与 smoke 都绿，因为场景 5 测的是隔离不是 ACL）。
   - **第 3 轮作废重发**：`cf2de31` 那一轮是按我读错的 D-list 发的（见上面那条教训），它验的仍是记录面那五条；本轮真正的 D9–D14 已在 `eab6bd7` 之后修完，重发的 prompt 把三条行为缺陷的复现方式写死（dense 腿零覆盖的包层计数、`_expected` 的向量计数、排期 supersede 的三种读法）。
+  - **评审第 3 轮 FAIL**（`cf2de31`，D15–D22，全部实测）：D15 = 我写「dense 腿也验了」时它其实是零覆盖——`test_as_of_search.py:105` 没开 `dense_embeddings_enabled`、没建 `VectorIndex`，在 `VectorIndex.search` 里加 `print("[CALL]")` 对该文件 **0 次**（`test_vector_index.py` 17 次），真开 dense 后 words 腿 `as_of=上周+1` 答抽屉、`index.search(as_of=…)` 返空，`expected_vectors` 只算出 1、`vec_atoms` 只有一行；把 `_allowed` 里 6 个 `as_of` 参数全绑成 `None` 仍 `5 passed`。⇒ 第 2 轮的 D9/D10 既没修也没记，却被写成已验。**已由 `92237c8` 修**（D9 测试真开 dense 并断言两条腿读同一窗口、D10 向量改读 `atoms`、D11 币种统一 + schema v12，五条变异各打红一条）。D16/D17 = 我上一行把功劳写错了对象：「在 `209f2d0` 重现旧 key 无身份成分」是**第 3 轮**做的，且实测比这句话多一层——`209f2d0:compiler.py:43` 根本不传 filters，那里没有「陌生人那份」可串，`compiler.py:33 if compiled.hits:` 说明空 pack 从不被端出；`as_holder or ""` 这个串**代码里从来没有**，只在 CHECKLIST 里出现过（`git log -S` 列出 `83dfc14`/`df812f3`/`cf2de31` 三个提交，都是改 CHECKLIST 的）。D18 = D-list 被换掉（已由 `92237c8` 恢复）。D19 = 「当前 411」是我自己工作树的数，被审的 `cf2de31` 是 410。D20 = 我那句「视频 harness 会拒写冻结报告」三处不实：`VIDEO_AUDIO.md`/`video_audio.json` 当时**未入库**（worktree 里根本没有可弄脏的冻结件）、worktree 跑不到写那步（死在 `no store to copy` / `123 store sources are not corpus files`）、守卫按 cwd 相对比较所以**绝对路径指向冻结件本身也能过**；且该 harness 是 `cf2de31` 自己引入的，「已修」修的是同一提交刚造的东西。D21 = `--limit 1 --probes 1`（含 `--probes 5`）死在 `aggregate needs at least one answerable query`、exit 1、零产物——冒烟旋钮在小数值时不是「少测」而是崩。D22 = 第 2 轮的 D13/D14 未动（已由 `92237c8` 修）。
+  - 本轮（D15–D22 收尾）harness 三修：`refuse_frozen_overwrite` 改成比较 `resolve()` 后的目录（绝对路径指向冻结件也拒，D20）；新增 `probe_sample(probes, limit)`，`--probes 0` 关掉该臂；当 `--limit` 把样本收到只剩视频时小 count 会整块跳过该模态（评审量的 `--limit 1 --probes 1` 与 `--probes 5` 就是），此时在**二十分钟 ingest 之前**就 `SystemExit` 点名「picked no answerable video query…Pass --probes 0 to skip the probe arm, or the shipped 300」（D21：分层样本按全库跨步，小 count 会整块跳过本模态）；`rank_arm` 空样本返 `None`，`_required` 保证视频臂不会静默变成一行零，`_score_of`/markdown 打印 `skipped`。新增 3 条测试（绝对路径守卫、`probe_sample` 两态、`_required`/`_score_of`）。
   - 本轮另修的两处产品事实（都是 D10/D11 带出来的）：一个还存在的记忆就保留它的向量（`vector_index.py:397`/`:451`/`:474` 改读 `atoms`），谁能看由 `_allowed` 决定；「现在开着」的谓词在视图与两条搜索腿里统一成 `valid_to IS NULL OR valid_to > now`，schema 以 **v12** 记录这次语义改变，升级时 `DROP VIEW`（`CREATE VIEW IF NOT EXISTS` 不会换身体）。`db.close_atoms` 的 `WHERE valid_to IS NULL` 不是现行读法而是写闸门（别去闭一条已经闭上的），故意保留。
   - 本轮门禁（D9–D14 修完，数字钉在本轮的 commit 上）：`pytest -q` **411 passed**（+1 排期 supersede 测试；`test_superseding_moves_the_vector_rather_than_keeping_two` 改成新契约 `test_a_superseded_memory_keeps_its_vector_but_leaves_the_current_answer_alone`：vectors **2**、现行搜索只回替换者、`as_of` 下回旧者）；`ruff check backend/src/ely_eye backend/tests bench` 干净；`bench/filesize_check` 干净；`bench.audit --product` → `audited 33 harness files, 2 reports, 6 set files` / `clean`；`./node_modules/.bin/pyright` 对 6 个改动文件 → **0 errors**；mypy 冷缓存对同 6 个文件 `Success: no issues found`；五个变异 M1–M5 各自只红一条测试（见 p2-3 那条 as_of）。
-  - 门禁（`83dfc14` 那轮量的；as_of 之后 406，D9–D14 修完当前 **411**）：`pytest -q` **401 passed**；`ruff check` 对 22 个改动文件干净；`bench/filesize_check` 干净；`bench.audit --product` → `audited 32 harness files, 2 reports, 6 set files` / `clean`；mypy 冷缓存 **24 errors in 9 files (checked 112 source files)**，与 `9cb46c1` 基线（24/9）**净新增 0**；`./node_modules/.bin/pyright` 对 17 个后端源文件 + 5 个测试文件 → **0 errors**；`go build` / `go vet` / `go test`（apiclient ok、mcpserver ok）+ `gofmt -l` 空；`npm run build` → `dist/assets/index-CSDHCFNo.js 221.48 kB`（bundle 哈希与 `246dae9` 相同，与「本轮没动前端」一致）+ eslint exit 0；`bash scripts/mcp_write_smoke.sh` 3.3s 两行 PASS。
+  - 门禁（`83dfc14` 那轮量的 401；`eab6bd7` 加 5 条 as_of 测试后 406，`cf2de31` 加 4 条视频 harness 测试后 **410**（评审第 3 轮 D19：我原先写「当前 411」是我工作树里的数，被审 commit 是 410），`92237c8` 加 1 条排期 supersede 测试后 **411**）：`pytest -q` **401 passed**；`ruff check` 对 22 个改动文件干净；`bench/filesize_check` 干净；`bench.audit --product` → `audited 32 harness files, 2 reports, 6 set files` / `clean`；mypy 冷缓存 **24 errors in 9 files (checked 112 source files)**，与 `9cb46c1` 基线（24/9）**净新增 0**；`./node_modules/.bin/pyright` 对 17 个后端源文件 + 5 个测试文件 → **0 errors**；`go build` / `go vet` / `go test`（apiclient ok、mcpserver ok）+ `gofmt -l` 空；`npm run build` → `dist/assets/index-CSDHCFNo.js 221.48 kB`（bundle 哈希与 `246dae9` 相同，与「本轮没动前端」一致）+ eslint exit 0；`bash scripts/mcp_write_smoke.sh` 3.3s 两行 PASS。
 - [x] **p2-5** 文档单一真相
   - [x] p2-5.1 `AGENTS.md` 重写并停止 gitignore
     - `.gitignore` 删了 `AGENTS.md` 那一行（`CLAUDE.md` 仍忽略，它不是本仓库的真相源），`git check-ignore AGENTS.md` 现在 exit 1。新文件 5.5KB，写的不是风格偏好而是**这个仓库付过学费的约束**：门禁命令清单（含 mypy 基线是 24 errors in 9 files、必须在分支点的 worktree 里量而不是猜）；一个 writer 一条规则（Go 不写 SQLite）；fail fast 必须点名（引真实报错 `nothing to forget: no atom or source was given`）；≤500 行、无死代码、注释单行、docstring 是契约；「缺席是一种状态」——`v_atoms_current` 隐藏而不删除。
@@ -460,7 +462,7 @@
     - 诚实处理两处派生数字：`+0.454` 不在 `BASELINE_EXTERNAL.md` 里，是它两行（0.866 / 0.412）相减得来，Gaps 里写明「该报告只给两行与‘这段差值是 holder 边界的价值’这句话，差值是这里减出来的」；ASR 速度改成当前报告的数（258.18s 音频 / 3.12s，每视频 68.4–111.2×），不再引旧报告的 63.9–112.2。
   - 偏差（顺手补的真缺口）：CLI `search` 只有 `--holder`（过滤器），**没有**办法用某个 holder 的眼睛看库——即无法回答「我的 agent 会看到什么」。加了 `--as-holder`（`SearchFilters.as_holder`）与两条断言（`--as-holder stranger` → `no_match`，`--as-holder wife` → 命中），否则三态眼睛只在 HTTP/MCP 两面成立而 CLI 面缺一个口。
   - 验收：`grep -rn "MMMU-Pro=\|OCRBench=\|1-hop=\|2-hop=" README.md TRAINING_ARTIFACTS.md Gaps.md AGENTS.md backend/README.md bench/README.md ely-eye-mcp/README.md` → **0 命中**（评审第 2 轮 D13：我原先写的命令带 `*.md`，会把 CHECKLIST 自己扫进去，实测 **2 命中**，两处都在本文件里——`:452` 与 `:458` 引用的正是被删掉的字符串；结论不变，命令与数字得改）。README 里每个数字都能在报告里找到（0.863→LEGS、0.866/0.412→BASELINE_EXTERNAL、0.0717/0.2177→SCALE、165973/153857/106 per second→DENSE、0.242/533/68.4–111.2→ASR、模态行→LEGS、148.9GB→Gaps 附录 B）。唯一留下的模型分数是 `TRAINING_ARTIFACTS.md:84` 的 `citation accuracy 1.00`：它在证据块里（那是当天产出的工件原文），且紧跟的段落写明了它测的是什么、不测什么。
-  - 门禁（`89c880f` 那轮量的；`eab6bd7` 加了 5 条 as_of 测试后 406，D9–D14 修完当前 **411**）：`pytest -q` **401 passed**；`ruff check backend/src/ely_eye backend/tests bench` 干净；`bench.filesize_check` 干净；`bench.audit --product` → `audited 32 harness files, 2 reports, 6 set files` / `clean`；mypy 冷缓存 **24 errors in 9 files (checked 112 source files)** = `9cb46c1` 基线，净新增 0；`./node_modules/.bin/pyright backend/src/ely_eye/cli.py backend/tests/test_search_api.py` → **0 errors**；本轮未动 Go/前端/报告，所以 go/npm 套件沿用 `83dfc14` 的绿。
+  - 门禁（`89c880f` 那轮量的 401；`eab6bd7` 加了 5 条 as_of 测试后 406，`cf2de31` 后 410，`92237c8` 加 1 条排期 supersede 测试后 **411**）：`pytest -q` **401 passed**；`ruff check backend/src/ely_eye backend/tests bench` 干净；`bench.filesize_check` 干净；`bench.audit --product` → `audited 32 harness files, 2 reports, 6 set files` / `clean`；mypy 冷缓存 **24 errors in 9 files (checked 112 source files)** = `9cb46c1` 基线，净新增 0；`./node_modules/.bin/pyright backend/src/ely_eye/cli.py backend/tests/test_search_api.py` → **0 errors**；本轮未动 Go/前端/报告，所以 go/npm 套件沿用 `83dfc14` 的绿。
 
 ---
 

bench/scratch.py

@@ -3,10 +3,40 @@ migrates whatever database it opens."""
 
 from __future__ import annotations
 
+import atexit
+import os
 import shutil
 import time
 from pathlib import Path
 
+HELD = "held_by"
+
+
+def _alive(pid: str) -> bool:
+    """A live process holds the copy. Without /proc there is nothing to check, so a stale lock cannot block a run."""
+    return pid.isdigit() and Path(f"/proc/{pid}").exists()
+
+
+def claim(scratch: Path) -> None:
+    """Refuse a copy another run is already using: two harnesses on one scratch overwrite each other's store mid-measurement."""
+    lock = scratch / HELD
+    if lock.exists():
+        holder = lock.read_text(encoding="utf-8").strip()
+        if _alive(holder) and holder != str(os.getpid()):
+            raise SystemExit(
+                f"{scratch} is held by pid {holder}; wait for that run or pass a different scratch path"
+            )
+    scratch.mkdir(parents=True, exist_ok=True)
+    lock.write_text(str(os.getpid()), encoding="utf-8")
+    atexit.register(_release, scratch)
+
+
+def _release(scratch: Path) -> None:
+    """Drop the claim, but only if it is still ours."""
+    lock = scratch / HELD
+    if lock.exists() and lock.read_text(encoding="utf-8").strip() == str(os.getpid()):
+        lock.unlink(missing_ok=True)
+
 
 def copy_store(home: Path, scratch: Path) -> tuple[Path, float]:
     """Work on a copy: the baseline store is frozen evidence and must not be migrated in place."""
@@ -15,6 +45,7 @@ def copy_store(home: Path, scratch: Path) -> tuple[Path, float]:
         raise SystemExit(
             f"{source}: no product database, build it with python -m bench.ingest_baseline"
         )
+    claim(scratch)
     data = scratch / "data"
     data.mkdir(parents=True, exist_ok=True)
     began = time.perf_counter()

bench/tests/test_scratch.py

@@ -0,0 +1,31 @@
+"""A scratch copy is one run's working store; two runs on one path silently measure each other's data."""
+
+import os
+from pathlib import Path
+
+import pytest
+
+from bench import scratch
+
+
+def test_a_live_holder_refuses_the_same_scratch(tmp_path: Path) -> None:
+    """The collision I hit: a second run re-copied the store under a run that was ingesting into it."""
+    if not Path("/proc/1").exists():
+        pytest.skip("liveness is read from /proc")
+    (tmp_path / scratch.HELD).write_text("1", encoding="utf-8")
+    with pytest.raises(SystemExit, match="is held by pid 1"):
+        scratch.claim(tmp_path)
+
+
+def test_a_stale_claim_is_taken_over(tmp_path: Path) -> None:
+    """A crashed run must not lock the scratch away forever."""
+    (tmp_path / scratch.HELD).write_text("999999", encoding="utf-8")
+    scratch.claim(tmp_path)
+    assert (tmp_path / scratch.HELD).read_text(encoding="utf-8") == str(os.getpid())
+
+
+def test_one_run_may_copy_more_than_once(tmp_path: Path) -> None:
+    """Harnesses that score several arms re-copy on purpose; that is not a collision."""
+    scratch.claim(tmp_path)
+    scratch.claim(tmp_path)
+    assert (tmp_path / scratch.HELD).read_text(encoding="utf-8") == str(os.getpid())

bench/tests/test_video_audio_check.py

@@ -11,8 +11,9 @@ from bench import video_audio_check
 def test_a_frozen_report_is_not_overwritten_by_a_harness(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> None:
-    """A reviewer runs harnesses in a worktree of its own; the committed reports stay put."""
+    """A reviewer runs harnesses in a worktree of its own; the committed evidence stays put."""
     monkeypatch.setattr(video_audio_check, "PAYLOAD", tmp_path / "video_audio.json")
+    monkeypatch.setattr(video_audio_check, "_tracked", lambda path: True)
     frozen = tmp_path / "VIDEO_AUDIO.md"
     frozen.write_text("committed evidence", encoding="utf-8")
     with pytest.raises(SystemExit, match="is a frozen report"):
@@ -23,6 +24,47 @@ def test_a_frozen_report_is_not_overwritten_by_a_harness(
     video_audio_check.refuse_frozen_overwrite(elsewhere)
 
 
+def test_a_harness_may_write_its_own_untracked_output(
+    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """The default destination is the reports directory; refusing it would stop the shipped run."""
+    monkeypatch.setattr(video_audio_check, "PAYLOAD", tmp_path / "video_audio.json")
+    monkeypatch.setattr(video_audio_check, "_tracked", lambda path: False)
+    fresh = tmp_path / "VIDEO_AUDIO.md"
+    fresh.write_text("not yet evidence", encoding="utf-8")
+    video_audio_check.refuse_frozen_overwrite(fresh)
+
+
+def test_an_absolute_path_to_the_frozen_directory_is_refused_too(
+    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """The guard compares where the file really lands, not how the path was spelled."""
+    monkeypatch.setattr(video_audio_check, "PAYLOAD", tmp_path / "video_audio.json")
+    monkeypatch.setattr(video_audio_check, "_tracked", lambda path: True)
+    frozen = tmp_path / "VIDEO_AUDIO.md"
+    frozen.write_text("committed evidence", encoding="utf-8")
+    with pytest.raises(SystemExit, match="is a frozen report"):
+        video_audio_check.refuse_frozen_overwrite(Path(frozen.resolve()))
+
+
+def test_the_probe_arm_switches_off_rather_than_ranking_nothing() -> None:
+    """--probes 0 means "skip that arm"; a small count with the smoke knob means "you measured nothing"."""
+    assert video_audio_check.probe_sample(0, 0) == []
+    # The shipped sample keeps every modality; the smoke knob narrows it to video, where a
+    # small count can miss the modality entirely (the reviewer's --limit 1 --probes 1 crash).
+    assert any(query.modality == video_audio_check.VIDEO_MODALITY for query in video_audio_check.probe_sample(300, 0))
+    for probes in (1, 5):
+        with pytest.raises(SystemExit, match=f"--probes {probes} picked no answerable video query"):
+            video_audio_check.probe_sample(probes, probes)
+
+
+def test_an_empty_video_arm_is_a_lie_not_a_zero() -> None:
+    """The video arm is the point of the harness; a row of zeros there must stop the run."""
+    with pytest.raises(SystemExit, match="nothing to rank"):
+        video_audio_check._required(None, "every video question")
+    assert video_audio_check._score_of(None) == "skipped"
+
+
 def test_a_missing_speech_snapshot_says_so_before_ranking(tmp_path: Path) -> None:
     """Without a snapshot every video defers, and the after row would equal the before row."""
     with pytest.raises(SystemExit, match="no speech snapshot"):

bench/video_audio_check.py

@@ -14,6 +14,7 @@ from __future__ import annotations
 import argparse
 import json
 import statistics
+import subprocess
 import sys
 import time
 from pathlib import Path
@@ -75,12 +76,14 @@ def speech_facts(store: Path) -> dict[str, Any]:
     return {"atoms": int(atoms), "characters": int(characters)}
 
 
-def rank_arm(home: Path, probes: list[Any], top_k: int) -> dict[str, Any]:
-    """Rank the probes through the shipped words-only path and score them."""
+def rank_arm(home: Path, probes: list[Any], top_k: int) -> dict[str, Any] | None:
+    """Rank the probes through the shipped words-only path and score them; None skips the arm."""
     from ely_eye.config import Settings
     from ely_eye.db import Database
     from ely_eye.retrieval import RetrievalService
 
+    if not probes:
+        return None
     settings = Settings(
         home=home,
         dense_embeddings_enabled=False,
@@ -140,15 +143,33 @@ def add_speech(home: Path, files: list[Path], asr: Path, limit: int) -> dict[str
     return {"files": batches, "seconds": time.perf_counter() - began}
 
 
+def probe_sample(probes: int, limit: int) -> list[Any]:
+    """The standard stratified sample, asked about video only when the smoke knob is on.
+
+    A small `--probes` is not "measure a little": the sample strides the whole suite, so a
+    count under a few dozen can miss this modality entirely and the arm would rank nothing.
+    """
+    if probes <= 0:
+        return []
+    picked = [query for query in scale_probes(load_queries(), probes) if query.gold]
+    if limit:
+        picked = [query for query in picked if query.modality == VIDEO_MODALITY][:limit]
+    if not picked:
+        raise SystemExit(
+            f"--probes {probes} picked no answerable video query: the sample strides the whole "
+            "suite, so a small count can skip this modality and the arm would rank nothing. "
+            "Pass --probes 0 to skip the probe arm, or the shipped 300 to keep every modality."
+        )
+    return picked
+
+
 def measure(home: Path, scratch: Path, videos: Path, asr: Path, probes: int, limit: int) -> dict[str, Any]:
     """Score the video questions and the standard probe sample before and after the speech pass."""
     if not (home / "data" / "ely_eye.sqlite").exists():
         raise SystemExit(f"{home}: no store to copy; run `python -m bench.dense_scale` first")
     files = guard_videos(videos, home)
     whole = video_queries() if not limit else video_queries()[:limit]
-    sample = [query for query in scale_probes(load_queries(), probes) if query.gold]
-    if limit:
-        sample = [query for query in sample if query.modality == VIDEO_MODALITY][:limit]
+    sample = probe_sample(probes, limit)
     print(f"{len(whole)} answerable video queries, {len(sample)} stratified probes, {len(files)} files", flush=True)
 
     copy_home, copied = copy_store(home, scratch)
@@ -156,11 +177,11 @@ def measure(home: Path, scratch: Path, videos: Path, asr: Path, probes: int, lim
     before_speech = speech_facts(copy_home)
     if before_speech["atoms"]:
         print(f"note: the copy already carries {before_speech['atoms']} speech atoms", flush=True)
-    before_video = rank_arm(copy_home, whole, SHIPPED_TOP_K)
+    before_video = _required(rank_arm(copy_home, whole, SHIPPED_TOP_K), "every video question")
     before_sample = rank_arm(copy_home, sample, SHIPPED_TOP_K)
     print(
         f"  before: video recall@10 {before_video['recall_at_10']:.3f}, "
-        f"probes {before_sample['recall_at_10']:.3f}",
+        f"probes {_score_of(before_sample)}",
         flush=True,
     )
 
@@ -171,11 +192,11 @@ def measure(home: Path, scratch: Path, videos: Path, asr: Path, probes: int, lim
             f"re-ingesting {len(files)} files added no speech atoms; the snapshot at {asr} is "
             "available but produced nothing, so the after row would be a measurement of nothing"
         )
-    after_video = rank_arm(copy_home, whole, SHIPPED_TOP_K)
+    after_video = _required(rank_arm(copy_home, whole, SHIPPED_TOP_K), "every video question")
     after_sample = rank_arm(copy_home, sample, SHIPPED_TOP_K)
     print(
         f"  after:  video recall@10 {after_video['recall_at_10']:.3f}, "
-        f"probes {after_sample['recall_at_10']:.3f}, speech atoms {after_speech['atoms']}",
+        f"probes {_score_of(after_sample)}, speech atoms {after_speech['atoms']}",
         flush=True,
     )
     return {
@@ -193,6 +214,18 @@ def measure(home: Path, scratch: Path, videos: Path, asr: Path, probes: int, lim
     }
 
 
+def _required(row: dict[str, Any] | None, label: str) -> dict[str, Any]:
+    """The video arm is never empty; a row of zeros there would be a silent lie."""
+    if row is None:
+        raise SystemExit(f"{label}: nothing to rank, so this run would print a row of zeros")
+    return row
+
+
+def _score_of(row: dict[str, Any] | None) -> str:
+    """One number for the progress line, or the reason there is none."""
+    return "skipped" if row is None else f"{row['recall_at_10']:.3f}"
+
+
 def write_markdown(report: dict[str, Any], destination: Path) -> None:
     """Print the two rows beside each other, with the load they were measured under."""
     load = report["machine"].get("load_average_1_5_15")
@@ -217,15 +250,19 @@ def write_markdown(report: dict[str, Any], destination: Path) -> None:
     for stage in ("before", "after"):
         for arm, label in (("video", "every video question"), ("probes", "the stratified probe sample")):
             row = report[stage][arm]
+            if row is None:
+                lines.append(f"| {label} ({stage}) | 0 | skipped (--probes 0) | — | — | — | — |")
+                continue
             lines.append(
                 f"| {label} ({stage}) | {row['queries']} | {row['recall_at_10']:.3f} | {row['mrr']:.3f} | "
                 f"{row['hit_at_10']:.3f} | {row['wilson_lower']:.3f} | {row['p50_seconds']:.3f} |"
             )
     gained = report["after"]["video"]["recall_at_10"] - report["before"]["video"]["recall_at_10"]
-    overall = report["after"]["probes"]["recall_at_10"] - report["before"]["probes"]["recall_at_10"]
+    probe_row = report["after"]["probes"]
+    overall = "not measured (--probes 0)" if probe_row is None else f"{probe_row['recall_at_10'] - report['before']['probes']['recall_at_10']:+.3f}"
     lines += [
         "",
-        f"Speech moved video recall@10 by {gained:+.3f} and the whole probe sample by {overall:+.3f}.",
+        f"Speech moved video recall@10 by {gained:+.3f} and the whole probe sample by {overall}.",
         "",
         "Latency is read from the same pass and moves with machine load; the recall columns do not.",
         "",
@@ -234,9 +271,21 @@ def write_markdown(report: dict[str, Any], destination: Path) -> None:
     destination.write_text("\n".join(lines), encoding="utf-8")
 
 
+def _tracked(path: Path) -> bool:
+    """Only committed evidence is frozen; a harness's own untracked output is fair game."""
+    return (
+        subprocess.run(
+            ["git", "ls-files", "--error-unmatch", str(path)], capture_output=True, check=False
+        ).returncode
+        == 0
+    )
+
+
 def refuse_frozen_overwrite(destination: Path) -> None:
     """A reviewer running this in a worktree of its own must not overwrite the committed evidence."""
-    if destination.parent == PAYLOAD.parent and destination.exists():
+    if destination.exists() and destination.resolve().parent == PAYLOAD.resolve().parent and _tracked(
+        destination
+    ):
         raise SystemExit(
             f"{destination} is a frozen report; point --report and --payload somewhere else, or "
             "delete the frozen copy on purpose rather than under a harness"