Stop the prose from drifting away from the report it cites
A second review found three numbers left behind when the report was regenerated with pinned decoding: - the transcriber docstring claimed 1.7 s for 88.9 s of speech; the report says 1.39 s, and 1.7 came from the unpinned run. The docstring now names no number, so it cannot go stale, and it stops calling container seconds speech with voice activity detection off. - the checklist still quoted 3.55 s and 61.6x / 73.0x / 97.7x; the frozen report says 3.26 s and 63.9x / 82.4x / 112.2x. - the checklist still said 31 speech segments for one mp4; with greedy decoding that video yields 38, which is where the store's 80 atoms come from (38 + 39 + 3). The report also credited the store with timing invariants that were measured on the model's answer. The harness now recomputes them from the words as stored (out_of_order_in_store, before_their_segment_in_store) and says so. Verification: 212 passed, ruff clean, mypy clean on both touched files with a cold cache, pyright 0 errors.
This commit is contained in:
1 parent
836a4908e1
commit
10a39c5125
3 files changed
+15
-5
No files matched your search
+12
-2
@@ -148,11 +148,20 @@ def speech_atoms_in_the_store(videos: list[Path]) -> dict[str, Any]:
|
||||
service.ingest_path(video)
|
||||
atoms = database.list_atoms()
|
||||
audio = [atom for atom in atoms if atom.modality == Modality.audio]
|
||||
out_of_order = 0
|
||||
before_their_segment = 0
|
||||
for atom in audio:
|
||||
segment_start = float(atom.metadata.get("start_seconds", 0.0))
|
||||
starts = [float(word["start"]) for word in atom.metadata.get("words", [])]
|
||||
out_of_order += sum(1 for first, second in pairwise(starts) if second < first)
|
||||
before_their_segment += sum(1 for start in starts if start < segment_start - 0.01)
|
||||
return {
|
||||
"atoms_all": len(atoms),
|
||||
"speech_atoms": len(audio),
|
||||
"word_timestamps": sum(len(atom.metadata.get("words", [])) for atom in audio),
|
||||
"sources": len({atom.source_id for atom in audio}),
|
||||
"out_of_order_in_store": out_of_order,
|
||||
"before_their_segment_in_store": before_their_segment,
|
||||
}
|
||||
|
||||
|
||||
@@ -265,8 +274,9 @@ def write_markdown(report: dict[str, Any]) -> None:
|
||||
(
|
||||
f"Ingested through the product path, the store holds {totals['speech_atoms']} audio atoms from "
|
||||
f"{totals['sources']} sources carrying {totals['word_timestamps']} word timestamps "
|
||||
f"({totals['out_of_order']} out of order, {totals['before_their_segment']} before their own segment) "
|
||||
f"out of {totals['atoms_all']} atoms in total."
|
||||
f"({totals['out_of_order_in_store']} out of order, "
|
||||
f"{totals['before_their_segment_in_store']} before their own segment) out of {totals['atoms_all']} "
|
||||
"atoms in total, checked on the words as stored rather than on the model's answer."
|
||||
),
|
||||
"",
|
||||
(
|
||||
|
||||
Reference in new issue
Block a user