Give spoken words a memory with an instant attached
An audio file used to produce no memory at all: _ingest_file returned an empty batch for any suffix it did not know, with no log and no reason. It now raises UnsupportedFile when you name the file and records a SkippedFile when it was a directory walk, so a refusal is visible either way. Speech is measured, not promised. Over 210.5 seconds of real Video-MME audio, faster-whisper-tiny on CPU int8 transcribed at 51x realtime and reached word error rate 0.242 against community subtitles nobody here wrote (a caption track drops words a transcript keeps, so that is an upper bound). 66 speech atoms now carry 550 word-level timestamps, 0 words out of order and 0 starting before their own segment. The snapshot is 78.2 MB, under the 500 MB cap, and the report is bench/reports/ASR.md. Two library facts were measured rather than assumed. faster-whisper's own audio loader is broken against PyAV 19 (av.open got an unexpected keyword 'metadata_errors') and it declares av with no version bound, so asr.decode_audio decodes with PyAV directly and hands the model a float32 array instead of pinning an old av with no cp314 wheel. And av.Error does not exist on PyAV 19 - the real base is av.error.FFmpegError - so that is the only thing caught and my own ValueErrors keep their messages. Files over the 500-line rule were split rather than excused: ingestion.py (748) into ingestion_common / ingestion_media / ingestion_repository / ingestion, and cli.py (544) into cli_senses plus cli_common, following the cli_transactive register pattern. MediaIngest declares _source_record abstract, so a half-wired host fails at construction instead of returning None. Verification: 207 tests pass; ruff clean on the 13 touched files; mypy clean on them, and repo-wide mypy fell from 29 errors to 24 (5 fixed, none added); pyright reports 0 errors on the touched files.
This commit is contained in:
1 parent
03ca35763f
commit
5289cb4b01
20 files changed
+1513
-611
No files matched your search
@@ -0,0 +1,16 @@
|
||||
# ASR: what the speech reader hears (commit 03ca35763)
|
||||
|
||||
- generated 2026-10-08T01:05:45+00:00
|
||||
- model: Systran/faster-whisper-tiny (cpu int8), model.bin sha256 dcb76c6586fc06cb…
|
||||
- reference: Video-MME community subtitles (third-party captions, not verbatim transcripts)
|
||||
- machine: {'system': 'Linux', 'release': '6.18.40.1-microsoft-standard-WSL2', 'cpus': 32, 'threads_per_query': 4, 'load_average_1_5_15': [1.84, 1.85, 6.99]}
|
||||
|
||||
| video | audio s | transcribe s | ×realtime | speech atoms | words | WER | mean word confidence |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| 0IdYJGBmguM | 88.9 | 1.73 | 51.3 | 31 | 218 | 0.179 | 0.879 |
|
||||
| 1q-5IIyZL20 | 99.9 | 1.94 | 51.6 | 32 | 300 | 0.28 | 0.83 |
|
||||
| 1tidNDIpKSY | 21.7 | 1.34 | 16.1 | 3 | 32 | 0.278 | 0.852 |
|
||||
|
||||
Over 3 videos and 210.5 seconds of audio: word error rate **0.242** against captions nobody here wrote, 66 speech atoms carrying 550 word-level timestamps, 0 words out of order and 0 starting before their own segment.
|
||||
|
||||
A caption track drops words a transcript keeps, so this WER is an upper bound on the model's real error. The point of the run is not the number: before this change an audio file produced no memory at all, and every word here now has an instant attached to it.
|
||||
@@ -0,0 +1,80 @@
|
||||
{
|
||||
"generated_utc": "2026-10-08T01:05:45+00:00",
|
||||
"commit": "03ca35763fd63eb8d8f0d2c721d02c6c3ded90a7",
|
||||
"model": {
|
||||
"id": "Systran/faster-whisper-tiny",
|
||||
"sha256_model_bin": "dcb76c6586fc06cbdac6dd21f14cfd129cc4cdd9dce19bf4ffa62e59cbe6e6d1",
|
||||
"compute": "cpu int8"
|
||||
},
|
||||
"reference": "Video-MME community subtitles (third-party captions, not verbatim transcripts)",
|
||||
"machine": {
|
||||
"system": "Linux",
|
||||
"release": "6.18.40.1-microsoft-standard-WSL2",
|
||||
"cpus": 32,
|
||||
"threads_per_query": 4,
|
||||
"load_average_1_5_15": [
|
||||
1.84,
|
||||
1.85,
|
||||
6.99
|
||||
]
|
||||
},
|
||||
"totals": {
|
||||
"videos": 3,
|
||||
"audio_seconds": 210.5,
|
||||
"word_error_rate": 0.242,
|
||||
"speech_atoms": 66,
|
||||
"words": 550,
|
||||
"out_of_order": 0,
|
||||
"before_their_segment": 0
|
||||
},
|
||||
"videos": [
|
||||
{
|
||||
"video": "0IdYJGBmguM",
|
||||
"seconds": 88.9,
|
||||
"transcribe_seconds": 1.73,
|
||||
"real_time_factor": 51.3,
|
||||
"segments": 31,
|
||||
"speech_atoms": 31,
|
||||
"modality": "audio",
|
||||
"hypothesis_words": 218,
|
||||
"reference_words": 201,
|
||||
"word_error_rate": 0.179,
|
||||
"mean_word_confidence": 0.879,
|
||||
"words": 218,
|
||||
"out_of_order": 0,
|
||||
"before_their_segment": 0
|
||||
},
|
||||
{
|
||||
"video": "1q-5IIyZL20",
|
||||
"seconds": 99.9,
|
||||
"transcribe_seconds": 1.94,
|
||||
"real_time_factor": 51.6,
|
||||
"segments": 32,
|
||||
"speech_atoms": 32,
|
||||
"modality": "audio",
|
||||
"hypothesis_words": 300,
|
||||
"reference_words": 296,
|
||||
"word_error_rate": 0.28,
|
||||
"mean_word_confidence": 0.83,
|
||||
"words": 300,
|
||||
"out_of_order": 0,
|
||||
"before_their_segment": 0
|
||||
},
|
||||
{
|
||||
"video": "1tidNDIpKSY",
|
||||
"seconds": 21.7,
|
||||
"transcribe_seconds": 1.34,
|
||||
"real_time_factor": 16.1,
|
||||
"segments": 3,
|
||||
"speech_atoms": 3,
|
||||
"modality": "audio",
|
||||
"hypothesis_words": 32,
|
||||
"reference_words": 36,
|
||||
"word_error_rate": 0.278,
|
||||
"mean_word_confidence": 0.852,
|
||||
"words": 32,
|
||||
"out_of_order": 0,
|
||||
"before_their_segment": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in new issue
Block a user