ZacharyZhang-NY e81923f62e Score recall over the whole suite and time only a stratified sample ...

The 300-probe sample this harness scored by modality read the screenshot
modality 0.818 where all 320 of its answerable queries read 0.659, because
striding a file-ordered list and truncating at 300 reshuffles the modality
mix. Recall is now scored over every answerable query (2,928) and latency
over a stratified sample, which is what latency needs and what recall does
not. The separate conversation sub-pass is gone: conversation is inside the
suite, and by_category already carries its six LongMemEval types.

2026-10-10 01:59:25 -04:00

..

test_audit.py

Require a token for writes and quarantine what nobody vouches for

2026-10-08 23:05:15 -04:00

test_counterpart.py

Add baseline harness: ingest, metrics, ablation arms, external counterpart, audit

2026-10-06 14:36:02 -04:00

test_joint.py

Keep every file inside the rules the repo already wrote down

2026-10-08 08:20:21 -04:00

test_leg_check.py

Score recall over the whole suite and time only a stratified sample

2026-10-10 01:59:25 -04:00

test_metrics.py

Time every modality and ask which memory came back first

2026-10-09 06:14:04 -04:00

test_query_set.py

Time every modality and ask which memory came back first

2026-10-09 06:14:04 -04:00

test_scratch.py

Stop two harnesses from measuring each other's store

2026-10-09 22:34:32 -04:00

test_systems.py

Keep every file inside the rules the repo already wrote down

2026-10-08 08:20:21 -04:00

test_video_audio_check.py

Stop two harnesses from measuring each other's store

2026-10-09 22:34:32 -04:00