Give every memory a holder, an origin, a time and a scenario

An EvidenceAtom could say what was remembered and nothing else: not whose memory it
is, not where it came from, not when it was true, not in what situation it may be
used. The baseline measured the cost of that: the same ranker scores recall@10 0.412
over the whole corpus and 0.866 inside one holder's memory, so the missing holder
column is worth more than any ranking change in the ablation.

- schemas.py: MemoryOrigin, TimePrecision, Visibility; EvidenceAtom gains holder_id,
  origin, scenario, event_time / event_time_end / event_time_precision /
  event_time_source, valid_from / valid_to, supersedes, visibility, retention_until.
  `time` is renamed `ingested_at` — ingestion time gets one name, not two.
  A validator refuses valid_to before valid_from, event_time_end before event_time,
  and an atom that supersedes itself.
- migrations.py: schema v2 with PRAGMA user_version. A first-version database is
  upgraded in place (ALTER + RENAME COLUMN), rows intact, and a database newer than
  this build refuses to open instead of being silently downgraded.
- v_atoms_current hides memories that are closed, lapsed or not yet begun; unknown
  bounds read as visible. Time travel is db.atoms_as_of(moment, holder) because a
  SQLite view cannot take a parameter.
- evaluation_schemas.py: the proof and benchmark report shapes move out of schemas.py
  to keep it under the file-size limit; importers follow.
- backend/tests/test_memory_schema.py: 8 tests, including the v1 -> v2 upgrade and the
  validity timeline.

Verification of record: pytest backend/tests bench/tests -q -> 70 passed; ruff check on
every touched file -> all checks passed; mypy (fresh cache) on db.py, migrations.py,
schemas.py, evaluation_schemas.py and backend/tests -> no issues. evals.py carries 40
pre-existing mypy errors at HEAD and 40 after this change (measured on both copies);
its pyright findings are the same debt, and p0-6 rewrites that file. `peft` and
`pyarrow.parquet` "could not be resolved" is the documented pyright artifact: pi-lens
runs it with 0 config files, and both import at runtime.
This commit is contained in:
ZacharyZhang-NY committed 2026-10-07 14:30:22 -04:00
1 parent 00dc9f8eaf
commit 181f6bf57e
12 files changed
+1100 -319

No files matched your search

+8 -6
View File
@@ -8,7 +8,7 @@ from ast import literal_eval
from pathlib import Path
from typing import Any
from .schemas import (
from .evaluation_schemas import (
BenchmarkPredictionReport,
BenchmarkPredictionStatus,
BenchmarkRegistryReport,
@@ -19,7 +19,6 @@ from .schemas import (
BenchmarkSpec,
)
STANDARD_BENCHMARKS = (
BenchmarkSpec(
name="MMMU-Pro",
@@ -125,7 +124,7 @@ def benchmark_source_report(timeout: int = 20) -> BenchmarkSourceReport:
sibling_count=len(siblings),
)
)
except Exception as exc:
except Exception as exc: # noqa: BLE001 # a probe records its failure instead of crashing the proof run
sources.append(
BenchmarkSourceStatus(
name=benchmark.name,
@@ -217,7 +216,7 @@ def load_benchmark_sample(benchmark: BenchmarkSpec, output_root: Path) -> Benchm
sample_sha256=sha256_json(normalized),
media_artifacts=media_artifacts,
)
except Exception as exc:
except Exception as exc: # noqa: BLE001 # a probe records its failure instead of crashing the proof run
message = str(exc).splitlines()[0]
status = "gated" if "gated dataset" in message.lower() or "authenticated" in message.lower() else "failed"
return BenchmarkSampleStatus(
@@ -274,7 +273,7 @@ def predict_benchmark_sample(benchmark: BenchmarkSpec, output_root: Path, runtim
output_tokens=generation.output_tokens,
media_artifacts=prediction_media,
)
except Exception as exc:
except Exception as exc: # noqa: BLE001 # a probe records its failure instead of crashing the proof run
message = str(exc).splitlines()[0]
status = "gated" if "gated dataset" in message.lower() or "authenticated" in message.lower() else "failed"
config, split = BENCHMARK_SAMPLE_LOADS.get(benchmark.name, (None, None))
@@ -351,7 +350,10 @@ def load_records(path: Path) -> list[dict[str, Any]]:
with path.open("r", encoding="utf-8") as handle:
return [json.loads(line) for line in handle if line.strip()]
if suffix == ".json":
data = json.loads(path.read_text(encoding="utf-8"))
try:
data = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as error:
raise ValueError(f"{path}: cannot read the benchmark file ({error})") from error
if isinstance(data, list):
return [dict(item) for item in data if isinstance(item, dict)]
if isinstance(data, dict):