Make a cartridge a scope you mount, not only a bundle you export

A cartridge used to be an export: materialize it, hash it, and never let it
change behaviour again. The transactive-memory promise needs the opposite —
mounting the home cartridge must make the work cartridge's memories
unfindable, and appending to a cartridge must make a new memory findable
through it.

Cartridge scope (schema v2 -> v3):
- cartridge_atoms (which atoms are in which cartridge) and cartridge_mounts
  (which cartridges this machine has mounted now); both rebuilt on reopen,
  because a mount is a machine-local state, not a fact about the data.
- mount / switch / unmount / append refuse a cartridge that was built for a
  different runtime, naming every differing field instead of a boolean.
- append rewrites atoms.parquet and sparse_index.bm25.zst, recomputes DNA and
  the KV fingerprint, and keeps the cartridges row in one SQLite transaction;
  the files are written first so a failed export cannot leave a half-cartridge
  in the database.
- RetrievalService.search() now ranks inside the mounted scope by default, and
  an explicit cartridge_ids filter overrides the mount. Nothing mounted means
  the whole library, so an empty store behaves as it did before.
- forget() removes the forgotten atoms from every cartridge's membership:
  forgetting must not leave a memory retrievable through a cartridge.

KV fingerprint (kv_fingerprint.py, stored in the manifest):
- model / tokenizer / chat-template hashes, the RoPE block (factor,
  original_max_position_embeddings, mrope_interleaved, mrope_section,
  partial_rotary_factor), the adapter hash, the context length, the engine
  version, and the fingerprint format version.
- A snapshot we cannot read is recorded as unverifiable and listed in
  unverifiable_fields; a blind machine refuses to mount rather than pretending
  the runtime matches, because a wrong RoPE factor silently corrupts retrieval.

Two defects found by running the new tests, not by reading:
- atoms.parquet could not export a memory at all: pyarrow refuses a struct with
  no child field, so metadata={} failed the whole table. The six JSON columns
  are now written as JSON text and decoded by the cartridge reader.
- materialize() inserted cartridge membership before the cartridges row existed,
  so the foreign key rejected every cartridge.

api.py crossed 500 lines, so the cartridge and mount routes moved to
api_cartridges.py. That module must not use `from __future__ import
annotations`: FastAPI then resolves Depends(settings_dep) against module
globals, where the injected dependency is only a parameter of register(), and
every route starts demanding settings as a query parameter.

Verification: 138 pytest passed (26 new: 10 fingerprint, 16 scope), Go
build/vet/gofmt/test clean, and scripts/mcp_write_smoke.sh now proves the whole
loop across two processes - mount narrows search, a memory written after the
upload is invisible until append_to_cartridge, unmount restores the library,
and rewriting the RoPE factor makes mount_cartridge refuse with "rope: factor:
cartridge 4.0 vs local 2.0". ruff clean on every touched file (the 20 remaining
package errors are in files this change did not touch; HEAD had 25). mypy: one
error, the pre-existing np.savez stub artifact at cartridge_assets.py:329
(HEAD:320). Repo pyright: 62 errors, unchanged from the p0-3 ledger.
This commit is contained in:
ZacharyZhang-NY committed 2026-10-07 16:40:27 -04:00
1 parent 674c88e84e
commit 9fe13fffb3
21 files changed
+1492 -153

No files matched your search

+33 -1
View File
@@ -168,7 +168,7 @@ def forget(atom_id: str = "", source_id: str = "", reason: str = "") -> None:
def build_dense_index() -> None:
index = EmbeddingService(get_settings()).build_index()
dimensions = int(index.vectors.shape[1]) if index.vectors.size else 0
typer.echo({"atoms": len(index.atom_ids), "dimensions": dimensions})
typer.echo(json_dump({"atoms": len(index.atom_ids), "dimensions": dimensions}))
@app.command("list-cartridges")
@@ -177,6 +177,38 @@ def list_cartridges() -> None:
typer.echo(manifest.model_dump_json(indent=2))
@app.command("mounts")
def mounts() -> None:
typer.echo(CartridgeService(get_settings()).mount_state().model_dump_json(indent=2))
@app.command()
def mount(cartridge_id: str) -> None:
"""Add a cartridge to the mounted set; a mounted search answers only from it."""
typer.echo(CartridgeService(get_settings()).mount(cartridge_id).model_dump_json(indent=2))
@app.command()
def switch(cartridge_id: str) -> None:
"""Mount one cartridge and nothing else."""
typer.echo(CartridgeService(get_settings()).switch(cartridge_id).model_dump_json(indent=2))
@app.command()
def unmount(cartridge_id: str) -> None:
typer.echo(CartridgeService(get_settings()).unmount(cartridge_id).model_dump_json(indent=2))
@app.command()
def append(
cartridge_id: str,
atom_id: list[str] = typer.Option(..., "--atom-id"), # noqa: B008 # typer reads option defaults itself
) -> None:
"""Add memories to an existing cartridge; the bundle and its DNA follow."""
service = CartridgeService(get_settings())
typer.echo(service.append(cartridge_id, list(atom_id)).model_dump_json(indent=2))
@app.command("hashhop")
def hashhop(kind: str = "visual", hops: int = 2, token_equivalent: int = 262_144) -> None:
evaluator = HashHopEvaluator(get_settings())