Make a cartridge a scope you mount, not only a bundle you export
A cartridge used to be an export: materialize it, hash it, and never let it
change behaviour again. The transactive-memory promise needs the opposite —
mounting the home cartridge must make the work cartridge's memories
unfindable, and appending to a cartridge must make a new memory findable
through it.
Cartridge scope (schema v2 -> v3):
- cartridge_atoms (which atoms are in which cartridge) and cartridge_mounts
(which cartridges this machine has mounted now); both rebuilt on reopen,
because a mount is a machine-local state, not a fact about the data.
- mount / switch / unmount / append refuse a cartridge that was built for a
different runtime, naming every differing field instead of a boolean.
- append rewrites atoms.parquet and sparse_index.bm25.zst, recomputes DNA and
the KV fingerprint, and keeps the cartridges row in one SQLite transaction;
the files are written first so a failed export cannot leave a half-cartridge
in the database.
- RetrievalService.search() now ranks inside the mounted scope by default, and
an explicit cartridge_ids filter overrides the mount. Nothing mounted means
the whole library, so an empty store behaves as it did before.
- forget() removes the forgotten atoms from every cartridge's membership:
forgetting must not leave a memory retrievable through a cartridge.
KV fingerprint (kv_fingerprint.py, stored in the manifest):
- model / tokenizer / chat-template hashes, the RoPE block (factor,
original_max_position_embeddings, mrope_interleaved, mrope_section,
partial_rotary_factor), the adapter hash, the context length, the engine
version, and the fingerprint format version.
- A snapshot we cannot read is recorded as unverifiable and listed in
unverifiable_fields; a blind machine refuses to mount rather than pretending
the runtime matches, because a wrong RoPE factor silently corrupts retrieval.
Two defects found by running the new tests, not by reading:
- atoms.parquet could not export a memory at all: pyarrow refuses a struct with
no child field, so metadata={} failed the whole table. The six JSON columns
are now written as JSON text and decoded by the cartridge reader.
- materialize() inserted cartridge membership before the cartridges row existed,
so the foreign key rejected every cartridge.
api.py crossed 500 lines, so the cartridge and mount routes moved to
api_cartridges.py. That module must not use `from __future__ import
annotations`: FastAPI then resolves Depends(settings_dep) against module
globals, where the injected dependency is only a parameter of register(), and
every route starts demanding settings as a query parameter.
Verification: 138 pytest passed (26 new: 10 fingerprint, 16 scope), Go
build/vet/gofmt/test clean, and scripts/mcp_write_smoke.sh now proves the whole
loop across two processes - mount narrows search, a memory written after the
upload is invisible until append_to_cartridge, unmount restores the library,
and rewriting the RoPE factor makes mount_cartridge refuse with "rope: factor:
cartridge 4.0 vs local 2.0". ruff clean on every touched file (the 20 remaining
package errors are in files this change did not touch; HEAD had 25). mypy: one
error, the pre-existing np.savez stub artifact at cartridge_assets.py:329
(HEAD:320). Repo pyright: 62 errors, unchanged from the p0-3 ledger.
This commit is contained in:
1 parent
674c88e84e
commit
9fe13fffb3
21 files changed
+1492
-153
No files matched your search
@@ -168,7 +168,7 @@ def forget(atom_id: str = "", source_id: str = "", reason: str = "") -> None:
|
||||
def build_dense_index() -> None:
|
||||
index = EmbeddingService(get_settings()).build_index()
|
||||
dimensions = int(index.vectors.shape[1]) if index.vectors.size else 0
|
||||
typer.echo({"atoms": len(index.atom_ids), "dimensions": dimensions})
|
||||
typer.echo(json_dump({"atoms": len(index.atom_ids), "dimensions": dimensions}))
|
||||
|
||||
|
||||
@app.command("list-cartridges")
|
||||
@@ -177,6 +177,38 @@ def list_cartridges() -> None:
|
||||
typer.echo(manifest.model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("mounts")
|
||||
def mounts() -> None:
|
||||
typer.echo(CartridgeService(get_settings()).mount_state().model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command()
|
||||
def mount(cartridge_id: str) -> None:
|
||||
"""Add a cartridge to the mounted set; a mounted search answers only from it."""
|
||||
typer.echo(CartridgeService(get_settings()).mount(cartridge_id).model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command()
|
||||
def switch(cartridge_id: str) -> None:
|
||||
"""Mount one cartridge and nothing else."""
|
||||
typer.echo(CartridgeService(get_settings()).switch(cartridge_id).model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command()
|
||||
def unmount(cartridge_id: str) -> None:
|
||||
typer.echo(CartridgeService(get_settings()).unmount(cartridge_id).model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command()
|
||||
def append(
|
||||
cartridge_id: str,
|
||||
atom_id: list[str] = typer.Option(..., "--atom-id"), # noqa: B008 # typer reads option defaults itself
|
||||
) -> None:
|
||||
"""Add memories to an existing cartridge; the bundle and its DNA follow."""
|
||||
service = CartridgeService(get_settings())
|
||||
typer.echo(service.append(cartridge_id, list(atom_id)).model_dump_json(indent=2))
|
||||
|
||||
|
||||
@app.command("hashhop")
|
||||
def hashhop(kind: str = "visual", hops: int = 2, token_equivalent: int = 262_144) -> None:
|
||||
evaluator = HashHopEvaluator(get_settings())
|
||||
|
||||
Reference in new issue
Block a user