Make a cartridge a scope you mount, not only a bundle you export
A cartridge used to be an export: materialize it, hash it, and never let it
change behaviour again. The transactive-memory promise needs the opposite —
mounting the home cartridge must make the work cartridge's memories
unfindable, and appending to a cartridge must make a new memory findable
through it.
Cartridge scope (schema v2 -> v3):
- cartridge_atoms (which atoms are in which cartridge) and cartridge_mounts
(which cartridges this machine has mounted now); both rebuilt on reopen,
because a mount is a machine-local state, not a fact about the data.
- mount / switch / unmount / append refuse a cartridge that was built for a
different runtime, naming every differing field instead of a boolean.
- append rewrites atoms.parquet and sparse_index.bm25.zst, recomputes DNA and
the KV fingerprint, and keeps the cartridges row in one SQLite transaction;
the files are written first so a failed export cannot leave a half-cartridge
in the database.
- RetrievalService.search() now ranks inside the mounted scope by default, and
an explicit cartridge_ids filter overrides the mount. Nothing mounted means
the whole library, so an empty store behaves as it did before.
- forget() removes the forgotten atoms from every cartridge's membership:
forgetting must not leave a memory retrievable through a cartridge.
KV fingerprint (kv_fingerprint.py, stored in the manifest):
- model / tokenizer / chat-template hashes, the RoPE block (factor,
original_max_position_embeddings, mrope_interleaved, mrope_section,
partial_rotary_factor), the adapter hash, the context length, the engine
version, and the fingerprint format version.
- A snapshot we cannot read is recorded as unverifiable and listed in
unverifiable_fields; a blind machine refuses to mount rather than pretending
the runtime matches, because a wrong RoPE factor silently corrupts retrieval.
Two defects found by running the new tests, not by reading:
- atoms.parquet could not export a memory at all: pyarrow refuses a struct with
no child field, so metadata={} failed the whole table. The six JSON columns
are now written as JSON text and decoded by the cartridge reader.
- materialize() inserted cartridge membership before the cartridges row existed,
so the foreign key rejected every cartridge.
api.py crossed 500 lines, so the cartridge and mount routes moved to
api_cartridges.py. That module must not use `from __future__ import
annotations`: FastAPI then resolves Depends(settings_dep) against module
globals, where the injected dependency is only a parameter of register(), and
every route starts demanding settings as a query parameter.
Verification: 138 pytest passed (26 new: 10 fingerprint, 16 scope), Go
build/vet/gofmt/test clean, and scripts/mcp_write_smoke.sh now proves the whole
loop across two processes - mount narrows search, a memory written after the
upload is invisible until append_to_cartridge, unmount restores the library,
and rewriting the RoPE factor makes mount_cartridge refuse with "rope: factor:
cartridge 4.0 vs local 2.0". ruff clean on every touched file (the 20 remaining
package errors are in files this change did not touch; HEAD had 25). mypy: one
error, the pre-existing np.savez stub artifact at cartridge_assets.py:329
(HEAD:320). Repo pyright: 62 errors, unchanged from the p0-3 ledger.
This commit is contained in:
1 parent
674c88e84e
commit
9fe13fffb3
21 files changed
+1492
-153
No files matched your search
@@ -11,6 +11,19 @@ export ELY_EYE_API="http://127.0.0.1:${port}"
|
||||
python_bin="$root/.venv/bin/python"
|
||||
[ -x "$python_bin" ] || python_bin="python3"
|
||||
|
||||
# A model snapshot the KV fingerprint can read: without it every mount is refused as
|
||||
# unverifiable, which is the behaviour, but the smoke test also needs a mount to succeed.
|
||||
model_dir="$home/model"
|
||||
mkdir -p "$model_dir"
|
||||
cat > "$model_dir/config.json" <<'JSON'
|
||||
{"model_type": "qwen3_5", "max_position_embeddings": 262144, "partial_rotary_factor": 0.25,
|
||||
"rope_parameters": {"rope_type": "yarn", "factor": 4.0, "mrope_section": [11, 11, 10]},
|
||||
"chat_template": "{% for message in messages %}{{ message['content'] }}{% endfor %}"}
|
||||
JSON
|
||||
printf '{"version": "1.0"}' > "$model_dir/tokenizer.json"
|
||||
printf '{"metadata": {}, "weight_map": {"model-00001.safetensors": "0"}}' > "$model_dir/model.safetensors.index.json"
|
||||
export ELY_EYE_MODEL_DIR="$model_dir"
|
||||
|
||||
api=""
|
||||
cleanup() {
|
||||
if [ -n "$api" ]; then kill "$api" 2>/dev/null || true; fi
|
||||
@@ -34,12 +47,24 @@ fi
|
||||
|
||||
(cd "$root/ely-eye-mcp" && go build -o "$home/ely-eye-mcp" ./cmd/ely-eye-mcp)
|
||||
|
||||
"$python_bin" - "$home/ely-eye-mcp" <<'PY'
|
||||
printf 'the pantry keeps flour\n' > "$home/pantry.txt"
|
||||
uploaded="$(curl -sf -X POST "http://127.0.0.1:${port}/api/ingest/upload?cartridge_name=home" \
|
||||
-F "files=@${home}/pantry.txt;type=text/plain")"
|
||||
cartridge_id="$(printf '%s' "$uploaded" | "$python_bin" -c 'import json,sys; print(json.load(sys.stdin)["cartridge_id"])')"
|
||||
if [ -z "$cartridge_id" ] || [ "$cartridge_id" = "None" ]; then
|
||||
echo "FAIL: the upload did not materialize a cartridge: $uploaded"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
"$python_bin" - "$home/ely-eye-mcp" "$cartridge_id" "$model_dir" <<'PY'
|
||||
import json
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
binary = sys.argv[1]
|
||||
cartridge_id = sys.argv[2]
|
||||
model_dir = pathlib.Path(sys.argv[3])
|
||||
proc = subprocess.Popen(
|
||||
[binary, "server"],
|
||||
stdin=subprocess.PIPE,
|
||||
@@ -82,7 +107,8 @@ call(1, "initialize", {
|
||||
send({"jsonrpc": "2.0", "method": "notifications/initialized"})
|
||||
|
||||
names = [item["name"] for item in call(2, "tools/list", {})["tools"]]
|
||||
for required in ("remember", "supersede_memory", "forget_memory"):
|
||||
for required in ("remember", "supersede_memory", "forget_memory", "mounted_cartridges",
|
||||
"mount_cartridge", "switch_cartridge", "unmount_cartridge", "append_to_cartridge"):
|
||||
if required not in names:
|
||||
raise SystemExit(f"FAIL: {required} is not registered; tools are {names}")
|
||||
|
||||
@@ -121,5 +147,40 @@ gone = call(3, "tools/call", {"name": "fetch_atom", "arguments": {"atom_id": ato
|
||||
if not gone.get("isError") or "not found" not in gone["content"][0]["text"]:
|
||||
raise SystemExit(f"FAIL: the forgotten memory is still readable: {gone}")
|
||||
|
||||
print("PASS: remember, supersede, search and forget work through MCP")
|
||||
state = tool("mounted_cartridges", {})["state"]
|
||||
if state["mounted"] or state["scoped"]:
|
||||
raise SystemExit(f"FAIL: a fresh store reports a scope: {state}")
|
||||
|
||||
outside = tool("remember", {"text": "the spare key hides in the garage", "scenario": "work"})["atom"]
|
||||
manifest = tool("mount_cartridge", {"cartridge_id": cartridge_id})["manifest"]
|
||||
if manifest["cartridge_id"] != cartridge_id or not manifest.get("dna"):
|
||||
raise SystemExit(f"FAIL: mount returned the wrong header: {manifest}")
|
||||
state = tool("mounted_cartridges", {})["state"]
|
||||
if not state["scoped"] or state["mounted"] != [cartridge_id]:
|
||||
raise SystemExit(f"FAIL: the mount did not take effect: {state}")
|
||||
if tool("search_atoms", {"query": "garage"})["hits"]:
|
||||
raise SystemExit("FAIL: a mounted cartridge let a memory outside it answer")
|
||||
|
||||
grown = tool("append_to_cartridge", {"cartridge_id": cartridge_id, "atom_ids": [outside["atom_id"]]})["manifest"]
|
||||
if grown["dna"] == manifest["dna"]:
|
||||
raise SystemExit("FAIL: append rewrote nothing; the DNA is unchanged")
|
||||
if not any(hit["atom"]["atom_id"] == outside["atom_id"] for hit in tool("search_atoms", {"query": "garage"})["hits"]):
|
||||
raise SystemExit("FAIL: an appended memory is still not reachable")
|
||||
|
||||
tool("unmount_cartridge", {"cartridge_id": cartridge_id})
|
||||
if tool("mounted_cartridges", {})["state"]["scoped"]:
|
||||
raise SystemExit("FAIL: unmount left the store scoped")
|
||||
|
||||
(model_dir / "config.json").write_text(json.dumps({
|
||||
"model_type": "qwen3_5",
|
||||
"max_position_embeddings": 262144,
|
||||
"partial_rotary_factor": 0.25,
|
||||
"rope_parameters": {"rope_type": "yarn", "factor": 2.0},
|
||||
"chat_template": "{% for message in messages %}{{ message['content'] }}{% endfor %}",
|
||||
}), encoding="utf-8")
|
||||
refused = call(3, "tools/call", {"name": "mount_cartridge", "arguments": {"cartridge_id": cartridge_id}})
|
||||
if not refused.get("isError") or "rope" not in refused["content"][0]["text"]:
|
||||
raise SystemExit(f"FAIL: a cartridge built for RoPE factor 4.0 mounted on a factor 2.0 machine: {refused}")
|
||||
|
||||
print("PASS: remember, supersede, search, forget and cartridge mounting work through MCP")
|
||||
PY
|
||||
Reference in new issue
Block a user