diff --git a/CHECKLIST.md b/CHECKLIST.md index 553f45d..0883fe9 100644 --- a/CHECKLIST.md +++ b/CHECKLIST.md @@ -421,11 +421,22 @@ - **评审第 4 轮 FAIL**（同一独立 pi，7 项全跑，~557 工具轮；D40/D41 的修法实测成立，16 对逐字复现、五对 D40 全记上）：三条——D42 就是上面 P1 那句被实验推翻（Go 层重发缺失字段，旧服务照样过）；D43 `:412` 的机制句**方向写反**，我写「主语之前的裸数字并入主语键」，而代码的改名槽是 `index + 1`（主语 token 的**下一个**，`trust.py:275-276`），实测 `3 room holds 22 degrees` 主语是 `3`、不会向前融合，三个例子里只有 `2026 saw …` 数字在前、走的是 `trust.py:270-272` 首数字分支；D44 `_numbers_follow` 的**漏报侧没披露**：改不改名取决于每条记忆自己后面还留着没有第二个数字，`温度 22 度` + `温度 24 度 湿度 40`（`温度`/`温度24`）、`温度 22 度 湿度 40` + `温度 24 度`、`温度 22 度 湿度 40` + `温度 24 度 湿度 60`（`温度22`/`温度24`）、`预算 5000 元 超了 8000` + `预算 6000 元`（`预算5000`/`预算`）四对全 0 条，方向 fail-closed（两条都留在页上）。D44 按评审建议只**披露 + 加测试**（`test_a_second_number_in_one_memory_hides_the_update`），不用「未改名主语兜底比对」，那会把 `Room 3 holds 22 degrees` + `Room 4 holds 24 degrees` 退回 D37 的误报。四条精度说明全部采纳：P-a `degrees` 键也吞年份（`in 2026` → `{'degrees':'2026'}`），P5 那句已改成「第二个数字，含年份」；P-b 反向实验的红名单是 4 条不是 5 条（全套 4 failed / 379 passed，无连带）；P-c 「数字只要站在某个词之后就是量不是名字」已就地加限定词（且不在紧挨主语那个位置）；P-d 8799 守卫仍只在开跑时探一次，现在指纹补上了**旧构建**答题这一口——同树残留的服务占住端口时指纹仍给 200（评审实测：同树同 `ELY_EYE_AUTH_TOKEN` 的服务把 5 条 atom 落进它自己的 home，直到挂载卡带才失败），那种情形仍只有 busy-port 守卫能挡。该轮同时实测确认：门禁全绿且 mypy 两边逐字相同、tokenize 0 块、最大文件 `bench/systems.py` 498 行；D41 的新测试文案与实测一致；P4 的 stamp 判断站得住（`bench/leg_report.py` 两处 blob 全同 `40596d29…`）。判定：p2-3.1/3.2/3.3 与两条验收都能 `[x]`，p2-3.4 因 D43/D44 的记录级缺陷暂缓。 - **评审第 5 轮 PASS**（同一独立 pi，7 项全跑，本轮 diff 仅 3 个文件 +31/−5、机制 blob 全同：`trust.py d0b1a090`、`memory.py 13b0ebbf`、`retrieval.py 305d6e3a`、`compiler.py 5d341ca5`、`api_memory.py 22a03666`）：D42/D43/D44 全部按实测改掉并被独立复现——D42 的判别子它自己起两边服务量过（`4c6af87` 的 `/api/memory/working` 404、HEAD 200）；D43 三个主语实测 `3` / `room3` / `2026`；D44 四对逐对 0 条且贴出了页面（两条都留着），新测试的两条断言它逐条读过（第二条真的检查了「都在页上」）。**「不用未改名主语兜底比对」这个判断它用实验支持了**：把 `quantity_conflict` 改成主语不等时退回比值键，`Room 3 holds 22 degrees` + `Room 4 holds 24 degrees` 立刻变回 `('holds','22','24')`，即第 2 轮 D37 的误报复活。反向实验（去掉 `_numbers_follow` 条件）红名单仍是 4 条、全套 4 failed / 379 passed，**新加那条不变红**（无条件改名下它两条断言仍成立）。20 对冲突 + D40 五对 + `Room 3 is big` 边界逐字复现。门禁全部重跑且与本报告逐格一致（384 passed、六文件 83、conflicts 27、三文件 35、mypy 与 `9cb46c1` 文件集合与去行号文本逐字相同、tokenize 148 个 `.py` 0 块、最大文件 `bench/systems.py` 498、go `-count=1` 全套 + gofmt 空、`index-CSDHCFNo.js 221.48 kB`、smoke PASS）。两条精度说明（不算缺陷）已采纳：P-e 把 D44 成因从「一侧改名」改成「改名后主语键各自吃进自己的数字，两侧都改名也永不相等」（我另测 `温度 22 度 湿度 40` + `温度 24 度 湿度 40` → 0 条证实对称情形）；P-f 把「指纹补上 8799 那一口」限定为旧构建那一口。判定：p2-3.1/3.2/3.3/3.4 与两条验收全部 `[x]`。 - 顺带：`cli.py` 495→432 行（写路径拆进 `cli_memory.py`），`bench/leg_check.py` 拆出 `bench/leg_report.py`；typer 0.27.2 用**关键字**调用回调，`*atom_ids: str` 形式的命令一调就 `TypeError`，改成 `list[str]`——这才是之前 `test_maintenance` 那条 TypeError 的真因，命令重名只是表象。报告 stamp 早于 harness 的缺陷也修了（`a52c104`）。 -- [ ] **p2-4** 共享记忆 - - [ ] p2-4.1 多用户/多 agent 作用域 + ACL - - [ ] p2-4.2 默认可见性：answer-only 转介 - - [ ] p2-4.3 能力卡：让外部 agent 能路由进 ely-eye - - 验收：两 holder 互不可见；越权被拒；一次真实跨 holder 会话证明能力卡可用 +- [x] **p2-4** 共享记忆 + - [x] p2-4.1 多用户/多 agent 作用域 + ACL（schema v11：holder token + 三态眼睛） + - 事实：`holders` 加 `token_hash` / `token_issued_at`（`migrations.py:92` DDL，`_NEW_HOLDER_COLUMNS` 给老库 ALTER）。**只存哈希**（`sha256`，`holder_store._token_hash`），所以拷走的库不能重放 token；明文只在 `POST /api/holders/{id}/token` 的响应里出现一次（`IssuedToken`），`GET /api/holders/wife` 只回 `token_issued_at`。生成用 `secrets.token_urlsafe(32)`，比对用 `secrets.compare_digest`。`register_minimal` 是 `INSERT … ON CONFLICT DO NOTHING`，`remember` 在写记忆前调它（`memory.py:120` 之后），所以被转介的人一定有目录行，且它绝不覆盖人已声明的 name/contact。 + - 身份是**服务器算的**：`api_auth.resolve_caller` 三选一——operator（全局 token，可指名任何人）、holder（自己的 token，`verify_token`）、anonymous（没 token）。`read_as(caller, claimed)` 把「调用者声称的 holder」降级成「服务器认定的眼睛」：operator 不声称 → `None`（整库），operator 声称 → 那个人，holder 声称自己 → 自己，**声称别人 → 403 并把两个人都点名**，anonymous 声称任何人 → 403。`/api/search`、`/api/atoms`、`/api/atoms/{id}/image`、`/api/chat`、`/api/memory/working`、`/api/memory/conflicts`、`/api/ask` 全部走它；`remember`/`supersede` 的 holder 也只能是 caller 自己（p0-2 那条「客户端不能声称自己是别人」现在由 token 保证，不再靠 `default_holder` 兜底）。 + - **眼睛是三态、不是一个字符串的两可**（跑出来的决定）：`NULL` = 整库（operator，或自己拥有这个 store 的进程），holder id = 那个人的眼睛，`''` = 谁也不是（无 token 的 HTTP 调用者）。让 `NULL` 也代表「谁也不是」会逼 CLI、proofs、compiler、bench harness、`who_knows`、`ask` 全都传眼睛，那跟 p0-2 定的「拥有 store 的进程按本机受信主体读」直接冲突——第一次这样改时 `pytest -q` 红了 29 条（`test_vector_index.py:46` `assert [] == ['mem-eaf3b790ca27bf3e:text']`、`test_proofs.py` 回放 0 hits、`test_visual_index.py:43` `ValueError: invalid literal for int() with base 10: 'widget gadget sprocket'`，因为一条本来被跳过的腿开始跑了）。`''` 能当「谁也不是」是因为本 schema 里 `holder_id = ''` 本来就是「无主」。 + - 修前是真的漏：`browse.py`（`GET /api/atoms` 与 `/api/atoms/{id}/image`）**完全没有可见性规则**，无 token 就能翻页读到别人的记忆正文和图片；`store_search.py:81-86` 与 `vector_index.py:245-252` 的规则是 `(? IS NULL OR holder_id = '' OR scope='everyone' OR holder_id = ? OR …)`，`as_holder IS NULL` 短路成「整库」，所以无 token 搜索也看见全部。现在五处副本统一成 `(? IS NULL OR holder_id = '' OR json_extract(visibility_json,'$.scope') = 'everyone' OR (? <> '' AND (holder_id = ? OR EXISTS (… value = ?))))`，**四个参数同一个值**。 + - 规则为什么写五份而不抽常量：semgrep `python.sqlalchemy.security.sqlalchemy-execute-raw-query` 对 `connection.execute(f"…{常量}…")` 报警，哪怕那个常量是模块级的，所以只能写整句字面量。防漂移靠 `backend/tests/test_visibility_rule.py`：`EXPECTED_COPIES = 5`，正则抓出每处、`_normalize`（去空白、去引号、去 `a.` 别名）后必须**只剩一个字符串**，第二条测试钉住 clause 里 `? IS NULL` / `holder_id = ''` / `$.scope')='everyone'` / `? <> ''` / `holder_id = ?` / `value = ?` 六个特征还在。第一版正则 `\(\? IS NULL[^;]*?` 会把 `(? IS NULL OR modality = ?)` 这类过滤谓词一起吃掉，读出 4 份不同 clause——那是正则错，不是规则漂移。 + - [x] p2-4.2 默认可见性：answer-only 转介（p0-5 的转介现在被 ACL 兜住） + - p0-5 只做了一半：`ask` 命中别人的时候只回指针不回正文（`_pointer`），但**搜索本身**还是能读到正文，所以「只给指针」是客气不是边界。现在无 token 的 `search_atoms` 搜不到那句私钥（smoke 实测），`ask` 的 `answer` 仍为空、`secret` 在整个 referral JSON 里不出现，而 `who_knows` 照旧能用——目录路由**故意不设卡**，因为它只出计数、分数、contact 和声明过的话题，不出记忆正文（`api_holders.py` 注册处一行注释写明）。`visibility` 三档实测：默认 `holder`（只有主人）、`everyone`（无 token 可搜到）、`visible_to: [husband]`（主人 + 被指名的人能读，陌生人和第三个 holder 不能）。 + - [x] p2-4.3 能力卡：让外部 agent 能路由进 ely-eye + - `GET /api/card`（整库）与 `GET /api/card/{holder}`（一个人），CLI `ely-eye card`，MCP `ely_eye_card`（Go `apiclient.Card` / `HolderCard`，空 id 在**发请求之前**拒）。卡是**派生的不是声明的**（`card.build_card`）：`PRAGMA user_version`、`sources` 计数、`v_atoms_current` 按模态分组、每人 `(条数, MAX(ingested_at))`、每人模态/场景、合并目录、`has_audio = "audio" in senses`、挂载状态。所以它不能承诺一个 store 里没有的模态——`has_audio: false` 就是把视觉/音频腿的缺口在**别人提问之前**说出来。 + - 无 token 就能读（smoke 实测 HTTP 与 MCP 两条路都通），且卡里**不含任何记忆正文**（smoke 断言 `secret not in json.dumps(card)`）；`token_issued` 只说「这个人有 token」，不发 token。卡的 schema 版本与整库卡一致这条也进了 smoke 的 tokenless 分支。 + - 偏差（都是跑出来的）：`/api/memory/working` 与 `/api/memory/conflicts` 现在拒绝「谁也不是」的调用者（422，消息含「somebody」），因为这两件事本来就属于某个人；`compiler.py:41` 的缓存 key 加了眼睛——先前写 `as_holder or ""` 会把 operator 的上下文和陌生人的**撞成同一个 key**（两边都哈希 `""`），于是陌生人那份空 pack 会端给 operator；`forget` 的级联、卡带成员、目录都不因 ACL 改动。 + - 验收：`pytest backend/tests/test_holder_acl.py backend/tests/test_visibility_rule.py backend/tests/test_transactive.py` → **13 + 2 + 19** 全绿（两 holder 互不可见、无 token 读不到私钥、`everyone` 无 token 可搜、`visible_to` 只放行被指名的人、声称别人 403 且点名两人、token 只出现一次且库里只有 sha256、吊销后旧 token 401、卡无 token 可读且不含正文、陌生人的图片 404 而无主的 200、`compile(as_holder="")` 返回 0 hits 且 `cache_trace_id` 与 `as_holder="wife"` / `None` 都不同）。跨进程：`bash scripts/mcp_write_smoke.sh` → 两行 PASS，第二行是「a caller with no token sees only public memory, a holder's token sees her own, naming somebody else is refused, and the capability card answers without a token」——它先证明无 token 搜不到私钥，再用 operator token 铸出妻子的 token、证明她能搜到自己的、搜不到 `local` 写的 `the oven needs descaling`，最后**把 MCP 进程不带 `ELY_EYE_AUTH_TOKEN` 重启一遍**，证明 Go 那条路守同一条规则（不是假设）。 + - 顺手修的两条自己写的断言缺陷（都是跑出来的）：smoke 里 `if secret in found(None, token=token)` 是同义反复（搜的还是那句话，当然命中），改成拿**别人写的**记忆当探针；指纹探测 `GET /api/memory/working?holder=fingerprint` 现在返回 **403**（路由在、但匿名不能声称 holder），所以探测必须带 operator token——旧树上没有这条路由仍是 404，判别力保住。 + - 门禁：`pytest -q` **401 passed**；`ruff check` 对 22 个改动文件干净；`bench/filesize_check` 干净；`bench.audit --product` → `audited 32 harness files, 2 reports, 6 set files` / `clean`；mypy 冷缓存 **24 errors in 9 files (checked 112 source files)**，与 `9cb46c1` 基线（24/9）**净新增 0**；`./node_modules/.bin/pyright` 对 17 个后端源文件 + 5 个测试文件 → **0 errors**；`go build` / `go vet` / `go test`（apiclient ok、mcpserver ok）+ `gofmt -l` 空；`npm run build` → `dist/assets/index-CSDHCFNo.js 221.48 kB`（bundle 哈希与 `246dae9` 相同，与「本轮没动前端」一致）+ eslint exit 0；`bash scripts/mcp_write_smoke.sh` 3.3s 两行 PASS。 - [ ] **p2-5** 文档单一真相 - [ ] p2-5.1 `AGENTS.md` 重写并停止 gitignore - [ ] p2-5.2 `README.md` 每条宣称指向可重跑命令 diff --git a/backend/src/ely_eye/api.py b/backend/src/ely_eye/api.py index 0a5eefc..fd66b8a 100644 --- a/backend/src/ely_eye/api.py +++ b/backend/src/ely_eye/api.py @@ -5,7 +5,7 @@ import shutil from pathlib import Path from typing import Annotated -from fastapi import Depends, FastAPI, File, HTTPException, Query, UploadFile +from fastapi import Depends, FastAPI, File, HTTPException, Query, Request, UploadFile from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import FileResponse from fastapi.staticfiles import StaticFiles @@ -13,7 +13,8 @@ from pydantic import BaseModel from . import __version__ from .adapters import plan as adapter_plan -from .api_auth import require_write +from .api_auth import read_as, require_write, resolve_caller +from .api_card import register as register_card_routes from .api_cartridges import register as register_cartridge_routes from .api_holders import register as register_holder_routes from .api_memory import register as register_memory_routes @@ -43,6 +44,7 @@ from .schemas import ( PathIngestRequest, RememberRequest, RuntimeProfile, + SearchFilters, SearchRequest, SearchResult, SupersedeRequest, @@ -166,12 +168,15 @@ def create_app() -> FastAPI: @app.post("/api/chat", response_model=AnswerResponse) def chat( request: ChatRequest, + http: Request, settings: Annotated[Settings, Depends(settings_dep)], _: Annotated[None, Depends(require_write)], ) -> AnswerResponse: db = Database(settings) compiler = ContextCompiler(settings, db) - context = compiler.compile(request.question, request.profile) + context = compiler.compile( + request.question, request.profile, as_holder=read_as(resolve_caller(http, settings), None) + ) if not context.hits: raise HTTPException(status_code=404, detail="No evidence atoms are available for this question.") runtime = QwenRuntime(settings) @@ -192,12 +197,24 @@ def create_app() -> FastAPI: ) @app.post("/api/search", response_model=SearchResult) - def search(request: SearchRequest, settings: Annotated[Settings, Depends(settings_dep)]) -> SearchResult: + def search( + request: SearchRequest, + http: Request, + settings: Annotated[Settings, Depends(settings_dep)], + ) -> SearchResult: + """Search through the eyes the token proves. A body that names another holder is refused, not obeyed.""" try: + caller = resolve_caller(http, settings) + eyes = read_as(caller, request.filters.as_holder if request.filters else None) + filters = request.filters + if filters is not None: + filters = filters.model_copy(update={"as_holder": eyes}) + elif eyes is not None: + filters = SearchFilters(as_holder=eyes) return RetrievalService(settings).search( request.query, request.top_k, - filters=request.filters, + filters=filters, cursor=request.cursor, image=request.image, ) @@ -207,13 +224,15 @@ def create_app() -> FastAPI: @app.post("/api/memory", response_model=EvidenceAtom) def remember( request: RememberRequest, + http: Request, settings: Annotated[Settings, Depends(settings_dep)], _: Annotated[None, Depends(require_write)], ) -> EvidenceAtom: + """The token decides who owns the memory; the body cannot say so. The operator token writes as the default holder.""" try: return MemoryService(settings).remember( request.text, - holder_id=settings.default_holder, + holder_id=resolve_caller(http, settings).holder_id or settings.default_holder, origin=MemoryOrigin.self_, scenario=request.scenario, event_time=request.event_time, @@ -229,6 +248,7 @@ def create_app() -> FastAPI: @app.post("/api/memory/supersede", response_model=EvidenceAtom) def supersede( request: SupersedeRequest, + http: Request, settings: Annotated[Settings, Depends(settings_dep)], _: Annotated[None, Depends(require_write)], ) -> EvidenceAtom: @@ -236,7 +256,7 @@ def create_app() -> FastAPI: return MemoryService(settings).supersede( request.atom_id, text=request.text, - holder_id=settings.default_holder, + holder_id=resolve_caller(http, settings).holder_id or settings.default_holder, scenario=request.scenario, event_time=request.event_time, event_time_precision=request.event_time_precision, @@ -263,6 +283,7 @@ def create_app() -> FastAPI: register_cartridge_routes(app, settings_dep) register_holder_routes(app, settings_dep) register_memory_routes(app, settings_dep) + register_card_routes(app, settings_dep) @app.get("/api/adapters", response_model=list[AdapterStatus]) def adapters(settings: Annotated[Settings, Depends(settings_dep)]) -> list[AdapterStatus]: @@ -270,6 +291,7 @@ def create_app() -> FastAPI: @app.get("/api/atoms", response_model=AtomPage) def atoms( + http: Request, settings: Annotated[Settings, Depends(settings_dep)], limit: Annotated[int, Query(ge=1, le=200)] = 50, cursor: str | None = None, @@ -277,6 +299,7 @@ def create_app() -> FastAPI: source_id: str | None = None, holder: str | None = None, ) -> AtomPage: + """`holder` picks whose shelf to look at; the token decides what is on it.""" try: return browse( settings, @@ -285,13 +308,16 @@ def create_app() -> FastAPI: modality=modality, source_id=source_id, holder=holder, + as_holder=read_as(resolve_caller(http, settings), None), ) except ValueError as exc: raise HTTPException(status_code=400, detail=str(exc)) from exc @app.get("/api/atoms/{atom_id}/image") - def atom_image(atom_id: str, settings: Annotated[Settings, Depends(settings_dep)]) -> FileResponse: - found = picture(settings, atom_id) + def atom_image( + atom_id: str, http: Request, settings: Annotated[Settings, Depends(settings_dep)] + ) -> FileResponse: + found = picture(settings, atom_id, as_holder=read_as(resolve_caller(http, settings), None)) if found is None: raise HTTPException(status_code=404, detail=f"{atom_id} has no picture to show") return FileResponse(found.path, media_type=found.mime) diff --git a/backend/src/ely_eye/api_auth.py b/backend/src/ely_eye/api_auth.py index db26f7d..888af0c 100644 --- a/backend/src/ely_eye/api_auth.py +++ b/backend/src/ely_eye/api_auth.py @@ -1,22 +1,94 @@ -"""Writes need a token; queries do not. +"""Writes need a token; a read needs one to be anybody at all. Anything that can change stored state — a memory, an ingested source, a mounted cartridge, a proof run — is refused unless the caller presents the token the server was started with. A browser must also come from an allowed origin. The CLI and the MCP server reach the same writer through the same rule, so no face of the product gets a cheaper path to the store. + +A read with no token is still allowed, but it is nobody's read: it sees what nobody owns and +what is marked for everyone, and nothing else. Naming a holder is a claim about who is asking, +so it needs that holder's token or the operator token — otherwise `as_holder` would be a way to +look over anyone's shoulder by typing their name. """ from __future__ import annotations import secrets -from typing import TYPE_CHECKING +from dataclasses import dataclass +from typing import TYPE_CHECKING, Literal from fastapi import HTTPException, Request +from .db import Database +from .holder_store import HolderStore + if TYPE_CHECKING: from .config import Settings +@dataclass(frozen=True) +class Caller: + """Who the server decided is asking, from the token in front of it.""" + + kind: Literal["operator", "holder", "anonymous"] + holder_id: str | None = None + + @property + def may_name_others(self) -> bool: + return self.kind == "operator" + + +def presented_token(request: Request) -> str: + """The bearer value on this request, or an empty string when nobody presented one.""" + + header = request.headers.get("authorization", "") + return header.removeprefix("Bearer ").strip() if header.startswith("Bearer ") else "" + + +def resolve_caller(request: Request, settings: Settings) -> Caller: + """Turn the bearer token into an identity. An unknown token is refused, not treated as nobody.""" + + token = presented_token(request) + if not token: + return Caller(kind="anonymous") + if settings.auth_token and secrets.compare_digest(token, settings.auth_token): + return Caller(kind="operator") + holder_id = HolderStore(Database(settings)).verify_token(token) + if holder_id is None: + raise HTTPException( + status_code=401, detail="this token is neither the operator token nor a holder token" + ) + return Caller(kind="holder", holder_id=holder_id) + + +def read_as(caller: Caller, claimed: str | None) -> str | None: + """Whose eyes a read runs under. A claim the token cannot back is refused, not ignored. + + The answer is the three states the store's visibility clause understands: a holder id, the + empty string for a caller who is nobody, and None for an operator who reads the whole library. + """ + + if claimed is None: + return "" if caller.kind == "anonymous" else caller.holder_id + if caller.holder_id == claimed or caller.may_name_others: + return claimed + if caller.holder_id is None: + raise HTTPException( + status_code=403, + detail=( + f"an anonymous caller may not read as {claimed!r}; " + "send that holder's token or the operator token" + ), + ) + raise HTTPException( + status_code=403, + detail=( + f"holder {caller.holder_id!r} may not read as {claimed!r}; " + "only the operator token may name another holder" + ), + ) + + def require_write(request: Request) -> None: """Gate a mutating request. The app carries the settings source, so this stays a plain global.""" diff --git a/backend/src/ely_eye/api_card.py b/backend/src/ely_eye/api_card.py new file mode 100644 index 0000000..0cc02fe --- /dev/null +++ b/backend/src/ely_eye/api_card.py @@ -0,0 +1,37 @@ +"""The capability card over HTTP: served without a token, because it is the reason to send one. + +A caller deciding whether to point an agent at this store should not have to guess whether the +store holds speech, who is in the directory, or which schema answered last. Everything here is +read back from the store, so it cannot advertise a sense the store has no atoms for. + +No `from __future__ import annotations` here: FastAPI reads these route signatures, and string +annotations would be resolved against module globals, where the injected settings_dep exists +only as a parameter of register(). +""" + +from collections.abc import Callable +from typing import Annotated + +from fastapi import Depends, FastAPI, HTTPException + +from .card import build_card +from .card_schemas import CapabilityCard, HolderCard +from .config import Settings + + +def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None: + """Register the card routes. Nothing here is secret: it is the store describing itself.""" + + @app.get("/api/card", response_model=CapabilityCard) + def card(settings: Annotated[Settings, Depends(settings_dep)]) -> CapabilityCard: + return build_card(settings) + + @app.get("/api/card/{holder_id}", response_model=HolderCard) + def holder_card( + holder_id: str, settings: Annotated[Settings, Depends(settings_dep)] + ) -> HolderCard: + """One directory entry with its counted evidence; 404 names the holder that is not there.""" + for entry in build_card(settings).holders: + if entry.holder_id == holder_id: + return entry + raise HTTPException(status_code=404, detail=f"no holder {holder_id!r} has memories or a directory entry") diff --git a/backend/src/ely_eye/api_holders.py b/backend/src/ely_eye/api_holders.py index 0429917..b45b23c 100644 --- a/backend/src/ely_eye/api_holders.py +++ b/backend/src/ely_eye/api_holders.py @@ -8,9 +8,9 @@ only as a parameter of register(). from collections.abc import Callable from typing import Annotated -from fastapi import Depends, FastAPI, HTTPException +from fastapi import Depends, FastAPI, HTTPException, Request -from .api_auth import require_write +from .api_auth import read_as, require_write, resolve_caller from .config import Settings from .db import Database from .holder_store import HolderStore @@ -20,6 +20,7 @@ from .transactive_schemas import ( AskRequest, Holder, HolderUpsertRequest, + IssuedToken, ReferralAnswer, WhoKnowsReport, WhoKnowsRequest, @@ -27,7 +28,12 @@ from .transactive_schemas import ( def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None: - """Register the transactive routes. The server owns the timestamps, the caller does not.""" + """Register the transactive routes. The server owns the timestamps, the caller does not. + + The directory routes answer "who knows this?" and carry counts, scores and contact details but + no memory text, so they stay readable without a token; `ask` reads the memories themselves, so + it runs through the eyes the token proves. + """ app.state.write_settings = settings_dep @@ -84,6 +90,34 @@ def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None: raise HTTPException(status_code=409, detail=str(exc)) from exc return entry + @app.post("/api/holders/{holder_id}/token", response_model=IssuedToken) + def issue_token( + holder_id: str, + settings: Annotated[Settings, Depends(settings_dep)], + _: Annotated[None, Depends(require_write)], + ) -> IssuedToken: + """Mint a bearer token for one holder. Only the operator token may do this, and the value is shown once.""" + try: + entry, token = store(settings).issue_token(holder_id) + except ValueError as exc: + raise HTTPException(status_code=422, detail=str(exc)) from exc + issued_at = entry.token_issued_at + if issued_at is None: + raise HTTPException(status_code=500, detail=f"holder {holder_id!r} holds a token but no issue time") + return IssuedToken(holder_id=entry.holder_id, token=token, issued_at=issued_at) + + @app.delete("/api/holders/{holder_id}/token", response_model=Holder) + def revoke_token( + holder_id: str, + settings: Annotated[Settings, Depends(settings_dep)], + _: Annotated[None, Depends(require_write)], + ) -> Holder: + """Drop a holder's token; the holder stays in the directory, only the proof of identity goes.""" + try: + return store(settings).revoke_token(holder_id) + except ValueError as exc: + raise HTTPException(status_code=404, detail=str(exc)) from exc + @app.get("/api/holders/{holder_id}/knows", response_model=WhoKnowsReport) def holder_knows( holder_id: str, @@ -111,12 +145,14 @@ def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None: @app.post("/api/ask", response_model=ReferralAnswer) def ask( request: AskRequest, + http: Request, settings: Annotated[Settings, Depends(settings_dep)], ) -> ReferralAnswer: + """Ask, and be referred. The memories read here belong to somebody, so the token decides whose.""" try: return TransactiveService(settings).ask( request.question, - as_holder=request.as_holder, + as_holder=read_as(resolve_caller(http, settings), request.as_holder), max_hops=request.max_hops, include_text=request.include_text, ) diff --git a/backend/src/ely_eye/api_memory.py b/backend/src/ely_eye/api_memory.py index 22a0366..4ba9675 100644 --- a/backend/src/ely_eye/api_memory.py +++ b/backend/src/ely_eye/api_memory.py @@ -12,8 +12,9 @@ only as a parameter of register(). from collections.abc import Callable from typing import Annotated -from fastapi import Depends, FastAPI, HTTPException +from fastapi import Depends, FastAPI, HTTPException, Request +from .api_auth import read_as, resolve_caller from .config import Settings from .db import Database from .memory import MemoryService @@ -21,28 +22,46 @@ from .schemas import ConflictRecord, EvidenceAtom def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None: - """Register the tier reads. The holder is named by the caller because the question is about - somebody's memory, not about the server's own identity.""" + """Register the tier reads. A tier belongs to somebody, so the token decides whose it is: the + `holder` parameter narrows the question to a name, and naming a name the token cannot back is + refused rather than obeyed.""" def service(settings: Settings) -> MemoryService: return MemoryService(settings, Database(settings)) @app.get("/api/memory/working", response_model=list[EvidenceAtom]) def working_set( - holder: str, session: str, + http: Request, settings: Annotated[Settings, Depends(settings_dep)], + holder: str | None = None, ) -> list[EvidenceAtom]: """What one session recorded, in the order it recorded it, with no search involved.""" + who = read_as(resolve_caller(http, settings), holder) + if not who: + raise HTTPException( + status_code=422, + detail="a working set belongs to somebody; send that holder's token, or the operator token with a holder", + ) try: - return service(settings).working_set(holder, session) + return service(settings).working_set(who, session) except ValueError as exc: raise HTTPException(status_code=422, detail=str(exc)) from exc @app.get("/api/memory/conflicts", response_model=list[ConflictRecord]) def conflicts( + http: Request, settings: Annotated[Settings, Depends(settings_dep)], holder: str | None = None, ) -> list[ConflictRecord]: """Every disagreement the store noticed and nobody has settled.""" - return service(settings).conflicts(holder) + who = read_as(resolve_caller(http, settings), holder) + if who == "": + raise HTTPException( + status_code=422, + detail=( + "a disagreement belongs to somebody; send that holder's token, " + "or the operator token with a holder" + ), + ) + return service(settings).conflicts(who) diff --git a/backend/src/ely_eye/browse.py b/backend/src/ely_eye/browse.py index 394144b..7a2f80d 100644 --- a/backend/src/ely_eye/browse.py +++ b/backend/src/ely_eye/browse.py @@ -11,10 +11,12 @@ from .retrieval import decode_cursor, encode_cursor from .schemas import AtomPage, Modality from .storage import ObjectStore +# The visibility rule is spelled out in four literal statements — two here, one in store_search, one in vector_index — because the SQL gate rejects a concatenated statement and a view cannot take a parameter; backend/tests/test_visibility_rule.py fails if the copies disagree. + @dataclass(frozen=True) class Picture: - """The bytes behind an atom and how to label them when serving.""" + """The stored bytes behind an atom and how to label them when serving.""" path: Path mime: str @@ -28,22 +30,48 @@ def browse( modality: Modality | None = None, source_id: str | None = None, holder: str | None = None, + as_holder: str | None = None, ) -> AtomPage: - """One page plus the total, so a list can say what it left out instead of looking endless.""" + """One page plus the total, so a list can say what it left out instead of looking endless. + + `holder` narrows the shelf to one holder's memories; `as_holder` is whose eyes the page is + read through: a holder id, the empty string for a caller who is nobody (only what nobody owns + and what is marked for everyone), or None to read the whole library as the operator does. + """ offset = decode_cursor(cursor) wanted = (modality.value if modality else None, source_id, holder) filters = tuple(value for bound in wanted for value in (bound, bound)) + eyes = (as_holder, as_holder, as_holder, as_holder) database = Database(settings) with database.connect() as connection: total = int( connection.execute( - "SELECT COUNT(*) FROM v_atoms_current WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?)", - filters, + "SELECT COUNT(*) FROM v_atoms_current" + " WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?)" + " AND (? IS NULL" + " OR holder_id = ''" + " OR json_extract(visibility_json, '$.scope') = 'everyone'" + " OR (? <> '' AND (holder_id = ?" + " OR EXISTS (" + " SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders'))" + " WHERE value = ?" + " ))))", + (*filters, *eyes), ).fetchone()[0] ) rows = connection.execute( - "SELECT * FROM v_atoms_current WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?) ORDER BY ingested_at DESC, atom_id DESC LIMIT ? OFFSET ?", - (*filters, limit + 1, offset), + "SELECT * FROM v_atoms_current" + " WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?)" + " AND (? IS NULL" + " OR holder_id = ''" + " OR json_extract(visibility_json, '$.scope') = 'everyone'" + " OR (? <> '' AND (holder_id = ?" + " OR EXISTS (" + " SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders'))" + " WHERE value = ?" + " ))))" + " ORDER BY ingested_at DESC, atom_id DESC LIMIT ? OFFSET ?", + (*filters, *eyes, limit + 1, offset), ).fetchall() atoms = [row_to_atom(row) for row in rows[:limit]] more = len(rows) > limit or offset + limit < total @@ -54,13 +82,22 @@ def browse( ) -def picture(settings: Settings, atom_id: str) -> Picture | None: +def picture(settings: Settings, atom_id: str, as_holder: str | None = None) -> Picture | None: """The stored bytes behind one atom; None means there is no picture to show, for any of the three reasons.""" database = Database(settings) with database.connect() as connection: row = connection.execute( - "SELECT image_ref, mime FROM v_atoms_current JOIN sources USING (source_id) WHERE atom_id = ?", - (atom_id,), + "SELECT image_ref, mime FROM v_atoms_current JOIN sources USING (source_id)" + " WHERE atom_id = ?" + " AND (? IS NULL" + " OR holder_id = ''" + " OR json_extract(visibility_json, '$.scope') = 'everyone'" + " OR (? <> '' AND (holder_id = ?" + " OR EXISTS (" + " SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders'))" + " WHERE value = ?" + " ))))", + (atom_id, as_holder, as_holder, as_holder, as_holder), ).fetchone() if row is None or not row["image_ref"]: return None diff --git a/backend/src/ely_eye/cache_fabric.py b/backend/src/ely_eye/cache_fabric.py index a79c388..eb8357f 100644 --- a/backend/src/ely_eye/cache_fabric.py +++ b/backend/src/ely_eye/cache_fabric.py @@ -17,9 +17,9 @@ class CacheFabric: self.settings = settings or get_settings() self.db = database or Database(self.settings) - def semantic_key(self, question: str, profile: str, store_revision: int) -> str: - """A compiled context is only true for the corpus it was compiled from, so the corpus is part of its name.""" - return hashlib.sha256(f"compiler-v2\n{profile}\n{store_revision}\n{question}".encode()).hexdigest() + def semantic_key(self, question: str, profile: str, store_revision: int, holder: str = "") -> str: + """A compiled context is only true for the corpus it was compiled from and the eyes that were allowed to see it, so both belong in its name.""" + return hashlib.sha256(f"compiler-v2\n{profile}\n{store_revision}\n{holder}\n{question}".encode()).hexdigest() def record_hit(self, layer: str, cache_key: str, bytes_estimate: int = 0, metadata: dict[str, Any] | None = None) -> None: self.db.log_cache_event(layer, cache_key, "hit", bytes_estimate, metadata) diff --git a/backend/src/ely_eye/card.py b/backend/src/ely_eye/card.py new file mode 100644 index 0000000..efe4cca --- /dev/null +++ b/backend/src/ely_eye/card.py @@ -0,0 +1,92 @@ +"""What this store can answer, read back from what it actually holds. + +A caller deciding whether to send its questions here needs three facts it cannot guess: who is in +the directory and what they claim to know, whether the memory it would get covers pictures and +speech or only words, and which schema the answers were produced under. Every field is read from +the store, so the card cannot promise a sense the store has no atoms for, and it cannot promise a +capacity a mount has not carried. +""" + +from __future__ import annotations + +from datetime import datetime + +from .card_schemas import CapabilityCard, HolderCard +from .cartridge import CartridgeService +from .config import Settings +from .db import Database +from .holder_store import HolderStore + + +def build_card(settings: Settings) -> CapabilityCard: + """The store-level card: schema, counts, senses present, and what is mounted read-only.""" + + database = Database(settings) + with database.connect() as connection: + schema_version = int(connection.execute("PRAGMA user_version").fetchone()[0]) + sources = int(connection.execute("SELECT COUNT(*) FROM sources").fetchone()[0]) + senses = { + str(row["modality"]): int(row["n"]) + for row in connection.execute( + "SELECT modality, COUNT(*) AS n FROM v_atoms_current GROUP BY modality ORDER BY n DESC" + ) + } + per_holder = { + str(row["holder_id"]): (int(row["n"]), str(row["newest"])) + for row in connection.execute( + "SELECT holder_id, COUNT(*) AS n, MAX(ingested_at) AS newest FROM v_atoms_current" + " GROUP BY holder_id" + ) + } + senses_by_holder: dict[str, dict[str, int]] = {} + scenarios_by_holder: dict[str, list[str]] = {} + for row in connection.execute( + "SELECT holder_id, modality, COUNT(*) AS n FROM v_atoms_current" + " GROUP BY holder_id, modality ORDER BY n DESC" + ): + senses_by_holder.setdefault(str(row["holder_id"]), {})[str(row["modality"])] = int(row["n"]) + for row in connection.execute( + "SELECT holder_id, scenario, COUNT(*) AS n FROM v_atoms_current" + " GROUP BY holder_id, scenario HAVING n > 0 ORDER BY n DESC" + ): + scenarios_by_holder.setdefault(str(row["holder_id"]), []).append(str(row["scenario"])) + + directory = {entry.holder_id: entry for entry in HolderStore(database).list_holders()} + cards: dict[str, HolderCard] = {} + for holder_id, (memories, newest) in per_holder.items(): + entry = directory.get(holder_id) + cards[holder_id] = HolderCard( + holder_id=holder_id, + name=entry.name if entry else "", + kind=entry.kind if entry else "person", + knows_about=list(entry.knows_about) if entry else [], + contact=dict(entry.contact) if entry else {}, + memories=memories, + modalities=senses_by_holder.get(holder_id, {}), + scenarios=scenarios_by_holder.get(holder_id, []), + newest_memory=datetime.fromisoformat(newest), + registered=entry is not None, + token_issued=entry is not None and entry.token_issued_at is not None, + ) + for holder_id, entry in directory.items(): + if holder_id not in cards: + cards[holder_id] = HolderCard( + holder_id=holder_id, + name=entry.name, + kind=entry.kind, + knows_about=list(entry.knows_about), + contact=dict(entry.contact), + newest_memory=None, + registered=True, + token_issued=entry.token_issued_at is not None, + ) + + return CapabilityCard( + schema_version=schema_version, + atoms=database.atom_count(), + sources=sources, + modalities=senses, + has_audio="audio" in senses, + holders=[cards[key] for key in sorted(cards)], + mounts=CartridgeService(settings).mount_state(), + ) diff --git a/backend/src/ely_eye/card_schemas.py b/backend/src/ely_eye/card_schemas.py new file mode 100644 index 0000000..5757382 --- /dev/null +++ b/backend/src/ely_eye/card_schemas.py @@ -0,0 +1,58 @@ +"""The capability card: what this store can answer, stated as data. + +Held apart from the memory schemas because the card is a promise made to a stranger — it is served +without a token — while the memory schemas describe what a holder's own memories look like. The +two change for different reasons, so they live in different files. +""" + +from __future__ import annotations + +from datetime import datetime + +from pydantic import BaseModel, Field + +from .schemas import MountState, utc_now +from .transactive_schemas import HolderKind + + +class HolderCard(BaseModel): + """One directory entry, plus the evidence that it is a holder rather than a typed-in name. + + `memories` and `newest_memory` are counted from the store, so a holder who declared ten + topics and recorded nothing shows as such. `token_issued` says whether anything can prove it + is this holder; `registered` says whether the directory knows it at all, which is false for a + holder_id that only ever appeared on a memory. + """ + + holder_id: str + name: str = "" + kind: HolderKind = "person" + knows_about: list[str] = Field(default_factory=list) + contact: dict[str, str] = Field(default_factory=dict) + memories: int = Field(ge=0, default=0) + modalities: dict[str, int] = Field(default_factory=dict) + scenarios: list[str] = Field(default_factory=list) + newest_memory: datetime | None = None + registered: bool = False + token_issued: bool = False + + +class CapabilityCard(BaseModel): + """What a caller can expect from this store before it sends anything into it. + + `has_audio` exists because a store that ingested only pictures and PDFs answers questions + about speech with OCR of a frame, and the caller should learn that before it relies on it. + `schema_version` is read from the open database, not from the build constant, so a store that + has not been migrated yet reports what it actually is. The card is served without a token: it + names holders, counts their memories and lists the scenario names they used, but no memory + text — a caller learns what this store can answer, not what it remembers. + """ + + schema_version: int + atoms: int = Field(ge=0) + sources: int = Field(ge=0) + modalities: dict[str, int] = Field(default_factory=dict) + has_audio: bool + holders: list[HolderCard] = Field(default_factory=list) + mounts: MountState + generated_at: datetime = Field(default_factory=utc_now) diff --git a/backend/src/ely_eye/cli_transactive.py b/backend/src/ely_eye/cli_transactive.py index b685693..6ce5cb7 100644 --- a/backend/src/ely_eye/cli_transactive.py +++ b/backend/src/ely_eye/cli_transactive.py @@ -10,12 +10,13 @@ from collections.abc import Callable import typer +from .card import build_card from .config import Settings from .db import Database from .holder_store import HolderStore from .schemas import utc_now from .transactive import TransactiveService -from .transactive_schemas import Holder, HolderKind +from .transactive_schemas import Holder, HolderKind, IssuedToken def register(app: typer.Typer, settings: Callable[[], Settings]) -> None: @@ -62,6 +63,25 @@ def register(app: typer.Typer, settings: Callable[[], Settings]) -> None: store().delete(holder_id) typer.echo(f'{{"deleted": "{holder_id}"}}') + @app.command("holder-issue") + def holder_issue(holder_id: str) -> None: + """Mint a bearer token for one holder. The value appears once: only its hash is stored.""" + entry, token = store().issue_token(holder_id) + if entry.token_issued_at is None: + raise ValueError(f"holder {holder_id!r} holds a token but no issue time") + issued = IssuedToken(holder_id=entry.holder_id, token=token, issued_at=entry.token_issued_at) + typer.echo(issued.model_dump_json(indent=2)) + + @app.command("holder-revoke") + def holder_revoke(holder_id: str) -> None: + """Drop a holder's token. The entry stays; nobody can read as them until a new one is issued.""" + typer.echo(store().revoke_token(holder_id).model_dump_json(indent=2)) + + @app.command() + def card() -> None: + """What this store can answer, read back from what it holds: holders, senses, schema, mounts.""" + typer.echo(build_card(settings()).model_dump_json(indent=2)) + @app.command("who-knows") def who_knows(query: str, as_holder: str = "", limit: int = 8) -> None: """Rank the holders whose memories answer this query.""" diff --git a/backend/src/ely_eye/compiler.py b/backend/src/ely_eye/compiler.py index 5d341ca..315a2ba 100644 --- a/backend/src/ely_eye/compiler.py +++ b/backend/src/ely_eye/compiler.py @@ -8,7 +8,14 @@ from .cache_fabric import SEMANTIC_LAYER, CacheFabric from .config import Settings, get_settings from .db import Database from .retrieval import RetrievalService -from .schemas import CompiledContext, ContextPlan, Modality, RetrievalHit, RuntimeProfile +from .schemas import ( + CompiledContext, + ContextPlan, + Modality, + RetrievalHit, + RuntimeProfile, + SearchFilters, +) PROFILE_TOKEN_BUDGETS = { RuntimeProfile.live_demo: 128_000, @@ -25,8 +32,18 @@ class ContextCompiler: self.retrieval = RetrievalService(self.settings, self.db) self.cache = CacheFabric(self.settings, self.db) - def compile(self, question: str, profile: RuntimeProfile) -> CompiledContext: - cache_key = self.cache.semantic_key(question, profile.value, self.db.revision()) + def compile(self, question: str, profile: RuntimeProfile, as_holder: str | None = None) -> CompiledContext: + """Build the context one question gets, seen through one holder's eyes. + + `as_holder` is passed to retrieval and folded into the cache key: a context compiled for + a holder carries that holder's own memories, and serving it to somebody else would leak. + """ + cache_key = self.cache.semantic_key( + question, + profile.value, + self.db.revision(), + "the whole library" if as_holder is None else f"as holder {as_holder!r}", + ) cached = self.cache.read_semantic_cache(cache_key) if cached is not None: compiled = CompiledContext.model_validate(cached) @@ -40,7 +57,9 @@ class ContextCompiler: else: self.cache.record_miss(SEMANTIC_LAYER, cache_key) plan = self.plan(question, profile) - hits = self.retrieval.search(plan.retrieval_query, plan.evidence_budget_atoms).hits + hits = self.retrieval.search( + plan.retrieval_query, plan.evidence_budget_atoms, filters=SearchFilters(as_holder=as_holder) + ).hits citable, held = self._split_citable(hits) packed_text, token_total = self._pack_hits(citable, plan.token_budget, plan.retrieval_query) trace_id = hashlib.sha256(f"{cache_key}:{token_total}".encode()).hexdigest()[:16] diff --git a/backend/src/ely_eye/holder_store.py b/backend/src/ely_eye/holder_store.py index ad24d0b..db4bd49 100644 --- a/backend/src/ely_eye/holder_store.py +++ b/backend/src/ely_eye/holder_store.py @@ -8,6 +8,8 @@ never a guess. from __future__ import annotations +import hashlib +import secrets import sqlite3 from dataclasses import dataclass from datetime import datetime @@ -30,6 +32,9 @@ class WhoKnowsCount: latest_time: str | None +_TOKEN_BYTES = 32 + + class HolderStore: """CRUD for registered holders plus the raw counts who_knows is built from.""" @@ -68,6 +73,63 @@ class HolderStore: ) return stored + def register_minimal(self, holder_id: str) -> None: + """Make a holder exist without overwriting what somebody already declared about them. + + A memory nobody registered still needs a directory entry, or a referral to its holder has + no target to point at. + """ + now = utc_now().isoformat() + with self.database.connect() as connection: + connection.execute( + """ + INSERT INTO holders (holder_id, name, kind, knows_about_json, contact_json, created_at, updated_at) + VALUES (?, '', 'person', '[]', '{}', ?, ?) + ON CONFLICT(holder_id) DO NOTHING + """, + (holder_id, now, now), + ) + + def issue_token(self, holder_id: str) -> tuple[Holder, str]: + """Give a holder a fresh bearer token. The plain value is returned once; the row keeps its hash.""" + self.register_minimal(holder_id) + token = secrets.token_urlsafe(_TOKEN_BYTES) + now = utc_now().isoformat() + with self.database.connect() as connection: + connection.execute( + "UPDATE holders SET token_hash = ?, token_issued_at = ?, updated_at = ? WHERE holder_id = ?", + (_token_hash(token), now, now, holder_id), + ) + entry = self.get(holder_id) + if entry is None: + raise ValueError(f"holder {holder_id!r} vanished while its token was being issued") + return entry, token + + def revoke_token(self, holder_id: str) -> Holder: + """Drop a holder's token: nobody can read as them until a new one is issued.""" + now = utc_now().isoformat() + with self.database.connect() as connection: + changed = connection.execute( + "UPDATE holders SET token_hash = NULL, token_issued_at = NULL, updated_at = ? WHERE holder_id = ?", + (now, holder_id), + ) + if changed.rowcount == 0: + raise ValueError(f"no holder {holder_id!r} is registered") + entry = self.get(holder_id) + if entry is None: + raise ValueError(f"holder {holder_id!r} vanished while its token was being revoked") + return entry + + def verify_token(self, token: str) -> str | None: + """Which holder a bearer token speaks for, or None when it speaks for nobody.""" + if not token: + return None + with self.database.connect() as connection: + row = connection.execute( + "SELECT holder_id FROM holders WHERE token_hash = ?", (_token_hash(token),) + ).fetchone() + return None if row is None else str(row["holder_id"]) + def get(self, holder_id: str) -> Holder | None: with self.database.connect() as connection: row = connection.execute("SELECT * FROM holders WHERE holder_id = ?", (holder_id,)).fetchone() @@ -145,6 +207,10 @@ class HolderStore: ] +def _token_hash(token: str) -> str: + return hashlib.sha256(token.encode("utf-8")).hexdigest() + + def _moment(value: str) -> datetime: parsed = datetime.fromisoformat(value) if parsed.tzinfo is None: @@ -168,6 +234,7 @@ def _row_to_holder(row: sqlite3.Row) -> Holder: kind=_kind(str(row["kind"])), knows_about=from_json(row["knows_about_json"], []), contact=from_json(row["contact_json"], {}), + token_issued_at=None if row["token_issued_at"] is None else _moment(str(row["token_issued_at"])), created_at=_moment(str(row["created_at"])), updated_at=_moment(str(row["updated_at"])), ) diff --git a/backend/src/ely_eye/memory.py b/backend/src/ely_eye/memory.py index 13b0ebb..bf2aca7 100644 --- a/backend/src/ely_eye/memory.py +++ b/backend/src/ely_eye/memory.py @@ -17,6 +17,7 @@ from . import lifecycle_store, trust from .cache_fabric import CacheFabric from .config import Settings, get_settings from .db import Database +from .holder_store import HolderStore from .schemas import ( ArchiveReport, ConflictRecord, @@ -75,7 +76,8 @@ class MemoryService: """Store one statement as a memory owned by a holder. Repeating the same sentence as the same holder updates the memory instead of - duplicating it; two holders holding the same sentence keep two memories. + duplicating it; two holders holding the same sentence keep two memories. A holder nobody + registered is added to the directory here, so a referral to it has something to point at. """ statement = text.strip() if not statement: @@ -89,6 +91,7 @@ class MemoryService: ) moment = utc_now() + HolderStore(self.db).register_minimal(holder_id) digest = hashlib.sha256(f"{holder_id}\n{statement}".encode()).hexdigest() source_id = f"mem-{digest[:16]}" self.db.upsert_sources( diff --git a/backend/src/ely_eye/migrations.py b/backend/src/ely_eye/migrations.py index 451ea0d..c61675a 100644 --- a/backend/src/ely_eye/migrations.py +++ b/backend/src/ely_eye/migrations.py @@ -16,7 +16,7 @@ import sqlite3 from .storage import token_equivalent, visual_token_equivalent from .text_index import index_text -SCHEMA_VERSION = 10 +SCHEMA_VERSION = 11 ATOM_COLUMNS_V2 = """ atom_id TEXT PRIMARY KEY, @@ -96,6 +96,8 @@ _BASE_TABLES = """ CHECK (kind IN ('person', 'agent', 'service')), knows_about_json TEXT NOT NULL DEFAULT '[]', contact_json TEXT NOT NULL DEFAULT '{}', + token_hash TEXT, + token_issued_at TEXT, created_at TEXT NOT NULL, updated_at TEXT NOT NULL ); @@ -290,6 +292,13 @@ def _recount_visual_tokens(connection: sqlite3.Connection) -> None: connection.executemany("UPDATE atoms SET token_equivalent = ? WHERE atom_id = ?", updates) +_NEW_HOLDER_COLUMNS: tuple[tuple[str, str], ...] = ( + ("token_hash", "TEXT"), + ("token_issued_at", "TEXT"), +) +"""A holder proves its identity with a bearer token; only its hash is kept, so a copied database cannot replay it.""" + + def apply(connection: sqlite3.Connection) -> None: """Bring the schema to SCHEMA_VERSION, in place, without losing rows.""" version = int(connection.execute("PRAGMA user_version").fetchone()[0]) @@ -314,6 +323,11 @@ def apply(connection: sqlite3.Connection) -> None: for name, definition in columns: if name not in existing: connection.execute(f"ALTER TABLE atoms ADD COLUMN {name} {definition}") + if version < 11: + existing_holders = _column_names(connection, "holders") + for name, definition in _NEW_HOLDER_COLUMNS: + if name not in existing_holders: + connection.execute(f"ALTER TABLE holders ADD COLUMN {name} {definition}") if version < 10: # CREATE VIEW IF NOT EXISTS keeps the old body, and v10 changed which rows are current, so an upgraded store must be told to look again. connection.execute("DROP VIEW IF EXISTS v_atoms_current") diff --git a/backend/src/ely_eye/store_search.py b/backend/src/ely_eye/store_search.py index 6a194ec..75ca0fc 100644 --- a/backend/src/ely_eye/store_search.py +++ b/backend/src/ely_eye/store_search.py @@ -46,10 +46,11 @@ def search_atoms( depends on string-built SQL, and the planner sees one shape. A memory with no event time cannot answer a time filter, so it drops out when one is asked for. - `as_holder` applies the sharing rule: a memory is readable by its owner, by anyone named - in its visible_to list, by everyone when it is public, and by anyone when nobody owns it. - A memory with no holder cannot be private to a holder that does not exist. Without an - asker the query runs over the whole library, which is what an operator debugging wants.""" + `as_holder` applies the sharing rule in three states: a holder id sees its own memories, the + ones naming it in visible_to, the ones marked for everyone, and the ones nobody owns; the + empty string is a caller who is nobody, so only the last two; None runs over the whole + library, which is what an operator or a process that owns the store reads with. A memory with + no holder cannot be private to a holder that does not exist.""" with database.connect() as connection: expression = fts_expression(rare_terms(connection, fts_terms(query))) rows = connection.execute( @@ -77,13 +78,14 @@ def search_atoms( WHERE ca.atom_id = f.atom_id AND ca.cartridge_id IN (SELECT value FROM json_each(?)) )) - AND (? IS NULL OR a.holder_id = '' + AND (? IS NULL + OR a.holder_id = '' OR json_extract(a.visibility_json, '$.scope') = 'everyone' - OR a.holder_id = ? - OR EXISTS ( - SELECT 1 FROM json_each(json_extract(a.visibility_json, '$.holders')) - WHERE value = ? - )) + OR (? <> '' AND (a.holder_id = ? + OR EXISTS ( + SELECT 1 FROM json_each(json_extract(a.visibility_json, '$.holders')) + WHERE value = ? + )))) ORDER BY score LIMIT ? OFFSET ? """, @@ -108,6 +110,7 @@ def search_atoms( as_holder, as_holder, as_holder, + as_holder, limit + 1, offset, ), diff --git a/backend/src/ely_eye/transactive_schemas.py b/backend/src/ely_eye/transactive_schemas.py index 3e361cf..06a0a14 100644 --- a/backend/src/ely_eye/transactive_schemas.py +++ b/backend/src/ely_eye/transactive_schemas.py @@ -24,6 +24,8 @@ class Holder(BaseModel): kind: HolderKind = "person" knows_about: list[str] = Field(default_factory=list) contact: dict[str, str] = Field(default_factory=dict) + token_issued_at: datetime | None = None + """When a bearer token was last issued here. The hash itself stays out of the answer.""" created_at: datetime updated_at: datetime @@ -35,6 +37,18 @@ class Holder(BaseModel): return value.strip() +class IssuedToken(BaseModel): + """A holder's bearer token, shown once at the moment it is minted. + + Only the hash is stored, so this answer is the only place the value will ever appear: a lost + token is replaced, not recovered. + """ + + holder_id: str + token: str + issued_at: datetime + + class HolderUpsertRequest(BaseModel): """What a caller may set about a holder. The id comes from the path, never the body.""" diff --git a/backend/src/ely_eye/vector_index.py b/backend/src/ely_eye/vector_index.py index c638d34..bf9bd54 100644 --- a/backend/src/ely_eye/vector_index.py +++ b/backend/src/ely_eye/vector_index.py @@ -225,7 +225,9 @@ class VectorIndex: question with a full scan (77ms over 153,857 atoms) while the atoms table answers it through the atom_id index in 0.2ms, and a search cannot afford the scan. The scope is checked here rather than in the engine so the rule lives in one place; the engine's id - list is only a narrowing. + list is only a narrowing. Visibility is here too, and it reads the same three states the + words leg reads: a holder id, the empty string for a caller who is nobody (unowned rows + and rows marked for everyone, no more), and None for an operator reading the whole library. """ rows = connection.execute( """ @@ -246,13 +248,14 @@ class VectorIndex: WHERE ca.atom_id = atoms.atom_id AND ca.cartridge_id IN (SELECT value FROM json_each(?)) )) - AND (? IS NULL OR holder_id = '' + AND (? IS NULL + OR holder_id = '' OR json_extract(visibility_json, '$.scope') = 'everyone' - OR holder_id = ? - OR EXISTS ( - SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders')) - WHERE value = ? - )) + OR (? <> '' AND (holder_id = ? + OR EXISTS ( + SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders')) + WHERE value = ? + )))) """, ( to_json(atom_ids), @@ -273,6 +276,7 @@ class VectorIndex: as_holder, as_holder, as_holder, + as_holder, ), ).fetchall() return {str(row["atom_id"]) for row in rows} diff --git a/backend/tests/test_cartridge_scope.py b/backend/tests/test_cartridge_scope.py index 3b446a8..77a071a 100644 --- a/backend/tests/test_cartridge_scope.py +++ b/backend/tests/test_cartridge_scope.py @@ -51,7 +51,8 @@ class Store: return self.cartridges.materialize(name, atoms, sources).cartridge_id def texts(self, query: str = "钥匙", **filters: Any) -> list[str]: - result = self.search.search(query, filters=SearchFilters(**filters) if filters else None) + eyes: dict[str, Any] = {"as_holder": "wife", **filters} + result = self.search.search(query, filters=SearchFilters(**eyes)) return [hit.atom.text for hit in result.hits] diff --git a/backend/tests/test_holder_acl.py b/backend/tests/test_holder_acl.py new file mode 100644 index 0000000..45c9501 --- /dev/null +++ b/backend/tests/test_holder_acl.py @@ -0,0 +1,268 @@ +"""The sharing rule over HTTP: a token says who is asking, and the store answers to that person.""" + +from __future__ import annotations + +import hashlib +from pathlib import Path +from typing import Any + +import pytest +from fastapi.testclient import TestClient + +import ely_eye.api as api_module +from ely_eye.compiler import ContextCompiler +from ely_eye.config import Settings +from ely_eye.db import Database +from ely_eye.holder_store import HolderStore +from ely_eye.memory import MemoryService +from ely_eye.schemas import ( + EvidenceAtom, + MemoryOrigin, + Modality, + RuntimeProfile, + SourceRecord, + TrustScores, + Visibility, +) +from ely_eye.storage import ObjectStore + +OPERATOR = "the-operator-token" +AS_OPERATOR = {"Authorization": f"Bearer {OPERATOR}"} +PNG = b"\x89PNG\r\n\x1a\n" +KEYS = "我们的钥匙放在玄关抽屉里" +THERMOSTAT = "恒温器设在二十二度" +OUTAGE = "上周三的停电持续了四十分钟" + + +def _settings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Settings: + resolved = Settings(home=tmp_path, default_holder="wife", auth_token=OPERATOR) + monkeypatch.setattr(api_module, "get_settings", lambda: resolved) + Database(resolved).init() + return resolved + + +@pytest.fixture +def client(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> TestClient: + return TestClient(api_module.create_app()) + + +@pytest.fixture +def settings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Settings: + return _settings(tmp_path, monkeypatch) + + +def _token_for(client: TestClient, holder_id: str) -> dict[str, Any]: + answer = client.post(f"/api/holders/{holder_id}/token", headers=AS_OPERATOR) + assert answer.status_code == 200, answer.json() + return dict(answer.json()) + + +def _remember(settings: Settings, text: str, holder_id: str, **kwargs: Any) -> EvidenceAtom: + return MemoryService(settings, Database(settings)).remember( + text, holder_id=holder_id, origin=MemoryOrigin.self_, **kwargs + ) + + +def _hits(client: TestClient, query: str, headers: dict[str, str] | None = None, **body: Any) -> list[str]: + answer = client.post("/api/search", json={"query": query, **body}, headers=headers or {}) + assert answer.status_code == 200, answer.json() + return [str(hit["atom"]["atom_id"]) for hit in answer.json()["hits"]] + + +def test_only_the_operator_mints_a_token(client: TestClient, settings: Settings) -> None: + _remember(settings, KEYS, "wife") + wife = _token_for(client, "wife") + assert client.post("/api/holders/husband/token").status_code == 401, "no token cannot mint one" + refused = client.post("/api/holders/husband/token", headers={"Authorization": f"Bearer {wife['token']}"}) + assert refused.status_code == 401, "a holder token is not an operator token" + assert "Bearer" in refused.json()["detail"] + + +def test_the_token_is_shown_once_and_stored_as_a_hash(client: TestClient, settings: Settings) -> None: + _remember(settings, KEYS, "wife") + issued = _token_for(client, "wife") + token = str(issued["token"]) + assert issued["holder_id"] == "wife" and issued["issued_at"] + + with Database(settings).connect() as connection: + row = connection.execute("SELECT token_hash FROM holders WHERE holder_id = 'wife'").fetchone() + stored = str(row["token_hash"]) + assert stored == hashlib.sha256(token.encode("utf-8")).hexdigest() + assert token not in stored and len(stored) == 64, "a copied database must not replay the token" + + listed = client.get("/api/holders/wife", headers=AS_OPERATOR).json() + assert "token" not in listed and listed["token_issued_at"] == issued["issued_at"] + + +def test_a_holder_token_reads_as_that_holder_and_nobody_else(client: TestClient, settings: Settings) -> None: + mine = _remember(settings, KEYS, "wife") + wife = _token_for(client, "wife") + _token_for(client, "husband") + + as_wife = _hits(client, KEYS, headers={"Authorization": f"Bearer {wife['token']}"}) + as_husband = _hits(client, KEYS, headers={"Authorization": f"Bearer {wife['token']}"}) + assert mine.atom_id in as_wife + assert as_husband == as_wife, "the same token reads the same way whatever the query says" + + anonymous = _hits(client, KEYS) + assert mine.atom_id not in anonymous, "a caller with no token is nobody" + + +def test_an_unknown_token_is_refused_rather_than_treated_as_nobody(client: TestClient, settings: Settings) -> None: + _remember(settings, KEYS, "wife") + refused = client.post("/api/search", json={"query": KEYS}, headers={"Authorization": "Bearer not-a-token"}) + assert refused.status_code == 401 + detail = str(refused.json()["detail"]) + assert "operator" in detail and "holder" in detail, "the refusal has to say which two things failed" + + +def test_a_public_memory_is_findable_by_anybody(client: TestClient, settings: Settings) -> None: + shared = _remember(settings, OUTAGE, "wife", visibility=Visibility(scope="everyone")) + assert shared.atom_id in _hits(client, OUTAGE), "no token, still visible" + + +def test_a_shared_memory_is_readable_only_by_the_holders_it_names(client: TestClient, settings: Settings) -> None: + private = _remember(settings, THERMOSTAT, "wife", visibility=Visibility(scope="visible_to", holders=["husband"])) + _remember(settings, KEYS, "husband") + wife = _token_for(client, "wife") + husband = _token_for(client, "husband") + + assert private.atom_id in _hits(client, THERMOSTAT, headers={"Authorization": f"Bearer {husband['token']}"}) + assert private.atom_id in _hits(client, THERMOSTAT, headers={"Authorization": f"Bearer {wife['token']}"}) + assert private.atom_id not in _hits(client, THERMOSTAT) + + outsider = _remember(settings, OUTAGE, "neighbour") + _token_for(client, "neighbour") + neighbour = _token_for(client, "neighbour") + assert outsider.atom_id + assert private.atom_id not in _hits(client, THERMOSTAT, headers={"Authorization": f"Bearer {neighbour['token']}"}) + + +def test_naming_another_holder_is_a_claim_the_token_has_to_back(client: TestClient, settings: Settings) -> None: + mine = _remember(settings, KEYS, "wife") + wife = _token_for(client, "wife") + _token_for(client, "husband") + as_wife = {"Authorization": f"Bearer {wife['token']}"} + + over = client.post("/api/search", json={"query": KEYS, "filters": {"as_holder": "husband"}}, headers=as_wife) + assert over.status_code == 403 + detail = str(over.json()["detail"]) + assert "wife" in detail and "husband" in detail, "a refusal that names neither side cannot be acted on" + + anonymous = client.post("/api/search", json={"query": KEYS, "filters": {"as_holder": "wife"}}) + assert anonymous.status_code == 403, "typing a name is not a token" + + operator = client.post("/api/search", json={"query": KEYS, "filters": {"as_holder": "wife"}}, headers=AS_OPERATOR) + assert operator.status_code == 200 + assert mine.atom_id in [str(hit["atom"]["atom_id"]) for hit in operator.json()["hits"]] + + +def test_revoking_a_token_stops_it_working(client: TestClient, settings: Settings) -> None: + _remember(settings, KEYS, "wife") + token = _token_for(client, "wife")["token"] + assert HolderStore(Database(settings)).verify_token(str(token)) == "wife" + + revoked = client.delete("/api/holders/wife/token", headers=AS_OPERATOR) + assert revoked.status_code == 200 and revoked.json()["holder_id"] == "wife" + assert client.post("/api/search", json={"query": KEYS}, headers={"Authorization": f"Bearer {token}"}).status_code == 401 + assert client.get("/api/holders/wife", headers=AS_OPERATOR).json()["token_issued_at"] is None + + +def test_a_working_set_refuses_to_belong_to_nobody(client: TestClient, settings: Settings) -> None: + _remember(settings, KEYS, "wife", session_id="kitchen-2026") + _remember(settings, THERMOSTAT, "husband", session_id="kitchen-2026") + + anonymous = client.get("/api/memory/working", params={"session": "kitchen-2026"}) + assert anonymous.status_code == 422 + assert "somebody" in str(anonymous.json()["detail"]) + + wife = _token_for(client, "wife") + hers = client.get("/api/memory/working", params={"session": "kitchen-2026"}, headers={"Authorization": f"Bearer {wife['token']}"}) + assert [atom["text"] for atom in hers.json()] == [KEYS], "a session sees its own lane, not the room's" + + both = client.get("/api/memory/working", params={"session": "kitchen-2026", "holder": "husband"}, headers=AS_OPERATOR) + assert [atom["text"] for atom in both.json()] == [THERMOSTAT] + + +def test_a_conflict_list_refuses_to_belong_to_nobody(client: TestClient, settings: Settings) -> None: + _remember(settings, "恒温器放在书房，设定 22 度", "wife") + _remember(settings, "恒温器调到 24 度", "wife") + assert client.get("/api/memory/conflicts").status_code == 422 + wife = _token_for(client, "wife") + hers = client.get("/api/memory/conflicts", headers={"Authorization": f"Bearer {wife['token']}"}) + assert hers.status_code == 200 and len(hers.json()) == 1 + + +def test_the_card_is_public_names_holders_and_carries_no_memory_text(client: TestClient, settings: Settings) -> None: + _remember(settings, KEYS, "wife", scenario="home") + _remember(settings, THERMOSTAT, "husband") + _token_for(client, "wife") + + answer = client.get("/api/card") + assert answer.status_code == 200, "a capability card exists to be read by a stranger" + card = answer.json() + assert card["schema_version"] >= 11 and card["has_audio"] is False + named = {holder["holder_id"]: holder for holder in card["holders"]} + assert set(named) == {"wife", "husband"} + assert named["wife"]["memories"] == 1 and named["wife"]["token_issued"] is True + assert named["husband"]["token_issued"] is False + assert named["wife"]["scenarios"] == ["home"] and named["wife"]["modalities"] == {"text": 1} + assert named["wife"]["newest_memory"], "a stranger needs to know how stale the memory is" + assert KEYS not in answer.text and THERMOSTAT not in answer.text + + one = client.get("/api/card/wife") + assert one.status_code == 200 and one.json()["holder_id"] == "wife" + assert client.get("/api/card/nobody").status_code == 404 + + +def test_a_picture_is_served_only_to_eyes_that_may_see_it(client: TestClient, settings: Settings) -> None: + database = Database(settings) + stored = ObjectStore(settings).put_bytes(PNG, ".png", "image") + database.upsert_sources( + [ + SourceRecord( + source_id=source, + path=f"file:///{source}.png", + kind="image", + mime="image/png", + sha256=source * 8, + size_bytes=len(PNG), + ) + for source in ("hers", "ours") + ] + ) + database.upsert_atoms( + [ + EvidenceAtom( + atom_id=f"{source}:frame", + modality=Modality.image, + source_id=source, + source=f"file:///{source}.png", + image_ref=stored, + trust=TrustScores(parser=1.0, ocr=0.0, model=0.0), + holder_id="wife" if source == "hers" else "", + ) + for source in ("hers", "ours") + ] + ) + + assert client.get("/api/atoms/ours:frame/image").status_code == 200, "nobody owns it, so anybody may look" + assert client.get("/api/atoms/hers:frame/image").status_code == 404, "the bytes are behind the same rule as the words" + assert client.get("/api/atoms", params={"modality": "image"}).json()["total"] == 1 + wife = _token_for(client, "wife") + page = client.get("/api/atoms", params={"modality": "image"}, headers={"Authorization": f"Bearer {wife['token']}"}) + assert page.json()["total"] == 2 + + +def test_an_operator_and_a_stranger_do_not_share_one_compiled_context(settings: Settings) -> None: + _remember(settings, KEYS, "wife") + compiler = ContextCompiler(settings, Database(settings)) + # The stranger compiles first: if the cache key ignored whose eyes were used, the operator would be served this. + stranger = compiler.compile(KEYS, RuntimeProfile.live_demo, as_holder="") + assert stranger.hits == [], "a memory nobody owns and nobody marked public is invisible to a stranger" + + owner = compiler.compile(KEYS, RuntimeProfile.live_demo, as_holder="wife") + assert KEYS in owner.packed_text and owner.cache_trace_id != stranger.cache_trace_id + + whole = compiler.compile(KEYS, RuntimeProfile.live_demo) + assert KEYS in whole.packed_text, "a process that owns the store still reads the whole library" diff --git a/backend/tests/test_product_api.py b/backend/tests/test_product_api.py index 5e07a83..60fae9e 100644 --- a/backend/tests/test_product_api.py +++ b/backend/tests/test_product_api.py @@ -18,6 +18,8 @@ from ely_eye.storage import ObjectStore PNG = b"\x89PNG\r\n\x1a\n" BASE = datetime(2026, 5, 1, 12, 0, tzinfo=UTC) +OPERATOR = "the-shelf-owner" +AS_OWNER = {"Authorization": f"Bearer {OPERATOR}"} HTTP_SURFACE = { "/api/adapters", @@ -32,11 +34,14 @@ HTTP_SURFACE = { "/api/cartridges/{cartridge_id}/refresh", "/api/cartridges/{cartridge_id}/switch", "/api/cartridges/{cartridge_id}/unmount", + "/api/card", + "/api/card/{holder_id}", "/api/chat", "/api/hashhop", "/api/holders", "/api/holders/{holder_id}", "/api/holders/{holder_id}/knows", + "/api/holders/{holder_id}/token", "/api/ingest/path", "/api/ingest/upload", "/api/memory", @@ -57,7 +62,7 @@ HTTP_SURFACE = { @pytest.fixture def settings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Settings: - resolved = Settings(home=tmp_path, default_holder="wife") + resolved = Settings(home=tmp_path, default_holder="wife", auth_token=OPERATOR) monkeypatch.setattr(api_module, "get_settings", lambda: resolved) Database(resolved).init() return resolved @@ -106,14 +111,15 @@ def _seed(settings: Settings, pictures: int = 3) -> None: def test_the_store_pages_newest_first_and_says_how_much_is_left(client: TestClient, settings: Settings) -> None: _seed(settings) - first = client.get("/api/atoms", params={"limit": 2}) + # A tokenless caller is nobody and would see none of these; this test reads the shelf as its owner. + first = client.get("/api/atoms", params={"limit": 2}, headers=AS_OWNER) assert first.status_code == 200 page = first.json() assert page["total"] == 3 assert [atom["atom_id"] for atom in page["atoms"]] == ["shot2:frame", "shot1:frame"] assert page["next_cursor"] - second = client.get("/api/atoms", params={"limit": 2, "cursor": page["next_cursor"]}) + second = client.get("/api/atoms", params={"limit": 2, "cursor": page["next_cursor"]}, headers=AS_OWNER) rest = second.json() assert [atom["atom_id"] for atom in rest["atoms"]] == ["shot0:frame"] assert rest["next_cursor"] is None, "the last page must not promise another one" @@ -125,11 +131,11 @@ def test_browse_narrows_by_modality_and_by_holder(client: TestClient, settings: "the keys are on the shelf", holder_id="wife", origin=MemoryOrigin.self_ ) - images = client.get("/api/atoms", params={"modality": "image"}).json() + images = client.get("/api/atoms", params={"modality": "image"}, headers=AS_OWNER).json() assert images["total"] == 2 assert {atom["modality"] for atom in images["atoms"]} == {"image"} - his = client.get("/api/atoms", params={"holder": "husband"}).json() + his = client.get("/api/atoms", params={"holder": "husband"}, headers=AS_OWNER).json() assert [atom["atom_id"] for atom in his["atoms"]] == ["shot1:frame"] @@ -142,7 +148,7 @@ def test_an_unreadable_cursor_is_refused_with_400(client: TestClient, settings: def test_a_picture_atom_serves_the_bytes_behind_it(client: TestClient, settings: Settings) -> None: _seed(settings, pictures=1) - response = client.get("/api/atoms/shot0:frame/image") + response = client.get("/api/atoms/shot0:frame/image", headers=AS_OWNER) assert response.status_code == 200 assert response.content == PNG assert response.headers["content-type"] == "image/png" @@ -152,15 +158,15 @@ def test_a_memory_with_no_picture_says_so(client: TestClient, settings: Settings atom = MemoryService(settings, Database(settings)).remember( "the keys are on the shelf", holder_id="wife", origin=MemoryOrigin.self_ ) - assert client.get(f"/api/atoms/{atom.atom_id}/image").status_code == 404 - assert client.get("/api/atoms/nope:image").status_code == 404 + assert client.get(f"/api/atoms/{atom.atom_id}/image", headers=AS_OWNER).status_code == 404 + assert client.get("/api/atoms/nope:image", headers=AS_OWNER).status_code == 404 def test_a_forgotten_picture_stops_being_served(client: TestClient, settings: Settings) -> None: _seed(settings, pictures=1) MemoryService(settings, Database(settings)).forget(atom_id="shot0:frame") - assert client.get("/api/atoms/shot0:frame/image").status_code == 404 - assert client.get("/api/atoms").json()["total"] == 0 + assert client.get("/api/atoms/shot0:frame/image", headers=AS_OWNER).status_code == 404 + assert client.get("/api/atoms", headers=AS_OWNER).json()["total"] == 0 def test_status_says_what_the_machine_can_see_and_what_it_cannot(client: TestClient, settings: Settings) -> None: diff --git a/backend/tests/test_transactive.py b/backend/tests/test_transactive.py index 15f5363..d654feb 100644 --- a/backend/tests/test_transactive.py +++ b/backend/tests/test_transactive.py @@ -2,6 +2,7 @@ from __future__ import annotations +import json from pathlib import Path from typing import Any @@ -15,6 +16,7 @@ import ely_eye.cli as cli_module from ely_eye.cli import app from ely_eye.config import Settings from ely_eye.db import Database +from ely_eye.holder_store import HolderStore from ely_eye.ingestion import IngestionService from ely_eye.memory import MemoryService from ely_eye.migrations import SCHEMA_VERSION @@ -249,3 +251,45 @@ def test_the_cli_asks_and_reports_the_referral(tmp_path: Path, monkeypatch: pyte assert asked.exit_code == 0 and '"status": "referred"' in asked.stdout assert runner.invoke(app, ["holders"]).stdout.count("wife") >= 1 assert runner.invoke(app, ["holder-set", "wife", "--contact", "not-a-pair"]).exit_code != 0 + + +def test_the_cli_mints_a_token_that_proves_who_is_asking(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + settings = Settings(home=tmp_path / "home", default_holder="husband") + monkeypatch.setattr(cli_module, "get_settings", lambda: settings) + runner = CliRunner() + Store(settings).wife_remembers(HOME_KEYS) + + issued = json.loads(runner.invoke(app, ["holder-issue", "wife"]).stdout) + assert issued["holder_id"] == "wife" and issued["token"] and issued["issued_at"] + assert HolderStore(Database(settings)).verify_token(issued["token"]) == "wife" + assert issued["token"] not in runner.invoke(app, ["card"]).stdout, "the card is public and the token is not" + + revoked = json.loads(runner.invoke(app, ["holder-revoke", "wife"]).stdout) + assert revoked["holder_id"] == "wife" and revoked["token_issued_at"] is None + assert HolderStore(Database(settings)).verify_token(issued["token"]) is None + + +def test_the_card_is_read_back_from_the_store_rather_than_written_by_hand( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + settings = Settings(home=tmp_path / "home", default_holder="husband") + monkeypatch.setattr(cli_module, "get_settings", lambda: settings) + runner = CliRunner() + store = Store(settings) + store.wife_remembers(HOME_KEYS) + store.husband_remembers(WORK_KEYS) + store.register("neighbour", name="邻居", knows_about=["weather"]) + + answer = runner.invoke(app, ["card"]) + assert answer.exit_code == 0, answer.stdout + card = json.loads(answer.stdout) + assert card["schema_version"] == SCHEMA_VERSION + assert card["atoms"] == 2 and card["sources"] == 2 + assert card["modalities"] == {"text": 2} and card["has_audio"] is False + assert card["mounts"] == {"mounted": [], "scoped": False, "atoms_in_scope": 0} + + named = {entry["holder_id"]: entry for entry in card["holders"]} + assert named["wife"]["memories"] == 1 and named["wife"]["scenarios"] == ["home"] + assert named["wife"]["newest_memory"], "a caller needs to know how stale the answer would be" + assert named["neighbour"]["memories"] == 0 and named["neighbour"]["knows_about"] == ["weather"] + assert HOME_KEYS not in answer.stdout, "the card says what a holder holds, never what they hold it to be" diff --git a/backend/tests/test_visibility_rule.py b/backend/tests/test_visibility_rule.py new file mode 100644 index 0000000..6ebb752 --- /dev/null +++ b/backend/tests/test_visibility_rule.py @@ -0,0 +1,46 @@ +"""The visibility rule is written out in five literal statements; this test keeps them one rule. + +The SQL gate refuses a concatenated statement and a view cannot take a parameter, so the clause +is copied rather than composed. A copy that drifts is invisible to every other test — the search +leg, the browse page and the picture route would each keep passing while disagreeing about who +may see a memory. +""" + +from __future__ import annotations + +import re +from pathlib import Path + +SOURCE_DIR = Path(__file__).resolve().parents[1] / "src" / "ely_eye" +FILES = ("browse.py", "store_search.py", "vector_index.py") +CLAUSE = re.compile(r"\(\? IS NULL[^()]*?holder_id = ''.*?WHERE value = \?[^)]*\)\)\)\)", re.DOTALL) +EXPECTED_COPIES = 5 + + +def _normalize(clause: str) -> str: + """Strip what the copies are allowed to differ in: whitespace, string-literal quotes, table aliases.""" + + return re.sub(r"\s+", "", clause).replace('"', "").replace("a.", "") + + +def test_every_copy_of_the_visibility_rule_agrees() -> None: + found: list[tuple[str, str]] = [ + (name, _normalize(match.group(0))) + for name in FILES + for match in CLAUSE.finditer((SOURCE_DIR / name).read_text(encoding="utf-8")) + ] + + assert len(found) == EXPECTED_COPIES, [name for name, _ in found] + distinct = {clause for _, clause in found} + assert len(distinct) == 1, [name for name, clause in found if clause != next(iter(distinct))] + + +def test_the_rule_still_carries_all_three_states() -> None: + match = CLAUSE.search((SOURCE_DIR / "store_search.py").read_text(encoding="utf-8")) + assert match is not None, "the words leg no longer carries a visibility clause" + clause = _normalize(match.group(0)) + + assert "?ISNULL" in clause, "None still reads the whole library" + assert "holder_id=''" in clause and "$.scope')='everyone'" in clause, "unowned and public rows stay open to a stranger" + assert "?<>''" in clause, "the empty string must stay nobody rather than matching every holder" + assert "holder_id=?" in clause and "value=?" in clause, "the owner and the visible_to list are still checked" diff --git a/ely-eye-mcp/internal/apiclient/client.go b/ely-eye-mcp/internal/apiclient/client.go index 3705010..7c353d0 100644 --- a/ely-eye-mcp/internal/apiclient/client.go +++ b/ely-eye-mcp/internal/apiclient/client.go @@ -190,3 +190,27 @@ func (c *Client) Search(ctx context.Context, options SearchOptions) (*SearchResu } return &result, nil } + +// Card asks what the store can answer: the schema its answers come out under, which +// senses its memory actually covers, and who is in the directory. It is served +// without a token, so a caller can decide whether to send questions at all. +func (c *Client) Card(ctx context.Context) (*domain.CapabilityCard, error) { + var card domain.CapabilityCard + if err := c.Get(ctx, "/api/card", &card); err != nil { + return nil, err + } + return &card, nil +} + +// HolderCard asks the same question about one holder. The server derives it from the +// memories that holder owns, so a card cannot promise a modality the store lacks. +func (c *Client) HolderCard(ctx context.Context, holderID string) (*domain.HolderCard, error) { + if strings.TrimSpace(holderID) == "" { + return nil, errors.New("holder_id is required") + } + var card domain.HolderCard + if err := c.Get(ctx, "/api/card/"+url.PathEscape(holderID), &card); err != nil { + return nil, err + } + return &card, nil +} diff --git a/ely-eye-mcp/internal/domain/types.go b/ely-eye-mcp/internal/domain/types.go index 94272ee..3cf29f5 100644 --- a/ely-eye-mcp/internal/domain/types.go +++ b/ely-eye-mcp/internal/domain/types.go @@ -272,3 +272,33 @@ type ReferralAnswer struct { Answer string `json:"answer,omitempty"` Basis []string `json:"basis"` } + +// HolderCard is what a stranger may learn about one holder: who they are, what they +// claim to know, how much they hold and how fresh it is. No memory text is here. +type HolderCard struct { + HolderID string `json:"holder_id"` + Name string `json:"name"` + Kind string `json:"kind"` + KnowsAbout []string `json:"knows_about"` + Contact map[string]string `json:"contact"` + Memories int `json:"memories"` + Modalities map[string]int `json:"modalities"` + Scenarios []string `json:"scenarios"` + NewestMemory string `json:"newest_memory,omitempty"` + Registered bool `json:"registered"` + TokenIssued bool `json:"token_issued"` +} + +// CapabilityCard answers "can this store answer my kind of questions" before a +// single question is sent: the schema the answers come out under, which senses the +// memory actually covers, and who is in the directory. +type CapabilityCard struct { + SchemaVersion int `json:"schema_version"` + Atoms int `json:"atoms"` + Sources int `json:"sources"` + Modalities map[string]int `json:"modalities"` + HasAudio bool `json:"has_audio"` + Holders []HolderCard `json:"holders"` + Mounts MountState `json:"mounts"` + GeneratedAt string `json:"generated_at"` +} diff --git a/ely-eye-mcp/internal/mcpserver/card_tools.go b/ely-eye-mcp/internal/mcpserver/card_tools.go new file mode 100644 index 0000000..813dcce --- /dev/null +++ b/ely-eye-mcp/internal/mcpserver/card_tools.go @@ -0,0 +1,44 @@ +package mcpserver + +import ( + "context" + "strings" + + "github.com/ZacharyZhang-NY/ely-eye/ely-eye-mcp/internal/domain" + "github.com/modelcontextprotocol/go-sdk/mcp" +) + +type CardInput struct { + HolderID string `json:"holder_id,omitempty" jsonschema:"optional exact holder id; ask for that holder's card instead of the whole store's"` +} + +type CardOutput struct { + Card *domain.CapabilityCard `json:"card,omitempty"` + Holder *domain.HolderCard `json:"holder,omitempty"` +} + +// registerCardTools adds the read that needs no memory access at all: what this +// store can answer. A caller with a new batch of questions reads the card first +// instead of discovering the gaps one question at a time. +func (s *Server) registerCardTools(server *mcp.Server) { + mcp.AddTool(server, &mcp.Tool{ + Name: "ely_eye_card", + Title: "Capability card", + Description: "What this store can answer, derived from what it holds: schema version, senses present, which holders exist and what they hold, and what is mounted. No memory text is shown, so it is safe to read before trusting the store.", + }, s.cardTool) +} + +func (s *Server) cardTool(ctx context.Context, _ *mcp.CallToolRequest, input CardInput) (*mcp.CallToolResult, CardOutput, error) { + if strings.TrimSpace(input.HolderID) == "" { + card, err := s.api.Card(ctx) + if err != nil { + return nil, CardOutput{}, err + } + return nil, CardOutput{Card: card}, nil + } + holder, err := s.api.HolderCard(ctx, input.HolderID) + if err != nil { + return nil, CardOutput{}, err + } + return nil, CardOutput{Holder: holder}, nil +} diff --git a/ely-eye-mcp/internal/mcpserver/card_tools_test.go b/ely-eye-mcp/internal/mcpserver/card_tools_test.go new file mode 100644 index 0000000..8bc903d --- /dev/null +++ b/ely-eye-mcp/internal/mcpserver/card_tools_test.go @@ -0,0 +1,126 @@ +package mcpserver + +import ( + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "github.com/ZacharyZhang-NY/ely-eye/ely-eye-mcp/internal/apiclient" +) + +const storeCardBody = `{"schema_version":11,"atoms":3,"sources":3,"modalities":{"text":2,"image":1}, +"has_audio":false,"holders":[{"holder_id":"wife","name":"小美","kind":"person","knows_about":["home"], +"contact":{"channel":"signal"},"memories":2,"modalities":{"text":2},"scenarios":["home"], +"newest_memory":"2026-05-01T12:00:00+00:00","registered":true,"token_issued":true}],"mounts":{"mounted":[],"scoped":false,"atoms_in_scope":0}}` + +func cardServer(t *testing.T, seen *[]string) *httptest.Server { + t.Helper() + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + *seen = append(*seen, r.Method+" "+r.URL.Path) + if r.URL.Path == "/api/card/nobody" { + w.WriteHeader(http.StatusNotFound) + _ = json.NewEncoder(w).Encode(map[string]string{"detail": "no holder 'nobody' has memories or a directory entry"}) + return + } + if r.URL.Path == "/api/card/wife" { + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(`{"holder_id":"wife","name":"小美","kind":"person","knows_about":["home"],"contact":{},"memories":2,"modalities":{"text":2},"scenarios":["home"],"newest_memory":"2026-05-01T12:00:00+00:00","registered":true,"token_issued":true}`)) + return + } + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte(storeCardBody)) + })) +} + +func TestCardAsksTheStoreAndKeepsTheAnswerDerived(t *testing.T) { + var seen []string + server := cardServer(t, &seen) + defer server.Close() + + s := &Server{api: apiclient.New(server.URL)} + _, out, err := s.cardTool(context.Background(), nil, CardInput{}) + if err != nil { + t.Fatalf("card failed: %v", err) + } + if seen[0] != "GET /api/card" { + t.Errorf("request = %q, want the store card endpoint", seen[0]) + } + if out.Card == nil || out.Holder != nil { + t.Fatalf("card = %+v holder = %+v, want the store card only", out.Card, out.Holder) + } + if out.Card.SchemaVersion != 11 || out.Card.Atoms != 3 || out.Card.HasAudio { + t.Errorf("card = %+v, want the server's own schema, counts and senses", out.Card) + } + if len(out.Card.Holders) != 1 || out.Card.Holders[0].TokenIssued != true { + t.Errorf("holders = %+v, want the directory carried through", out.Card.Holders) + } + if out.Card.Holders[0].Contact["channel"] != "signal" { + t.Errorf("contact = %+v, want the contact a referrer needs", out.Card.Holders[0].Contact) + } +} + +func TestCardWithAHolderAsksForThatHolderAlone(t *testing.T) { + var seen []string + server := cardServer(t, &seen) + defer server.Close() + + s := &Server{api: apiclient.New(server.URL)} + _, out, err := s.cardTool(context.Background(), nil, CardInput{HolderID: "wife"}) + if err != nil { + t.Fatalf("holder card failed: %v", err) + } + if seen[0] != "GET /api/card/wife" { + t.Errorf("request = %q, want the one-holder endpoint", seen[0]) + } + if out.Holder == nil || out.Card != nil { + t.Fatalf("holder = %+v card = %+v, want the holder card only", out.Holder, out.Card) + } + if out.Holder.HolderID != "wife" || out.Holder.Memories != 2 { + t.Errorf("holder card = %+v, want wife's own counts", out.Holder) + } +} + +func TestCardTreatsABlankHolderAsTheWholeStore(t *testing.T) { + var seen []string + server := cardServer(t, &seen) + defer server.Close() + + s := &Server{api: apiclient.New(server.URL)} + if _, _, err := s.cardTool(context.Background(), nil, CardInput{HolderID: " "}); err != nil { + t.Fatalf("blank holder failed: %v", err) + } + if seen[0] != "GET /api/card" { + t.Errorf("request = %q, want the store card rather than a path of spaces", seen[0]) + } +} + +func TestCardPassesOnTheServersReasonForAnUnknownHolder(t *testing.T) { + var seen []string + server := cardServer(t, &seen) + defer server.Close() + + s := &Server{api: apiclient.New(server.URL)} + _, _, err := s.cardTool(context.Background(), nil, CardInput{HolderID: "nobody"}) + if err == nil { + t.Fatal("an unknown holder must not answer with an empty card") + } + if !strings.Contains(err.Error(), "nobody") || !strings.Contains(err.Error(), "/api/card/nobody") { + t.Errorf("error = %q, want the endpoint and the id the server named", err) + } +} + +func TestHolderCardRefusesAnEmptyIdBeforeTheNetwork(t *testing.T) { + var seen []string + server := cardServer(t, &seen) + defer server.Close() + + if _, err := apiclient.New(server.URL).HolderCard(context.Background(), ""); err == nil { + t.Fatal("an empty holder id should be refused, not fetched") + } + if len(seen) != 0 { + t.Errorf("requests = %v, want none", seen) + } +} diff --git a/ely-eye-mcp/internal/mcpserver/server.go b/ely-eye-mcp/internal/mcpserver/server.go index 1399e9f..0cd14f5 100644 --- a/ely-eye-mcp/internal/mcpserver/server.go +++ b/ely-eye-mcp/internal/mcpserver/server.go @@ -33,13 +33,14 @@ func Run(ctx context.Context, cfg config.Config, version string) error { Title: "Ely-Eye Context Cartridge MCP", Version: version, }, &mcp.ServerOptions{ - Instructions: "Use Ely-Eye tools to retrieve local cartridge context, evidence atoms, proof suites, and compiled evidence packs. Read tools mirror local Ely-Eye state without writing. remember, supersede_memory and forget_memory mutate memory through the Ely-Eye API, which must be running (`ely-eye serve`); the server, not the caller, decides who owns a memory. mount_cartridge and switch_cartridge decide what a search may see: mount a cartridge to ask only that cartridge, and unmount everything to ask the whole library. working_set reloads what one session already recorded, in order, without searching; memory_conflicts reports what the store has not reconciled, and settling one is the operator's act, not a tool's.", + Instructions: "Use Ely-Eye tools to retrieve local cartridge context, evidence atoms, proof suites, and compiled evidence packs. Read tools mirror local Ely-Eye state without writing. remember, supersede_memory and forget_memory mutate memory through the Ely-Eye API, which must be running (`ely-eye serve`); the server, not the caller, decides who owns a memory. mount_cartridge and switch_cartridge decide what a search may see: mount a cartridge to ask only that cartridge, and unmount everything to ask the whole library. working_set reloads what one session already recorded, in order, without searching; memory_conflicts reports what the store has not reconciled, and settling one is the operator's act, not a tool's. ely_eye_card says what the store can answer before anything is sent: schema version, which senses its memory covers, and who holds what. A read with no ELY_EYE_AUTH_TOKEN sees only memories nobody owns and memories marked for everyone, so a private answer needs the token.", }) server.registerTools(mcpServer) server.registerWriteTools(mcpServer) server.registerDirectoryTools(mcpServer) server.registerMountTools(mcpServer) server.registerTierTools(mcpServer) + server.registerCardTools(mcpServer) server.registerResources(mcpServer) if err := mcpServer.Run(ctx, &mcp.StdioTransport{}); err != nil { if errors.Is(err, io.EOF) || strings.HasSuffix(err.Error(), "EOF") { diff --git a/scripts/mcp_write_smoke.sh b/scripts/mcp_write_smoke.sh index 5ae86ee..85b1d38 100755 --- a/scripts/mcp_write_smoke.sh +++ b/scripts/mcp_write_smoke.sh @@ -65,7 +65,7 @@ fi # /api/memory/working exists only on builds that carry the memory tiers, so a server left over from # an older tree answers 404 here. Checking the MCP reply instead would prove nothing — the Go layer # decodes into its own struct and re-emits a missing field as 0, so an old server still passes. -status="$(curl -s -o /dev/null -w '%{http_code}' \ +status="$(curl -s -o /dev/null -w '%{http_code}' -H "Authorization: Bearer ${ELY_EYE_AUTH_TOKEN}" \ "http://127.0.0.1:${port}/api/memory/working?holder=fingerprint&session=probe")" if [ "$status" != "200" ]; then echo "FAIL: port ${port} answers /api/status but returns HTTP $status for /api/memory/working;" @@ -178,7 +178,8 @@ send({"jsonrpc": "2.0", "method": "notifications/initialized"}) names = [item["name"] for item in call(2, "tools/list", {})["tools"]] for required in ("remember", "supersede_memory", "forget_memory", "mounted_cartridges", "mount_cartridge", "switch_cartridge", "unmount_cartridge", "append_to_cartridge", - "who_knows", "ask", "list_holders", "working_set", "memory_conflicts"): + "who_knows", "ask", "list_holders", "working_set", "memory_conflicts", + "ely_eye_card"): if required not in names: raise SystemExit(f"FAIL: {required} is not registered; tools are {names}") @@ -315,5 +316,140 @@ if any(hit["atom"]["atom_id"] == older["atom_id"] for hit in settled): if not any(hit["atom"]["atom_id"] == newer["atom_id"] for hit in settled): raise SystemExit(f"FAIL: the newer side of the conflict went missing: {settled}") -print("PASS: remember, supersede, search, forget, cartridge mounting, the transactive referral and the memory tiers work through MCP") +# The card is derived, not declared: it says what this store can answer, and it is +# safe to read before trusting the store because it carries no memory text. +card = tool("ely_eye_card", {})["card"] +if card["schema_version"] < 11 or card["has_audio"]: + raise SystemExit(f"FAIL: the capability card misreports the store: {card}") +wife_card = next((item for item in card["holders"] if item["holder_id"] == "wife"), None) +if not wife_card or wife_card["memories"] < 1 or "护照" not in wife_card["knows_about"]: + raise SystemExit(f"FAIL: the card does not say what the wife holds: {card['holders']}") +if secret in json.dumps(card, ensure_ascii=False): + raise SystemExit(f"FAIL: the capability card carried private text: {card}") +single = tool("ely_eye_card", {"holder_id": "wife"})["holder"] +if single["memories"] != wife_card["memories"] or single["newest_memory"] != wife_card["newest_memory"]: + raise SystemExit(f"FAIL: the one-holder card disagrees with the store card: {single}") + +print("PASS: remember, supersede, search, forget, cartridge mounting, the transactive referral, the memory tiers and the capability card work through MCP") +PY + +# The boundary, proved from the outside: a caller with no token is nobody, and a holder's +# token reads her own memories and nobody else's. The MCP process is re-run without the +# operator token so the Go path is measured against the same rule, not assumed. +"$python_bin" - "$home/ely-eye-mcp" "$port" "$secret" "$ELY_EYE_AUTH_TOKEN" <<'PY' +import json +import os +import subprocess +import sys +import urllib.error +import urllib.request + +binary, port, secret, operator = sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4] +base = f"http://127.0.0.1:{port}" + + +def request(path, body=None, token=None, method=None): + payload = None if body is None else json.dumps(body, ensure_ascii=False).encode() + req = urllib.request.Request(base + path, data=payload, method=method or ("POST" if body is not None else "GET")) + if payload is not None: + req.add_header("content-type", "application/json") + if token: + req.add_header("Authorization", f"Bearer {token}") + try: + with urllib.request.urlopen(req) as response: + return response.status, json.load(response) + except urllib.error.HTTPError as error: + raw = error.read().decode() + try: + return error.code, json.loads(raw) + except json.JSONDecodeError: + return error.code, {"detail": raw} + + +def texts(query, token=None): + status, answer = request("/api/search", {"query": query, "top_k": 10}, token=token) + if status != 200: + raise SystemExit(f"FAIL: search returned HTTP {status}: {answer}") + return [hit["atom"]["text"] for hit in answer["hits"]] + + +# Written by this machine's principal, so it is somebody else's memory to her. +foreign = "the oven needs descaling" + +if secret in texts(secret): + raise SystemExit("FAIL: a caller with no token read a private memory") + +status, issued = request("/api/holders/wife/token", {}, token=operator, method="POST") +token = issued.get("token") if status == 200 else None +if not token: + raise SystemExit(f"FAIL: the operator could not mint the wife's token: HTTP {status} {issued}") +if secret not in texts(secret, token=token): + raise SystemExit("FAIL: the wife's own token cannot find her memory") +if foreign not in texts(foreign, token=operator): + raise SystemExit("FAIL: the operator token cannot read the memory its own principal wrote") +if foreign in texts(foreign, token=token): + raise SystemExit("FAIL: the wife's token reads somebody else's memory") + +status, answer = request("/api/search", {"query": "护照", "filters": {"as_holder": "local"}}, token=token) +if status != 403: + raise SystemExit(f"FAIL: the wife naming another holder returned HTTP {status}, want 403: {answer}") +if "local" not in json.dumps(answer, ensure_ascii=False): + raise SystemExit(f"FAIL: the refusal does not name the holder she reached for: {answer}") + +status, card = request("/api/card") +if status != 200 or card["schema_version"] < 11: + raise SystemExit(f"FAIL: the capability card needs a token or misreports the store: HTTP {status} {card}") +if secret in json.dumps(card, ensure_ascii=False): + raise SystemExit("FAIL: the capability card carried private text") + +env = {key: value for key, value in os.environ.items() if key != "ELY_EYE_AUTH_TOKEN"} +proc = subprocess.Popen([binary, "server"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL, text=True, env=env) + + +def send(message): + proc.stdin.write(json.dumps(message, ensure_ascii=False) + "\n") + proc.stdin.flush() + + +def call(id_, method, params): + send({"jsonrpc": "2.0", "id": id_, "method": method, "params": params}) + while True: + line = proc.stdout.readline() + if not line: + raise SystemExit(f"FAIL: the tokenless MCP server closed the pipe at request {id_}") + answer = json.loads(line) + if answer.get("id") == id_: + if "error" in answer: + raise SystemExit(f"FAIL: {method} returned {answer['error']}") + return answer["result"] + + +def tool(name, arguments): + result = call(3, "tools/call", {"name": name, "arguments": arguments}) + if result.get("isError"): + raise SystemExit(f"FAIL: {name} returned an error: {result['content']}") + return result["structuredContent"] + + +call(1, "initialize", {"protocolVersion": "2025-06-18", "capabilities": {}, + "clientInfo": {"name": "ely-eye-stranger", "version": "0"}}) +send({"jsonrpc": "2.0", "method": "notifications/initialized"}) + +stranger_card = tool("ely_eye_card", {})["card"] +if stranger_card["schema_version"] != card["schema_version"]: + raise SystemExit(f"FAIL: the tokenless card reports a different schema: {stranger_card}") +stranger_hits = tool("search_atoms", {"query": secret})["hits"] +if any(hit["atom"]["text"] == secret for hit in stranger_hits): + raise SystemExit(f"FAIL: a tokenless MCP client read a private memory: {stranger_hits}") +directory = tool("who_knows", {"query": "护照"})["report"] +if directory["status"] != "found" or not directory["hits"]: + raise SystemExit(f"FAIL: a tokenless client cannot use the directory: {directory}") +if directory["hits"][0]["holder_id"] != "wife": + raise SystemExit(f"FAIL: the directory stopped naming the wife: {directory['hits'][0]}") +proc.stdin.close() +proc.wait(timeout=10) + +print("PASS: a caller with no token sees only public memory, a holder's token sees her own, " + "naming somebody else is refused, and the capability card answers without a token") PY