Let the server decide whose memory a caller may read
A holder is now somebody who can prove it. Schema v11 stores a sha256 of a per-holder token beside the directory row, so a copied database cannot replay it and the plaintext is shown exactly once. The operator token mints and revokes those tokens; a holder token reads that holder's own memories and nothing else; naming somebody else without the operator token is refused with both ids in the message. The visibility rule was leaking before this: browse had none at all, and the search rules short-circuited to the whole library whenever no holder was named. The rule is now written out in five copies with one parameter carried four times, and a test compares the copies so they cannot drift — semgrep rejects an interpolated constant, so the copies are not a choice. Eyes carry three states on purpose. NULL means the whole library, which is what a process that owns the store is entitled to; a holder id means that holder; the empty string means nobody, which is what a tokenless HTTP caller is. Making NULL mean nobody instead would have forced every internal caller — CLI, proofs, the compiler, the bench harnesses — to name eyes, contradicting the rule that a local process reads as its own trusted principal. The capability card answers "can this store answer my kind of question" before anything is sent. It is derived from what the store holds, so it cannot promise a modality the store lacks, and it carries no memory text, so it is safe to read before trusting the store. HTTP, CLI and one MCP tool all serve it without a token.
This commit is contained in:
ZacharyZhang-NY committed
1 parent
209f2d0a1c
commit
83dfc14daa
29 files changed
+1335
-77
No files matched your search
+16
-5
@@ -421,11 +421,22 @@
|
||||
- **评审第 4 轮 FAIL**（同一独立 pi，7 项全跑，~557 工具轮；D40/D41 的修法实测成立，16 对逐字复现、五对 D40 全记上）：三条——D42 就是上面 P1 那句被实验推翻（Go 层重发缺失字段，旧服务照样过）；D43 `:412` 的机制句**方向写反**，我写「主语之前的裸数字并入主语键」，而代码的改名槽是 `index + 1`（主语 token 的**下一个**，`trust.py:275-276`），实测 `3 room holds 22 degrees` 主语是 `3`、不会向前融合，三个例子里只有 `2026 saw …` 数字在前、走的是 `trust.py:270-272` 首数字分支；D44 `_numbers_follow` 的**漏报侧没披露**：改不改名取决于每条记忆自己后面还留着没有第二个数字，`温度 22 度` + `温度 24 度 湿度 40`（`温度`/`温度24`）、`温度 22 度 湿度 40` + `温度 24 度`、`温度 22 度 湿度 40` + `温度 24 度 湿度 60`（`温度22`/`温度24`）、`预算 5000 元 超了 8000` + `预算 6000 元`（`预算5000`/`预算`）四对全 0 条，方向 fail-closed（两条都留在页上）。D44 按评审建议只**披露 + 加测试**（`test_a_second_number_in_one_memory_hides_the_update`），不用「未改名主语兜底比对」，那会把 `Room 3 holds 22 degrees` + `Room 4 holds 24 degrees` 退回 D37 的误报。四条精度说明全部采纳：P-a `degrees` 键也吞年份（`in 2026` → `{'degrees':'2026'}`），P5 那句已改成「第二个数字，含年份」；P-b 反向实验的红名单是 4 条不是 5 条（全套 4 failed / 379 passed，无连带）；P-c 「数字只要站在某个词之后就是量不是名字」已就地加限定词（且不在紧挨主语那个位置）；P-d 8799 守卫仍只在开跑时探一次，现在指纹补上了**旧构建**答题这一口——同树残留的服务占住端口时指纹仍给 200（评审实测：同树同 `ELY_EYE_AUTH_TOKEN` 的服务把 5 条 atom 落进它自己的 home，直到挂载卡带才失败），那种情形仍只有 busy-port 守卫能挡。该轮同时实测确认：门禁全绿且 mypy 两边逐字相同、tokenize 0 块、最大文件 `bench/systems.py` 498 行；D41 的新测试文案与实测一致；P4 的 stamp 判断站得住（`bench/leg_report.py` 两处 blob 全同 `40596d29…`）。判定：p2-3.1/3.2/3.3 与两条验收都能 `[x]`，p2-3.4 因 D43/D44 的记录级缺陷暂缓。
|
||||
- **评审第 5 轮 PASS**（同一独立 pi，7 项全跑，本轮 diff 仅 3 个文件 +31/−5、机制 blob 全同：`trust.py d0b1a090`、`memory.py 13b0ebbf`、`retrieval.py 305d6e3a`、`compiler.py 5d341ca5`、`api_memory.py 22a03666`）：D42/D43/D44 全部按实测改掉并被独立复现——D42 的判别子它自己起两边服务量过（`4c6af87` 的 `/api/memory/working` 404、HEAD 200）；D43 三个主语实测 `3` / `room3` / `2026`；D44 四对逐对 0 条且贴出了页面（两条都留着），新测试的两条断言它逐条读过（第二条真的检查了「都在页上」）。**「不用未改名主语兜底比对」这个判断它用实验支持了**：把 `quantity_conflict` 改成主语不等时退回比值键，`Room 3 holds 22 degrees` + `Room 4 holds 24 degrees` 立刻变回 `('holds','22','24')`，即第 2 轮 D37 的误报复活。反向实验（去掉 `_numbers_follow` 条件）红名单仍是 4 条、全套 4 failed / 379 passed，**新加那条不变红**（无条件改名下它两条断言仍成立）。20 对冲突 + D40 五对 + `Room 3 is big` 边界逐字复现。门禁全部重跑且与本报告逐格一致（384 passed、六文件 83、conflicts 27、三文件 35、mypy 与 `9cb46c1` 文件集合与去行号文本逐字相同、tokenize 148 个 `.py` 0 块、最大文件 `bench/systems.py` 498、go `-count=1` 全套 + gofmt 空、`index-CSDHCFNo.js 221.48 kB`、smoke PASS）。两条精度说明（不算缺陷）已采纳：P-e 把 D44 成因从「一侧改名」改成「改名后主语键各自吃进自己的数字，两侧都改名也永不相等」（我另测 `温度 22 度 湿度 40` + `温度 24 度 湿度 40` → 0 条证实对称情形）；P-f 把「指纹补上 8799 那一口」限定为旧构建那一口。判定：p2-3.1/3.2/3.3/3.4 与两条验收全部 `[x]`。
|
||||
- 顺带：`cli.py` 495→432 行（写路径拆进 `cli_memory.py`），`bench/leg_check.py` 拆出 `bench/leg_report.py`；typer 0.27.2 用**关键字**调用回调，`*atom_ids: str` 形式的命令一调就 `TypeError`，改成 `list[str]`——这才是之前 `test_maintenance` 那条 TypeError 的真因，命令重名只是表象。报告 stamp 早于 harness 的缺陷也修了（`a52c104`）。
|
||||
- [ ] **p2-4** 共享记忆
|
||||
- [ ] p2-4.1 多用户/多 agent 作用域 + ACL
|
||||
- [ ] p2-4.2 默认可见性：answer-only 转介
|
||||
- [ ] p2-4.3 能力卡：让外部 agent 能路由进 ely-eye
|
||||
- 验收：两 holder 互不可见；越权被拒；一次真实跨 holder 会话证明能力卡可用
|
||||
- [x] **p2-4** 共享记忆
|
||||
- [x] p2-4.1 多用户/多 agent 作用域 + ACL（schema v11：holder token + 三态眼睛）
|
||||
- 事实：`holders` 加 `token_hash` / `token_issued_at`（`migrations.py:92` DDL，`_NEW_HOLDER_COLUMNS` 给老库 ALTER）。**只存哈希**（`sha256`，`holder_store._token_hash`），所以拷走的库不能重放 token；明文只在 `POST /api/holders/{id}/token` 的响应里出现一次（`IssuedToken`），`GET /api/holders/wife` 只回 `token_issued_at`。生成用 `secrets.token_urlsafe(32)`，比对用 `secrets.compare_digest`。`register_minimal` 是 `INSERT … ON CONFLICT DO NOTHING`，`remember` 在写记忆前调它（`memory.py:120` 之后），所以被转介的人一定有目录行，且它绝不覆盖人已声明的 name/contact。
|
||||
- 身份是**服务器算的**：`api_auth.resolve_caller` 三选一——operator（全局 token，可指名任何人）、holder（自己的 token，`verify_token`）、anonymous（没 token）。`read_as(caller, claimed)` 把「调用者声称的 holder」降级成「服务器认定的眼睛」：operator 不声称 → `None`（整库），operator 声称 → 那个人，holder 声称自己 → 自己，**声称别人 → 403 并把两个人都点名**，anonymous 声称任何人 → 403。`/api/search`、`/api/atoms`、`/api/atoms/{id}/image`、`/api/chat`、`/api/memory/working`、`/api/memory/conflicts`、`/api/ask` 全部走它；`remember`/`supersede` 的 holder 也只能是 caller 自己（p0-2 那条「客户端不能声称自己是别人」现在由 token 保证，不再靠 `default_holder` 兜底）。
|
||||
- **眼睛是三态、不是一个字符串的两可**（跑出来的决定）：`NULL` = 整库（operator，或自己拥有这个 store 的进程），holder id = 那个人的眼睛，`''` = 谁也不是（无 token 的 HTTP 调用者）。让 `NULL` 也代表「谁也不是」会逼 CLI、proofs、compiler、bench harness、`who_knows`、`ask` 全都传眼睛，那跟 p0-2 定的「拥有 store 的进程按本机受信主体读」直接冲突——第一次这样改时 `pytest -q` 红了 29 条（`test_vector_index.py:46` `assert [] == ['mem-eaf3b790ca27bf3e:text']`、`test_proofs.py` 回放 0 hits、`test_visual_index.py:43` `ValueError: invalid literal for int() with base 10: 'widget gadget sprocket'`，因为一条本来被跳过的腿开始跑了）。`''` 能当「谁也不是」是因为本 schema 里 `holder_id = ''` 本来就是「无主」。
|
||||
- 修前是真的漏：`browse.py`（`GET /api/atoms` 与 `/api/atoms/{id}/image`）**完全没有可见性规则**，无 token 就能翻页读到别人的记忆正文和图片；`store_search.py:81-86` 与 `vector_index.py:245-252` 的规则是 `(? IS NULL OR holder_id = '' OR scope='everyone' OR holder_id = ? OR …)`，`as_holder IS NULL` 短路成「整库」，所以无 token 搜索也看见全部。现在五处副本统一成 `(? IS NULL OR holder_id = '' OR json_extract(visibility_json,'$.scope') = 'everyone' OR (? <> '' AND (holder_id = ? OR EXISTS (… value = ?))))`，**四个参数同一个值**。
|
||||
- 规则为什么写五份而不抽常量：semgrep `python.sqlalchemy.security.sqlalchemy-execute-raw-query` 对 `connection.execute(f"…{常量}…")` 报警，哪怕那个常量是模块级的，所以只能写整句字面量。防漂移靠 `backend/tests/test_visibility_rule.py`：`EXPECTED_COPIES = 5`，正则抓出每处、`_normalize`（去空白、去引号、去 `a.` 别名）后必须**只剩一个字符串**，第二条测试钉住 clause 里 `? IS NULL` / `holder_id = ''` / `$.scope')='everyone'` / `? <> ''` / `holder_id = ?` / `value = ?` 六个特征还在。第一版正则 `\(\? IS NULL[^;]*?` 会把 `(? IS NULL OR modality = ?)` 这类过滤谓词一起吃掉，读出 4 份不同 clause——那是正则错，不是规则漂移。
|
||||
- [x] p2-4.2 默认可见性：answer-only 转介（p0-5 的转介现在被 ACL 兜住）
|
||||
- p0-5 只做了一半：`ask` 命中别人的时候只回指针不回正文（`_pointer`），但**搜索本身**还是能读到正文，所以「只给指针」是客气不是边界。现在无 token 的 `search_atoms` 搜不到那句私钥（smoke 实测），`ask` 的 `answer` 仍为空、`secret` 在整个 referral JSON 里不出现，而 `who_knows` 照旧能用——目录路由**故意不设卡**，因为它只出计数、分数、contact 和声明过的话题，不出记忆正文（`api_holders.py` 注册处一行注释写明）。`visibility` 三档实测：默认 `holder`（只有主人）、`everyone`（无 token 可搜到）、`visible_to: [husband]`（主人 + 被指名的人能读，陌生人和第三个 holder 不能）。
|
||||
- [x] p2-4.3 能力卡：让外部 agent 能路由进 ely-eye
|
||||
- `GET /api/card`（整库）与 `GET /api/card/{holder}`（一个人），CLI `ely-eye card`，MCP `ely_eye_card`（Go `apiclient.Card` / `HolderCard`，空 id 在**发请求之前**拒）。卡是**派生的不是声明的**（`card.build_card`）：`PRAGMA user_version`、`sources` 计数、`v_atoms_current` 按模态分组、每人 `(条数, MAX(ingested_at))`、每人模态/场景、合并目录、`has_audio = "audio" in senses`、挂载状态。所以它不能承诺一个 store 里没有的模态——`has_audio: false` 就是把视觉/音频腿的缺口在**别人提问之前**说出来。
|
||||
- 无 token 就能读（smoke 实测 HTTP 与 MCP 两条路都通），且卡里**不含任何记忆正文**（smoke 断言 `secret not in json.dumps(card)`）；`token_issued` 只说「这个人有 token」，不发 token。卡的 schema 版本与整库卡一致这条也进了 smoke 的 tokenless 分支。
|
||||
- 偏差（都是跑出来的）：`/api/memory/working` 与 `/api/memory/conflicts` 现在拒绝「谁也不是」的调用者（422，消息含「somebody」），因为这两件事本来就属于某个人；`compiler.py:41` 的缓存 key 加了眼睛——先前写 `as_holder or ""` 会把 operator 的上下文和陌生人的**撞成同一个 key**（两边都哈希 `""`），于是陌生人那份空 pack 会端给 operator；`forget` 的级联、卡带成员、目录都不因 ACL 改动。
|
||||
- 验收：`pytest backend/tests/test_holder_acl.py backend/tests/test_visibility_rule.py backend/tests/test_transactive.py` → **13 + 2 + 19** 全绿（两 holder 互不可见、无 token 读不到私钥、`everyone` 无 token 可搜、`visible_to` 只放行被指名的人、声称别人 403 且点名两人、token 只出现一次且库里只有 sha256、吊销后旧 token 401、卡无 token 可读且不含正文、陌生人的图片 404 而无主的 200、`compile(as_holder="")` 返回 0 hits 且 `cache_trace_id` 与 `as_holder="wife"` / `None` 都不同）。跨进程：`bash scripts/mcp_write_smoke.sh` → 两行 PASS，第二行是「a caller with no token sees only public memory, a holder's token sees her own, naming somebody else is refused, and the capability card answers without a token」——它先证明无 token 搜不到私钥，再用 operator token 铸出妻子的 token、证明她能搜到自己的、搜不到 `local` 写的 `the oven needs descaling`，最后**把 MCP 进程不带 `ELY_EYE_AUTH_TOKEN` 重启一遍**，证明 Go 那条路守同一条规则（不是假设）。
|
||||
- 顺手修的两条自己写的断言缺陷（都是跑出来的）：smoke 里 `if secret in found(None, token=token)` 是同义反复（搜的还是那句话，当然命中），改成拿**别人写的**记忆当探针；指纹探测 `GET /api/memory/working?holder=fingerprint` 现在返回 **403**（路由在、但匿名不能声称 holder），所以探测必须带 operator token——旧树上没有这条路由仍是 404，判别力保住。
|
||||
- 门禁：`pytest -q` **401 passed**；`ruff check` 对 22 个改动文件干净；`bench/filesize_check` 干净；`bench.audit --product` → `audited 32 harness files, 2 reports, 6 set files` / `clean`；mypy 冷缓存 **24 errors in 9 files (checked 112 source files)**，与 `9cb46c1` 基线（24/9）**净新增 0**；`./node_modules/.bin/pyright` 对 17 个后端源文件 + 5 个测试文件 → **0 errors**；`go build` / `go vet` / `go test`（apiclient ok、mcpserver ok）+ `gofmt -l` 空；`npm run build` → `dist/assets/index-CSDHCFNo.js 221.48 kB`（bundle 哈希与 `246dae9` 相同，与「本轮没动前端」一致）+ eslint exit 0；`bash scripts/mcp_write_smoke.sh` 3.3s 两行 PASS。
|
||||
- [ ] **p2-5** 文档单一真相
|
||||
- [ ] p2-5.1 `AGENTS.md` 重写并停止 gitignore
|
||||
- [ ] p2-5.2 `README.md` 每条宣称指向可重跑命令
|
||||
|
||||
@@ -5,7 +5,7 @@ import shutil
|
||||
from pathlib import Path
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import Depends, FastAPI, File, HTTPException, Query, UploadFile
|
||||
from fastapi import Depends, FastAPI, File, HTTPException, Query, Request, UploadFile
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
@@ -13,7 +13,8 @@ from pydantic import BaseModel
|
||||
|
||||
from . import __version__
|
||||
from .adapters import plan as adapter_plan
|
||||
from .api_auth import require_write
|
||||
from .api_auth import read_as, require_write, resolve_caller
|
||||
from .api_card import register as register_card_routes
|
||||
from .api_cartridges import register as register_cartridge_routes
|
||||
from .api_holders import register as register_holder_routes
|
||||
from .api_memory import register as register_memory_routes
|
||||
@@ -43,6 +44,7 @@ from .schemas import (
|
||||
PathIngestRequest,
|
||||
RememberRequest,
|
||||
RuntimeProfile,
|
||||
SearchFilters,
|
||||
SearchRequest,
|
||||
SearchResult,
|
||||
SupersedeRequest,
|
||||
@@ -166,12 +168,15 @@ def create_app() -> FastAPI:
|
||||
@app.post("/api/chat", response_model=AnswerResponse)
|
||||
def chat(
|
||||
request: ChatRequest,
|
||||
http: Request,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
_: Annotated[None, Depends(require_write)],
|
||||
) -> AnswerResponse:
|
||||
db = Database(settings)
|
||||
compiler = ContextCompiler(settings, db)
|
||||
context = compiler.compile(request.question, request.profile)
|
||||
context = compiler.compile(
|
||||
request.question, request.profile, as_holder=read_as(resolve_caller(http, settings), None)
|
||||
)
|
||||
if not context.hits:
|
||||
raise HTTPException(status_code=404, detail="No evidence atoms are available for this question.")
|
||||
runtime = QwenRuntime(settings)
|
||||
@@ -192,12 +197,24 @@ def create_app() -> FastAPI:
|
||||
)
|
||||
|
||||
@app.post("/api/search", response_model=SearchResult)
|
||||
def search(request: SearchRequest, settings: Annotated[Settings, Depends(settings_dep)]) -> SearchResult:
|
||||
def search(
|
||||
request: SearchRequest,
|
||||
http: Request,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
) -> SearchResult:
|
||||
"""Search through the eyes the token proves. A body that names another holder is refused, not obeyed."""
|
||||
try:
|
||||
caller = resolve_caller(http, settings)
|
||||
eyes = read_as(caller, request.filters.as_holder if request.filters else None)
|
||||
filters = request.filters
|
||||
if filters is not None:
|
||||
filters = filters.model_copy(update={"as_holder": eyes})
|
||||
elif eyes is not None:
|
||||
filters = SearchFilters(as_holder=eyes)
|
||||
return RetrievalService(settings).search(
|
||||
request.query,
|
||||
request.top_k,
|
||||
filters=request.filters,
|
||||
filters=filters,
|
||||
cursor=request.cursor,
|
||||
image=request.image,
|
||||
)
|
||||
@@ -207,13 +224,15 @@ def create_app() -> FastAPI:
|
||||
@app.post("/api/memory", response_model=EvidenceAtom)
|
||||
def remember(
|
||||
request: RememberRequest,
|
||||
http: Request,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
_: Annotated[None, Depends(require_write)],
|
||||
) -> EvidenceAtom:
|
||||
"""The token decides who owns the memory; the body cannot say so. The operator token writes as the default holder."""
|
||||
try:
|
||||
return MemoryService(settings).remember(
|
||||
request.text,
|
||||
holder_id=settings.default_holder,
|
||||
holder_id=resolve_caller(http, settings).holder_id or settings.default_holder,
|
||||
origin=MemoryOrigin.self_,
|
||||
scenario=request.scenario,
|
||||
event_time=request.event_time,
|
||||
@@ -229,6 +248,7 @@ def create_app() -> FastAPI:
|
||||
@app.post("/api/memory/supersede", response_model=EvidenceAtom)
|
||||
def supersede(
|
||||
request: SupersedeRequest,
|
||||
http: Request,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
_: Annotated[None, Depends(require_write)],
|
||||
) -> EvidenceAtom:
|
||||
@@ -236,7 +256,7 @@ def create_app() -> FastAPI:
|
||||
return MemoryService(settings).supersede(
|
||||
request.atom_id,
|
||||
text=request.text,
|
||||
holder_id=settings.default_holder,
|
||||
holder_id=resolve_caller(http, settings).holder_id or settings.default_holder,
|
||||
scenario=request.scenario,
|
||||
event_time=request.event_time,
|
||||
event_time_precision=request.event_time_precision,
|
||||
@@ -263,6 +283,7 @@ def create_app() -> FastAPI:
|
||||
register_cartridge_routes(app, settings_dep)
|
||||
register_holder_routes(app, settings_dep)
|
||||
register_memory_routes(app, settings_dep)
|
||||
register_card_routes(app, settings_dep)
|
||||
|
||||
@app.get("/api/adapters", response_model=list[AdapterStatus])
|
||||
def adapters(settings: Annotated[Settings, Depends(settings_dep)]) -> list[AdapterStatus]:
|
||||
@@ -270,6 +291,7 @@ def create_app() -> FastAPI:
|
||||
|
||||
@app.get("/api/atoms", response_model=AtomPage)
|
||||
def atoms(
|
||||
http: Request,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
limit: Annotated[int, Query(ge=1, le=200)] = 50,
|
||||
cursor: str | None = None,
|
||||
@@ -277,6 +299,7 @@ def create_app() -> FastAPI:
|
||||
source_id: str | None = None,
|
||||
holder: str | None = None,
|
||||
) -> AtomPage:
|
||||
"""`holder` picks whose shelf to look at; the token decides what is on it."""
|
||||
try:
|
||||
return browse(
|
||||
settings,
|
||||
@@ -285,13 +308,16 @@ def create_app() -> FastAPI:
|
||||
modality=modality,
|
||||
source_id=source_id,
|
||||
holder=holder,
|
||||
as_holder=read_as(resolve_caller(http, settings), None),
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
@app.get("/api/atoms/{atom_id}/image")
|
||||
def atom_image(atom_id: str, settings: Annotated[Settings, Depends(settings_dep)]) -> FileResponse:
|
||||
found = picture(settings, atom_id)
|
||||
def atom_image(
|
||||
atom_id: str, http: Request, settings: Annotated[Settings, Depends(settings_dep)]
|
||||
) -> FileResponse:
|
||||
found = picture(settings, atom_id, as_holder=read_as(resolve_caller(http, settings), None))
|
||||
if found is None:
|
||||
raise HTTPException(status_code=404, detail=f"{atom_id} has no picture to show")
|
||||
return FileResponse(found.path, media_type=found.mime)
|
||||
|
||||
@@ -1,22 +1,94 @@
|
||||
"""Writes need a token; queries do not.
|
||||
"""Writes need a token; a read needs one to be anybody at all.
|
||||
|
||||
Anything that can change stored state — a memory, an ingested source, a mounted cartridge, a
|
||||
proof run — is refused unless the caller presents the token the server was started with. A
|
||||
browser must also come from an allowed origin. The CLI and the MCP server reach the same
|
||||
writer through the same rule, so no face of the product gets a cheaper path to the store.
|
||||
|
||||
A read with no token is still allowed, but it is nobody's read: it sees what nobody owns and
|
||||
what is marked for everyone, and nothing else. Naming a holder is a claim about who is asking,
|
||||
so it needs that holder's token or the operator token — otherwise `as_holder` would be a way to
|
||||
look over anyone's shoulder by typing their name.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import secrets
|
||||
from typing import TYPE_CHECKING
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Literal
|
||||
|
||||
from fastapi import HTTPException, Request
|
||||
|
||||
from .db import Database
|
||||
from .holder_store import HolderStore
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .config import Settings
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Caller:
|
||||
"""Who the server decided is asking, from the token in front of it."""
|
||||
|
||||
kind: Literal["operator", "holder", "anonymous"]
|
||||
holder_id: str | None = None
|
||||
|
||||
@property
|
||||
def may_name_others(self) -> bool:
|
||||
return self.kind == "operator"
|
||||
|
||||
|
||||
def presented_token(request: Request) -> str:
|
||||
"""The bearer value on this request, or an empty string when nobody presented one."""
|
||||
|
||||
header = request.headers.get("authorization", "")
|
||||
return header.removeprefix("Bearer ").strip() if header.startswith("Bearer ") else ""
|
||||
|
||||
|
||||
def resolve_caller(request: Request, settings: Settings) -> Caller:
|
||||
"""Turn the bearer token into an identity. An unknown token is refused, not treated as nobody."""
|
||||
|
||||
token = presented_token(request)
|
||||
if not token:
|
||||
return Caller(kind="anonymous")
|
||||
if settings.auth_token and secrets.compare_digest(token, settings.auth_token):
|
||||
return Caller(kind="operator")
|
||||
holder_id = HolderStore(Database(settings)).verify_token(token)
|
||||
if holder_id is None:
|
||||
raise HTTPException(
|
||||
status_code=401, detail="this token is neither the operator token nor a holder token"
|
||||
)
|
||||
return Caller(kind="holder", holder_id=holder_id)
|
||||
|
||||
|
||||
def read_as(caller: Caller, claimed: str | None) -> str | None:
|
||||
"""Whose eyes a read runs under. A claim the token cannot back is refused, not ignored.
|
||||
|
||||
The answer is the three states the store's visibility clause understands: a holder id, the
|
||||
empty string for a caller who is nobody, and None for an operator who reads the whole library.
|
||||
"""
|
||||
|
||||
if claimed is None:
|
||||
return "" if caller.kind == "anonymous" else caller.holder_id
|
||||
if caller.holder_id == claimed or caller.may_name_others:
|
||||
return claimed
|
||||
if caller.holder_id is None:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=(
|
||||
f"an anonymous caller may not read as {claimed!r}; "
|
||||
"send that holder's token or the operator token"
|
||||
),
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail=(
|
||||
f"holder {caller.holder_id!r} may not read as {claimed!r}; "
|
||||
"only the operator token may name another holder"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def require_write(request: Request) -> None:
|
||||
"""Gate a mutating request. The app carries the settings source, so this stays a plain global."""
|
||||
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
"""The capability card over HTTP: served without a token, because it is the reason to send one.
|
||||
|
||||
A caller deciding whether to point an agent at this store should not have to guess whether the
|
||||
store holds speech, who is in the directory, or which schema answered last. Everything here is
|
||||
read back from the store, so it cannot advertise a sense the store has no atoms for.
|
||||
|
||||
No `from __future__ import annotations` here: FastAPI reads these route signatures, and string
|
||||
annotations would be resolved against module globals, where the injected settings_dep exists
|
||||
only as a parameter of register().
|
||||
"""
|
||||
|
||||
from collections.abc import Callable
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import Depends, FastAPI, HTTPException
|
||||
|
||||
from .card import build_card
|
||||
from .card_schemas import CapabilityCard, HolderCard
|
||||
from .config import Settings
|
||||
|
||||
|
||||
def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None:
|
||||
"""Register the card routes. Nothing here is secret: it is the store describing itself."""
|
||||
|
||||
@app.get("/api/card", response_model=CapabilityCard)
|
||||
def card(settings: Annotated[Settings, Depends(settings_dep)]) -> CapabilityCard:
|
||||
return build_card(settings)
|
||||
|
||||
@app.get("/api/card/{holder_id}", response_model=HolderCard)
|
||||
def holder_card(
|
||||
holder_id: str, settings: Annotated[Settings, Depends(settings_dep)]
|
||||
) -> HolderCard:
|
||||
"""One directory entry with its counted evidence; 404 names the holder that is not there."""
|
||||
for entry in build_card(settings).holders:
|
||||
if entry.holder_id == holder_id:
|
||||
return entry
|
||||
raise HTTPException(status_code=404, detail=f"no holder {holder_id!r} has memories or a directory entry")
|
||||
@@ -8,9 +8,9 @@ only as a parameter of register().
|
||||
from collections.abc import Callable
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import Depends, FastAPI, HTTPException
|
||||
from fastapi import Depends, FastAPI, HTTPException, Request
|
||||
|
||||
from .api_auth import require_write
|
||||
from .api_auth import read_as, require_write, resolve_caller
|
||||
from .config import Settings
|
||||
from .db import Database
|
||||
from .holder_store import HolderStore
|
||||
@@ -20,6 +20,7 @@ from .transactive_schemas import (
|
||||
AskRequest,
|
||||
Holder,
|
||||
HolderUpsertRequest,
|
||||
IssuedToken,
|
||||
ReferralAnswer,
|
||||
WhoKnowsReport,
|
||||
WhoKnowsRequest,
|
||||
@@ -27,7 +28,12 @@ from .transactive_schemas import (
|
||||
|
||||
|
||||
def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None:
|
||||
"""Register the transactive routes. The server owns the timestamps, the caller does not."""
|
||||
"""Register the transactive routes. The server owns the timestamps, the caller does not.
|
||||
|
||||
The directory routes answer "who knows this?" and carry counts, scores and contact details but
|
||||
no memory text, so they stay readable without a token; `ask` reads the memories themselves, so
|
||||
it runs through the eyes the token proves.
|
||||
"""
|
||||
|
||||
app.state.write_settings = settings_dep
|
||||
|
||||
@@ -84,6 +90,34 @@ def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
return entry
|
||||
|
||||
@app.post("/api/holders/{holder_id}/token", response_model=IssuedToken)
|
||||
def issue_token(
|
||||
holder_id: str,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
_: Annotated[None, Depends(require_write)],
|
||||
) -> IssuedToken:
|
||||
"""Mint a bearer token for one holder. Only the operator token may do this, and the value is shown once."""
|
||||
try:
|
||||
entry, token = store(settings).issue_token(holder_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||
issued_at = entry.token_issued_at
|
||||
if issued_at is None:
|
||||
raise HTTPException(status_code=500, detail=f"holder {holder_id!r} holds a token but no issue time")
|
||||
return IssuedToken(holder_id=entry.holder_id, token=token, issued_at=issued_at)
|
||||
|
||||
@app.delete("/api/holders/{holder_id}/token", response_model=Holder)
|
||||
def revoke_token(
|
||||
holder_id: str,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
_: Annotated[None, Depends(require_write)],
|
||||
) -> Holder:
|
||||
"""Drop a holder's token; the holder stays in the directory, only the proof of identity goes."""
|
||||
try:
|
||||
return store(settings).revoke_token(holder_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
@app.get("/api/holders/{holder_id}/knows", response_model=WhoKnowsReport)
|
||||
def holder_knows(
|
||||
holder_id: str,
|
||||
@@ -111,12 +145,14 @@ def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None:
|
||||
@app.post("/api/ask", response_model=ReferralAnswer)
|
||||
def ask(
|
||||
request: AskRequest,
|
||||
http: Request,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
) -> ReferralAnswer:
|
||||
"""Ask, and be referred. The memories read here belong to somebody, so the token decides whose."""
|
||||
try:
|
||||
return TransactiveService(settings).ask(
|
||||
request.question,
|
||||
as_holder=request.as_holder,
|
||||
as_holder=read_as(resolve_caller(http, settings), request.as_holder),
|
||||
max_hops=request.max_hops,
|
||||
include_text=request.include_text,
|
||||
)
|
||||
|
||||
@@ -12,8 +12,9 @@ only as a parameter of register().
|
||||
from collections.abc import Callable
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import Depends, FastAPI, HTTPException
|
||||
from fastapi import Depends, FastAPI, HTTPException, Request
|
||||
|
||||
from .api_auth import read_as, resolve_caller
|
||||
from .config import Settings
|
||||
from .db import Database
|
||||
from .memory import MemoryService
|
||||
@@ -21,28 +22,46 @@ from .schemas import ConflictRecord, EvidenceAtom
|
||||
|
||||
|
||||
def register(app: FastAPI, settings_dep: Callable[[], Settings]) -> None:
|
||||
"""Register the tier reads. The holder is named by the caller because the question is about
|
||||
somebody's memory, not about the server's own identity."""
|
||||
"""Register the tier reads. A tier belongs to somebody, so the token decides whose it is: the
|
||||
`holder` parameter narrows the question to a name, and naming a name the token cannot back is
|
||||
refused rather than obeyed."""
|
||||
|
||||
def service(settings: Settings) -> MemoryService:
|
||||
return MemoryService(settings, Database(settings))
|
||||
|
||||
@app.get("/api/memory/working", response_model=list[EvidenceAtom])
|
||||
def working_set(
|
||||
holder: str,
|
||||
session: str,
|
||||
http: Request,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
holder: str | None = None,
|
||||
) -> list[EvidenceAtom]:
|
||||
"""What one session recorded, in the order it recorded it, with no search involved."""
|
||||
who = read_as(resolve_caller(http, settings), holder)
|
||||
if not who:
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail="a working set belongs to somebody; send that holder's token, or the operator token with a holder",
|
||||
)
|
||||
try:
|
||||
return service(settings).working_set(holder, session)
|
||||
return service(settings).working_set(who, session)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||
|
||||
@app.get("/api/memory/conflicts", response_model=list[ConflictRecord])
|
||||
def conflicts(
|
||||
http: Request,
|
||||
settings: Annotated[Settings, Depends(settings_dep)],
|
||||
holder: str | None = None,
|
||||
) -> list[ConflictRecord]:
|
||||
"""Every disagreement the store noticed and nobody has settled."""
|
||||
return service(settings).conflicts(holder)
|
||||
who = read_as(resolve_caller(http, settings), holder)
|
||||
if who == "":
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
"a disagreement belongs to somebody; send that holder's token, "
|
||||
"or the operator token with a holder"
|
||||
),
|
||||
)
|
||||
return service(settings).conflicts(who)
|
||||
@@ -11,10 +11,12 @@ from .retrieval import decode_cursor, encode_cursor
|
||||
from .schemas import AtomPage, Modality
|
||||
from .storage import ObjectStore
|
||||
|
||||
# The visibility rule is spelled out in four literal statements — two here, one in store_search, one in vector_index — because the SQL gate rejects a concatenated statement and a view cannot take a parameter; backend/tests/test_visibility_rule.py fails if the copies disagree.
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Picture:
|
||||
"""The bytes behind an atom and how to label them when serving."""
|
||||
"""The stored bytes behind an atom and how to label them when serving."""
|
||||
|
||||
path: Path
|
||||
mime: str
|
||||
@@ -28,22 +30,48 @@ def browse(
|
||||
modality: Modality | None = None,
|
||||
source_id: str | None = None,
|
||||
holder: str | None = None,
|
||||
as_holder: str | None = None,
|
||||
) -> AtomPage:
|
||||
"""One page plus the total, so a list can say what it left out instead of looking endless."""
|
||||
"""One page plus the total, so a list can say what it left out instead of looking endless.
|
||||
|
||||
`holder` narrows the shelf to one holder's memories; `as_holder` is whose eyes the page is
|
||||
read through: a holder id, the empty string for a caller who is nobody (only what nobody owns
|
||||
and what is marked for everyone), or None to read the whole library as the operator does.
|
||||
"""
|
||||
offset = decode_cursor(cursor)
|
||||
wanted = (modality.value if modality else None, source_id, holder)
|
||||
filters = tuple(value for bound in wanted for value in (bound, bound))
|
||||
eyes = (as_holder, as_holder, as_holder, as_holder)
|
||||
database = Database(settings)
|
||||
with database.connect() as connection:
|
||||
total = int(
|
||||
connection.execute(
|
||||
"SELECT COUNT(*) FROM v_atoms_current WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?)",
|
||||
filters,
|
||||
"SELECT COUNT(*) FROM v_atoms_current"
|
||||
" WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?)"
|
||||
" AND (? IS NULL"
|
||||
" OR holder_id = ''"
|
||||
" OR json_extract(visibility_json, '$.scope') = 'everyone'"
|
||||
" OR (? <> '' AND (holder_id = ?"
|
||||
" OR EXISTS ("
|
||||
" SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders'))"
|
||||
" WHERE value = ?"
|
||||
" ))))",
|
||||
(*filters, *eyes),
|
||||
).fetchone()[0]
|
||||
)
|
||||
rows = connection.execute(
|
||||
"SELECT * FROM v_atoms_current WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?) ORDER BY ingested_at DESC, atom_id DESC LIMIT ? OFFSET ?",
|
||||
(*filters, limit + 1, offset),
|
||||
"SELECT * FROM v_atoms_current"
|
||||
" WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?)"
|
||||
" AND (? IS NULL"
|
||||
" OR holder_id = ''"
|
||||
" OR json_extract(visibility_json, '$.scope') = 'everyone'"
|
||||
" OR (? <> '' AND (holder_id = ?"
|
||||
" OR EXISTS ("
|
||||
" SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders'))"
|
||||
" WHERE value = ?"
|
||||
" ))))"
|
||||
" ORDER BY ingested_at DESC, atom_id DESC LIMIT ? OFFSET ?",
|
||||
(*filters, *eyes, limit + 1, offset),
|
||||
).fetchall()
|
||||
atoms = [row_to_atom(row) for row in rows[:limit]]
|
||||
more = len(rows) > limit or offset + limit < total
|
||||
@@ -54,13 +82,22 @@ def browse(
|
||||
)
|
||||
|
||||
|
||||
def picture(settings: Settings, atom_id: str) -> Picture | None:
|
||||
def picture(settings: Settings, atom_id: str, as_holder: str | None = None) -> Picture | None:
|
||||
"""The stored bytes behind one atom; None means there is no picture to show, for any of the three reasons."""
|
||||
database = Database(settings)
|
||||
with database.connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT image_ref, mime FROM v_atoms_current JOIN sources USING (source_id) WHERE atom_id = ?",
|
||||
(atom_id,),
|
||||
"SELECT image_ref, mime FROM v_atoms_current JOIN sources USING (source_id)"
|
||||
" WHERE atom_id = ?"
|
||||
" AND (? IS NULL"
|
||||
" OR holder_id = ''"
|
||||
" OR json_extract(visibility_json, '$.scope') = 'everyone'"
|
||||
" OR (? <> '' AND (holder_id = ?"
|
||||
" OR EXISTS ("
|
||||
" SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders'))"
|
||||
" WHERE value = ?"
|
||||
" ))))",
|
||||
(atom_id, as_holder, as_holder, as_holder, as_holder),
|
||||
).fetchone()
|
||||
if row is None or not row["image_ref"]:
|
||||
return None
|
||||
|
||||
@@ -17,9 +17,9 @@ class CacheFabric:
|
||||
self.settings = settings or get_settings()
|
||||
self.db = database or Database(self.settings)
|
||||
|
||||
def semantic_key(self, question: str, profile: str, store_revision: int) -> str:
|
||||
"""A compiled context is only true for the corpus it was compiled from, so the corpus is part of its name."""
|
||||
return hashlib.sha256(f"compiler-v2\n{profile}\n{store_revision}\n{question}".encode()).hexdigest()
|
||||
def semantic_key(self, question: str, profile: str, store_revision: int, holder: str = "") -> str:
|
||||
"""A compiled context is only true for the corpus it was compiled from and the eyes that were allowed to see it, so both belong in its name."""
|
||||
return hashlib.sha256(f"compiler-v2\n{profile}\n{store_revision}\n{holder}\n{question}".encode()).hexdigest()
|
||||
|
||||
def record_hit(self, layer: str, cache_key: str, bytes_estimate: int = 0, metadata: dict[str, Any] | None = None) -> None:
|
||||
self.db.log_cache_event(layer, cache_key, "hit", bytes_estimate, metadata)
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
"""What this store can answer, read back from what it actually holds.
|
||||
|
||||
A caller deciding whether to send its questions here needs three facts it cannot guess: who is in
|
||||
the directory and what they claim to know, whether the memory it would get covers pictures and
|
||||
speech or only words, and which schema the answers were produced under. Every field is read from
|
||||
the store, so the card cannot promise a sense the store has no atoms for, and it cannot promise a
|
||||
capacity a mount has not carried.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from .card_schemas import CapabilityCard, HolderCard
|
||||
from .cartridge import CartridgeService
|
||||
from .config import Settings
|
||||
from .db import Database
|
||||
from .holder_store import HolderStore
|
||||
|
||||
|
||||
def build_card(settings: Settings) -> CapabilityCard:
|
||||
"""The store-level card: schema, counts, senses present, and what is mounted read-only."""
|
||||
|
||||
database = Database(settings)
|
||||
with database.connect() as connection:
|
||||
schema_version = int(connection.execute("PRAGMA user_version").fetchone()[0])
|
||||
sources = int(connection.execute("SELECT COUNT(*) FROM sources").fetchone()[0])
|
||||
senses = {
|
||||
str(row["modality"]): int(row["n"])
|
||||
for row in connection.execute(
|
||||
"SELECT modality, COUNT(*) AS n FROM v_atoms_current GROUP BY modality ORDER BY n DESC"
|
||||
)
|
||||
}
|
||||
per_holder = {
|
||||
str(row["holder_id"]): (int(row["n"]), str(row["newest"]))
|
||||
for row in connection.execute(
|
||||
"SELECT holder_id, COUNT(*) AS n, MAX(ingested_at) AS newest FROM v_atoms_current"
|
||||
" GROUP BY holder_id"
|
||||
)
|
||||
}
|
||||
senses_by_holder: dict[str, dict[str, int]] = {}
|
||||
scenarios_by_holder: dict[str, list[str]] = {}
|
||||
for row in connection.execute(
|
||||
"SELECT holder_id, modality, COUNT(*) AS n FROM v_atoms_current"
|
||||
" GROUP BY holder_id, modality ORDER BY n DESC"
|
||||
):
|
||||
senses_by_holder.setdefault(str(row["holder_id"]), {})[str(row["modality"])] = int(row["n"])
|
||||
for row in connection.execute(
|
||||
"SELECT holder_id, scenario, COUNT(*) AS n FROM v_atoms_current"
|
||||
" GROUP BY holder_id, scenario HAVING n > 0 ORDER BY n DESC"
|
||||
):
|
||||
scenarios_by_holder.setdefault(str(row["holder_id"]), []).append(str(row["scenario"]))
|
||||
|
||||
directory = {entry.holder_id: entry for entry in HolderStore(database).list_holders()}
|
||||
cards: dict[str, HolderCard] = {}
|
||||
for holder_id, (memories, newest) in per_holder.items():
|
||||
entry = directory.get(holder_id)
|
||||
cards[holder_id] = HolderCard(
|
||||
holder_id=holder_id,
|
||||
name=entry.name if entry else "",
|
||||
kind=entry.kind if entry else "person",
|
||||
knows_about=list(entry.knows_about) if entry else [],
|
||||
contact=dict(entry.contact) if entry else {},
|
||||
memories=memories,
|
||||
modalities=senses_by_holder.get(holder_id, {}),
|
||||
scenarios=scenarios_by_holder.get(holder_id, []),
|
||||
newest_memory=datetime.fromisoformat(newest),
|
||||
registered=entry is not None,
|
||||
token_issued=entry is not None and entry.token_issued_at is not None,
|
||||
)
|
||||
for holder_id, entry in directory.items():
|
||||
if holder_id not in cards:
|
||||
cards[holder_id] = HolderCard(
|
||||
holder_id=holder_id,
|
||||
name=entry.name,
|
||||
kind=entry.kind,
|
||||
knows_about=list(entry.knows_about),
|
||||
contact=dict(entry.contact),
|
||||
newest_memory=None,
|
||||
registered=True,
|
||||
token_issued=entry.token_issued_at is not None,
|
||||
)
|
||||
|
||||
return CapabilityCard(
|
||||
schema_version=schema_version,
|
||||
atoms=database.atom_count(),
|
||||
sources=sources,
|
||||
modalities=senses,
|
||||
has_audio="audio" in senses,
|
||||
holders=[cards[key] for key in sorted(cards)],
|
||||
mounts=CartridgeService(settings).mount_state(),
|
||||
)
|
||||
@@ -0,0 +1,58 @@
|
||||
"""The capability card: what this store can answer, stated as data.
|
||||
|
||||
Held apart from the memory schemas because the card is a promise made to a stranger — it is served
|
||||
without a token — while the memory schemas describe what a holder's own memories look like. The
|
||||
two change for different reasons, so they live in different files.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from .schemas import MountState, utc_now
|
||||
from .transactive_schemas import HolderKind
|
||||
|
||||
|
||||
class HolderCard(BaseModel):
|
||||
"""One directory entry, plus the evidence that it is a holder rather than a typed-in name.
|
||||
|
||||
`memories` and `newest_memory` are counted from the store, so a holder who declared ten
|
||||
topics and recorded nothing shows as such. `token_issued` says whether anything can prove it
|
||||
is this holder; `registered` says whether the directory knows it at all, which is false for a
|
||||
holder_id that only ever appeared on a memory.
|
||||
"""
|
||||
|
||||
holder_id: str
|
||||
name: str = ""
|
||||
kind: HolderKind = "person"
|
||||
knows_about: list[str] = Field(default_factory=list)
|
||||
contact: dict[str, str] = Field(default_factory=dict)
|
||||
memories: int = Field(ge=0, default=0)
|
||||
modalities: dict[str, int] = Field(default_factory=dict)
|
||||
scenarios: list[str] = Field(default_factory=list)
|
||||
newest_memory: datetime | None = None
|
||||
registered: bool = False
|
||||
token_issued: bool = False
|
||||
|
||||
|
||||
class CapabilityCard(BaseModel):
|
||||
"""What a caller can expect from this store before it sends anything into it.
|
||||
|
||||
`has_audio` exists because a store that ingested only pictures and PDFs answers questions
|
||||
about speech with OCR of a frame, and the caller should learn that before it relies on it.
|
||||
`schema_version` is read from the open database, not from the build constant, so a store that
|
||||
has not been migrated yet reports what it actually is. The card is served without a token: it
|
||||
names holders, counts their memories and lists the scenario names they used, but no memory
|
||||
text — a caller learns what this store can answer, not what it remembers.
|
||||
"""
|
||||
|
||||
schema_version: int
|
||||
atoms: int = Field(ge=0)
|
||||
sources: int = Field(ge=0)
|
||||
modalities: dict[str, int] = Field(default_factory=dict)
|
||||
has_audio: bool
|
||||
holders: list[HolderCard] = Field(default_factory=list)
|
||||
mounts: MountState
|
||||
generated_at: datetime = Field(default_factory=utc_now)
|
||||
@@ -10,12 +10,13 @@ from collections.abc import Callable
|
||||
|
||||
import typer
|
||||
|
||||
from .card import build_card
|
||||
from .config import Settings
|
||||
from .db import Database
|
||||
from .holder_store import HolderStore
|
||||
from .schemas import utc_now
|
||||
from .transactive import TransactiveService
|
||||
from .transactive_schemas import Holder, HolderKind
|
||||
from .transactive_schemas import Holder, HolderKind, IssuedToken
|
||||
|
||||
|
||||
def register(app: typer.Typer, settings: Callable[[], Settings]) -> None:
|
||||
@@ -62,6 +63,25 @@ def register(app: typer.Typer, settings: Callable[[], Settings]) -> None:
|
||||
store().delete(holder_id)
|
||||
typer.echo(f'{{"deleted": "{holder_id}"}}')
|
||||
|
||||
@app.command("holder-issue")
|
||||
def holder_issue(holder_id: str) -> None:
|
||||
"""Mint a bearer token for one holder. The value appears once: only its hash is stored."""
|
||||
entry, token = store().issue_token(holder_id)
|
||||
if entry.token_issued_at is None:
|
||||
raise ValueError(f"holder {holder_id!r} holds a token but no issue time")
|
||||
issued = IssuedToken(holder_id=entry.holder_id, token=token, issued_at=entry.token_issued_at)
|
||||
typer.echo(issued.model_dump_json(indent=2))
|
||||
|
||||
@app.command("holder-revoke")
|
||||
def holder_revoke(holder_id: str) -> None:
|
||||
"""Drop a holder's token. The entry stays; nobody can read as them until a new one is issued."""
|
||||
typer.echo(store().revoke_token(holder_id).model_dump_json(indent=2))
|
||||
|
||||
@app.command()
|
||||
def card() -> None:
|
||||
"""What this store can answer, read back from what it holds: holders, senses, schema, mounts."""
|
||||
typer.echo(build_card(settings()).model_dump_json(indent=2))
|
||||
|
||||
@app.command("who-knows")
|
||||
def who_knows(query: str, as_holder: str = "", limit: int = 8) -> None:
|
||||
"""Rank the holders whose memories answer this query."""
|
||||
|
||||
@@ -8,7 +8,14 @@ from .cache_fabric import SEMANTIC_LAYER, CacheFabric
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .retrieval import RetrievalService
|
||||
from .schemas import CompiledContext, ContextPlan, Modality, RetrievalHit, RuntimeProfile
|
||||
from .schemas import (
|
||||
CompiledContext,
|
||||
ContextPlan,
|
||||
Modality,
|
||||
RetrievalHit,
|
||||
RuntimeProfile,
|
||||
SearchFilters,
|
||||
)
|
||||
|
||||
PROFILE_TOKEN_BUDGETS = {
|
||||
RuntimeProfile.live_demo: 128_000,
|
||||
@@ -25,8 +32,18 @@ class ContextCompiler:
|
||||
self.retrieval = RetrievalService(self.settings, self.db)
|
||||
self.cache = CacheFabric(self.settings, self.db)
|
||||
|
||||
def compile(self, question: str, profile: RuntimeProfile) -> CompiledContext:
|
||||
cache_key = self.cache.semantic_key(question, profile.value, self.db.revision())
|
||||
def compile(self, question: str, profile: RuntimeProfile, as_holder: str | None = None) -> CompiledContext:
|
||||
"""Build the context one question gets, seen through one holder's eyes.
|
||||
|
||||
`as_holder` is passed to retrieval and folded into the cache key: a context compiled for
|
||||
a holder carries that holder's own memories, and serving it to somebody else would leak.
|
||||
"""
|
||||
cache_key = self.cache.semantic_key(
|
||||
question,
|
||||
profile.value,
|
||||
self.db.revision(),
|
||||
"the whole library" if as_holder is None else f"as holder {as_holder!r}",
|
||||
)
|
||||
cached = self.cache.read_semantic_cache(cache_key)
|
||||
if cached is not None:
|
||||
compiled = CompiledContext.model_validate(cached)
|
||||
@@ -40,7 +57,9 @@ class ContextCompiler:
|
||||
else:
|
||||
self.cache.record_miss(SEMANTIC_LAYER, cache_key)
|
||||
plan = self.plan(question, profile)
|
||||
hits = self.retrieval.search(plan.retrieval_query, plan.evidence_budget_atoms).hits
|
||||
hits = self.retrieval.search(
|
||||
plan.retrieval_query, plan.evidence_budget_atoms, filters=SearchFilters(as_holder=as_holder)
|
||||
).hits
|
||||
citable, held = self._split_citable(hits)
|
||||
packed_text, token_total = self._pack_hits(citable, plan.token_budget, plan.retrieval_query)
|
||||
trace_id = hashlib.sha256(f"{cache_key}:{token_total}".encode()).hexdigest()[:16]
|
||||
|
||||
@@ -8,6 +8,8 @@ never a guess.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import secrets
|
||||
import sqlite3
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
@@ -30,6 +32,9 @@ class WhoKnowsCount:
|
||||
latest_time: str | None
|
||||
|
||||
|
||||
_TOKEN_BYTES = 32
|
||||
|
||||
|
||||
class HolderStore:
|
||||
"""CRUD for registered holders plus the raw counts who_knows is built from."""
|
||||
|
||||
@@ -68,6 +73,63 @@ class HolderStore:
|
||||
)
|
||||
return stored
|
||||
|
||||
def register_minimal(self, holder_id: str) -> None:
|
||||
"""Make a holder exist without overwriting what somebody already declared about them.
|
||||
|
||||
A memory nobody registered still needs a directory entry, or a referral to its holder has
|
||||
no target to point at.
|
||||
"""
|
||||
now = utc_now().isoformat()
|
||||
with self.database.connect() as connection:
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT INTO holders (holder_id, name, kind, knows_about_json, contact_json, created_at, updated_at)
|
||||
VALUES (?, '', 'person', '[]', '{}', ?, ?)
|
||||
ON CONFLICT(holder_id) DO NOTHING
|
||||
""",
|
||||
(holder_id, now, now),
|
||||
)
|
||||
|
||||
def issue_token(self, holder_id: str) -> tuple[Holder, str]:
|
||||
"""Give a holder a fresh bearer token. The plain value is returned once; the row keeps its hash."""
|
||||
self.register_minimal(holder_id)
|
||||
token = secrets.token_urlsafe(_TOKEN_BYTES)
|
||||
now = utc_now().isoformat()
|
||||
with self.database.connect() as connection:
|
||||
connection.execute(
|
||||
"UPDATE holders SET token_hash = ?, token_issued_at = ?, updated_at = ? WHERE holder_id = ?",
|
||||
(_token_hash(token), now, now, holder_id),
|
||||
)
|
||||
entry = self.get(holder_id)
|
||||
if entry is None:
|
||||
raise ValueError(f"holder {holder_id!r} vanished while its token was being issued")
|
||||
return entry, token
|
||||
|
||||
def revoke_token(self, holder_id: str) -> Holder:
|
||||
"""Drop a holder's token: nobody can read as them until a new one is issued."""
|
||||
now = utc_now().isoformat()
|
||||
with self.database.connect() as connection:
|
||||
changed = connection.execute(
|
||||
"UPDATE holders SET token_hash = NULL, token_issued_at = NULL, updated_at = ? WHERE holder_id = ?",
|
||||
(now, holder_id),
|
||||
)
|
||||
if changed.rowcount == 0:
|
||||
raise ValueError(f"no holder {holder_id!r} is registered")
|
||||
entry = self.get(holder_id)
|
||||
if entry is None:
|
||||
raise ValueError(f"holder {holder_id!r} vanished while its token was being revoked")
|
||||
return entry
|
||||
|
||||
def verify_token(self, token: str) -> str | None:
|
||||
"""Which holder a bearer token speaks for, or None when it speaks for nobody."""
|
||||
if not token:
|
||||
return None
|
||||
with self.database.connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT holder_id FROM holders WHERE token_hash = ?", (_token_hash(token),)
|
||||
).fetchone()
|
||||
return None if row is None else str(row["holder_id"])
|
||||
|
||||
def get(self, holder_id: str) -> Holder | None:
|
||||
with self.database.connect() as connection:
|
||||
row = connection.execute("SELECT * FROM holders WHERE holder_id = ?", (holder_id,)).fetchone()
|
||||
@@ -145,6 +207,10 @@ class HolderStore:
|
||||
]
|
||||
|
||||
|
||||
def _token_hash(token: str) -> str:
|
||||
return hashlib.sha256(token.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _moment(value: str) -> datetime:
|
||||
parsed = datetime.fromisoformat(value)
|
||||
if parsed.tzinfo is None:
|
||||
@@ -168,6 +234,7 @@ def _row_to_holder(row: sqlite3.Row) -> Holder:
|
||||
kind=_kind(str(row["kind"])),
|
||||
knows_about=from_json(row["knows_about_json"], []),
|
||||
contact=from_json(row["contact_json"], {}),
|
||||
token_issued_at=None if row["token_issued_at"] is None else _moment(str(row["token_issued_at"])),
|
||||
created_at=_moment(str(row["created_at"])),
|
||||
updated_at=_moment(str(row["updated_at"])),
|
||||
)
|
||||
@@ -17,6 +17,7 @@ from . import lifecycle_store, trust
|
||||
from .cache_fabric import CacheFabric
|
||||
from .config import Settings, get_settings
|
||||
from .db import Database
|
||||
from .holder_store import HolderStore
|
||||
from .schemas import (
|
||||
ArchiveReport,
|
||||
ConflictRecord,
|
||||
@@ -75,7 +76,8 @@ class MemoryService:
|
||||
"""Store one statement as a memory owned by a holder.
|
||||
|
||||
Repeating the same sentence as the same holder updates the memory instead of
|
||||
duplicating it; two holders holding the same sentence keep two memories.
|
||||
duplicating it; two holders holding the same sentence keep two memories. A holder nobody
|
||||
registered is added to the directory here, so a referral to it has something to point at.
|
||||
"""
|
||||
statement = text.strip()
|
||||
if not statement:
|
||||
@@ -89,6 +91,7 @@ class MemoryService:
|
||||
)
|
||||
|
||||
moment = utc_now()
|
||||
HolderStore(self.db).register_minimal(holder_id)
|
||||
digest = hashlib.sha256(f"{holder_id}\n{statement}".encode()).hexdigest()
|
||||
source_id = f"mem-{digest[:16]}"
|
||||
self.db.upsert_sources(
|
||||
|
||||
@@ -16,7 +16,7 @@ import sqlite3
|
||||
from .storage import token_equivalent, visual_token_equivalent
|
||||
from .text_index import index_text
|
||||
|
||||
SCHEMA_VERSION = 10
|
||||
SCHEMA_VERSION = 11
|
||||
|
||||
ATOM_COLUMNS_V2 = """
|
||||
atom_id TEXT PRIMARY KEY,
|
||||
@@ -96,6 +96,8 @@ _BASE_TABLES = """
|
||||
CHECK (kind IN ('person', 'agent', 'service')),
|
||||
knows_about_json TEXT NOT NULL DEFAULT '[]',
|
||||
contact_json TEXT NOT NULL DEFAULT '{}',
|
||||
token_hash TEXT,
|
||||
token_issued_at TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
@@ -290,6 +292,13 @@ def _recount_visual_tokens(connection: sqlite3.Connection) -> None:
|
||||
connection.executemany("UPDATE atoms SET token_equivalent = ? WHERE atom_id = ?", updates)
|
||||
|
||||
|
||||
_NEW_HOLDER_COLUMNS: tuple[tuple[str, str], ...] = (
|
||||
("token_hash", "TEXT"),
|
||||
("token_issued_at", "TEXT"),
|
||||
)
|
||||
"""A holder proves its identity with a bearer token; only its hash is kept, so a copied database cannot replay it."""
|
||||
|
||||
|
||||
def apply(connection: sqlite3.Connection) -> None:
|
||||
"""Bring the schema to SCHEMA_VERSION, in place, without losing rows."""
|
||||
version = int(connection.execute("PRAGMA user_version").fetchone()[0])
|
||||
@@ -314,6 +323,11 @@ def apply(connection: sqlite3.Connection) -> None:
|
||||
for name, definition in columns:
|
||||
if name not in existing:
|
||||
connection.execute(f"ALTER TABLE atoms ADD COLUMN {name} {definition}")
|
||||
if version < 11:
|
||||
existing_holders = _column_names(connection, "holders")
|
||||
for name, definition in _NEW_HOLDER_COLUMNS:
|
||||
if name not in existing_holders:
|
||||
connection.execute(f"ALTER TABLE holders ADD COLUMN {name} {definition}")
|
||||
if version < 10:
|
||||
# CREATE VIEW IF NOT EXISTS keeps the old body, and v10 changed which rows are current, so an upgraded store must be told to look again.
|
||||
connection.execute("DROP VIEW IF EXISTS v_atoms_current")
|
||||
|
||||
@@ -46,10 +46,11 @@ def search_atoms(
|
||||
depends on string-built SQL, and the planner sees one shape. A memory with no event time
|
||||
cannot answer a time filter, so it drops out when one is asked for.
|
||||
|
||||
`as_holder` applies the sharing rule: a memory is readable by its owner, by anyone named
|
||||
in its visible_to list, by everyone when it is public, and by anyone when nobody owns it.
|
||||
A memory with no holder cannot be private to a holder that does not exist. Without an
|
||||
asker the query runs over the whole library, which is what an operator debugging wants."""
|
||||
`as_holder` applies the sharing rule in three states: a holder id sees its own memories, the
|
||||
ones naming it in visible_to, the ones marked for everyone, and the ones nobody owns; the
|
||||
empty string is a caller who is nobody, so only the last two; None runs over the whole
|
||||
library, which is what an operator or a process that owns the store reads with. A memory with
|
||||
no holder cannot be private to a holder that does not exist."""
|
||||
with database.connect() as connection:
|
||||
expression = fts_expression(rare_terms(connection, fts_terms(query)))
|
||||
rows = connection.execute(
|
||||
@@ -77,13 +78,14 @@ def search_atoms(
|
||||
WHERE ca.atom_id = f.atom_id
|
||||
AND ca.cartridge_id IN (SELECT value FROM json_each(?))
|
||||
))
|
||||
AND (? IS NULL OR a.holder_id = ''
|
||||
AND (? IS NULL
|
||||
OR a.holder_id = ''
|
||||
OR json_extract(a.visibility_json, '$.scope') = 'everyone'
|
||||
OR a.holder_id = ?
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM json_each(json_extract(a.visibility_json, '$.holders'))
|
||||
WHERE value = ?
|
||||
))
|
||||
OR (? <> '' AND (a.holder_id = ?
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM json_each(json_extract(a.visibility_json, '$.holders'))
|
||||
WHERE value = ?
|
||||
))))
|
||||
ORDER BY score
|
||||
LIMIT ? OFFSET ?
|
||||
""",
|
||||
@@ -108,6 +110,7 @@ def search_atoms(
|
||||
as_holder,
|
||||
as_holder,
|
||||
as_holder,
|
||||
as_holder,
|
||||
limit + 1,
|
||||
offset,
|
||||
),
|
||||
|
||||
@@ -24,6 +24,8 @@ class Holder(BaseModel):
|
||||
kind: HolderKind = "person"
|
||||
knows_about: list[str] = Field(default_factory=list)
|
||||
contact: dict[str, str] = Field(default_factory=dict)
|
||||
token_issued_at: datetime | None = None
|
||||
"""When a bearer token was last issued here. The hash itself stays out of the answer."""
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
@@ -35,6 +37,18 @@ class Holder(BaseModel):
|
||||
return value.strip()
|
||||
|
||||
|
||||
class IssuedToken(BaseModel):
|
||||
"""A holder's bearer token, shown once at the moment it is minted.
|
||||
|
||||
Only the hash is stored, so this answer is the only place the value will ever appear: a lost
|
||||
token is replaced, not recovered.
|
||||
"""
|
||||
|
||||
holder_id: str
|
||||
token: str
|
||||
issued_at: datetime
|
||||
|
||||
|
||||
class HolderUpsertRequest(BaseModel):
|
||||
"""What a caller may set about a holder. The id comes from the path, never the body."""
|
||||
|
||||
|
||||
@@ -225,7 +225,9 @@ class VectorIndex:
|
||||
question with a full scan (77ms over 153,857 atoms) while the atoms table answers it
|
||||
through the atom_id index in 0.2ms, and a search cannot afford the scan. The scope is
|
||||
checked here rather than in the engine so the rule lives in one place; the engine's id
|
||||
list is only a narrowing.
|
||||
list is only a narrowing. Visibility is here too, and it reads the same three states the
|
||||
words leg reads: a holder id, the empty string for a caller who is nobody (unowned rows
|
||||
and rows marked for everyone, no more), and None for an operator reading the whole library.
|
||||
"""
|
||||
rows = connection.execute(
|
||||
"""
|
||||
@@ -246,13 +248,14 @@ class VectorIndex:
|
||||
WHERE ca.atom_id = atoms.atom_id
|
||||
AND ca.cartridge_id IN (SELECT value FROM json_each(?))
|
||||
))
|
||||
AND (? IS NULL OR holder_id = ''
|
||||
AND (? IS NULL
|
||||
OR holder_id = ''
|
||||
OR json_extract(visibility_json, '$.scope') = 'everyone'
|
||||
OR holder_id = ?
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders'))
|
||||
WHERE value = ?
|
||||
))
|
||||
OR (? <> '' AND (holder_id = ?
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM json_each(json_extract(visibility_json, '$.holders'))
|
||||
WHERE value = ?
|
||||
))))
|
||||
""",
|
||||
(
|
||||
to_json(atom_ids),
|
||||
@@ -273,6 +276,7 @@ class VectorIndex:
|
||||
as_holder,
|
||||
as_holder,
|
||||
as_holder,
|
||||
as_holder,
|
||||
),
|
||||
).fetchall()
|
||||
return {str(row["atom_id"]) for row in rows}
|
||||
|
||||
@@ -51,7 +51,8 @@ class Store:
|
||||
return self.cartridges.materialize(name, atoms, sources).cartridge_id
|
||||
|
||||
def texts(self, query: str = "钥匙", **filters: Any) -> list[str]:
|
||||
result = self.search.search(query, filters=SearchFilters(**filters) if filters else None)
|
||||
eyes: dict[str, Any] = {"as_holder": "wife", **filters}
|
||||
result = self.search.search(query, filters=SearchFilters(**eyes))
|
||||
return [hit.atom.text for hit in result.hits]
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,268 @@
|
||||
"""The sharing rule over HTTP: a token says who is asking, and the store answers to that person."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
import ely_eye.api as api_module
|
||||
from ely_eye.compiler import ContextCompiler
|
||||
from ely_eye.config import Settings
|
||||
from ely_eye.db import Database
|
||||
from ely_eye.holder_store import HolderStore
|
||||
from ely_eye.memory import MemoryService
|
||||
from ely_eye.schemas import (
|
||||
EvidenceAtom,
|
||||
MemoryOrigin,
|
||||
Modality,
|
||||
RuntimeProfile,
|
||||
SourceRecord,
|
||||
TrustScores,
|
||||
Visibility,
|
||||
)
|
||||
from ely_eye.storage import ObjectStore
|
||||
|
||||
OPERATOR = "the-operator-token"
|
||||
AS_OPERATOR = {"Authorization": f"Bearer {OPERATOR}"}
|
||||
PNG = b"\x89PNG\r\n\x1a\n"
|
||||
KEYS = "我们的钥匙放在玄关抽屉里"
|
||||
THERMOSTAT = "恒温器设在二十二度"
|
||||
OUTAGE = "上周三的停电持续了四十分钟"
|
||||
|
||||
|
||||
def _settings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Settings:
|
||||
resolved = Settings(home=tmp_path, default_holder="wife", auth_token=OPERATOR)
|
||||
monkeypatch.setattr(api_module, "get_settings", lambda: resolved)
|
||||
Database(resolved).init()
|
||||
return resolved
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> TestClient:
|
||||
return TestClient(api_module.create_app())
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def settings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Settings:
|
||||
return _settings(tmp_path, monkeypatch)
|
||||
|
||||
|
||||
def _token_for(client: TestClient, holder_id: str) -> dict[str, Any]:
|
||||
answer = client.post(f"/api/holders/{holder_id}/token", headers=AS_OPERATOR)
|
||||
assert answer.status_code == 200, answer.json()
|
||||
return dict(answer.json())
|
||||
|
||||
|
||||
def _remember(settings: Settings, text: str, holder_id: str, **kwargs: Any) -> EvidenceAtom:
|
||||
return MemoryService(settings, Database(settings)).remember(
|
||||
text, holder_id=holder_id, origin=MemoryOrigin.self_, **kwargs
|
||||
)
|
||||
|
||||
|
||||
def _hits(client: TestClient, query: str, headers: dict[str, str] | None = None, **body: Any) -> list[str]:
|
||||
answer = client.post("/api/search", json={"query": query, **body}, headers=headers or {})
|
||||
assert answer.status_code == 200, answer.json()
|
||||
return [str(hit["atom"]["atom_id"]) for hit in answer.json()["hits"]]
|
||||
|
||||
|
||||
def test_only_the_operator_mints_a_token(client: TestClient, settings: Settings) -> None:
|
||||
_remember(settings, KEYS, "wife")
|
||||
wife = _token_for(client, "wife")
|
||||
assert client.post("/api/holders/husband/token").status_code == 401, "no token cannot mint one"
|
||||
refused = client.post("/api/holders/husband/token", headers={"Authorization": f"Bearer {wife['token']}"})
|
||||
assert refused.status_code == 401, "a holder token is not an operator token"
|
||||
assert "Bearer" in refused.json()["detail"]
|
||||
|
||||
|
||||
def test_the_token_is_shown_once_and_stored_as_a_hash(client: TestClient, settings: Settings) -> None:
|
||||
_remember(settings, KEYS, "wife")
|
||||
issued = _token_for(client, "wife")
|
||||
token = str(issued["token"])
|
||||
assert issued["holder_id"] == "wife" and issued["issued_at"]
|
||||
|
||||
with Database(settings).connect() as connection:
|
||||
row = connection.execute("SELECT token_hash FROM holders WHERE holder_id = 'wife'").fetchone()
|
||||
stored = str(row["token_hash"])
|
||||
assert stored == hashlib.sha256(token.encode("utf-8")).hexdigest()
|
||||
assert token not in stored and len(stored) == 64, "a copied database must not replay the token"
|
||||
|
||||
listed = client.get("/api/holders/wife", headers=AS_OPERATOR).json()
|
||||
assert "token" not in listed and listed["token_issued_at"] == issued["issued_at"]
|
||||
|
||||
|
||||
def test_a_holder_token_reads_as_that_holder_and_nobody_else(client: TestClient, settings: Settings) -> None:
|
||||
mine = _remember(settings, KEYS, "wife")
|
||||
wife = _token_for(client, "wife")
|
||||
_token_for(client, "husband")
|
||||
|
||||
as_wife = _hits(client, KEYS, headers={"Authorization": f"Bearer {wife['token']}"})
|
||||
as_husband = _hits(client, KEYS, headers={"Authorization": f"Bearer {wife['token']}"})
|
||||
assert mine.atom_id in as_wife
|
||||
assert as_husband == as_wife, "the same token reads the same way whatever the query says"
|
||||
|
||||
anonymous = _hits(client, KEYS)
|
||||
assert mine.atom_id not in anonymous, "a caller with no token is nobody"
|
||||
|
||||
|
||||
def test_an_unknown_token_is_refused_rather_than_treated_as_nobody(client: TestClient, settings: Settings) -> None:
|
||||
_remember(settings, KEYS, "wife")
|
||||
refused = client.post("/api/search", json={"query": KEYS}, headers={"Authorization": "Bearer not-a-token"})
|
||||
assert refused.status_code == 401
|
||||
detail = str(refused.json()["detail"])
|
||||
assert "operator" in detail and "holder" in detail, "the refusal has to say which two things failed"
|
||||
|
||||
|
||||
def test_a_public_memory_is_findable_by_anybody(client: TestClient, settings: Settings) -> None:
|
||||
shared = _remember(settings, OUTAGE, "wife", visibility=Visibility(scope="everyone"))
|
||||
assert shared.atom_id in _hits(client, OUTAGE), "no token, still visible"
|
||||
|
||||
|
||||
def test_a_shared_memory_is_readable_only_by_the_holders_it_names(client: TestClient, settings: Settings) -> None:
|
||||
private = _remember(settings, THERMOSTAT, "wife", visibility=Visibility(scope="visible_to", holders=["husband"]))
|
||||
_remember(settings, KEYS, "husband")
|
||||
wife = _token_for(client, "wife")
|
||||
husband = _token_for(client, "husband")
|
||||
|
||||
assert private.atom_id in _hits(client, THERMOSTAT, headers={"Authorization": f"Bearer {husband['token']}"})
|
||||
assert private.atom_id in _hits(client, THERMOSTAT, headers={"Authorization": f"Bearer {wife['token']}"})
|
||||
assert private.atom_id not in _hits(client, THERMOSTAT)
|
||||
|
||||
outsider = _remember(settings, OUTAGE, "neighbour")
|
||||
_token_for(client, "neighbour")
|
||||
neighbour = _token_for(client, "neighbour")
|
||||
assert outsider.atom_id
|
||||
assert private.atom_id not in _hits(client, THERMOSTAT, headers={"Authorization": f"Bearer {neighbour['token']}"})
|
||||
|
||||
|
||||
def test_naming_another_holder_is_a_claim_the_token_has_to_back(client: TestClient, settings: Settings) -> None:
|
||||
mine = _remember(settings, KEYS, "wife")
|
||||
wife = _token_for(client, "wife")
|
||||
_token_for(client, "husband")
|
||||
as_wife = {"Authorization": f"Bearer {wife['token']}"}
|
||||
|
||||
over = client.post("/api/search", json={"query": KEYS, "filters": {"as_holder": "husband"}}, headers=as_wife)
|
||||
assert over.status_code == 403
|
||||
detail = str(over.json()["detail"])
|
||||
assert "wife" in detail and "husband" in detail, "a refusal that names neither side cannot be acted on"
|
||||
|
||||
anonymous = client.post("/api/search", json={"query": KEYS, "filters": {"as_holder": "wife"}})
|
||||
assert anonymous.status_code == 403, "typing a name is not a token"
|
||||
|
||||
operator = client.post("/api/search", json={"query": KEYS, "filters": {"as_holder": "wife"}}, headers=AS_OPERATOR)
|
||||
assert operator.status_code == 200
|
||||
assert mine.atom_id in [str(hit["atom"]["atom_id"]) for hit in operator.json()["hits"]]
|
||||
|
||||
|
||||
def test_revoking_a_token_stops_it_working(client: TestClient, settings: Settings) -> None:
|
||||
_remember(settings, KEYS, "wife")
|
||||
token = _token_for(client, "wife")["token"]
|
||||
assert HolderStore(Database(settings)).verify_token(str(token)) == "wife"
|
||||
|
||||
revoked = client.delete("/api/holders/wife/token", headers=AS_OPERATOR)
|
||||
assert revoked.status_code == 200 and revoked.json()["holder_id"] == "wife"
|
||||
assert client.post("/api/search", json={"query": KEYS}, headers={"Authorization": f"Bearer {token}"}).status_code == 401
|
||||
assert client.get("/api/holders/wife", headers=AS_OPERATOR).json()["token_issued_at"] is None
|
||||
|
||||
|
||||
def test_a_working_set_refuses_to_belong_to_nobody(client: TestClient, settings: Settings) -> None:
|
||||
_remember(settings, KEYS, "wife", session_id="kitchen-2026")
|
||||
_remember(settings, THERMOSTAT, "husband", session_id="kitchen-2026")
|
||||
|
||||
anonymous = client.get("/api/memory/working", params={"session": "kitchen-2026"})
|
||||
assert anonymous.status_code == 422
|
||||
assert "somebody" in str(anonymous.json()["detail"])
|
||||
|
||||
wife = _token_for(client, "wife")
|
||||
hers = client.get("/api/memory/working", params={"session": "kitchen-2026"}, headers={"Authorization": f"Bearer {wife['token']}"})
|
||||
assert [atom["text"] for atom in hers.json()] == [KEYS], "a session sees its own lane, not the room's"
|
||||
|
||||
both = client.get("/api/memory/working", params={"session": "kitchen-2026", "holder": "husband"}, headers=AS_OPERATOR)
|
||||
assert [atom["text"] for atom in both.json()] == [THERMOSTAT]
|
||||
|
||||
|
||||
def test_a_conflict_list_refuses_to_belong_to_nobody(client: TestClient, settings: Settings) -> None:
|
||||
_remember(settings, "恒温器放在书房，设定 22 度", "wife")
|
||||
_remember(settings, "恒温器调到 24 度", "wife")
|
||||
assert client.get("/api/memory/conflicts").status_code == 422
|
||||
wife = _token_for(client, "wife")
|
||||
hers = client.get("/api/memory/conflicts", headers={"Authorization": f"Bearer {wife['token']}"})
|
||||
assert hers.status_code == 200 and len(hers.json()) == 1
|
||||
|
||||
|
||||
def test_the_card_is_public_names_holders_and_carries_no_memory_text(client: TestClient, settings: Settings) -> None:
|
||||
_remember(settings, KEYS, "wife", scenario="home")
|
||||
_remember(settings, THERMOSTAT, "husband")
|
||||
_token_for(client, "wife")
|
||||
|
||||
answer = client.get("/api/card")
|
||||
assert answer.status_code == 200, "a capability card exists to be read by a stranger"
|
||||
card = answer.json()
|
||||
assert card["schema_version"] >= 11 and card["has_audio"] is False
|
||||
named = {holder["holder_id"]: holder for holder in card["holders"]}
|
||||
assert set(named) == {"wife", "husband"}
|
||||
assert named["wife"]["memories"] == 1 and named["wife"]["token_issued"] is True
|
||||
assert named["husband"]["token_issued"] is False
|
||||
assert named["wife"]["scenarios"] == ["home"] and named["wife"]["modalities"] == {"text": 1}
|
||||
assert named["wife"]["newest_memory"], "a stranger needs to know how stale the memory is"
|
||||
assert KEYS not in answer.text and THERMOSTAT not in answer.text
|
||||
|
||||
one = client.get("/api/card/wife")
|
||||
assert one.status_code == 200 and one.json()["holder_id"] == "wife"
|
||||
assert client.get("/api/card/nobody").status_code == 404
|
||||
|
||||
|
||||
def test_a_picture_is_served_only_to_eyes_that_may_see_it(client: TestClient, settings: Settings) -> None:
|
||||
database = Database(settings)
|
||||
stored = ObjectStore(settings).put_bytes(PNG, ".png", "image")
|
||||
database.upsert_sources(
|
||||
[
|
||||
SourceRecord(
|
||||
source_id=source,
|
||||
path=f"file:///{source}.png",
|
||||
kind="image",
|
||||
mime="image/png",
|
||||
sha256=source * 8,
|
||||
size_bytes=len(PNG),
|
||||
)
|
||||
for source in ("hers", "ours")
|
||||
]
|
||||
)
|
||||
database.upsert_atoms(
|
||||
[
|
||||
EvidenceAtom(
|
||||
atom_id=f"{source}:frame",
|
||||
modality=Modality.image,
|
||||
source_id=source,
|
||||
source=f"file:///{source}.png",
|
||||
image_ref=stored,
|
||||
trust=TrustScores(parser=1.0, ocr=0.0, model=0.0),
|
||||
holder_id="wife" if source == "hers" else "",
|
||||
)
|
||||
for source in ("hers", "ours")
|
||||
]
|
||||
)
|
||||
|
||||
assert client.get("/api/atoms/ours:frame/image").status_code == 200, "nobody owns it, so anybody may look"
|
||||
assert client.get("/api/atoms/hers:frame/image").status_code == 404, "the bytes are behind the same rule as the words"
|
||||
assert client.get("/api/atoms", params={"modality": "image"}).json()["total"] == 1
|
||||
wife = _token_for(client, "wife")
|
||||
page = client.get("/api/atoms", params={"modality": "image"}, headers={"Authorization": f"Bearer {wife['token']}"})
|
||||
assert page.json()["total"] == 2
|
||||
|
||||
|
||||
def test_an_operator_and_a_stranger_do_not_share_one_compiled_context(settings: Settings) -> None:
|
||||
_remember(settings, KEYS, "wife")
|
||||
compiler = ContextCompiler(settings, Database(settings))
|
||||
# The stranger compiles first: if the cache key ignored whose eyes were used, the operator would be served this.
|
||||
stranger = compiler.compile(KEYS, RuntimeProfile.live_demo, as_holder="")
|
||||
assert stranger.hits == [], "a memory nobody owns and nobody marked public is invisible to a stranger"
|
||||
|
||||
owner = compiler.compile(KEYS, RuntimeProfile.live_demo, as_holder="wife")
|
||||
assert KEYS in owner.packed_text and owner.cache_trace_id != stranger.cache_trace_id
|
||||
|
||||
whole = compiler.compile(KEYS, RuntimeProfile.live_demo)
|
||||
assert KEYS in whole.packed_text, "a process that owns the store still reads the whole library"
|
||||
@@ -18,6 +18,8 @@ from ely_eye.storage import ObjectStore
|
||||
|
||||
PNG = b"\x89PNG\r\n\x1a\n"
|
||||
BASE = datetime(2026, 5, 1, 12, 0, tzinfo=UTC)
|
||||
OPERATOR = "the-shelf-owner"
|
||||
AS_OWNER = {"Authorization": f"Bearer {OPERATOR}"}
|
||||
|
||||
HTTP_SURFACE = {
|
||||
"/api/adapters",
|
||||
@@ -32,11 +34,14 @@ HTTP_SURFACE = {
|
||||
"/api/cartridges/{cartridge_id}/refresh",
|
||||
"/api/cartridges/{cartridge_id}/switch",
|
||||
"/api/cartridges/{cartridge_id}/unmount",
|
||||
"/api/card",
|
||||
"/api/card/{holder_id}",
|
||||
"/api/chat",
|
||||
"/api/hashhop",
|
||||
"/api/holders",
|
||||
"/api/holders/{holder_id}",
|
||||
"/api/holders/{holder_id}/knows",
|
||||
"/api/holders/{holder_id}/token",
|
||||
"/api/ingest/path",
|
||||
"/api/ingest/upload",
|
||||
"/api/memory",
|
||||
@@ -57,7 +62,7 @@ HTTP_SURFACE = {
|
||||
|
||||
@pytest.fixture
|
||||
def settings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Settings:
|
||||
resolved = Settings(home=tmp_path, default_holder="wife")
|
||||
resolved = Settings(home=tmp_path, default_holder="wife", auth_token=OPERATOR)
|
||||
monkeypatch.setattr(api_module, "get_settings", lambda: resolved)
|
||||
Database(resolved).init()
|
||||
return resolved
|
||||
@@ -106,14 +111,15 @@ def _seed(settings: Settings, pictures: int = 3) -> None:
|
||||
|
||||
def test_the_store_pages_newest_first_and_says_how_much_is_left(client: TestClient, settings: Settings) -> None:
|
||||
_seed(settings)
|
||||
first = client.get("/api/atoms", params={"limit": 2})
|
||||
# A tokenless caller is nobody and would see none of these; this test reads the shelf as its owner.
|
||||
first = client.get("/api/atoms", params={"limit": 2}, headers=AS_OWNER)
|
||||
assert first.status_code == 200
|
||||
page = first.json()
|
||||
assert page["total"] == 3
|
||||
assert [atom["atom_id"] for atom in page["atoms"]] == ["shot2:frame", "shot1:frame"]
|
||||
assert page["next_cursor"]
|
||||
|
||||
second = client.get("/api/atoms", params={"limit": 2, "cursor": page["next_cursor"]})
|
||||
second = client.get("/api/atoms", params={"limit": 2, "cursor": page["next_cursor"]}, headers=AS_OWNER)
|
||||
rest = second.json()
|
||||
assert [atom["atom_id"] for atom in rest["atoms"]] == ["shot0:frame"]
|
||||
assert rest["next_cursor"] is None, "the last page must not promise another one"
|
||||
@@ -125,11 +131,11 @@ def test_browse_narrows_by_modality_and_by_holder(client: TestClient, settings:
|
||||
"the keys are on the shelf", holder_id="wife", origin=MemoryOrigin.self_
|
||||
)
|
||||
|
||||
images = client.get("/api/atoms", params={"modality": "image"}).json()
|
||||
images = client.get("/api/atoms", params={"modality": "image"}, headers=AS_OWNER).json()
|
||||
assert images["total"] == 2
|
||||
assert {atom["modality"] for atom in images["atoms"]} == {"image"}
|
||||
|
||||
his = client.get("/api/atoms", params={"holder": "husband"}).json()
|
||||
his = client.get("/api/atoms", params={"holder": "husband"}, headers=AS_OWNER).json()
|
||||
assert [atom["atom_id"] for atom in his["atoms"]] == ["shot1:frame"]
|
||||
|
||||
|
||||
@@ -142,7 +148,7 @@ def test_an_unreadable_cursor_is_refused_with_400(client: TestClient, settings:
|
||||
|
||||
def test_a_picture_atom_serves_the_bytes_behind_it(client: TestClient, settings: Settings) -> None:
|
||||
_seed(settings, pictures=1)
|
||||
response = client.get("/api/atoms/shot0:frame/image")
|
||||
response = client.get("/api/atoms/shot0:frame/image", headers=AS_OWNER)
|
||||
assert response.status_code == 200
|
||||
assert response.content == PNG
|
||||
assert response.headers["content-type"] == "image/png"
|
||||
@@ -152,15 +158,15 @@ def test_a_memory_with_no_picture_says_so(client: TestClient, settings: Settings
|
||||
atom = MemoryService(settings, Database(settings)).remember(
|
||||
"the keys are on the shelf", holder_id="wife", origin=MemoryOrigin.self_
|
||||
)
|
||||
assert client.get(f"/api/atoms/{atom.atom_id}/image").status_code == 404
|
||||
assert client.get("/api/atoms/nope:image").status_code == 404
|
||||
assert client.get(f"/api/atoms/{atom.atom_id}/image", headers=AS_OWNER).status_code == 404
|
||||
assert client.get("/api/atoms/nope:image", headers=AS_OWNER).status_code == 404
|
||||
|
||||
|
||||
def test_a_forgotten_picture_stops_being_served(client: TestClient, settings: Settings) -> None:
|
||||
_seed(settings, pictures=1)
|
||||
MemoryService(settings, Database(settings)).forget(atom_id="shot0:frame")
|
||||
assert client.get("/api/atoms/shot0:frame/image").status_code == 404
|
||||
assert client.get("/api/atoms").json()["total"] == 0
|
||||
assert client.get("/api/atoms/shot0:frame/image", headers=AS_OWNER).status_code == 404
|
||||
assert client.get("/api/atoms", headers=AS_OWNER).json()["total"] == 0
|
||||
|
||||
|
||||
def test_status_says_what_the_machine_can_see_and_what_it_cannot(client: TestClient, settings: Settings) -> None:
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -15,6 +16,7 @@ import ely_eye.cli as cli_module
|
||||
from ely_eye.cli import app
|
||||
from ely_eye.config import Settings
|
||||
from ely_eye.db import Database
|
||||
from ely_eye.holder_store import HolderStore
|
||||
from ely_eye.ingestion import IngestionService
|
||||
from ely_eye.memory import MemoryService
|
||||
from ely_eye.migrations import SCHEMA_VERSION
|
||||
@@ -249,3 +251,45 @@ def test_the_cli_asks_and_reports_the_referral(tmp_path: Path, monkeypatch: pyte
|
||||
assert asked.exit_code == 0 and '"status": "referred"' in asked.stdout
|
||||
assert runner.invoke(app, ["holders"]).stdout.count("wife") >= 1
|
||||
assert runner.invoke(app, ["holder-set", "wife", "--contact", "not-a-pair"]).exit_code != 0
|
||||
|
||||
|
||||
def test_the_cli_mints_a_token_that_proves_who_is_asking(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
settings = Settings(home=tmp_path / "home", default_holder="husband")
|
||||
monkeypatch.setattr(cli_module, "get_settings", lambda: settings)
|
||||
runner = CliRunner()
|
||||
Store(settings).wife_remembers(HOME_KEYS)
|
||||
|
||||
issued = json.loads(runner.invoke(app, ["holder-issue", "wife"]).stdout)
|
||||
assert issued["holder_id"] == "wife" and issued["token"] and issued["issued_at"]
|
||||
assert HolderStore(Database(settings)).verify_token(issued["token"]) == "wife"
|
||||
assert issued["token"] not in runner.invoke(app, ["card"]).stdout, "the card is public and the token is not"
|
||||
|
||||
revoked = json.loads(runner.invoke(app, ["holder-revoke", "wife"]).stdout)
|
||||
assert revoked["holder_id"] == "wife" and revoked["token_issued_at"] is None
|
||||
assert HolderStore(Database(settings)).verify_token(issued["token"]) is None
|
||||
|
||||
|
||||
def test_the_card_is_read_back_from_the_store_rather_than_written_by_hand(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
settings = Settings(home=tmp_path / "home", default_holder="husband")
|
||||
monkeypatch.setattr(cli_module, "get_settings", lambda: settings)
|
||||
runner = CliRunner()
|
||||
store = Store(settings)
|
||||
store.wife_remembers(HOME_KEYS)
|
||||
store.husband_remembers(WORK_KEYS)
|
||||
store.register("neighbour", name="邻居", knows_about=["weather"])
|
||||
|
||||
answer = runner.invoke(app, ["card"])
|
||||
assert answer.exit_code == 0, answer.stdout
|
||||
card = json.loads(answer.stdout)
|
||||
assert card["schema_version"] == SCHEMA_VERSION
|
||||
assert card["atoms"] == 2 and card["sources"] == 2
|
||||
assert card["modalities"] == {"text": 2} and card["has_audio"] is False
|
||||
assert card["mounts"] == {"mounted": [], "scoped": False, "atoms_in_scope": 0}
|
||||
|
||||
named = {entry["holder_id"]: entry for entry in card["holders"]}
|
||||
assert named["wife"]["memories"] == 1 and named["wife"]["scenarios"] == ["home"]
|
||||
assert named["wife"]["newest_memory"], "a caller needs to know how stale the answer would be"
|
||||
assert named["neighbour"]["memories"] == 0 and named["neighbour"]["knows_about"] == ["weather"]
|
||||
assert HOME_KEYS not in answer.stdout, "the card says what a holder holds, never what they hold it to be"
|
||||
@@ -0,0 +1,46 @@
|
||||
"""The visibility rule is written out in five literal statements; this test keeps them one rule.
|
||||
|
||||
The SQL gate refuses a concatenated statement and a view cannot take a parameter, so the clause
|
||||
is copied rather than composed. A copy that drifts is invisible to every other test — the search
|
||||
leg, the browse page and the picture route would each keep passing while disagreeing about who
|
||||
may see a memory.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
SOURCE_DIR = Path(__file__).resolve().parents[1] / "src" / "ely_eye"
|
||||
FILES = ("browse.py", "store_search.py", "vector_index.py")
|
||||
CLAUSE = re.compile(r"\(\? IS NULL[^()]*?holder_id = ''.*?WHERE value = \?[^)]*\)\)\)\)", re.DOTALL)
|
||||
EXPECTED_COPIES = 5
|
||||
|
||||
|
||||
def _normalize(clause: str) -> str:
|
||||
"""Strip what the copies are allowed to differ in: whitespace, string-literal quotes, table aliases."""
|
||||
|
||||
return re.sub(r"\s+", "", clause).replace('"', "").replace("a.", "")
|
||||
|
||||
|
||||
def test_every_copy_of_the_visibility_rule_agrees() -> None:
|
||||
found: list[tuple[str, str]] = [
|
||||
(name, _normalize(match.group(0)))
|
||||
for name in FILES
|
||||
for match in CLAUSE.finditer((SOURCE_DIR / name).read_text(encoding="utf-8"))
|
||||
]
|
||||
|
||||
assert len(found) == EXPECTED_COPIES, [name for name, _ in found]
|
||||
distinct = {clause for _, clause in found}
|
||||
assert len(distinct) == 1, [name for name, clause in found if clause != next(iter(distinct))]
|
||||
|
||||
|
||||
def test_the_rule_still_carries_all_three_states() -> None:
|
||||
match = CLAUSE.search((SOURCE_DIR / "store_search.py").read_text(encoding="utf-8"))
|
||||
assert match is not None, "the words leg no longer carries a visibility clause"
|
||||
clause = _normalize(match.group(0))
|
||||
|
||||
assert "?ISNULL" in clause, "None still reads the whole library"
|
||||
assert "holder_id=''" in clause and "$.scope')='everyone'" in clause, "unowned and public rows stay open to a stranger"
|
||||
assert "?<>''" in clause, "the empty string must stay nobody rather than matching every holder"
|
||||
assert "holder_id=?" in clause and "value=?" in clause, "the owner and the visible_to list are still checked"
|
||||
@@ -190,3 +190,27 @@ func (c *Client) Search(ctx context.Context, options SearchOptions) (*SearchResu
|
||||
}
|
||||
return &result, nil
|
||||
}
|
||||
|
||||
// Card asks what the store can answer: the schema its answers come out under, which
|
||||
// senses its memory actually covers, and who is in the directory. It is served
|
||||
// without a token, so a caller can decide whether to send questions at all.
|
||||
func (c *Client) Card(ctx context.Context) (*domain.CapabilityCard, error) {
|
||||
var card domain.CapabilityCard
|
||||
if err := c.Get(ctx, "/api/card", &card); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &card, nil
|
||||
}
|
||||
|
||||
// HolderCard asks the same question about one holder. The server derives it from the
|
||||
// memories that holder owns, so a card cannot promise a modality the store lacks.
|
||||
func (c *Client) HolderCard(ctx context.Context, holderID string) (*domain.HolderCard, error) {
|
||||
if strings.TrimSpace(holderID) == "" {
|
||||
return nil, errors.New("holder_id is required")
|
||||
}
|
||||
var card domain.HolderCard
|
||||
if err := c.Get(ctx, "/api/card/"+url.PathEscape(holderID), &card); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &card, nil
|
||||
}
|
||||
@@ -272,3 +272,33 @@ type ReferralAnswer struct {
|
||||
Answer string `json:"answer,omitempty"`
|
||||
Basis []string `json:"basis"`
|
||||
}
|
||||
|
||||
// HolderCard is what a stranger may learn about one holder: who they are, what they
|
||||
// claim to know, how much they hold and how fresh it is. No memory text is here.
|
||||
type HolderCard struct {
|
||||
HolderID string `json:"holder_id"`
|
||||
Name string `json:"name"`
|
||||
Kind string `json:"kind"`
|
||||
KnowsAbout []string `json:"knows_about"`
|
||||
Contact map[string]string `json:"contact"`
|
||||
Memories int `json:"memories"`
|
||||
Modalities map[string]int `json:"modalities"`
|
||||
Scenarios []string `json:"scenarios"`
|
||||
NewestMemory string `json:"newest_memory,omitempty"`
|
||||
Registered bool `json:"registered"`
|
||||
TokenIssued bool `json:"token_issued"`
|
||||
}
|
||||
|
||||
// CapabilityCard answers "can this store answer my kind of questions" before a
|
||||
// single question is sent: the schema the answers come out under, which senses the
|
||||
// memory actually covers, and who is in the directory.
|
||||
type CapabilityCard struct {
|
||||
SchemaVersion int `json:"schema_version"`
|
||||
Atoms int `json:"atoms"`
|
||||
Sources int `json:"sources"`
|
||||
Modalities map[string]int `json:"modalities"`
|
||||
HasAudio bool `json:"has_audio"`
|
||||
Holders []HolderCard `json:"holders"`
|
||||
Mounts MountState `json:"mounts"`
|
||||
GeneratedAt string `json:"generated_at"`
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
package mcpserver
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
|
||||
"github.com/ZacharyZhang-NY/ely-eye/ely-eye-mcp/internal/domain"
|
||||
"github.com/modelcontextprotocol/go-sdk/mcp"
|
||||
)
|
||||
|
||||
type CardInput struct {
|
||||
HolderID string `json:"holder_id,omitempty" jsonschema:"optional exact holder id; ask for that holder's card instead of the whole store's"`
|
||||
}
|
||||
|
||||
type CardOutput struct {
|
||||
Card *domain.CapabilityCard `json:"card,omitempty"`
|
||||
Holder *domain.HolderCard `json:"holder,omitempty"`
|
||||
}
|
||||
|
||||
// registerCardTools adds the read that needs no memory access at all: what this
|
||||
// store can answer. A caller with a new batch of questions reads the card first
|
||||
// instead of discovering the gaps one question at a time.
|
||||
func (s *Server) registerCardTools(server *mcp.Server) {
|
||||
mcp.AddTool(server, &mcp.Tool{
|
||||
Name: "ely_eye_card",
|
||||
Title: "Capability card",
|
||||
Description: "What this store can answer, derived from what it holds: schema version, senses present, which holders exist and what they hold, and what is mounted. No memory text is shown, so it is safe to read before trusting the store.",
|
||||
}, s.cardTool)
|
||||
}
|
||||
|
||||
func (s *Server) cardTool(ctx context.Context, _ *mcp.CallToolRequest, input CardInput) (*mcp.CallToolResult, CardOutput, error) {
|
||||
if strings.TrimSpace(input.HolderID) == "" {
|
||||
card, err := s.api.Card(ctx)
|
||||
if err != nil {
|
||||
return nil, CardOutput{}, err
|
||||
}
|
||||
return nil, CardOutput{Card: card}, nil
|
||||
}
|
||||
holder, err := s.api.HolderCard(ctx, input.HolderID)
|
||||
if err != nil {
|
||||
return nil, CardOutput{}, err
|
||||
}
|
||||
return nil, CardOutput{Holder: holder}, nil
|
||||
}
|
||||
@@ -0,0 +1,126 @@
|
||||
package mcpserver
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/ZacharyZhang-NY/ely-eye/ely-eye-mcp/internal/apiclient"
|
||||
)
|
||||
|
||||
const storeCardBody = `{"schema_version":11,"atoms":3,"sources":3,"modalities":{"text":2,"image":1},
|
||||
"has_audio":false,"holders":[{"holder_id":"wife","name":"小美","kind":"person","knows_about":["home"],
|
||||
"contact":{"channel":"signal"},"memories":2,"modalities":{"text":2},"scenarios":["home"],
|
||||
"newest_memory":"2026-05-01T12:00:00+00:00","registered":true,"token_issued":true}],"mounts":{"mounted":[],"scoped":false,"atoms_in_scope":0}}`
|
||||
|
||||
func cardServer(t *testing.T, seen *[]string) *httptest.Server {
|
||||
t.Helper()
|
||||
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
*seen = append(*seen, r.Method+" "+r.URL.Path)
|
||||
if r.URL.Path == "/api/card/nobody" {
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
_ = json.NewEncoder(w).Encode(map[string]string{"detail": "no holder 'nobody' has memories or a directory entry"})
|
||||
return
|
||||
}
|
||||
if r.URL.Path == "/api/card/wife" {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(`{"holder_id":"wife","name":"小美","kind":"person","knows_about":["home"],"contact":{},"memories":2,"modalities":{"text":2},"scenarios":["home"],"newest_memory":"2026-05-01T12:00:00+00:00","registered":true,"token_issued":true}`))
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(storeCardBody))
|
||||
}))
|
||||
}
|
||||
|
||||
func TestCardAsksTheStoreAndKeepsTheAnswerDerived(t *testing.T) {
|
||||
var seen []string
|
||||
server := cardServer(t, &seen)
|
||||
defer server.Close()
|
||||
|
||||
s := &Server{api: apiclient.New(server.URL)}
|
||||
_, out, err := s.cardTool(context.Background(), nil, CardInput{})
|
||||
if err != nil {
|
||||
t.Fatalf("card failed: %v", err)
|
||||
}
|
||||
if seen[0] != "GET /api/card" {
|
||||
t.Errorf("request = %q, want the store card endpoint", seen[0])
|
||||
}
|
||||
if out.Card == nil || out.Holder != nil {
|
||||
t.Fatalf("card = %+v holder = %+v, want the store card only", out.Card, out.Holder)
|
||||
}
|
||||
if out.Card.SchemaVersion != 11 || out.Card.Atoms != 3 || out.Card.HasAudio {
|
||||
t.Errorf("card = %+v, want the server's own schema, counts and senses", out.Card)
|
||||
}
|
||||
if len(out.Card.Holders) != 1 || out.Card.Holders[0].TokenIssued != true {
|
||||
t.Errorf("holders = %+v, want the directory carried through", out.Card.Holders)
|
||||
}
|
||||
if out.Card.Holders[0].Contact["channel"] != "signal" {
|
||||
t.Errorf("contact = %+v, want the contact a referrer needs", out.Card.Holders[0].Contact)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCardWithAHolderAsksForThatHolderAlone(t *testing.T) {
|
||||
var seen []string
|
||||
server := cardServer(t, &seen)
|
||||
defer server.Close()
|
||||
|
||||
s := &Server{api: apiclient.New(server.URL)}
|
||||
_, out, err := s.cardTool(context.Background(), nil, CardInput{HolderID: "wife"})
|
||||
if err != nil {
|
||||
t.Fatalf("holder card failed: %v", err)
|
||||
}
|
||||
if seen[0] != "GET /api/card/wife" {
|
||||
t.Errorf("request = %q, want the one-holder endpoint", seen[0])
|
||||
}
|
||||
if out.Holder == nil || out.Card != nil {
|
||||
t.Fatalf("holder = %+v card = %+v, want the holder card only", out.Holder, out.Card)
|
||||
}
|
||||
if out.Holder.HolderID != "wife" || out.Holder.Memories != 2 {
|
||||
t.Errorf("holder card = %+v, want wife's own counts", out.Holder)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCardTreatsABlankHolderAsTheWholeStore(t *testing.T) {
|
||||
var seen []string
|
||||
server := cardServer(t, &seen)
|
||||
defer server.Close()
|
||||
|
||||
s := &Server{api: apiclient.New(server.URL)}
|
||||
if _, _, err := s.cardTool(context.Background(), nil, CardInput{HolderID: " "}); err != nil {
|
||||
t.Fatalf("blank holder failed: %v", err)
|
||||
}
|
||||
if seen[0] != "GET /api/card" {
|
||||
t.Errorf("request = %q, want the store card rather than a path of spaces", seen[0])
|
||||
}
|
||||
}
|
||||
|
||||
func TestCardPassesOnTheServersReasonForAnUnknownHolder(t *testing.T) {
|
||||
var seen []string
|
||||
server := cardServer(t, &seen)
|
||||
defer server.Close()
|
||||
|
||||
s := &Server{api: apiclient.New(server.URL)}
|
||||
_, _, err := s.cardTool(context.Background(), nil, CardInput{HolderID: "nobody"})
|
||||
if err == nil {
|
||||
t.Fatal("an unknown holder must not answer with an empty card")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "nobody") || !strings.Contains(err.Error(), "/api/card/nobody") {
|
||||
t.Errorf("error = %q, want the endpoint and the id the server named", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHolderCardRefusesAnEmptyIdBeforeTheNetwork(t *testing.T) {
|
||||
var seen []string
|
||||
server := cardServer(t, &seen)
|
||||
defer server.Close()
|
||||
|
||||
if _, err := apiclient.New(server.URL).HolderCard(context.Background(), ""); err == nil {
|
||||
t.Fatal("an empty holder id should be refused, not fetched")
|
||||
}
|
||||
if len(seen) != 0 {
|
||||
t.Errorf("requests = %v, want none", seen)
|
||||
}
|
||||
}
|
||||
@@ -33,13 +33,14 @@ func Run(ctx context.Context, cfg config.Config, version string) error {
|
||||
Title: "Ely-Eye Context Cartridge MCP",
|
||||
Version: version,
|
||||
}, &mcp.ServerOptions{
|
||||
Instructions: "Use Ely-Eye tools to retrieve local cartridge context, evidence atoms, proof suites, and compiled evidence packs. Read tools mirror local Ely-Eye state without writing. remember, supersede_memory and forget_memory mutate memory through the Ely-Eye API, which must be running (`ely-eye serve`); the server, not the caller, decides who owns a memory. mount_cartridge and switch_cartridge decide what a search may see: mount a cartridge to ask only that cartridge, and unmount everything to ask the whole library. working_set reloads what one session already recorded, in order, without searching; memory_conflicts reports what the store has not reconciled, and settling one is the operator's act, not a tool's.",
|
||||
Instructions: "Use Ely-Eye tools to retrieve local cartridge context, evidence atoms, proof suites, and compiled evidence packs. Read tools mirror local Ely-Eye state without writing. remember, supersede_memory and forget_memory mutate memory through the Ely-Eye API, which must be running (`ely-eye serve`); the server, not the caller, decides who owns a memory. mount_cartridge and switch_cartridge decide what a search may see: mount a cartridge to ask only that cartridge, and unmount everything to ask the whole library. working_set reloads what one session already recorded, in order, without searching; memory_conflicts reports what the store has not reconciled, and settling one is the operator's act, not a tool's. ely_eye_card says what the store can answer before anything is sent: schema version, which senses its memory covers, and who holds what. A read with no ELY_EYE_AUTH_TOKEN sees only memories nobody owns and memories marked for everyone, so a private answer needs the token.",
|
||||
})
|
||||
server.registerTools(mcpServer)
|
||||
server.registerWriteTools(mcpServer)
|
||||
server.registerDirectoryTools(mcpServer)
|
||||
server.registerMountTools(mcpServer)
|
||||
server.registerTierTools(mcpServer)
|
||||
server.registerCardTools(mcpServer)
|
||||
server.registerResources(mcpServer)
|
||||
if err := mcpServer.Run(ctx, &mcp.StdioTransport{}); err != nil {
|
||||
if errors.Is(err, io.EOF) || strings.HasSuffix(err.Error(), "EOF") {
|
||||
|
||||
+139
-3
@@ -65,7 +65,7 @@ fi
|
||||
# /api/memory/working exists only on builds that carry the memory tiers, so a server left over from
|
||||
# an older tree answers 404 here. Checking the MCP reply instead would prove nothing — the Go layer
|
||||
# decodes into its own struct and re-emits a missing field as 0, so an old server still passes.
|
||||
status="$(curl -s -o /dev/null -w '%{http_code}' \
|
||||
status="$(curl -s -o /dev/null -w '%{http_code}' -H "Authorization: Bearer ${ELY_EYE_AUTH_TOKEN}" \
|
||||
"http://127.0.0.1:${port}/api/memory/working?holder=fingerprint&session=probe")"
|
||||
if [ "$status" != "200" ]; then
|
||||
echo "FAIL: port ${port} answers /api/status but returns HTTP $status for /api/memory/working;"
|
||||
@@ -178,7 +178,8 @@ send({"jsonrpc": "2.0", "method": "notifications/initialized"})
|
||||
names = [item["name"] for item in call(2, "tools/list", {})["tools"]]
|
||||
for required in ("remember", "supersede_memory", "forget_memory", "mounted_cartridges",
|
||||
"mount_cartridge", "switch_cartridge", "unmount_cartridge", "append_to_cartridge",
|
||||
"who_knows", "ask", "list_holders", "working_set", "memory_conflicts"):
|
||||
"who_knows", "ask", "list_holders", "working_set", "memory_conflicts",
|
||||
"ely_eye_card"):
|
||||
if required not in names:
|
||||
raise SystemExit(f"FAIL: {required} is not registered; tools are {names}")
|
||||
|
||||
@@ -315,5 +316,140 @@ if any(hit["atom"]["atom_id"] == older["atom_id"] for hit in settled):
|
||||
if not any(hit["atom"]["atom_id"] == newer["atom_id"] for hit in settled):
|
||||
raise SystemExit(f"FAIL: the newer side of the conflict went missing: {settled}")
|
||||
|
||||
print("PASS: remember, supersede, search, forget, cartridge mounting, the transactive referral and the memory tiers work through MCP")
|
||||
# The card is derived, not declared: it says what this store can answer, and it is
|
||||
# safe to read before trusting the store because it carries no memory text.
|
||||
card = tool("ely_eye_card", {})["card"]
|
||||
if card["schema_version"] < 11 or card["has_audio"]:
|
||||
raise SystemExit(f"FAIL: the capability card misreports the store: {card}")
|
||||
wife_card = next((item for item in card["holders"] if item["holder_id"] == "wife"), None)
|
||||
if not wife_card or wife_card["memories"] < 1 or "护照" not in wife_card["knows_about"]:
|
||||
raise SystemExit(f"FAIL: the card does not say what the wife holds: {card['holders']}")
|
||||
if secret in json.dumps(card, ensure_ascii=False):
|
||||
raise SystemExit(f"FAIL: the capability card carried private text: {card}")
|
||||
single = tool("ely_eye_card", {"holder_id": "wife"})["holder"]
|
||||
if single["memories"] != wife_card["memories"] or single["newest_memory"] != wife_card["newest_memory"]:
|
||||
raise SystemExit(f"FAIL: the one-holder card disagrees with the store card: {single}")
|
||||
|
||||
print("PASS: remember, supersede, search, forget, cartridge mounting, the transactive referral, the memory tiers and the capability card work through MCP")
|
||||
PY
|
||||
|
||||
# The boundary, proved from the outside: a caller with no token is nobody, and a holder's
|
||||
# token reads her own memories and nobody else's. The MCP process is re-run without the
|
||||
# operator token so the Go path is measured against the same rule, not assumed.
|
||||
"$python_bin" - "$home/ely-eye-mcp" "$port" "$secret" "$ELY_EYE_AUTH_TOKEN" <<'PY'
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
binary, port, secret, operator = sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4]
|
||||
base = f"http://127.0.0.1:{port}"
|
||||
|
||||
|
||||
def request(path, body=None, token=None, method=None):
|
||||
payload = None if body is None else json.dumps(body, ensure_ascii=False).encode()
|
||||
req = urllib.request.Request(base + path, data=payload, method=method or ("POST" if body is not None else "GET"))
|
||||
if payload is not None:
|
||||
req.add_header("content-type", "application/json")
|
||||
if token:
|
||||
req.add_header("Authorization", f"Bearer {token}")
|
||||
try:
|
||||
with urllib.request.urlopen(req) as response:
|
||||
return response.status, json.load(response)
|
||||
except urllib.error.HTTPError as error:
|
||||
raw = error.read().decode()
|
||||
try:
|
||||
return error.code, json.loads(raw)
|
||||
except json.JSONDecodeError:
|
||||
return error.code, {"detail": raw}
|
||||
|
||||
|
||||
def texts(query, token=None):
|
||||
status, answer = request("/api/search", {"query": query, "top_k": 10}, token=token)
|
||||
if status != 200:
|
||||
raise SystemExit(f"FAIL: search returned HTTP {status}: {answer}")
|
||||
return [hit["atom"]["text"] for hit in answer["hits"]]
|
||||
|
||||
|
||||
# Written by this machine's principal, so it is somebody else's memory to her.
|
||||
foreign = "the oven needs descaling"
|
||||
|
||||
if secret in texts(secret):
|
||||
raise SystemExit("FAIL: a caller with no token read a private memory")
|
||||
|
||||
status, issued = request("/api/holders/wife/token", {}, token=operator, method="POST")
|
||||
token = issued.get("token") if status == 200 else None
|
||||
if not token:
|
||||
raise SystemExit(f"FAIL: the operator could not mint the wife's token: HTTP {status} {issued}")
|
||||
if secret not in texts(secret, token=token):
|
||||
raise SystemExit("FAIL: the wife's own token cannot find her memory")
|
||||
if foreign not in texts(foreign, token=operator):
|
||||
raise SystemExit("FAIL: the operator token cannot read the memory its own principal wrote")
|
||||
if foreign in texts(foreign, token=token):
|
||||
raise SystemExit("FAIL: the wife's token reads somebody else's memory")
|
||||
|
||||
status, answer = request("/api/search", {"query": "护照", "filters": {"as_holder": "local"}}, token=token)
|
||||
if status != 403:
|
||||
raise SystemExit(f"FAIL: the wife naming another holder returned HTTP {status}, want 403: {answer}")
|
||||
if "local" not in json.dumps(answer, ensure_ascii=False):
|
||||
raise SystemExit(f"FAIL: the refusal does not name the holder she reached for: {answer}")
|
||||
|
||||
status, card = request("/api/card")
|
||||
if status != 200 or card["schema_version"] < 11:
|
||||
raise SystemExit(f"FAIL: the capability card needs a token or misreports the store: HTTP {status} {card}")
|
||||
if secret in json.dumps(card, ensure_ascii=False):
|
||||
raise SystemExit("FAIL: the capability card carried private text")
|
||||
|
||||
env = {key: value for key, value in os.environ.items() if key != "ELY_EYE_AUTH_TOKEN"}
|
||||
proc = subprocess.Popen([binary, "server"], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
||||
stderr=subprocess.DEVNULL, text=True, env=env)
|
||||
|
||||
|
||||
def send(message):
|
||||
proc.stdin.write(json.dumps(message, ensure_ascii=False) + "\n")
|
||||
proc.stdin.flush()
|
||||
|
||||
|
||||
def call(id_, method, params):
|
||||
send({"jsonrpc": "2.0", "id": id_, "method": method, "params": params})
|
||||
while True:
|
||||
line = proc.stdout.readline()
|
||||
if not line:
|
||||
raise SystemExit(f"FAIL: the tokenless MCP server closed the pipe at request {id_}")
|
||||
answer = json.loads(line)
|
||||
if answer.get("id") == id_:
|
||||
if "error" in answer:
|
||||
raise SystemExit(f"FAIL: {method} returned {answer['error']}")
|
||||
return answer["result"]
|
||||
|
||||
|
||||
def tool(name, arguments):
|
||||
result = call(3, "tools/call", {"name": name, "arguments": arguments})
|
||||
if result.get("isError"):
|
||||
raise SystemExit(f"FAIL: {name} returned an error: {result['content']}")
|
||||
return result["structuredContent"]
|
||||
|
||||
|
||||
call(1, "initialize", {"protocolVersion": "2025-06-18", "capabilities": {},
|
||||
"clientInfo": {"name": "ely-eye-stranger", "version": "0"}})
|
||||
send({"jsonrpc": "2.0", "method": "notifications/initialized"})
|
||||
|
||||
stranger_card = tool("ely_eye_card", {})["card"]
|
||||
if stranger_card["schema_version"] != card["schema_version"]:
|
||||
raise SystemExit(f"FAIL: the tokenless card reports a different schema: {stranger_card}")
|
||||
stranger_hits = tool("search_atoms", {"query": secret})["hits"]
|
||||
if any(hit["atom"]["text"] == secret for hit in stranger_hits):
|
||||
raise SystemExit(f"FAIL: a tokenless MCP client read a private memory: {stranger_hits}")
|
||||
directory = tool("who_knows", {"query": "护照"})["report"]
|
||||
if directory["status"] != "found" or not directory["hits"]:
|
||||
raise SystemExit(f"FAIL: a tokenless client cannot use the directory: {directory}")
|
||||
if directory["hits"][0]["holder_id"] != "wife":
|
||||
raise SystemExit(f"FAIL: the directory stopped naming the wife: {directory['hits'][0]}")
|
||||
proc.stdin.close()
|
||||
proc.wait(timeout=10)
|
||||
|
||||
print("PASS: a caller with no token sees only public memory, a holder's token sees her own, "
|
||||
"naming somebody else is refused, and the capability card answers without a token")
|
||||
PY
|
||||
Reference in new issue
Block a user