Say what the store actually does, and put a receipt under each claim
The README described a project that no longer exists: a Windows-only Qwen runtime with nine read-only tools, a console as the only way to operate the store, and a 100M-token capsule as if it were measured. It is now a description of what is here, with the report and the command that re-measures each number, and a section that names what has not been run instead of leaving the reader to discover it. TRAINING_ARTIFACTS.md keeps its artifact tables and loses the scores that were never run: MMMU-Pro and OCRBench each read one sample, and the HashHop gate collected proofs until it hit its target. What stays is annotated with what it actually measures — citation accuracy there means the cited id existed in the context that was handed over, not that the passage supports the answer. Gaps.md gains a re-audit section: ten of the twelve gaps closed, two partial, each row naming the command or report behind it, and the remaining work stated plainly — the 0.90 recall bar is unmet and the deficit is entirely in the visual and audio legs. AGENTS.md stops being ignored. It records the constraints this repository paid for: the measurement rules, the environment traps, and the review protocol, so the next agent does not pay them again. The CLI could filter by holder but could not look through a holder's eyes, so search gains --as-holder: three states of eyes on every face, not two.
This commit is contained in:
8 files changed
+349
-125
No files matched your search
@@ -22,7 +22,6 @@ unsloth_compiled_cache/
|
|||||||
*.sqlite-wal
|
*.sqlite-wal
|
||||||
*.sqlite-shm
|
*.sqlite-shm
|
||||||
PRD.md
|
PRD.md
|
||||||
AGENTS.md
|
|
||||||
CLAUDE.md
|
CLAUDE.md
|
||||||
designreference.html
|
designreference.html
|
||||||
# Local landing site — deployed to Cloudflare, not part of this repo
|
# Local landing site — deployed to Cloudflare, not part of this repo
|
||||||
|
|||||||
@@ -0,0 +1,98 @@
|
|||||||
|
# Agent notes for this repository
|
||||||
|
|
||||||
|
Read `CHECKLIST.md` for the ordered work list and `Gaps.md` for the audit the
|
||||||
|
work started from. Measured results live in `bench/reports/`. If a sentence in
|
||||||
|
this repository states a capability, there must be a command that demonstrates
|
||||||
|
it — otherwise the sentence belongs in the "not measured" section.
|
||||||
|
|
||||||
|
## Verify before you claim
|
||||||
|
|
||||||
|
Run the gates, and compare against the recorded baseline rather than "it built":
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pytest -q # both launch styles must work
|
||||||
|
ruff check backend/src/ely_eye backend/tests bench
|
||||||
|
.venv/bin/python -m bench.filesize_check # fails on any tracked file > 500 lines
|
||||||
|
.venv/bin/python -m bench.audit --product # the harness must not justify the product
|
||||||
|
.venv/bin/python -m mypy backend/src/ely_eye backend/tests # cold cache; baseline is 24 errors in 9 files
|
||||||
|
./node_modules/.bin/pyright <the files you touched> # must be 0 errors on touched files
|
||||||
|
cd ely-eye-mcp && go build ./... && go vet ./... && go test ./... && gofmt -l .
|
||||||
|
cd frontend && npm run build && npx eslint src --max-warnings 0
|
||||||
|
bash scripts/mcp_write_smoke.sh # two processes, one store
|
||||||
|
```
|
||||||
|
|
||||||
|
mypy carries a pre-existing error baseline. A change is clean when the count and
|
||||||
|
the file set match the recorded baseline; measure the baseline in a worktree of
|
||||||
|
the commit you branched from, do not assume it.
|
||||||
|
|
||||||
|
## Rules this repo enforces
|
||||||
|
|
||||||
|
- One writer per fact. Memory writes go through `MemoryService`; the Go MCP
|
||||||
|
server forwards over HTTP instead of touching SQLite, so the rules cannot
|
||||||
|
drift into two dialects.
|
||||||
|
- Fail fast and name the thing. `nothing to forget: no atom or source was given`,
|
||||||
|
`database schema is version 99, this build understands 11`. A silent default
|
||||||
|
that hides a caller's mistake is a defect, not a convenience.
|
||||||
|
- A file over 500 lines gets split before it gets commented.
|
||||||
|
- No dead code, no `v1`/`v2`/`v3` shadows of a thing that no longer exists.
|
||||||
|
- A `#` comment is one line. A docstring is a contract and may be longer.
|
||||||
|
- Absence is a state. `v_atoms_current` hides a closed, lapsed, or not-yet-valid
|
||||||
|
memory; it does not delete it, and `atom_count()` still counts it.
|
||||||
|
|
||||||
|
## Measurement rules
|
||||||
|
|
||||||
|
- A number that moves with machine load is quoted as a range across runs, or it
|
||||||
|
points at the report that holds it. Never quote a single load-dependent value
|
||||||
|
as if it were stable.
|
||||||
|
- A report's commit stamp must name the commit that carries the harness that
|
||||||
|
produced it. That is why a harness change and its report land as two commits: a
|
||||||
|
report cannot stamp its own commit.
|
||||||
|
- Every recall number sits next to its chance floor and, where one exists, an
|
||||||
|
external counterpart. `bench/reports/BASELINE_EXTERNAL.md` is the counterpart;
|
||||||
|
the chance row is printed in the same table.
|
||||||
|
- One sample is not a benchmark. `next(iter(stream))` with `streaming=True` was
|
||||||
|
how the old proof suite "scored" MMMU-Pro and OCRBench.
|
||||||
|
- A probe sample drawn by file order can silently drop a whole modality.
|
||||||
|
`bench/query_set.py:scale_probes` is modality-stratified for that reason, and a
|
||||||
|
test pins that every modality appears.
|
||||||
|
- A harness works on a copy of the frozen store (`bench/scratch.py:copy_store`).
|
||||||
|
Never migrate `bench/runs/baseline/data/ely_eye.sqlite` in place.
|
||||||
|
- When two correct engines disagree, suspect the metric before the engine:
|
||||||
|
byte-identical atoms produce tied scores, so id-level recall can read 0.83 for
|
||||||
|
no real reason. `bench/reports/QUANT.md` scores a hit as "any returned atom
|
||||||
|
whose true cosine is at least the K-th best".
|
||||||
|
|
||||||
|
## Environment traps already paid for
|
||||||
|
|
||||||
|
- pi-lens runs pyright with **0 config files**, so its `Import could not be
|
||||||
|
resolved` findings for `pydantic`, `fastapi`, `torch`, `sqlite_vec` are an
|
||||||
|
environment artifact. `./node_modules/.bin/pyright` reads `pyrightconfig.json`
|
||||||
|
and is the one that counts.
|
||||||
|
- pi-lens auto-runs a `ruff format`-style formatter on files you touch. This
|
||||||
|
repository is not ruff-format clean and its gate is `ruff check`, so revert
|
||||||
|
that drift (`git checkout --`) before committing.
|
||||||
|
- Run `ruff check --fix` on explicit paths, never on the package: it will
|
||||||
|
reformat a dozen unrelated files and you will commit them by accident.
|
||||||
|
- `bench/tests` import `bench.*`; the root `pyproject.toml` supplies
|
||||||
|
`pythonpath = ["."]`, which is why bare `pytest` works.
|
||||||
|
- `import fitz` prints a deprecation warning to **stdout**, which corrupts every
|
||||||
|
CLI JSON answer. Import it as `import pymupdf as fitz`.
|
||||||
|
- SQLite statements must be literals at the call site: the repo's semgrep rule
|
||||||
|
rejects `connection.execute(variable, …)` even when the variable is a module
|
||||||
|
constant. Where a rule must be copied, a test compares the copies
|
||||||
|
(`backend/tests/test_visibility_rule.py`).
|
||||||
|
|
||||||
|
## Review protocol
|
||||||
|
|
||||||
|
Every task lands through an independent reviewer running in a `herdr` pane
|
||||||
|
against the commit, not against a description of it. Findings are numbered
|
||||||
|
`D1…Dn`; fix the mechanism, not the wording, and re-measure. A checklist item is
|
||||||
|
not `[x]` before the reviewer returns PASS. Read the reviewer's verdict from its
|
||||||
|
session JSONL (the last assistant text block), not from a pane scroll-back that
|
||||||
|
may show an older round.
|
||||||
|
|
||||||
|
## Cross-language contract
|
||||||
|
|
||||||
|
`backend/src/ely_eye/schemas.py` is the source of truth. `ely-eye-mcp/internal/domain/types.go`
|
||||||
|
and `frontend/src/types.ts` mirror it; when a field changes there, it changes in
|
||||||
|
all three, and the smoke script is what proves the Go path agrees.
|
||||||
+17
-6
@@ -437,12 +437,23 @@
|
|||||||
- 验收:`pytest backend/tests/test_holder_acl.py backend/tests/test_visibility_rule.py backend/tests/test_transactive.py` → **13 + 2 + 19** 全绿(两 holder 互不可见、无 token 读不到私钥、`everyone` 无 token 可搜、`visible_to` 只放行被指名的人、声称别人 403 且点名两人、token 只出现一次且库里只有 sha256、吊销后旧 token 401、卡无 token 可读且不含正文、陌生人的图片 404 而无主的 200、`compile(as_holder="")` 返回 0 hits 且 `cache_trace_id` 与 `as_holder="wife"` / `None` 都不同)。跨进程:`bash scripts/mcp_write_smoke.sh` → 两行 PASS,第二行是「a caller with no token sees only public memory, a holder's token sees her own, naming somebody else is refused, and the capability card answers without a token」——它先证明无 token 搜不到私钥,再用 operator token 铸出妻子的 token、证明她能搜到自己的、搜不到 `local` 写的 `the oven needs descaling`,最后**把 MCP 进程不带 `ELY_EYE_AUTH_TOKEN` 重启一遍**,证明 Go 那条路守同一条规则(不是假设)。
|
- 验收:`pytest backend/tests/test_holder_acl.py backend/tests/test_visibility_rule.py backend/tests/test_transactive.py` → **13 + 2 + 19** 全绿(两 holder 互不可见、无 token 读不到私钥、`everyone` 无 token 可搜、`visible_to` 只放行被指名的人、声称别人 403 且点名两人、token 只出现一次且库里只有 sha256、吊销后旧 token 401、卡无 token 可读且不含正文、陌生人的图片 404 而无主的 200、`compile(as_holder="")` 返回 0 hits 且 `cache_trace_id` 与 `as_holder="wife"` / `None` 都不同)。跨进程:`bash scripts/mcp_write_smoke.sh` → 两行 PASS,第二行是「a caller with no token sees only public memory, a holder's token sees her own, naming somebody else is refused, and the capability card answers without a token」——它先证明无 token 搜不到私钥,再用 operator token 铸出妻子的 token、证明她能搜到自己的、搜不到 `local` 写的 `the oven needs descaling`,最后**把 MCP 进程不带 `ELY_EYE_AUTH_TOKEN` 重启一遍**,证明 Go 那条路守同一条规则(不是假设)。
|
||||||
- 顺手修的两条自己写的断言缺陷(都是跑出来的):smoke 里 `if secret in found(None, token=token)` 是同义反复(搜的还是那句话,当然命中),改成拿**别人写的**记忆当探针;指纹探测 `GET /api/memory/working?holder=fingerprint` 现在返回 **403**(路由在、但匿名不能声称 holder),所以探测必须带 operator token——旧树上没有这条路由仍是 404,判别力保住。
|
- 顺手修的两条自己写的断言缺陷(都是跑出来的):smoke 里 `if secret in found(None, token=token)` 是同义反复(搜的还是那句话,当然命中),改成拿**别人写的**记忆当探针;指纹探测 `GET /api/memory/working?holder=fingerprint` 现在返回 **403**(路由在、但匿名不能声称 holder),所以探测必须带 operator token——旧树上没有这条路由仍是 404,判别力保住。
|
||||||
- 门禁:`pytest -q` **401 passed**;`ruff check` 对 22 个改动文件干净;`bench/filesize_check` 干净;`bench.audit --product` → `audited 32 harness files, 2 reports, 6 set files` / `clean`;mypy 冷缓存 **24 errors in 9 files (checked 112 source files)**,与 `9cb46c1` 基线(24/9)**净新增 0**;`./node_modules/.bin/pyright` 对 17 个后端源文件 + 5 个测试文件 → **0 errors**;`go build` / `go vet` / `go test`(apiclient ok、mcpserver ok)+ `gofmt -l` 空;`npm run build` → `dist/assets/index-CSDHCFNo.js 221.48 kB`(bundle 哈希与 `246dae9` 相同,与「本轮没动前端」一致)+ eslint exit 0;`bash scripts/mcp_write_smoke.sh` 3.3s 两行 PASS。
|
- 门禁:`pytest -q` **401 passed**;`ruff check` 对 22 个改动文件干净;`bench/filesize_check` 干净;`bench.audit --product` → `audited 32 harness files, 2 reports, 6 set files` / `clean`;mypy 冷缓存 **24 errors in 9 files (checked 112 source files)**,与 `9cb46c1` 基线(24/9)**净新增 0**;`./node_modules/.bin/pyright` 对 17 个后端源文件 + 5 个测试文件 → **0 errors**;`go build` / `go vet` / `go test`(apiclient ok、mcpserver ok)+ `gofmt -l` 空;`npm run build` → `dist/assets/index-CSDHCFNo.js 221.48 kB`(bundle 哈希与 `246dae9` 相同,与「本轮没动前端」一致)+ eslint exit 0;`bash scripts/mcp_write_smoke.sh` 3.3s 两行 PASS。
|
||||||
- [ ] **p2-5** 文档单一真相
|
- [x] **p2-5** 文档单一真相
|
||||||
- [ ] p2-5.1 `AGENTS.md` 重写并停止 gitignore
|
- [x] p2-5.1 `AGENTS.md` 重写并停止 gitignore
|
||||||
- [ ] p2-5.2 `README.md` 每条宣称指向可重跑命令
|
- `.gitignore` 删了 `AGENTS.md` 那一行(`CLAUDE.md` 仍忽略,它不是本仓库的真相源),`git check-ignore AGENTS.md` 现在 exit 1。新文件 5.5KB,写的不是风格偏好而是**这个仓库付过学费的约束**:门禁命令清单(含 mypy 基线是 24 errors in 9 files、必须在分支点的 worktree 里量而不是猜);一个 writer 一条规则(Go 不写 SQLite);fail fast 必须点名(引真实报错 `nothing to forget: no atom or source was given`);≤500 行、无死代码、注释单行、docstring 是契约;「缺席是一种状态」——`v_atoms_current` 隐藏而不删除。
|
||||||
- [ ] p2-5.3 `TRAINING_ARTIFACTS.md` 未跑分数删除
|
- 测量规则单独一节,都是本轮踩过的坑:**跟机器负载走的数字只能写区间或指向报告**;报告的 commit 戳必须指向含该 harness 的那个 commit(所以 harness 改动与报告是两个 commit);每个 recall 必须与 chance 行和外部对手段并排;**n=1 不是基准**(旧套件就是靠 `next(iter(stream))` 「跑」了 MMMU-Pro 与 OCRBench);按文件序抽样会静默丢掉整个模态(`bench/query_set.py:scale_probes` 已按模态分层并有测试钉住);harness 只动副本(`bench/scratch.py:copy_store`),绝不原地迁移冻结库;两个正确引擎不一致时先怀疑度量(字节相同的 atom 造成并列,id 级 recall 会读出 0.83 的假差距)。
|
||||||
- [ ] p2-5.4 `Gaps.md` 更新为已闭合状态
|
- 环境陷阱一节:pi-lens 跑 pyright 时是 **0 config files**,所以它对 `pydantic/fastapi/torch/sqlite_vec` 的 `Import could not be resolved` 是环境伪影,算数的是读 `pyrightconfig.json` 的 `./node_modules/.bin/pyright`;pi-lens 会在你改过的文件上自动跑 `ruff format` 风格格式化,而本仓库不是 ruff-format clean、门禁是 `ruff check`,所以提交前要 `git checkout --` 回退;`ruff check --fix` 只跑显式路径;`import fitz` 把警告打到 **stdout** 会弄坏每一条 CLI JSON,必须 `import pymupdf as fitz`;semgrep 拒绝 `connection.execute(变量)`,所以规则副本必须存在、并由测试比对(`test_visibility_rule.py`)。
|
||||||
- 验收:`grep` 无未跑模型分数残留;每条数字可重跑
|
- [x] p2-5.2 `README.md` 每条宣称指向可重跑命令
|
||||||
|
- 十类旧宣称全部替换(不是加脚注):(1)「built on Qwen3.5-9B … fifteen-check proof suite exercises the whole thing against the live model」→ 生成侧列入「**没在本仓库跑过**」清单,每项一条可重跑命令;(2)「retrieves with sparse and Qwen embedding search」→ dense 默认**关**,并给出 LEGS 实测:24 页上 0.863→0.860 而 p50 122→713ms,宽页两边都 0.873 而 p50 84→579ms,它买的是记忆类问题(multi-session 0.857→0.920、preference 0.800→0.960)、赔的是图像类(0.720→0.680);(3)「checks every answer with a verifier」→ 引用准确性是 id 存在不是蕴含(写在 `TRAINING_ARTIFACTS.md` 那条证据旁边);(4)「100M-token memory capsule」→ 实测 165,973 atoms / 153,857 vectors,1M 是外推不是测量;(5) HashHop / Contradiction Lens / 标准基准 → 全部进「未测」清单;(6)「nine read-only tools … never writes」→ 23 个工具,写走 API;(7)「the console … is the only place」→ CLI 做得到同样的事;(8)「Requirements: Windows」→ Linux/macOS `.sh` 双胞胎 + Docker + Linux CI;(9) `.ely_eye/` 布局与卡带内容 → 今天的真实形状(vec0 在库内、卡带是可挂载作用域);(10) 记忆模型 / bench / ACL / 能力卡 / 验收线状态在旧 README **完全没提**,现在都在。
|
||||||
|
- 验收线写成明字的**未达**:0.863 对 0.90,并列出缺口在哪(video 0.455 / image 0.720 / ui_screenshot 0.818,而 code 0.945 / pdf 0.885 / conversation 1.000)以及为什么(冻结库一个 audio atom 都没有;CLIP 不能在整屏里定位图标)。
|
||||||
|
- [x] p2-5.3 `TRAINING_ARTIFACTS.md` 未跑分数删除
|
||||||
|
- 删了:`MMMU-Pro=1.00` 与 `OCRBench=1.00`(每个基准只读了 **一个** 样本,`next(iter(stream))`)和「Visual HashHop gate passed 7/7 … 1-hop=1.00 2-hop=1.00」的 gate 口径(那个检查是**重试到达标为止**,现在重试已删)。改而写明:文件是 2026-05-28 在 Windows+4090 上从已不存在的 `PRD.md` 生成的,adapter 与所有 `.ely_eye/` 路径**不被 git 跟踪**,所以 clone 不能重现。
|
||||||
|
- 保留但说清它到底测了什么:`citation accuracy 1.00` = 引用的 id 在送进去的上下文里存在,不是蕴含(且那道题把答案写在题面里);100M capsule = 检查自己生成的 capsule 自哈希自洽,不是 100M 真实内容;repository input proof 是当天快照(仓库现在更大,且任何 git 仓库都能满足它);runtime profile「verified」限定在那台机器的固定 WSL 发行版(现在读 `ELY_EYE_WSL_DISTRO`,且有 `.sh` 双胞胎与 Linux CI);15 行 `Checks:` 块标成「当天写下的行,不是今天的结论」并指向 Gaps.md 附录 A 的逐条判定;卡带资产清单换成 `cartridge.py` 今天真正写的五个文件,并点名 `kv_snapshots.kivi` / `ttt_vl_adapter.safetensors` 是从未接上 loader 的占位。
|
||||||
|
- [x] p2-5.4 `Gaps.md` 更新为已闭合状态
|
||||||
|
- 新增 §0.5(不改写 §0–§4 的历史判断,只加今天复核):12 项差距逐项一行——**10 项闭合、2 项部分闭合**,每行带跑过的命令或报告;四个根因逐一说现在为什么不成立;末尾「还剩什么」只列三件事:验收线 0.90 未达(缺口全在视觉/音频腿)、卡带的算力节省(前缀/KV 复用)没做、生成侧在本仓库一次没跑过。
|
||||||
|
- 诚实处理两处派生数字:`+0.454` 不在 `BASELINE_EXTERNAL.md` 里,是它两行(0.866 / 0.412)相减得来,Gaps 里写明「该报告只给两行与‘这段差值是 holder 边界的价值’这句话,差值是这里减出来的」;ASR 速度改成当前报告的数(258.18s 音频 / 3.12s,每视频 68.4–111.2×),不再引旧报告的 63.9–112.2。
|
||||||
|
- 偏差(顺手补的真缺口):CLI `search` 只有 `--holder`(过滤器),**没有**办法用某个 holder 的眼睛看库——即无法回答「我的 agent 会看到什么」。加了 `--as-holder`(`SearchFilters.as_holder`)与两条断言(`--as-holder stranger` → `no_match`,`--as-holder wife` → 命中),否则三态眼睛只在 HTTP/MCP 两面成立而 CLI 面缺一个口。
|
||||||
|
- 验收:`grep -rn "MMMU-Pro=\|OCRBench=\|1-hop=\|2-hop=" *.md */README.md` → **0 命中**;README 里每个数字都能在报告里找到(0.863→LEGS、0.866/0.412→BASELINE_EXTERNAL、0.0717/0.2177→SCALE、165973/153857/106 per second→DENSE、0.242/533/68.4–111.2→ASR、模态行→LEGS、148.9GB→Gaps 附录 B)。唯一留下的模型分数是 `TRAINING_ARTIFACTS.md:84` 的 `citation accuracy 1.00`:它在证据块里(那是当天产出的工件原文),且紧跟的段落写明了它测的是什么、不测什么。
|
||||||
|
- 门禁:`pytest -q` **401 passed**;`ruff check backend/src/ely_eye backend/tests bench` 干净;`bench.filesize_check` 干净;`bench.audit --product` → `audited 32 harness files, 2 reports, 6 set files` / `clean`;mypy 冷缓存 **24 errors in 9 files (checked 112 source files)** = `9cb46c1` 基线,净新增 0;`./node_modules/.bin/pyright backend/src/ely_eye/cli.py backend/tests/test_search_api.py` → **0 errors**;本轮未动 Go/前端/报告,所以 go/npm 套件沿用 `83dfc14` 的绿。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -20,6 +20,31 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 0.5 现在的状态(2026-10-09 复核,逐项跑过)
|
||||||
|
|
||||||
|
上面那三句结论在审查当天成立。经过 `CHECKLIST.md` 上的 b→P0→P1→P2 之后,**12 项差距里 10 项已闭合、2 项部分闭合**;未闭合的两项卡在同一个地方:视觉与音频腿的能力,不是架构。
|
||||||
|
|
||||||
|
| 差距 | 状态 | 证据(跑过的命令 / 报告) |
|
||||||
|
|---|---|---|
|
||||||
|
| 3.1 记忆模型:谁、何时、何事、对谁可见 | **闭合** | schema 已到 v11;`holder_id/origin/scenario/event_time(+precision)/valid_from/valid_to/supersedes/visibility/retention_until` 全部落库并有往返测试(`backend/tests/test_memory_schema.py`);会话、重要度、归档层、冲突清单在 p2-3 落地(`test_memory_tiers.py`、`test_memory_conflicts.py`) |
|
||||||
|
| 3.2 没有写入回路 | **闭合** | `remember/supersede/forget` 一个 Python writer,HTTP/CLI/MCP 三面共用(`3d79241`);写入需 token,第三方来源无人佐证即隔离不可引用(p2-2,`test_trust_quarantine.py`) |
|
||||||
|
| 3.3 O(语料库) 检索 + 两套矛盾排序 | **闭合,但验收线未达** | 检索走 SQL FTS + RRF 融合(`674c88e`),向量在库内 vec0、与 atom 同一事务写入(`cd9851a`);165,973 atoms 上 p50 **0.0717 s**,替换掉的是 **20.16 s**(`bench/reports/SCALE.md`)。recall@10 **0.863**,验收线 0.90 **未达**(`bench/reports/LEGS.md`) |
|
||||||
|
| 3.4 卡带是导出包不是卡带 | **作用域闭合,算力未闭合** | `mount/switch/unmount/append` 让挂载真的改变检索域,指纹不符拒绝挂载(`9fe13ff`)。原文里「省下真实算力(前缀/KV 复用)」**没做**:`profiles.py` 的 YaRN/量化/CPU-offload 仍是引擎启动参数,没有前缀复用实现 |
|
||||||
|
| 3.5 没有 holder,就没有「找老婆」 | **闭合** | `who_knows` / `ask` / 目录 / 「没人知道」终态(`dfc6cd4`);p2-4 用 token 把「只给指针」变成真边界:无 token 搜不到私钥,MCP 进程不带 token 重启一遍仍守规则(`83dfc14`、`bash scripts/mcp_write_smoke.sh` 两行 PASS) |
|
||||||
|
| 3.6 没有工作记忆与巩固 | **闭合** | `working / consolidate / archive / archived / memory-restore / conflicts / resolve-conflicts`(p2-3) |
|
||||||
|
| 3.7 无文字图像结构性不可见 | **部分闭合** | CLIP 图文互查(`03ca357`)与语音+词级时间戳(`9f0f799`)已实现,但实测 recall@10 仍是 video **0.455** / image **0.720** / ui_screenshot **0.818**(code 0.945、pdf 0.885、conversation 1.000),且冻结库**一个 audio atom 都没有**——视频是抽帧+OCR,靠说话回答的问题无从匹配;CLIP 也无法在整屏截图里定位一个图标 |
|
||||||
|
| 3.8 6 个 adapter 只有 1 个真被加载 | **加载闭合,效果未测** | `adapters.py` 统一加载/校验/缓存(p1-3)。9B 生成、HashHop、标准基准在本仓库**没跑过**,所以 README 与 `TRAINING_ARTIFACTS.md` 不再引用任何未跑分数 |
|
||||||
|
| 3.9 证明外壳比能力跑得远 | **闭合** | `evals.py` 重写:跑不动的检查报告它跑不动,不再静默通过;第三方语料 50,789 sources / 5,308 queries 冻结在 `bench/sets/`,`python -m bench.validate_sets` 校验;`python -m bench.audit --product` 专抓「harness 自证」 |
|
||||||
|
| 3.10 工程与可复现性 | **闭合** | 15 个 `.sh` 双胞胎 + Docker + Linux CI + `backup/restore/doctor`(p1-4);每个 tracked 源文件 ≤500 行、死代码清零、注释单行(p1-6,`bench/filesize_check`) |
|
||||||
|
| 3.11 能取不能记,能看不能操作 | **闭合** | HTTP 路由 21 → **38**;CLI 命令 **59**;MCP 工具 9 只读 → **23**(写/挂载/分层/目录/能力卡) |
|
||||||
|
| 3.12 安全与隐私 | **闭合** | 写入门禁 + 每 holder 独立 token(库里只存 sha256)+ 服务器决定眼睛 + 能力卡不含正文(p2-2、p2-4) |
|
||||||
|
|
||||||
|
四个根因现在的状态:**「命名即实现」**被 `bench.audit --product` 和「每条宣称指向一条可重跑命令」压住,README 已重写;**「生产者即验证者」**被第三方冻结语料 + chance 行 + 仓内 Mem0/Zep 对手段子破掉(`bench/reports/BASELINE_EXTERNAL.md`:同一 ranker 同一批条目,holder 作用域 0.866 对全库 0.412,**相差 0.454**——该报告只给两行与「这段差值是 holder 边界的价值、不是更好排序器的价值」这句话,差值是这里减出来的);**「语料库而非记忆」**随 3.1/3.5/3.6 闭合不再成立;**「单用户·单轮·只读」**随 3.2/3.5/3.11/3.12 闭合不再成立。
|
||||||
|
|
||||||
|
**还剩什么**:验收线 0.90 未达,缺口全在视觉/音频腿;卡带的算力节省(前缀/KV 复用)没做;生成侧(9B + adapter)在本仓库一次都没跑过。§6 的钥匙测试逐条跑通是终验,未跑通不标绿。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 1. 目的对齐(我理解的目标,以及「做到」长什么样)
|
## 1. 目的对齐(我理解的目标,以及「做到」长什么样)
|
||||||
|
|
||||||
你要的机制有两层,仓库目前只碰到了第一层的边:
|
你要的机制有两层,仓库目前只碰到了第一层的边:
|
||||||
|
|||||||
@@ -1,133 +1,150 @@
|
|||||||
# Ely-Eye
|
# Ely-Eye
|
||||||
|
|
||||||
Ely-Eye is a local multimodal context operating system built on Qwen3.5-9B. It
|
Ely-Eye is a local memory store for AI. Every memory it keeps carries who it
|
||||||
runs on a single consumer GPU and gives a small model three tiers of context: a
|
belongs to, where it came from, when it was true, in what situation, and who may
|
||||||
262K native window, a 1.01M extended window, and a 100M-token memory capsule it
|
read it. Retrieval answers as the person asking, so a store shared by several
|
||||||
can retrieve from, cite, and recall across sessions.
|
people and several agents does not blend them into one voice. A Context
|
||||||
|
Cartridge is a scope you mount, not only a bundle you export. Coding agents
|
||||||
|
reach the same store over MCP.
|
||||||
|
|
||||||
Under the hood it ingests PDFs, screenshots, video, and code into Evidence
|
The difference between this and a vector database is the referral: "I don't know
|
||||||
Atoms, packs them into a Context Cartridge, retrieves with sparse and Qwen
|
where the keys are, I know my wife knows, and I can find my wife." When the
|
||||||
embedding search, runs Qwen3.5-9B with trained LoRA adapters, and checks every
|
store cannot answer, it says nobody knows and names who does — instead of
|
||||||
answer with a verifier. A fifteen-check proof suite exercises the whole thing
|
inventing a plausible answer.
|
||||||
against the live model, so a real run can be told apart from a fabricated one.
|
|
||||||
|
|
||||||
The landing site lives at https://model.elydora.com/
|
Everything below is either measured in this repository, with the report and the
|
||||||
(its source is under `ely-eye-web/`, which this repository does not track).
|
command that re-measures it, or it is marked as not measured.
|
||||||
|
|
||||||
## Requirements
|
## What is measured
|
||||||
|
|
||||||
- Windows with an NVIDIA GPU and CUDA. Training and the proof suite were built
|
| question | answer | report |
|
||||||
on an RTX 4090; deployment targets a 16GB card such as an RTX 4080.
|
| --- | --- | --- |
|
||||||
- The extended-context runtimes (SGLang, vLLM, KTransformers, llama.cpp) are
|
| can it find the memory that answers a question? | recall@10 **0.863**, MRR 0.682, over 300 answerable probes drawn from six third-party corpora, each asked as the holder that corpus assigns it | `bench/reports/LEGS.md` |
|
||||||
optional and run under WSL. The default Windows runtime uses Transformers and
|
| is that any good? | a Mem0-style counterpart on the same corpus scores **0.866** inside one holder's memory and **0.412** across all of them; chance is 0.0002 whole-corpus and 0.369 inside one holder | `bench/reports/BASELINE_EXTERNAL.md` |
|
||||||
needs none of them.
|
| does the holder boundary matter? | **+0.454 recall@10** with an identical ranker and identical items — the gain is scoping, not ranking | same report |
|
||||||
|
| how fast at corpus size? | p50 **0.072 s**, p95 0.218 s over 165,973 atoms; the search this replaced took 20.16 s on the same corpus | `bench/reports/SCALE.md` |
|
||||||
|
| how much does it hold? | 165,973 atoms and 153,857 vectors in one SQLite file; the vector build runs 106 memories/s. One million is an extrapolation, not a measurement | `bench/reports/DENSE.md` |
|
||||||
|
| can a wordless picture be found? | yes for icon crops, weakly for whole screenshots — the report shows every arm, including the ones that hurt | `bench/reports/VISUAL.md` |
|
||||||
|
| what does speech cost? | WER **0.242** over 533 reference words; 258.18 s of audio transcribed in 3.12 s (68.4–111.2× realtime per video, CPU int8), byte-identical across two passes | `bench/reports/ASR.md` |
|
||||||
|
| is the acceptance bar met? | **no.** The bar is recall@10 ≥ 0.90; shipped is 0.863 | `bench/reports/LEGS.md` |
|
||||||
|
|
||||||
## Run it locally
|
Where the deficit lives, by modality (recall@10, same run): code 0.945,
|
||||||
|
conversation 1.000, pdf 0.885, ui_screenshot 0.818, image 0.720, video 0.455.
|
||||||
|
Video is the worst because the frozen stores hold no audio atoms — a video is
|
||||||
|
sampled frames plus OCR, so a question answered by speech has nothing to match.
|
||||||
|
Images and screenshots are limited because CLIP cannot localize an icon inside a
|
||||||
|
screenshot. `bench/reports/LEGS.md` names the arms; nothing here rounds those
|
||||||
|
numbers up.
|
||||||
|
|
||||||
```powershell
|
Re-measure any of them:
|
||||||
.\scripts\install.ps1 # virtual env, PyTorch + CUDA, Transformers, PEFT, bitsandbytes
|
|
||||||
.\scripts\download-models.ps1 # Qwen3.5-9B, the 0.8B planner, Qwen3-VL-Embedding-8B
|
```bash
|
||||||
.\scripts\serve-backend.ps1 # API on http://127.0.0.1:8765
|
python -m bench.validate_sets # the frozen corpus and its gold
|
||||||
.\scripts\serve-frontend.ps1 # dashboard on http://127.0.0.1:5173
|
python -m bench.leg_check --probes 300 # the shipped retrieval path
|
||||||
|
python -m bench.run_external # the counterparts and the chance floor
|
||||||
|
python -m bench.dense_scale # the vector leg at corpus size
|
||||||
```
|
```
|
||||||
|
|
||||||
When `ELY_EYE_ADAPTER_DIR` is empty, the runtime picks the latest trained
|
## What is not measured here
|
||||||
HME-Core-LoRA adapter for `Qwen/Qwen3.5-9B` on its own; `ely-eye status` prints
|
|
||||||
which adapter it resolved.
|
|
||||||
|
|
||||||
For the 262K and 1.01M extended-context profiles, set up the Linux runtimes once
|
The generator side is written and runnable but has not been run in this
|
||||||
and launch one:
|
repository's reports, so no score from it is quoted anywhere in this README:
|
||||||
|
|
||||||
```powershell
|
- generation with Qwen3.5-9B and the trained LoRA adapters — `ely-eye runtime-generation-proof`
|
||||||
.\scripts\setup-linux-runtimes.ps1
|
- Visual HashHop multi-hop visual addressing — `ely-eye hashhop --kind visual`
|
||||||
.\scripts\launch-sglang-live.ps1 # 262K
|
- the Visual Contradiction Lens — `ely-eye visual-contradiction-proof`
|
||||||
.\scripts\launch-sglang-extreme.ps1 # 1.01M via YaRN
|
- standard benchmarks (MMMU-Pro, OCRBench, and the rest) — `ely-eye run-benchmark`
|
||||||
|
- the fifteen-check proof suite as a whole — `ely-eye proof-suite`
|
||||||
|
|
||||||
|
Each needs a CUDA machine and a model download. They are marked DEFERRED in
|
||||||
|
`CHECKLIST.md`, and a run that has not happened is not allowed to look like a
|
||||||
|
result: `ely-eye proof-suite` writes what it observed, and the checks that
|
||||||
|
depend on a model report what they could not do rather than passing quietly.
|
||||||
|
|
||||||
|
`TRAINING_ARTIFACTS.md` keeps the six-adapter training table, which is
|
||||||
|
artifact-backed on the machine that trained them, and no longer quotes benchmark
|
||||||
|
scores that were never run.
|
||||||
|
|
||||||
|
## What a memory carries
|
||||||
|
|
||||||
|
`holder_id`, `origin` (self / observed / artifact / third_party), `scenario`,
|
||||||
|
`event_time` with its precision and how it was learned, `valid_from` /
|
||||||
|
`valid_to`, `supersedes`, `visibility` (holder / visible_to / everyone),
|
||||||
|
`retention_until`, plus ingestion time and trust scores. Ingestion stamps
|
||||||
|
provenance; a client cannot claim to be somebody else, because the server
|
||||||
|
decides who a caller is from the token it presents.
|
||||||
|
|
||||||
|
Four verbs, one writer: `remember`, `supersede`, `forget`, `search`. They exist
|
||||||
|
over HTTP, the CLI, and MCP, and they all go through the same Python writer, so
|
||||||
|
the rules cannot drift apart. `forget` cascades to everything derived — vectors,
|
||||||
|
FTS rows, edges, cache entries — and reports which cartridges contained the
|
||||||
|
memory rather than silently rewriting an export whose digest somebody already
|
||||||
|
quoted.
|
||||||
|
|
||||||
|
On top of that: a working tier (what one session recorded, in order),
|
||||||
|
consolidation into a standing summary, an archive tier for memories that are
|
||||||
|
neither used nor valued, and a conflict list for dated memories that disagree
|
||||||
|
and that nobody has settled. A third-party memory that no other memory vouches
|
||||||
|
for is quarantined: it stays in the store, and it is not citable.
|
||||||
|
|
||||||
|
## Run it
|
||||||
|
|
||||||
|
Linux and macOS:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/install.sh
|
||||||
|
bash scripts/serve-backend.sh # API on http://127.0.0.1:8765
|
||||||
|
bash scripts/serve-frontend.sh # dashboard on http://127.0.0.1:5173
|
||||||
```
|
```
|
||||||
|
|
||||||
## Core commands
|
Windows: the same names as `.ps1`. There is also a container image
|
||||||
|
(`docker build -t ely-eye .`), and CI runs the whole suite on Linux.
|
||||||
|
|
||||||
```powershell
|
```bash
|
||||||
.\.venv\Scripts\ely-eye.exe status
|
ely-eye ingest ~/some/package --cartridge-name project
|
||||||
.\.venv\Scripts\ely-eye.exe ingest C:\path\to\package --cartridge-name project
|
ely-eye search "where did we put the archive key" --as-holder wife
|
||||||
.\.venv\Scripts\ely-eye.exe hashhop --kind visual --hops 2 --token-equivalent 262144
|
ely-eye remember "the archive key is in the second drawer" --holder wife --scenario home
|
||||||
.\.venv\Scripts\ely-eye.exe proof-suite
|
ely-eye who-knows "archive key"
|
||||||
|
ely-eye ask "where is the archive key" --as-holder husband
|
||||||
|
ely-eye card
|
||||||
|
ely-eye mounts && ely-eye mount project && ely-eye unmount
|
||||||
|
ely-eye backup ./backup && ely-eye doctor
|
||||||
|
bash scripts/mcp_write_smoke.sh # two processes, one store, one SQLite
|
||||||
```
|
```
|
||||||
|
|
||||||
The console is served by the API itself once `ELY_EYE_UI_DIR` points at
|
The dashboard is served by the API itself once `ELY_EYE_UI_DIR` points at
|
||||||
`frontend/dist`. It shows the state the CLI prints — runtime, GPU, library,
|
`frontend/dist`. `scripts/ui_smoke.sh` drives it in a real browser against a
|
||||||
cartridges, senses, adapters, proof suite — and it is the only place to search
|
fresh store and fails if any of it stops working. The CLI does everything the
|
||||||
the store as somebody, page through atoms, forget one, and mount a cartridge.
|
console does, so nothing requires a browser.
|
||||||
`scripts/ui_smoke.sh` drives that console in a real browser against a fresh
|
|
||||||
store and fails if any of it stops working.
|
|
||||||
|
|
||||||
## Fine-tuning
|
## Retrieval
|
||||||
|
|
||||||
Training runs real BF16 LoRA updates on the GPU through Unsloth-patched
|
Three legs, fused by rank. The words leg always runs. The dense leg runs only
|
||||||
Transformers, PEFT, and bitsandbytes with paged 8-bit AdamW. Each run writes the
|
when `ELY_EYE_DENSE_EMBEDDINGS_ENABLED=1` and an embedding model is configured.
|
||||||
PEFT adapter alongside a manifest, a per-step training trace, a summary, and a
|
On the shipped page that leg moves recall from 0.863 to 0.860 while p50 goes
|
||||||
proof file that records dataset and weight hashes, per-step loss, optimizer-update
|
from 122 ms to 713 ms; on a wider page both reach 0.873 while p50 goes from
|
||||||
norms, and the CUDA/BF16 environment.
|
84 ms to 579 ms. It buys the memory questions (multi-session 0.857 → 0.920,
|
||||||
|
preference 0.800 → 0.960) and costs the picture ones (image 0.720 → 0.680),
|
||||||
The adapter set:
|
which is why it is opt-in rather than the default. The picture leg runs only
|
||||||
|
when the words page comes back thin and `ELY_EYE_CLIP_MODEL_DIR` is set.
|
||||||
```text
|
Vectors live in the database beside the atoms, written in the same transaction
|
||||||
HME-Core-LoRA Qwen3.5-9B attention/MLP
|
as the memory itself, so a memory is findable the moment it is stored and there
|
||||||
HME-Vision-LoRA Qwen3.5-9B visual projector and selected vision layers
|
is no index to rebuild.
|
||||||
HME-TTT-VL Qwen3.5-9B mutable MLP and visual adapter
|
|
||||||
HME-Visual-MTP Qwen3.5-9B draft-head LoRA
|
|
||||||
HME-Router Qwen3.5-0.8B context planner
|
|
||||||
HME-Retrieval Qwen3-VL-Embedding-8B contrastive retrieval adapter
|
|
||||||
```
|
|
||||||
|
|
||||||
## Proof suite
|
|
||||||
|
|
||||||
`ely-eye proof-suite` runs fifteen checks against the live model and writes the
|
|
||||||
evidence under `.ely_eye/data/eval_proofs/`. Among them:
|
|
||||||
|
|
||||||
- real CUDA BF16 LoRA training — changed weight hashes, nonzero optimizer
|
|
||||||
updates, per-step loss
|
|
||||||
- the runtime loading the model and adapter and generating a cited answer
|
|
||||||
- Visual HashHop: multi-hop visual addressing the model has to actually solve,
|
|
||||||
with no OCR shortcut
|
|
||||||
- the 100M-token memory capsule committing to a reproducible Memory DNA
|
|
||||||
- the Visual Contradiction Lens localizing cross-version drift (design-token,
|
|
||||||
layout, typography, copy, visual-code, temporal) at >=80% recall with no false
|
|
||||||
positives
|
|
||||||
- standard benchmark samples (MMMU-Pro, OCRBench, and others) scored against
|
|
||||||
ground truth
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
.\.venv\Scripts\ely-eye.exe proof-suite
|
|
||||||
.\.venv\Scripts\ely-eye.exe latest-proof-suite
|
|
||||||
.\.venv\Scripts\ely-eye.exe runtime-generation-proof
|
|
||||||
.\.venv\Scripts\ely-eye.exe visual-contradiction-proof
|
|
||||||
```
|
|
||||||
|
|
||||||
## MCP server for coding agents
|
## MCP server for coding agents
|
||||||
|
|
||||||
`ely-eye-mcp/` is a local Model Context Protocol server that exposes the
|
`ely-eye-mcp/` is a Go MCP server. One command builds a native binary and
|
||||||
`.ely_eye` memory store to coding agents such as Codex and Claude Code. One
|
registers it with Codex and Claude Code. Reads mirror the store; writes go
|
||||||
command builds a native Go binary (or downloads a prebuilt one), installs it
|
through the API, so the write rules live in one place. A read with no
|
||||||
under `.ely_eye/bin/`, and registers it with both clients. No Docker.
|
`ELY_EYE_AUTH_TOKEN` sees only memories nobody owns and memories marked for
|
||||||
|
everyone; `ely_eye_card` answers "can this store answer my kind of question"
|
||||||
Windows:
|
before anything sensitive is sent, and it carries no memory text.
|
||||||
|
|
||||||
```powershell
|
|
||||||
powershell -ExecutionPolicy Bypass -File .\ely-eye-mcp\install.ps1 -Client both
|
|
||||||
```
|
|
||||||
|
|
||||||
Linux and macOS:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash ./ely-eye-mcp/install.sh --client both
|
bash ./ely-eye-mcp/install.sh --client both
|
||||||
```
|
```
|
||||||
|
|
||||||
The agent then has nine read-only tools over the live memory — status,
|
See `ely-eye-mcp/README.md` for the full tool list.
|
||||||
cartridge listing and detail, Evidence Atom search and fetch, evidence-pack
|
|
||||||
compilation, proof suites, and HashHop proofs. The server opens the database
|
|
||||||
read-only and never writes to it. The same installer works against any
|
|
||||||
`.ely_eye` store, not only this repository. See `ely-eye-mcp/README.md` for the
|
|
||||||
full tool list and the release pipeline.
|
|
||||||
|
|
||||||
## Data layout
|
## Data layout
|
||||||
|
|
||||||
@@ -135,27 +152,31 @@ Runtime state lives under `.ely_eye/` (git-ignored):
|
|||||||
|
|
||||||
```text
|
```text
|
||||||
.ely_eye/
|
.ely_eye/
|
||||||
├─ adapters/ trained LoRA adapters
|
├─ data/ the SQLite store: atoms, FTS, vectors (vec0), edges, mounts
|
||||||
├─ data/ sqlite database and eval proofs
|
|
||||||
├─ training/ datasets and training traces
|
|
||||||
├─ objects/ ingested source files
|
├─ objects/ ingested source files
|
||||||
├─ cache/ cache fabric
|
├─ cache/ compiled-context cache and the picture index
|
||||||
└─ cartridges/ materialized Context Cartridges
|
├─ cartridges/ materialized Context Cartridges
|
||||||
|
├─ adapters/ trained LoRA adapters
|
||||||
|
└─ training/ datasets and training traces
|
||||||
```
|
```
|
||||||
|
|
||||||
A Context Cartridge is a self-contained memory package: the Evidence Atoms, a
|
A Context Cartridge is a self-contained memory package: the Evidence Atoms, the
|
||||||
sparse index and vector indexes, a temporal graph, KV snapshots, the memory
|
sparse and vector indexes, the temporal graph, and a DNA digest of the files.
|
||||||
capsule, any attached adapters, and the eval proofs.
|
Mounting one scopes every subsequent search to it; switching is one command.
|
||||||
|
|
||||||
## Project layout
|
## Project layout
|
||||||
|
|
||||||
```text
|
```text
|
||||||
backend/ Python: ingestion, retrieval, runtime, training, proof suite, FastAPI
|
backend/ Python: ingestion, retrieval, memory, runtime, training, FastAPI
|
||||||
frontend/ the local dashboard (Vite + React)
|
bench/ the benchmark suite: frozen third-party corpora, harnesses, reports
|
||||||
scripts/ install, model download, serving, runtimes, fine-tuning (.ps1 on Windows, .sh on Linux)
|
frontend/ the local dashboard (Vite + React)
|
||||||
ely-eye-mcp/ Go: local MCP server exposing the memory store to coding agents
|
scripts/ install, serving, runtimes, smoke (.sh on Linux/macOS, .ps1 on Windows)
|
||||||
|
ely-eye-mcp/ Go: the MCP server
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`CHECKLIST.md` is the ordered work list with the acceptance line for every item;
|
||||||
|
`Gaps.md` is the audit this work started from.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
AGPL-3.0-only. Copyright © 2026 雷电芽衣 <zhangyanghaha0407@outlook.com>.
|
AGPL-3.0-only. Copyright © 2026 雷电芽衣 <zhangyanghaha0407@outlook.com>.
|
||||||
|
|||||||
+66
-3
@@ -1,5 +1,21 @@
|
|||||||
# Ely-Eye Training Artifacts
|
# Ely-Eye Training Artifacts
|
||||||
|
|
||||||
|
## What this file is, and what it is not
|
||||||
|
|
||||||
|
It is a record of what was produced on **2026-05-28** on the Windows + RTX 4090
|
||||||
|
machine that trained the adapters, from a `PRD.md` that no longer exists in this
|
||||||
|
repository. The adapters themselves, the cartridge, and every `.ely_eye/` path
|
||||||
|
listed below are runtime state and are **not tracked by git**, so a clone of this
|
||||||
|
repository cannot reproduce them.
|
||||||
|
|
||||||
|
What is **measured in this repository today** lives in `bench/reports/` and is
|
||||||
|
reproducible with the commands in `README.md`. Nothing in this file belongs to
|
||||||
|
that set. Scores that were never run against ground truth have been deleted
|
||||||
|
rather than kept with a hedge; where a number is kept, the sentence next to it
|
||||||
|
says what the number actually measures.
|
||||||
|
|
||||||
|
## Artifacts
|
||||||
|
|
||||||
Generated on 2026-05-28 from the local `PRD.md`, ingested Evidence Atom
|
Generated on 2026-05-28 from the local `PRD.md`, ingested Evidence Atom
|
||||||
`PRD_0b88b12114634a56:text`, visual Evidence Atom
|
`PRD_0b88b12114634a56:text`, visual Evidence Atom
|
||||||
`ely_eye_dashboard_0180927ffdd3f8b0:image`, local UI proof images, and Qwen
|
`ely_eye_dashboard_0180927ffdd3f8b0:image`, local UI proof images, and Qwen
|
||||||
@@ -68,6 +84,11 @@ Runtime generation proof:
|
|||||||
transformers generated 218 tokens with hme_core_lora citation accuracy 1.00
|
transformers generated 218 tokens with hme_core_lora citation accuracy 1.00
|
||||||
```
|
```
|
||||||
|
|
||||||
|
"Citation accuracy 1.00" there means every atom id the answer cited existed in
|
||||||
|
the context that was handed to the model. It is not entailment: the check cannot
|
||||||
|
tell a correct citation from a citation of the wrong passage, and the question
|
||||||
|
used by that check contained its own answer.
|
||||||
|
|
||||||
100M memory capsule proof:
|
100M memory capsule proof:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
@@ -75,6 +96,13 @@ transformers generated 218 tokens with hme_core_lora citation accuracy 1.00
|
|||||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/memory_capsule_index.json
|
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/memory_capsule_index.json
|
||||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/memory_capsule.json
|
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/memory_capsule.json
|
||||||
The Context Cartridge materialized a 100,000,000 token-equivalent memory capsule with 4,096 committed segments and a verified segment Merkle root.
|
The Context Cartridge materialized a 100,000,000 token-equivalent memory capsule with 4,096 committed segments and a verified segment Merkle root.
|
||||||
|
|
||||||
|
The 100M is a property of a capsule the check generated for itself, not 100
|
||||||
|
million tokens of ingested content: the segments were synthesized, and the check
|
||||||
|
verified that the capsule hashed to its own recorded digest. What the store
|
||||||
|
actually holds today is measured in `bench/reports/DENSE.md` (165,973 atoms,
|
||||||
|
153,857 vectors), and `Gaps.md` appendix B prices one million tokens of live KV
|
||||||
|
cache at 148.9 GB in fp16.
|
||||||
```
|
```
|
||||||
|
|
||||||
Model-solved HashHop proofs:
|
Model-solved HashHop proofs:
|
||||||
@@ -87,7 +115,12 @@ Model-solved HashHop proofs:
|
|||||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/4f24efd13342694c/model_visual_hashhop.json
|
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/4f24efd13342694c/model_visual_hashhop.json
|
||||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/a3f49402ef2710a3/model_visual_hashhop.json
|
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/a3f49402ef2710a3/model_visual_hashhop.json
|
||||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/0bd17526ab51392b/model_visual_hashhop.json
|
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/hashhop/0bd17526ab51392b/model_visual_hashhop.json
|
||||||
Qwen/Qwen3.5-9B with HME-Core-LoRA solved a 1.01M token-equivalent 2-hop text HashHop and 6 Visual HashHop arenas at 262K token-equivalent. The Visual HashHop gate passed 7/7 model-solved proofs, with 262K 1-hop=1.00 and 2-hop=1.00.
|
Qwen/Qwen3.5-9B with HME-Core-LoRA solved a 1.01M token-equivalent 2-hop text HashHop and 6 Visual HashHop arenas at 262K token-equivalent.
|
||||||
|
|
||||||
|
The check that produced this collected proofs until its pass-rate target was
|
||||||
|
reached, so the file records that the model could solve these arenas, not a
|
||||||
|
measured solve rate. The retry loop was removed in the audit pass; a re-run now
|
||||||
|
reports the rate it got.
|
||||||
```
|
```
|
||||||
|
|
||||||
Benchmark source and sample proof:
|
Benchmark source and sample proof:
|
||||||
@@ -96,7 +129,14 @@ Benchmark source and sample proof:
|
|||||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/standard_benchmark_sources.json
|
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/standard_benchmark_sources.json
|
||||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/standard_benchmark_samples.json
|
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/standard_benchmark_samples.json
|
||||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/standard_benchmark_predictions.json
|
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/standard_benchmark_predictions.json
|
||||||
MMMU-Pro, OCRBench, MMLongBench-Doc, VideoMME, LongVideoBench, RULER, and LongBench v2 loaded official streaming samples or official public metadata samples with sample hashes and source dataset SHA values. LongVideoBench uses `longvideobench/LongVideoBench-Meta` validation metadata with canonical `longvideobench/LongVideoBench` registry tracking. MMMU-Pro and OCRBench generated local Qwen/Qwen3.5-9B + HME-Core-LoRA predictions from official image samples, with local sample scores MMMU-Pro=1.00 and OCRBench=1.00.
|
MMMU-Pro, OCRBench, MMLongBench-Doc, VideoMME, LongVideoBench, RULER, and LongBench v2 loaded official streaming samples or official public metadata samples with sample hashes and source dataset SHA values. LongVideoBench uses `longvideobench/LongVideoBench-Meta` validation metadata with canonical `longvideobench/LongVideoBench` registry tracking.
|
||||||
|
|
||||||
|
No benchmark score is quoted here. The prediction step read **one** sample per
|
||||||
|
benchmark (`next(iter(stream))` with `streaming=True`), so MMMU-Pro and OCRBench
|
||||||
|
produced a single local prediction each; a score computed from one sample is not
|
||||||
|
a benchmark result and the two `=1.00` lines that used to sit in this file have
|
||||||
|
been deleted. `bench/reports/` is where this repository
|
||||||
|
reports measured numbers, on corpora with hundreds of queries.
|
||||||
```
|
```
|
||||||
|
|
||||||
Repository input proof:
|
Repository input proof:
|
||||||
@@ -104,6 +144,10 @@ Repository input proof:
|
|||||||
```text
|
```text
|
||||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/repository_input.json
|
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/repository_input.json
|
||||||
The repository input proof indexed 94 files, 49 code files, 11,977 call graph edges, and 26 local git commits for PRD 10.1 code repository coverage.
|
The repository input proof indexed 94 files, 49 code files, 11,977 call graph edges, and 26 local git commits for PRD 10.1 code repository coverage.
|
||||||
|
|
||||||
|
That is a snapshot of the repository as it stood on 2026-05-28; the repository is
|
||||||
|
larger now, and the check passed on any git history, which is why it is not
|
||||||
|
treated as evidence of anything today.
|
||||||
```
|
```
|
||||||
|
|
||||||
Runtime profile readiness:
|
Runtime profile readiness:
|
||||||
@@ -111,6 +155,11 @@ Runtime profile readiness:
|
|||||||
```text
|
```text
|
||||||
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/runtime_profiles.json
|
.ely_eye/data/eval_proofs/prd_proof_suite_b03b88d7cfe5ba96/runtime_profiles.json
|
||||||
SGLang 0.5.12.post1, vLLM 0.21.0, KTransformers 0.6.2.post4, WSL RTX 4090 GPU visibility, launcher module probes, and winget llama-server help output are verified.
|
SGLang 0.5.12.post1, vLLM 0.21.0, KTransformers 0.6.2.post4, WSL RTX 4090 GPU visibility, launcher module probes, and winget llama-server help output are verified.
|
||||||
|
|
||||||
|
Verified inside WSL on that machine, against the fixed distro name the launcher
|
||||||
|
used at the time. The launch profiles now read `ELY_EYE_WSL_DISTRO`, every
|
||||||
|
`.ps1` has a `.sh` twin for Linux and macOS, and CI runs the suite on Linux — so
|
||||||
|
this line is a record of one machine, not a portability claim.
|
||||||
```
|
```
|
||||||
|
|
||||||
Checks:
|
Checks:
|
||||||
@@ -123,7 +172,7 @@ Cartridge Physical Assets passed
|
|||||||
100M Memory Capsule passed
|
100M Memory Capsule passed
|
||||||
Context Cartridge Replay passed
|
Context Cartridge Replay passed
|
||||||
Visual HashHop passed # 7/7 model-solved proofs
|
Visual HashHop passed # 7/7 model-solved proofs
|
||||||
Standard Benchmark Registry passed # MMMU-Pro=1.00, OCRBench=1.00 local samples
|
Standard Benchmark Registry passed # one local sample per benchmark
|
||||||
Repository Input Coverage passed # 94 files + 11977 call graph edges + local git history
|
Repository Input Coverage passed # 94 files + 11977 call graph edges + local git history
|
||||||
Runtime Profile Delivery passed
|
Runtime Profile Delivery passed
|
||||||
Research Module Delivery passed # executable module gates, evidence graph edge, probability sum
|
Research Module Delivery passed # executable module gates, evidence graph edge, probability sum
|
||||||
@@ -132,6 +181,13 @@ Memory DNA passed
|
|||||||
Cache Trace passed
|
Cache Trace passed
|
||||||
```
|
```
|
||||||
|
|
||||||
|
These are the rows that suite wrote on 2026-05-28, not a current verdict. `Gaps.md`
|
||||||
|
appendix A judges each of the fifteen checks individually, and several of them
|
||||||
|
were self-referential — the replay check compiled the same question twice against
|
||||||
|
the live store, the recall check only proved an adapter file loads, the cache check
|
||||||
|
passed if any layer had any entry. The suite was rewritten afterwards so a check
|
||||||
|
that cannot run says what it could not run instead of passing.
|
||||||
|
|
||||||
Cartridge physical assets:
|
Cartridge physical assets:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
@@ -142,3 +198,10 @@ Cartridge physical assets:
|
|||||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/ttt_vl_adapter.safetensors
|
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/ttt_vl_adapter.safetensors
|
||||||
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/visual_mtp_head.safetensors
|
.ely_eye/cartridges/ely_eye_prd_7e5cd27b8284/visual_mtp_head.safetensors
|
||||||
```
|
```
|
||||||
|
|
||||||
|
A cartridge written today is not this list. `cartridge.py` now writes
|
||||||
|
`manifest.json`, `atoms.parquet`, `sources.json`, `sparse_index.bm25.zst` and
|
||||||
|
`temporal_graph.sqlite`, and a cartridge is a scope you mount rather than only a
|
||||||
|
bundle you export. The files above are what that 2026-05-28 export contained;
|
||||||
|
`kv_snapshots.kivi` and `ttt_vl_adapter.safetensors` were placeholders for a
|
||||||
|
runtime layout that was never wired to a loader.
|
||||||
@@ -88,6 +88,7 @@ def search(
|
|||||||
query: str,
|
query: str,
|
||||||
top_k: int = typer.Option(0, "--top-k", help="hits to return; 0 uses the configured default"),
|
top_k: int = typer.Option(0, "--top-k", help="hits to return; 0 uses the configured default"),
|
||||||
holder: str = typer.Option("", "--holder", help="only this holder's memories may answer"),
|
holder: str = typer.Option("", "--holder", help="only this holder's memories may answer"),
|
||||||
|
as_holder: str = typer.Option("", "--as-holder", help="see the store through this holder's eyes, the way their agent would"),
|
||||||
scenario: str = typer.Option("", "--scenario"),
|
scenario: str = typer.Option("", "--scenario"),
|
||||||
source_id: str = typer.Option("", "--source-id"),
|
source_id: str = typer.Option("", "--source-id"),
|
||||||
since: str = typer.Option("", "--since", help="event time at or after, ISO with offset"),
|
since: str = typer.Option("", "--since", help="event time at or after, ISO with offset"),
|
||||||
@@ -103,6 +104,7 @@ def search(
|
|||||||
top_k or None,
|
top_k or None,
|
||||||
filters=SearchFilters(
|
filters=SearchFilters(
|
||||||
holder=holder or None,
|
holder=holder or None,
|
||||||
|
as_holder=as_holder or None,
|
||||||
scenario=scenario or None,
|
scenario=scenario or None,
|
||||||
source_id=source_id or None,
|
source_id=source_id or None,
|
||||||
since=moment(since, "--since"),
|
since=moment(since, "--since"),
|
||||||
|
|||||||
@@ -112,3 +112,8 @@ def test_the_cli_answers_in_json_a_machine_can_read(tmp_path: Path, monkeypatch:
|
|||||||
|
|
||||||
empty = json.loads(runner.invoke(app, ["search", "钥匙", "--scenario", "travel"]).stdout)
|
empty = json.loads(runner.invoke(app, ["search", "钥匙", "--scenario", "travel"]).stdout)
|
||||||
assert empty["status"] == "no_match"
|
assert empty["status"] == "no_match"
|
||||||
|
|
||||||
|
stranger = json.loads(runner.invoke(app, ["search", "钥匙", "--as-holder", "stranger"]).stdout)
|
||||||
|
assert stranger["status"] == "no_match", "--as-holder must show what a stranger would see"
|
||||||
|
owner = json.loads(runner.invoke(app, ["search", "钥匙", "--as-holder", "wife"]).stdout)
|
||||||
|
assert [hit["atom"]["text"] for hit in owner["hits"]] == [KEYS]
|
||||||
Reference in new issue
Block a user