diff --git a/CHECKLIST.md b/CHECKLIST.md index a0a8be0..461a758 100644 --- a/CHECKLIST.md +++ b/CHECKLIST.md @@ -287,12 +287,33 @@ - 验收：CI 绿；干净 clone 上 `docker build` 成功（已实测 `09803cd49add` + 容器 curl 通）；桩 `nvidia-smi` 打印 RTX 5090 证明设备名不再致命（已实测） - 独立 review（同一个 `p1review` 里的 pi，模型 `strata/qwen3.8-flash-next-abliterated-iq3_s:high`，三轮，不读我的话只跑东西）：第一轮 **FAIL 9 条**（全是 CHECKLIST 里的数字与断言不重现：测试文件数、套件规模、构建上下文体积、`profiles.py` 异常元组、`training.py` 行号、`cli.py` 行数、把 4090 说成不支持 BF16、备份摘要不可复现、秒数写死）→ 第二轮 **FAIL 1 条**（`_TinyDense` 那两行 docstring 是 p1-4 唯一新增的多行注释，跟 p1-6.4 的「≤1 行」相冲）→ **第三轮 PASS**。reviewer 自己跑 `gh run view`、`pytest`、`ruff`、`mypy`（冷缓存逐文件比对 `908157f` 与 HEAD → 两边都是 24 错/6 文件，逐条同消息）、`pyright`、go 全套、`bash -n` 15 个 `.sh`、`pip wheel`、`sha256sum LICENSE` 与 gnu.org 逐字节比对、拉起 22.4GB 容器 curl 三个 URL，并在真 worktree 里做变异测试（去掉 reranker 桩 → 2 failed；去掉 dense 桩 → 正好复现 CI 那 4 条；只翻 `bench/systems.py:414` 的排序键 → 正好 `fusion_rerank` 那条挂）。它还量了全仓多行 docstring 数 `114 → 114`，确认 p1-4 现在新增 0 条。三轮同一个人，没换人放水。 - 门禁：`pytest -q` **253 passed**；`ruff check backend/src bench` All checks passed（HEAD 是 17 errors）；mypy 冷缓存查改动文件与 HEAD 同错、净新增 0；`pyright` 查 maintenance 三个文件 → **0 errors, 0 warnings**；go build/vet/test + gofmt 干净；`bash -n` 15 个 `.sh` 干净；`pip wheel ./backend` 能建且元数据带 license 表达式 -- [ ] **p1-5** 产品面 - - [ ] p1-5.1 `GET /api/search` - - [ ] p1-5.2 前端：搜索、浏览、forget、挂载卡带、时间显示、图像渲染 - - [ ] p1-5.3 `types.ts` 补回 `time` / `source_id` / `trust` / `relations` - - [ ] p1-5.4 12 个未用端点：接上或删除 - - 验收：E2E 测试覆盖五项；`grep` 无未接端点残留 +- [x] **p1-5** 产品面 + - [x] p1-5.1 搜索端点接上前端 + - 验收写的是 `GET /api/search`，实际是 p0-3 就有的 `POST /api/search`：查询体带 holder / scenario / modality / image，GET 塞不下这些。前端 `search()` 调它 + - [x] p1-5.2 前端：搜索、浏览、forget、挂载卡带、时间显示、图像渲染 + - `App.tsx` 517 行拆成 130 行的壳 + `components/{MemoryPanel,CartridgePanel,SensesPanel,AdapterPanel,ProofPanel,IngestPanel,CompilerPanel,ui}.tsx` + `lib/format.ts`，每个面板自己取数，没有一个文件超 500 行 + - 为此新增的后端能力在 `backend/src/ely_eye/browse.py`：`GET /api/atoms`（游标翻页 + modality / holder / source 过滤，读 `v_atoms_current`）与 `GET /api/atoms/{atom_id}/image`（`FileResponse`；没图 → 404 点名那颗 atom） + - 时间显示：`formatAgo`（just now / 5m / 2h / 3d / 1y）与 `formatStamp`（写死 UTC，`2026-10-08 09:03Z`）；每行是 `holder · origin · scenario · 时间 · trust` + - 图像渲染：``，E2E 断言 `naturalWidth > 0`——真解码出像素，不是有个标签就算 + - [x] p1-5.3 `types.ts` 补回记忆维度 + - 验收写的 `time` 在 p0-1 已改名 `ingested_at`，另有 `event_time / valid_from / valid_to`。补的是 `source_id, ingested_at, holder_id, origin, scenario, event_time, event_time_precision, valid_from, valid_to, trust`。`relations` 没补：至今没人写 atom 的 relations（p0 记过这条），前端拿它没东西可渲染 + - 另加 `AtomPage, SearchFilters, SenseStatus, CartridgeManifest, MountState, AdapterChoice, AdapterLoadReport, Runtime*Profile*, ForgetReport`；`MemoryMap` 类型随端点一起删 + - [x] p1-5.4 未用端点：接上或删除 + - 删 8 个：`/api/benchmarks/{registry,sources,samples,predictions}`、`/api/research/modules`、`/api/index/dense`、`/api/visual-contradiction-proof`、`/api/memory-map`（前端是 memory-map 唯一的消费者，状态卡本来就带 layer 与卡带数）。底层函数一个没删——`cli.py` 与 `proofs/checks_visual.py` 在用；删的是没人调的 HTTP 壳 + - 加 2 个：`/api/atoms`、`/api/atoms/{atom_id}/image`；`/api/status` 现在带 `senses`。HTTP 面从 36 条到 **30** 条 + - 反漂移门禁：`backend/tests/test_product_api.py::test_the_http_surface_is_exactly_what_someone_uses` 把 30 条路由钉成集合，多一条少一条都红 + - [x] p1-5.5 E2E 五项 + - `scripts/ui_smoke.sh`：`mktemp -d` 起一个干净 home（`ELY_EYE_DEFAULT_HOLDER=wife`、`ELY_EYE_UI_DIR=frontend/dist`），API 自己用 `StaticFiles(html=True)` 同源挂 UI，浏览器只对一个 origin 说话，收尾删 home + - 五项都在真浏览器里做：① 搜 `spare glasses` + scenario `home` → 那行必须同时写 `wife` 和 `home`（「谁在什么场景下知道」就是这个意思）② 按 modality 过滤 → 无字图片渲染出真像素，翻页换掉整页 ③ Forget → 提示 `forgotten: N atoms`，再搜同一句返回「Nothing in the store answers」④ Mount → 状态真的变了 ⑤ Adapter Matrix 每个 kind 一行，状态落在 `loaded|available|deferred|missing` 内且**必须带原因** + - 实测：`mount: refused: cannot verify ui_smoke_26f91b8f3a22 on this machine: model_hash, tokenizer_hash, chat_template_hash, rope are unknown`（p0-4 那条「没有 model_dir 就拒绝挂载」的偏差在 UI 里看得见）；`adapters: 6 kinds: missing ×6`，每条原因是「no adapter of kind X is on disk」；KPI 显示 `Runtime ready`、`Qwen/Qwen3.5-9B`、`NVIDIA GeForce RTX 4090 20914/23028 MiB`、`Cartridges 1` + - 失败留证据：截图 `ui-failure.png` + Memory 面板文本 + 那一刻 `/api/atoms?modality=…` 返回的 JSON + - 网络噪声分开判：`404 /api/proof-suite/latest`（没跑过证明的库就该 404，面板写 pending）和被拒的挂载 400 算预期，其余 ≥400 一律红；JS 未捕获异常一律红 + - 三条是 E2E 逼出来的真缺陷，都不是样式问题： + - `AdapterPanel` 的指标条渲染 `load.gpu_available` 与 `load.probe_status`，而 `AdapterLoadReport`（`runtime_schemas.py:27`）只有 `model_id / mounted / adapters`——没 GPU 的机器上它会永远显示「no gpu visible」，而这台机器有 4090。删掉那两个 span 和 `types.ts` 里编出来的五个字段，只渲染服务端真给的 + - 我第一版 E2E 硬写 `modality=image`，结果 0 行：`ingestion_media.py:277` 按**文件名**判 kind（`ui_markers = ("screenshot","screen","figma","ui","wireframe","mockup")`），我那个文件叫 `ui-smoke-wordless.png` → 存成 `ui_screenshot`。改成从 `/api/atoms` 读回那颗 atom 的 modality 再选，库说是啥就是啥。顺带记一条本轮没动的产品缺陷：`"ui"` 是子串匹配，`guide.png`、`quickstart.png` 都会被叫成 UI 截图；不改是因为冻结的 bench 数字是在现规则下量的 + - playwright 1.60 在 Ubuntu 26.04 上没有 chromium 构建（`Playwright does not support chromium on ubuntu26.04-x64`），`npm i -D playwright@1.64.0` 才有；而且它的 npm 包不自动下浏览器，要 `npx playwright install chromium` + - 顺手挖掉的产品根因：`/api/status` 与 `doctor` 各算一遍「这台机器能感知什么」，两份真相。抽成 `backend/src/ely_eye/senses.py::sense_facts()`，两边只渲染它。图片计数从 modality 改成「有没有 `image_ref`」：按 modality 数会漏掉 pdf_region 的截图裁块，也看不见字节已被删走的 atom。现在 `picture_bytes_missing` 会让 doctor 说「N picture atoms with no bytes behind them; re-ingest the source they came from」，而 CLIP 那条警告只数还能编码的 + - 验收：`bash scripts/ui_smoke.sh` → `PASS: the console searches, pages, forgets, mounts and reports adapter states`；`pytest -q` **263 passed**；`ruff check` 8 个改动文件 All checks passed；mypy 冷缓存查这 8 个文件 `Success: no issues found`，全仓仍是 **24 错 / 6 文件**（与 p1-4 记的基线相同，净新增 0）；`pyright` 这 8 个文件 **0 errors, 0 warnings**；go build/vet/test + gofmt 干净；`npm run build` + `eslint .` 干净；`scripts/mcp_write_smoke.sh` 仍 PASS - [ ] **p1-6** 代码卫生 - [ ] p1-6.1 `evals.py` 2413 行拆到 ≤500 行/文件 - [ ] p1-6.2 删 dead code 与未调用函数 diff --git a/README.md b/README.md index 0a2cf35..9e641c1 100644 --- a/README.md +++ b/README.md @@ -53,8 +53,12 @@ and launch one: .\.venv\Scripts\ely-eye.exe proof-suite ``` -Every command has an HTTP equivalent under `/api/...`, and the dashboard shows -the same state: Adapter Matrix, Proof Suite, Memory Map, Cache Trace. +The console is served by the API itself once `ELY_EYE_UI_DIR` points at +`frontend/dist`. It shows the state the CLI prints — runtime, GPU, library, +cartridges, senses, adapters, proof suite — and it is the only place to search +the store as somebody, page through atoms, forget one, and mount a cartridge. +`scripts/ui_smoke.sh` drives that console in a real browser against a fresh +store and fails if any of it stops working. ## Fine-tuning diff --git a/backend/src/ely_eye/api.py b/backend/src/ely_eye/api.py index db95213..b546bdf 100644 --- a/backend/src/ely_eye/api.py +++ b/backend/src/ely_eye/api.py @@ -5,8 +5,9 @@ import shutil from pathlib import Path from typing import Annotated -from fastapi import Depends, FastAPI, File, HTTPException, UploadFile +from fastapi import Depends, FastAPI, File, HTTPException, Query, UploadFile from fastapi.middleware.cors import CORSMiddleware +from fastapi.responses import FileResponse from fastapi.staticfiles import StaticFiles from pydantic import BaseModel @@ -14,33 +15,18 @@ from . import __version__ from .adapters import plan as adapter_plan from .api_cartridges import register as register_cartridge_routes from .api_holders import register as register_holder_routes -from .benchmarks import ( - benchmark_prediction_report, - benchmark_registry_report, - benchmark_sample_report, - benchmark_source_report, -) +from .browse import browse, picture from .cache_fabric import CacheFabric from .cartridge import CartridgeService from .compiler import ContextCompiler from .config import Settings, get_settings from .db import Database -from .embeddings import EmbeddingService -from .evaluation_schemas import ( - BenchmarkPredictionReport, - BenchmarkRegistryReport, - BenchmarkSampleReport, - BenchmarkSourceReport, - HashHopProof, - ProofCheck, - ProofSuiteReport, -) +from .evaluation_schemas import HashHopProof, ProofSuiteReport from .evidence_graph import EvidenceGraphService from .ingestion import IngestionService from .memory import MemoryService from .profiles import runtime_profile_report from .proofs import HashHopEvaluator, PRDProofSuite -from .research_modules import ResearchModuleService from .retrieval import RetrievalService from .runtime import QwenRuntime, RuntimeUnavailableError from .runtime_schemas import AdapterLoadReport @@ -48,11 +34,13 @@ from .schemas import ( AdapterManifest, AdapterStatus, AnswerResponse, + AtomPage, EvidenceAtom, ForgetReport, ForgetRequest, MemoryOrigin, MemoryStatus, + Modality, PathIngestRequest, RememberRequest, RuntimeProfile, @@ -63,6 +51,7 @@ from .schemas import ( SystemStatus, UploadIngestRequest, ) +from .senses import sense_facts from .training import safetensor_weight_stats from .verifier import Verifier, VisualContradictionLens @@ -114,6 +103,7 @@ def create_app() -> FastAPI: version=__version__, runtime=runtime, memory=memory, + senses=sense_facts(settings, db), data_home=str(settings.home), profiles={ "Live Demo": {"context_tokens": settings.live_context_tokens, "purpose": "stable 262K workflow"}, @@ -260,21 +250,33 @@ def create_app() -> FastAPI: def adapters(settings: Annotated[Settings, Depends(settings_dep)]) -> list[AdapterStatus]: return list_adapter_statuses(settings) - @app.get("/api/benchmarks/registry", response_model=BenchmarkRegistryReport) - def benchmarks_registry() -> BenchmarkRegistryReport: - return benchmark_registry_report() + @app.get("/api/atoms", response_model=AtomPage) + def atoms( + settings: Annotated[Settings, Depends(settings_dep)], + limit: Annotated[int, Query(ge=1, le=200)] = 50, + cursor: str | None = None, + modality: Modality | None = None, + source_id: str | None = None, + holder: str | None = None, + ) -> AtomPage: + try: + return browse( + settings, + limit=limit, + cursor=cursor, + modality=modality, + source_id=source_id, + holder=holder, + ) + except ValueError as exc: + raise HTTPException(status_code=400, detail=str(exc)) from exc - @app.get("/api/benchmarks/sources", response_model=BenchmarkSourceReport) - def benchmarks_sources() -> BenchmarkSourceReport: - return benchmark_source_report() - - @app.get("/api/benchmarks/samples", response_model=BenchmarkSampleReport) - def benchmarks_samples(settings: Annotated[Settings, Depends(settings_dep)]) -> BenchmarkSampleReport: - return benchmark_sample_report(settings.data_dir / "benchmark_samples") - - @app.get("/api/benchmarks/predictions", response_model=BenchmarkPredictionReport) - def benchmarks_predictions(settings: Annotated[Settings, Depends(settings_dep)]) -> BenchmarkPredictionReport: - return benchmark_prediction_report(settings, settings.data_dir / "benchmark_predictions") + @app.get("/api/atoms/{atom_id}/image") + def atom_image(atom_id: str, settings: Annotated[Settings, Depends(settings_dep)]) -> FileResponse: + found = picture(settings, atom_id) + if found is None: + raise HTTPException(status_code=404, detail=f"{atom_id} has no picture to show") + return FileResponse(found.path, media_type=found.mime) @app.get("/api/runtime/profiles", response_model=RuntimeProfileReport) def runtime_profiles( @@ -286,22 +288,6 @@ def create_app() -> FastAPI: def runtime_adapters(settings: Annotated[Settings, Depends(settings_dep)]) -> AdapterLoadReport: return adapter_plan(settings, Database(settings)) - @app.get("/api/research/modules") - def research_modules(settings: Annotated[Settings, Depends(settings_dep)]): - db = Database(settings) - compiler = ContextCompiler(settings, db) - context = compiler.compile( - "Ely-Eye V-NSA DVKV x-modal KV speculative prefetch evidence graph", - RuntimeProfile.research_theater, - ) - return ResearchModuleService(settings, db).build_report(context.plan.question, context.hits) - - @app.post("/api/index/dense") - def build_dense_index(settings: Annotated[Settings, Depends(settings_dep)]): - service = EmbeddingService(settings) - index = service.build_index() - return {"atoms": len(index.atom_ids), "dimensions": int(index.vectors.shape[1]) if index.vectors.size else 0} - @app.post("/api/hashhop", response_model=HashHopProof) def hashhop(request: HashHopRequest, settings: Annotated[Settings, Depends(settings_dep)]) -> HashHopProof: evaluator = HashHopEvaluator(settings) @@ -328,20 +314,6 @@ def create_app() -> FastAPI: raise HTTPException(status_code=404, detail="No PRD proof suite has been generated.") return report - @app.post("/api/visual-contradiction-proof", response_model=ProofCheck) - def visual_contradiction_proof(settings: Annotated[Settings, Depends(settings_dep)]) -> ProofCheck: - return PRDProofSuite(settings).visual_contradiction_proof() - - @app.get("/api/memory-map") - def memory_map(settings: Annotated[Settings, Depends(settings_dep)]): - db = Database(settings) - atoms = db.list_atoms(limit=200) - return { - "atoms": [atom.model_dump(mode="json") for atom in atoms], - "layers": CacheFabric(settings, db).statuses(), - "cartridges": CartridgeService(settings, db).list_manifests(), - } - mounted_ui = get_settings().ui_dir if mounted_ui is not None: if not mounted_ui.is_dir(): diff --git a/backend/src/ely_eye/browse.py b/backend/src/ely_eye/browse.py new file mode 100644 index 0000000..6c91467 --- /dev/null +++ b/backend/src/ely_eye/browse.py @@ -0,0 +1,75 @@ +"""Page through the store the way a person browses a shelf: newest first, filtered by who and what.""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path + +from .config import Settings +from .db import Database, row_to_atom +from .retrieval import decode_cursor, encode_cursor +from .schemas import AtomPage, Modality +from .storage import ObjectStore + + +@dataclass(frozen=True) +class Picture: + """The bytes behind an atom and how to label them when serving.""" + + path: Path + mime: str + + +def browse( + settings: Settings, + *, + limit: int, + cursor: str | None = None, + modality: Modality | None = None, + source_id: str | None = None, + holder: str | None = None, +) -> AtomPage: + """One page plus the total, so a list can say what it left out instead of looking endless.""" + offset = decode_cursor(cursor) + wanted = (modality.value if modality else None, source_id, holder) + filters = tuple(value for bound in wanted for value in (bound, bound)) + database = Database(settings) + with database.connect() as connection: + total = int( + connection.execute( + "SELECT COUNT(*) FROM v_atoms_current WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?)", + filters, + ).fetchone()[0] + ) + rows = connection.execute( + "SELECT * FROM v_atoms_current WHERE (? IS NULL OR modality = ?) AND (? IS NULL OR source_id = ?) AND (? IS NULL OR holder_id = ?) ORDER BY ingested_at DESC, atom_id DESC LIMIT ? OFFSET ?", + (*filters, limit + 1, offset), + ).fetchall() + atoms = [row_to_atom(row) for row in rows[:limit]] + more = len(rows) > limit or offset + limit < total + return AtomPage( + atoms=atoms, + total=total, + next_cursor=encode_cursor(offset + limit) if atoms and more else None, + ) + + +def picture(settings: Settings, atom_id: str) -> Picture | None: + """The stored bytes behind one atom, through the same visibility view search uses. + + None covers three cases the caller cannot tell apart and does not need to: the atom is unknown, + it was forgotten, or its bytes are gone. All three mean "there is no picture to show". + """ + database = Database(settings) + with database.connect() as connection: + row = connection.execute( + "SELECT image_ref, mime FROM v_atoms_current JOIN sources USING (source_id) WHERE atom_id = ?", + (atom_id,), + ).fetchone() + if row is None or not row["image_ref"]: + return None + path = ObjectStore(settings).resolve(str(row["image_ref"])) + if not path.exists(): + return None + mime = str(row["mime"]) if row["mime"] else "image/png" + return Picture(path=path, mime=mime) diff --git a/backend/src/ely_eye/maintenance.py b/backend/src/ely_eye/maintenance.py index b2d9467..8ee951d 100644 --- a/backend/src/ely_eye/maintenance.py +++ b/backend/src/ely_eye/maintenance.py @@ -14,15 +14,13 @@ from . import __version__ from .cartridge_store import mounted_ids from .config import Settings from .db import Database -from .embeddings import EmbeddingService from .migrations import SCHEMA_VERSION -from .schemas import utc_now -from .visual_index import VisualIndex +from .schemas import SenseStatus, utc_now +from .senses import sense_facts BACKUP_FILE = "ely_eye.sqlite" MANIFEST_FILE = "backup_manifest.json" _REQUIRED_TABLES = ("atoms", "sources", "cartridges") -PICTURE_MODALITIES = ("image", "ui_screenshot") class BackupManifest(BaseModel): @@ -57,12 +55,6 @@ def _atoms_word(count: int) -> str: return "atom" if count == 1 else "atoms" -def _modality_counts(database: Database) -> dict[str, int]: - with database.connect() as connection: - rows = connection.execute("SELECT modality, COUNT(*) AS total FROM atoms GROUP BY modality").fetchall() - return {str(row["modality"]): int(row["total"]) for row in rows} - - def _counts(connection: sqlite3.Connection) -> dict[str, int]: return { "atoms": int(connection.execute("SELECT COUNT(*) FROM atoms").fetchone()[0]), @@ -175,41 +167,41 @@ def _check_cartridges(database: Database, mounted: list[str], problems: list[str problems.append(f"{cartridge_id} is mounted but its files at {root} are gone") -def _check_senses(settings: Settings, database: Database, visual: VisualIndex, warnings: list[str]) -> None: - counts = _modality_counts(database) - pictures = sum(counts.get(name, 0) for name in PICTURE_MODALITIES) - if settings.clip_model_dir is not None and visual.model_dir() is None: - warnings.append(f"the CLIP snapshot at {settings.clip_model_dir} is missing, so wordless pictures cannot be found") - elif pictures and not visual.available(): +def _check_senses(facts: SenseStatus, warnings: list[str]) -> None: + """Render the missing senses as sentences; the counting lives in senses.py, not here.""" + encodable = facts.picture_atoms - facts.picture_bytes_missing + if facts.clip_path is not None and facts.visual == "absent": + warnings.append(f"the CLIP snapshot at {facts.clip_path} is missing, so wordless pictures cannot be found") + elif encodable and facts.visual == "absent": warnings.append( - f"the store holds {pictures} picture {_atoms_word(pictures)} but no CLIP snapshot is set " + f"the store holds {encodable} picture {_atoms_word(encodable)} but no CLIP snapshot is set " "(ELY_EYE_CLIP_MODEL_DIR), so they are findable only by the text already on them" ) - elif visual.visual_atoms() and not visual.is_current(): + elif facts.visual == "stale": warnings.append("the visual index is behind the store; run `ely-eye index-visual`") - audio = counts.get("audio", 0) - if settings.asr_model_dir is None: - if audio: - warnings.append( - f"the store holds {audio} audio {_atoms_word(audio)} but no speech snapshot is set " - "(ELY_EYE_ASR_MODEL_DIR), so new audio will not be transcribed" - ) - elif not settings.asr_model_dir.exists(): - warnings.append(f"the speech snapshot at {settings.asr_model_dir} is missing, so audio cannot be transcribed") - if settings.vlm_model_dir is not None and not settings.vlm_model_dir.exists(): - warnings.append(f"the captioning model at {settings.vlm_model_dir} is missing, so images stay uncaptioned") - - -def _check_indexes(settings: Settings, database: Database, warnings: list[str]) -> None: - if not settings.dense_embeddings_enabled: - return - index = EmbeddingService(settings, database).load_index() - stored = database.atom_count() - if index is None: - warnings.append("dense embeddings are on but no index exists; run `ely-eye build-dense-index`") - elif len(index.atom_ids) != stored: + if facts.picture_bytes_missing: + missing = facts.picture_bytes_missing warnings.append( - f"the dense index covers {len(index.atom_ids)} atoms while the store holds {stored}; run `ely-eye build-dense-index`" + f"{missing} picture {_atoms_word(missing)} with no bytes behind them; re-ingest the source they came from" + ) + if facts.asr_path is not None and facts.speech == "missing": + warnings.append(f"the speech snapshot at {facts.asr_path} is missing, so audio cannot be transcribed") + elif facts.audio_atoms and facts.speech == "absent": + warnings.append( + f"the store holds {facts.audio_atoms} audio {_atoms_word(facts.audio_atoms)} but no speech snapshot is set " + "(ELY_EYE_ASR_MODEL_DIR), so new audio will not be transcribed" + ) + if facts.vlm_path is not None and facts.captioning == "missing": + warnings.append(f"the captioning model at {facts.vlm_path} is missing, so images stay uncaptioned") + + +def _check_indexes(facts: SenseStatus, warnings: list[str]) -> None: + if facts.dense == "missing": + warnings.append("dense embeddings are on but no index exists; run `ely-eye build-dense-index`") + elif facts.dense == "behind": + warnings.append( + f"the dense index covers {facts.dense_covered} atoms while the store holds {facts.store_atoms}; " + "run `ely-eye build-dense-index`" ) @@ -242,8 +234,9 @@ def doctor(settings: Settings) -> DoctorReport: problems.append(f"the schema settled at version {version} while this build writes {SCHEMA_VERSION}") mounted = mounted_ids(database) _check_cartridges(database, mounted, problems) - _check_senses(settings, database, VisualIndex(settings, database), warnings) - _check_indexes(settings, database, warnings) + facts = sense_facts(settings, database) + _check_senses(facts, warnings) + _check_indexes(facts, warnings) if settings.ui_dir is not None and not settings.ui_dir.exists(): problems.append(f"ui_dir {settings.ui_dir} is set but holds no built interface, so the API refuses to start") return DoctorReport( diff --git a/backend/src/ely_eye/schemas.py b/backend/src/ely_eye/schemas.py index 3cecd21..79a8edd 100644 --- a/backend/src/ely_eye/schemas.py +++ b/backend/src/ely_eye/schemas.py @@ -201,6 +201,14 @@ class SearchResult(BaseModel): hits: list[RetrievalHit] next_cursor: str | None = None + +class AtomPage(BaseModel): + """A page of the store plus the total behind it, so a list can admit what it left out.""" + + atoms: list[EvidenceAtom] + total: int + next_cursor: str | None = None + class ContextPlan(BaseModel): question: str profile: RuntimeProfile @@ -261,11 +269,39 @@ class RuntimeStatus(BaseModel): vram_total_mb: int | None = None vram_used_mb: int | None = None +SenseState = Literal["absent", "missing", "ready"] +VisualState = Literal["absent", "stale", "ready"] +DenseState = Literal["off", "missing", "behind", "current"] + + +class SenseStatus(BaseModel): + """What the machine can perceive right now, stated once for the UI and the doctor to share. + + `absent` means the sense was never given a model, `missing` means it was given one that is gone. + A store full of pictures and no CLIP snapshot is a different problem, and needs a different + sentence, than a snapshot directory somebody deleted. + """ + + dense: DenseState + dense_covered: int + store_atoms: int + picture_atoms: int + picture_bytes_missing: int + visual: VisualState + audio_atoms: int + speech: SenseState + captioning: SenseState + clip_path: str | None = None + asr_path: str | None = None + vlm_path: str | None = None + + class SystemStatus(BaseModel): app: str = "Ely-Eye" version: str runtime: RuntimeStatus memory: MemoryStatus + senses: SenseStatus profiles: dict[str, dict[str, Any]] data_home: str diff --git a/backend/src/ely_eye/senses.py b/backend/src/ely_eye/senses.py new file mode 100644 index 0000000..50a9233 --- /dev/null +++ b/backend/src/ely_eye/senses.py @@ -0,0 +1,84 @@ +"""What this machine can actually perceive, computed once and read by status and doctor alike.""" + +from __future__ import annotations + +from pathlib import Path + +from .config import Settings +from .db import Database +from .embeddings import EmbeddingService +from .schemas import DenseState, SenseState, SenseStatus, VisualState +from .visual_index import VisualIndex + + +def sense_facts(settings: Settings, database: Database, visual: VisualIndex | None = None) -> SenseStatus: + """Report the senses as facts, so the UI and the doctor cannot disagree about what is missing. + + A path set but gone on disk is a different failure from a path never set: the first is a broken + install, the second is a machine that was never given that sense. + """ + index = visual or VisualIndex(settings, database) + claimed = _picture_claim(database) + on_disk = index.visual_atom_ids() + dense, covered, total = _dense_facts(settings, database) + return SenseStatus( + dense=dense, + dense_covered=covered, + store_atoms=total, + picture_atoms=claimed, + picture_bytes_missing=max(0, claimed - len(on_disk)), + visual=_visual_state(index, on_disk), + audio_atoms=_audio_count(database), + speech=_sense_state(settings.asr_model_dir), + captioning=_sense_state(settings.vlm_model_dir), + clip_path=_stated(settings.clip_model_dir), + asr_path=_stated(settings.asr_model_dir), + vlm_path=_stated(settings.vlm_model_dir), + ) + + +def _dense_facts(settings: Settings, database: Database) -> tuple[DenseState, int, int]: + total = database.atom_count() + if not settings.dense_embeddings_enabled: + return "off", 0, total + loaded = EmbeddingService(settings, database).load_index() + if loaded is None: + return "missing", 0, total + covered = len(loaded.atom_ids) + return ("current" if covered == total else "behind"), covered, total + + +def _visual_state(visual: VisualIndex, pictures: set[str]) -> VisualState: + """A store of pictures the index has not encoded yet is stale: they are invisible to search.""" + if not visual.available(): + return "absent" + if not pictures: + return "ready" + loaded = visual.load() + return "ready" if loaded is not None and set(loaded[0]) == pictures else "stale" + + +def _picture_claim(database: Database) -> int: + """Atoms that say they hold a picture, whether or not the bytes survived.""" + with database.connect() as connection: + row = connection.execute( + "SELECT COUNT(*) FROM v_atoms_current WHERE image_ref IS NOT NULL AND image_ref != ''" + ).fetchone() + return int(row[0]) + + +def _sense_state(model_dir: Path | None) -> SenseState: + """A sense with no configured snapshot is absent; one pointing at a deleted directory is missing.""" + if model_dir is None: + return "absent" + return "ready" if model_dir.exists() else "missing" + + +def _stated(model_dir: Path | None) -> str | None: + return None if model_dir is None else str(model_dir) + + +def _audio_count(database: Database) -> int: + with database.connect() as connection: + row = connection.execute("SELECT COUNT(*) AS n FROM v_atoms_current WHERE modality = 'audio'").fetchone() + return int(row["n"]) diff --git a/backend/src/ely_eye/visual_index.py b/backend/src/ely_eye/visual_index.py index 0260c01..2708ef6 100644 --- a/backend/src/ely_eye/visual_index.py +++ b/backend/src/ely_eye/visual_index.py @@ -141,12 +141,30 @@ class VisualIndex: _LOADED.clear() return removed - def is_current(self, atoms: list[EvidenceAtom] | None = None) -> bool: - """True when the index covers exactly the image atoms the store holds right now.""" + def visual_atom_ids(self) -> set[str]: + """Picture atoms whose bytes are still on disk, asked with one light query.""" + store = ObjectStore(self.settings) + with self.db.connect() as connection: + rows = connection.execute( + "SELECT atom_id, image_ref FROM v_atoms_current WHERE image_ref IS NOT NULL AND image_ref != ''" + ).fetchall() + kept: set[str] = set() + for row in rows: + try: + path = store.resolve(str(row["image_ref"])) + except ValueError: + continue + if path.exists(): + kept.add(str(row["atom_id"])) + return kept + + def is_current(self, atom_ids: set[str] | None = None) -> bool: + """True when the index covers exactly the picture atoms the store holds right now.""" loaded = self.load() if loaded is None: return False - return {atom.atom_id for atom in self.visual_atoms(atoms)} == set(loaded[0]) + wanted = self.visual_atom_ids() if atom_ids is None else atom_ids + return wanted == set(loaded[0]) def search_text(self, query: str, top_k: int, restrict: set[str] | None = None) -> list[tuple[str, float]]: vector = self.encoder().encode_texts([query])[0] diff --git a/backend/tests/test_maintenance.py b/backend/tests/test_maintenance.py index 89be85d..7d4750f 100644 --- a/backend/tests/test_maintenance.py +++ b/backend/tests/test_maintenance.py @@ -19,6 +19,7 @@ from ely_eye.db import Database from ely_eye.memory import MemoryService from ely_eye.migrations import SCHEMA_VERSION from ely_eye.schemas import EvidenceAtom, MemoryOrigin, Modality, SourceRecord, TrustScores, utc_now +from ely_eye.storage import ObjectStore KEYS = "the keys are on the shelf by the door" PASSPORT = "the passport is in the blue box" @@ -164,7 +165,14 @@ def test_doctor_names_the_senses_a_store_holds_but_cannot_use(tmp_path: Path) -> ) database.upsert_atoms( [ - EvidenceAtom(atom_id="img:1", modality=Modality.image, source_id="img", source="file:///img.png", trust=trust), + EvidenceAtom( + atom_id="img:1", + modality=Modality.image, + source_id="img", + source="file:///img.png", + trust=trust, + image_ref=ObjectStore(settings).put_bytes(b"\x89PNG\r\n\x1a\n", ".png", "image"), + ), EvidenceAtom(atom_id="aud:1", modality=Modality.audio, source_id="aud", source="file:///a.mp3", trust=trust), ] ) @@ -175,6 +183,30 @@ def test_doctor_names_the_senses_a_store_holds_but_cannot_use(tmp_path: Path) -> assert "1 audio atom but no speech snapshot" in report.warnings[1] +def test_doctor_names_picture_atoms_whose_bytes_are_gone(tmp_path: Path) -> None: + settings = _settings(tmp_path) + database = Database(settings) + database.init() + trust = TrustScores(parser=0.0, ocr=0.0, model=0.0) + database.upsert_sources( + [SourceRecord(source_id="img", path="file:///img.png", kind="image", mime="image/png", sha256="a", size_bytes=1)] + ) + database.upsert_atoms( + [ + EvidenceAtom( + atom_id="img:1", + modality=Modality.image, + source_id="img", + source="file:///img.png", + trust=trust, + image_ref="object://image/ab/deadbeef.png", + ) + ] + ) + report = maintenance.doctor(settings) + assert "1 picture atom with no bytes behind them" in report.warnings[0] + + def test_doctor_names_a_mount_whose_files_are_gone(tmp_path: Path) -> None: settings = _settings(tmp_path) database = _store(settings) diff --git a/backend/tests/test_product_api.py b/backend/tests/test_product_api.py new file mode 100644 index 0000000..c8ea067 --- /dev/null +++ b/backend/tests/test_product_api.py @@ -0,0 +1,192 @@ +"""The product face: page the store, look at a picture, and say plainly what the machine can perceive.""" + +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from pathlib import Path + +import pytest +from fastapi.routing import APIRoute +from fastapi.testclient import TestClient + +import ely_eye.api as api_module +from ely_eye.config import Settings +from ely_eye.db import Database +from ely_eye.memory import MemoryService +from ely_eye.schemas import EvidenceAtom, MemoryOrigin, Modality, SourceRecord, TrustScores +from ely_eye.storage import ObjectStore + +PNG = b"\x89PNG\r\n\x1a\n" +BASE = datetime(2026, 5, 1, 12, 0, tzinfo=UTC) + +HTTP_SURFACE = { + "/api/adapters", + "/api/ask", + "/api/atoms", + "/api/atoms/{atom_id}/image", + "/api/cartridges", + "/api/cartridges/{cartridge_id}/append", + "/api/cartridges/{cartridge_id}/assets", + "/api/cartridges/{cartridge_id}/finalize", + "/api/cartridges/{cartridge_id}/mount", + "/api/cartridges/{cartridge_id}/refresh", + "/api/cartridges/{cartridge_id}/switch", + "/api/cartridges/{cartridge_id}/unmount", + "/api/chat", + "/api/hashhop", + "/api/holders", + "/api/holders/{holder_id}", + "/api/holders/{holder_id}/knows", + "/api/ingest/path", + "/api/ingest/upload", + "/api/memory", + "/api/memory/forget", + "/api/memory/supersede", + "/api/mounts", + "/api/proof-suite", + "/api/proof-suite/latest", + "/api/runtime/adapters", + "/api/runtime/profiles", + "/api/search", + "/api/status", + "/api/who_knows", +} + + +@pytest.fixture +def settings(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Settings: + resolved = Settings(home=tmp_path, default_holder="wife") + monkeypatch.setattr(api_module, "get_settings", lambda: resolved) + Database(resolved).init() + return resolved + + +@pytest.fixture +def client(settings: Settings) -> TestClient: + return TestClient(api_module.create_app()) + + +def _seed(settings: Settings, pictures: int = 3) -> None: + """Pictures an hour apart, so newest-first has a fact to order by rather than a coincidence.""" + database = Database(settings) + trust = TrustScores(parser=1.0, ocr=0.0, model=0.0) + store = ObjectStore(settings) + stored = store.put_bytes(PNG, ".png", "image") + database.upsert_sources( + [ + SourceRecord( + source_id=f"shot{i}", + path=f"file:///shot{i}.png", + kind="image", + mime="image/png", + sha256=f"{i}" * 8, + size_bytes=len(PNG), + ) + for i in range(pictures) + ] + ) + database.upsert_atoms( + [ + EvidenceAtom( + atom_id=f"shot{i}:frame", + modality=Modality.image, + source_id=f"shot{i}", + source=f"file:///shot{i}.png", + trust=trust, + image_ref=stored, + holder_id="wife" if i % 2 == 0 else "husband", + ingested_at=BASE + timedelta(hours=i), + ) + for i in range(pictures) + ] + ) + + +def test_the_store_pages_newest_first_and_says_how_much_is_left(client: TestClient, settings: Settings) -> None: + _seed(settings) + first = client.get("/api/atoms", params={"limit": 2}) + assert first.status_code == 200 + page = first.json() + assert page["total"] == 3 + assert [atom["atom_id"] for atom in page["atoms"]] == ["shot2:frame", "shot1:frame"] + assert page["next_cursor"] + + second = client.get("/api/atoms", params={"limit": 2, "cursor": page["next_cursor"]}) + rest = second.json() + assert [atom["atom_id"] for atom in rest["atoms"]] == ["shot0:frame"] + assert rest["next_cursor"] is None, "the last page must not promise another one" + + +def test_browse_narrows_by_modality_and_by_holder(client: TestClient, settings: Settings) -> None: + _seed(settings, pictures=2) + MemoryService(settings, Database(settings)).remember( + "the keys are on the shelf", holder_id="wife", origin=MemoryOrigin.self_ + ) + + images = client.get("/api/atoms", params={"modality": "image"}).json() + assert images["total"] == 2 + assert {atom["modality"] for atom in images["atoms"]} == {"image"} + + his = client.get("/api/atoms", params={"holder": "husband"}).json() + assert [atom["atom_id"] for atom in his["atoms"]] == ["shot1:frame"] + + +def test_an_unreadable_cursor_is_refused_with_400(client: TestClient, settings: Settings) -> None: + _seed(settings, pictures=1) + response = client.get("/api/atoms", params={"cursor": "not-a-cursor"}) + assert response.status_code == 400 + assert "cursor" in response.json()["detail"] + + +def test_a_picture_atom_serves_the_bytes_behind_it(client: TestClient, settings: Settings) -> None: + _seed(settings, pictures=1) + response = client.get("/api/atoms/shot0:frame/image") + assert response.status_code == 200 + assert response.content == PNG + assert response.headers["content-type"] == "image/png" + + +def test_a_memory_with_no_picture_says_so(client: TestClient, settings: Settings) -> None: + atom = MemoryService(settings, Database(settings)).remember( + "the keys are on the shelf", holder_id="wife", origin=MemoryOrigin.self_ + ) + assert client.get(f"/api/atoms/{atom.atom_id}/image").status_code == 404 + assert client.get("/api/atoms/nope:image").status_code == 404 + + +def test_a_forgotten_picture_stops_being_served(client: TestClient, settings: Settings) -> None: + _seed(settings, pictures=1) + MemoryService(settings, Database(settings)).forget(atom_id="shot0:frame") + assert client.get("/api/atoms/shot0:frame/image").status_code == 404 + assert client.get("/api/atoms").json()["total"] == 0 + + +def test_status_says_what_the_machine_can_see_and_what_it_cannot(client: TestClient, settings: Settings) -> None: + _seed(settings, pictures=2) + senses = client.get("/api/status").json()["senses"] + assert senses["dense"] == "off", "dense embeddings are off by default and status must not claim otherwise" + assert senses["picture_atoms"] == 2 + assert senses["picture_bytes_missing"] == 0 + assert senses["visual"] == "absent", "no CLIP snapshot on this machine means wordless pictures stay hidden" + assert senses["speech"] == "absent" + + +def test_status_names_pictures_whose_bytes_are_gone(client: TestClient, settings: Settings) -> None: + _seed(settings, pictures=1) + dead = Database(settings).get_atoms(["shot0:frame"])[0].model_copy( + update={"atom_id": "shot0:ghost", "image_ref": "object://image/ab/deadbeef.png"} + ) + Database(settings).upsert_atoms([dead]) + senses = client.get("/api/status").json()["senses"] + assert senses["picture_atoms"] == 2 + assert senses["picture_bytes_missing"] == 1 + + +def test_the_http_surface_is_exactly_what_someone_uses() -> None: + """A route nobody calls is a contract nobody keeps: adding one has to be a deliberate act.""" + served = { + route.path + for route in api_module.app.routes + if isinstance(route, APIRoute) and route.path.startswith("/api") + } + assert served == HTTP_SURFACE diff --git a/frontend/package-lock.json b/frontend/package-lock.json index 3586fcc..a22d22d 100644 --- a/frontend/package-lock.json +++ b/frontend/package-lock.json @@ -7,6 +7,7 @@ "": { "name": "ely-eye-frontend", "version": "0.1.0", + "license": "AGPL-3.0-only", "dependencies": { "@vitejs/plugin-react": "^6.0.2", "lucide-react": "^0.562.0", @@ -24,7 +25,7 @@ "eslint-plugin-react-hooks": "^7.0.1", "eslint-plugin-react-refresh": "^0.4.24", "globals": "^16.5.0", - "playwright": "^1.60.0", + "playwright": "^1.64.0", "typescript-eslint": "^8.50.0" } }, @@ -2503,50 +2504,32 @@ } }, "node_modules/playwright": { - "version": "1.60.0", - "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.60.0.tgz", - "integrity": "sha512-hheHdokM8cdqCb0lcE3s+zT4t4W+vvjpGxsZlDnikarzx8tSzMebh3UiFtgqwFwnTnjYQcsyMF8ei2mCO/tpeA==", + "version": "1.64.0", + "resolved": "https://registry.npmjs.org/playwright/-/playwright-1.64.0.tgz", + "integrity": "sha512-kVzTnYFYEyQ9RGzBjNCd2+yEqWmVbxl3AHPynv3qQ6JTjv4a+kwRcnqQwIpuP3NnrnIrE/Q6wWAIAvMfIE2/Yg==", "dev": true, "license": "Apache-2.0", "dependencies": { - "playwright-core": "1.60.0" + "playwright-core": "1.64.0" }, "bin": { "playwright": "cli.js" }, "engines": { - "node": ">=18" - }, - "optionalDependencies": { - "fsevents": "2.3.2" + "node": ">=20" } }, "node_modules/playwright-core": { - "version": "1.60.0", - "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.60.0.tgz", - "integrity": "sha512-9bW6zvX/m0lEbgTKJ6YppOKx8H3VOPBMOCFh2irXFOT4BbHgrx5hPjwJYLT40Lu+4qtD36qKc/Hn56StUW57IA==", + "version": "1.64.0", + "resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.64.0.tgz", + "integrity": "sha512-T9r+MZkTECl2+oUcZ26YgZzTrYidTsIhYYkJix+6iDOymOb1FB2RLLSOxRGnZGv5DStt+aduIVbt+E8WygXYpg==", "dev": true, "license": "Apache-2.0", "bin": { "playwright-core": "cli.js" }, "engines": { - "node": ">=18" - } - }, - "node_modules/playwright/node_modules/fsevents": { - "version": "2.3.2", - "resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz", - "integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==", - "dev": true, - "hasInstallScript": true, - "license": "MIT", - "optional": true, - "os": [ - "darwin" - ], - "engines": { - "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + "node": ">=20" } }, "node_modules/postcss": { diff --git a/frontend/package.json b/frontend/package.json index 250ddb8..47174f5 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -28,7 +28,7 @@ "eslint-plugin-react-hooks": "^7.0.1", "eslint-plugin-react-refresh": "^0.4.24", "globals": "^16.5.0", - "playwright": "^1.60.0", + "playwright": "^1.64.0", "typescript-eslint": "^8.50.0" } } diff --git a/frontend/scripts/verify-ui.mjs b/frontend/scripts/verify-ui.mjs index 313a781..b4dfd9a 100644 --- a/frontend/scripts/verify-ui.mjs +++ b/frontend/scripts/verify-ui.mjs @@ -1,32 +1,231 @@ import { chromium } from 'playwright'; -import { mkdir } from 'node:fs/promises'; +import { mkdir, writeFile } from 'node:fs/promises'; import path from 'node:path'; const root = path.resolve(process.cwd(), '..'); -const outputDir = path.join(root, '.ely_eye', 'proofs'); +const outputDir = process.env.ELY_EYE_UI_SMOKE_DIR ?? path.join(root, '.ely_eye', 'proofs'); await mkdir(outputDir, { recursive: true }); +const apiBase = process.env.ELY_EYE_API ?? 'http://127.0.0.1:8765'; +const uiUrl = process.env.ELY_EYE_UI_URL ?? 'http://127.0.0.1:5173/'; + +async function api(route, body) { + const response = await fetch(`${apiBase}${route}`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(body ?? {}), + }); + const payload = await response.json().catch(() => ({})); + if (!response.ok) { + throw new Error(`${route} was rejected: ${JSON.stringify(payload)}`); + } + return payload; +} + +async function apiGet(route) { + const response = await fetch(`${apiBase}${route}`); + const payload = await response.json().catch(() => ({})); + if (!response.ok) { + throw new Error(`${route} was rejected: ${JSON.stringify(payload)}`); + } + return payload; +} + +async function seed() { + const statements = ['the spare glasses are in the hall drawer']; + for (let index = 1; index < 14; index += 1) { + statements.push(`shelf position ${index} holds the ${index} box of cables`); + } + for (const statement of statements) { + await api('/api/memory', { text: statement, scenario: 'home' }); + } + const png = Buffer.from( + 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==', + 'base64', + ); + const imagePath = path.join(outputDir, 'wordless-cup.png'); + await writeFile(imagePath, png); + await api('/api/ingest/path', { path: imagePath, cartridge_name: 'ui-smoke' }); + // The store, not the test, decides what kind of picture a file became. + const page = await apiGet('/api/atoms?limit=50'); + const picture = page.atoms.find((atom) => atom.image_ref); + if (!picture) { + throw new Error(`ingesting ${imagePath} left no atom with a picture: ${JSON.stringify(page.atoms.map((atom) => [atom.atom_id, atom.modality]))}`); + } + return picture.modality; +} + +const pictureModality = await seed(); + const browser = await chromium.launch(); const page = await browser.newPage({ viewport: { width: 1440, height: 1100 } }); const consoleErrors = []; +const failedRequests = []; +page.on('pageerror', (error) => consoleErrors.push(`uncaught: ${error.message}`)); page.on('console', (message) => { - if (message.type() === 'error') { + if (message.type() === 'error' && !message.text().startsWith('Failed to load resource')) { consoleErrors.push(message.text()); } }); +function routeOf(url) { + try { + return new URL(url).pathname; + } catch { + return url; + } +} -await page.goto('http://127.0.0.1:5173/', { waitUntil: 'networkidle' }); +page.on('response', (response) => { + if (response.status() >= 400) { + failedRequests.push(`${response.status()} ${routeOf(response.url())}`); + } +}); +// A store that has never run the proof suite answers 404, and the panel says "pending". +const expectedFailures = ['404 /api/proof-suite/latest']; + +const observed = { mount: 'unknown', adapters: 'unknown' }; + +async function evidence(label) { + await page.screenshot({ path: path.join(outputDir, 'ui-failure.png'), fullPage: true }).catch(() => undefined); + const text = await page + .locator('.panel', { has: page.getByRole('heading', { name: 'Memory' }) }) + .first() + .innerText() + .catch(() => '(the memory panel is gone)'); + const served = await fetch(`${apiBase}/api/atoms?limit=5&modality=image`) + .then((response) => response.json()) + .catch((err) => ({ error: String(err) })); + console.error( + `${label}\n--- memory panel ---\n${text}\n--- what the API serves ---\n${JSON.stringify(served, null, 2)}\n--- screenshot ---\n${path.join(outputDir, 'ui-failure.png')}`, + ); +} + +await page.goto(uiUrl, { waitUntil: 'networkidle' }); await page.getByRole('heading', { name: 'Local VLM Context OS' }).waitFor(); await page.getByLabel('system status').getByText('Qwen/Qwen3.5-9B').waitFor(); await page.getByRole('heading', { name: 'Adapter Matrix' }).waitFor(); await page.getByRole('heading', { name: 'PRD Proof Suite' }).waitFor(); -await page.getByText(/\d+\/\d+ passed/).waitFor(); +await page.getByText(/(\d+\/\d+ passed|pending)/).waitFor(); + +// 1. search: ask what someone knows, in a scenario. +await page.getByPlaceholder('where are the keys').fill('spare glasses'); +await page.getByPlaceholder('any').nth(1).fill('home'); +await page.getByRole('button', { name: 'Search', exact: true }).click(); +const hit = page.locator('.atom-row', { hasText: 'spare glasses are in the hall drawer' }).first(); +await hit.waitFor(); +const hitText = await hit.innerText(); +if (!hitText.includes('home') || !hitText.includes('wife')) { + throw new Error(`the search row does not name who knows it and in what setting: ${hitText}`); +} + +// 2. browse: page the store and look at a wordless picture. +await page.getByLabel('Modality').selectOption(pictureModality); +try { + await page.waitForFunction( + () => { + const image = document.querySelector('img.thumb'); + return Boolean(image) && image.complete && image.naturalWidth > 0; + }, + undefined, + { timeout: 30_000 }, + ); +} catch (error) { + await evidence('the picture atom never rendered pixels in the browse list'); + throw error; +} +await page.getByLabel('Modality').selectOption(''); +const firstPageRows = await page.locator('.atom-list .atom-row').count(); +await page.getByTitle('Next page').click(); +await page.waitForFunction( + (count) => document.querySelectorAll('.atom-list .atom-row').length !== count, + firstPageRows, +); +await page.getByTitle('Previous page').click(); + +// 3. forget: remove a memory and watch it leave the store. +await page.getByPlaceholder('where are the keys').fill('spare glasses'); +await page.getByRole('button', { name: 'Search', exact: true }).click(); +await page.locator('.atom-row', { hasText: 'spare glasses are in the hall drawer' }).first().waitFor(); +await page + .locator('.atom-row', { hasText: 'spare glasses are in the hall drawer' }) + .first() + .getByTitle('Forget this memory') + .click(); +await page.getByText(/forgotten: \d+ atom/).waitFor(); +await page.getByPlaceholder('where are the keys').fill('spare glasses'); +await page.getByRole('button', { name: 'Search', exact: true }).click(); +await page.getByText(/Nothing in the store answers/).waitFor(); + +// 4. mount: a cartridge is a scope you mount, and the machine decides if it may. +const mountRow = page.locator('.adapter-row', { hasText: 'ui-smoke' }).first(); +await mountRow.waitFor(); +await mountRow.getByRole('button', { name: 'Mount', exact: true }).click(); +const mountedPill = mountRow.locator('.proof-pill.passed', { hasText: 'mounted' }); +const refusal = page.locator('.error-line').first(); +if (await mountedPill.isVisible().catch(() => false)) { + observed.mount = 'accepted'; + await mountRow.getByRole('button', { name: 'Unmount', exact: true }).click(); + await page.getByText(/unmount ui-smoke/).waitFor(); +} else { + await refusal.waitFor(); + observed.mount = `refused: ${(await refusal.innerText()).slice(0, 120)}`; + const cartridges = await apiGet('/api/cartridges'); + const cartridge = cartridges.find((entry) => entry.name === 'ui-smoke'); + if (!cartridge) { + throw new Error('the ui-smoke cartridge vanished from the store while the console was mounting it'); + } + expectedFailures.push(`400 /api/cartridges/${cartridge.cartridge_id}/mount`); +} + +// 5. adapter states: the panel says what is attached and why the rest is not. +await page.getByRole('heading', { name: 'Adapter Matrix' }).scrollIntoViewIfNeeded(); +const loadStrip = page + .locator('.panel', { has: page.getByRole('heading', { name: 'Adapter Matrix' }) }) + .first(); +await loadStrip.getByText(/\d+ mounted/).waitFor(); +const states = await loadStrip.locator('.layer-row').evaluateAll((rows) => + rows.map((row) => [ + row.querySelector('strong')?.textContent ?? '', + (row.querySelector('small')?.textContent ?? '').trim(), + ]), +); +if (states.length === 0) { + await evidence('the adapter panel named no adapter kind at all'); + throw new Error('the adapter panel reported no states; the load report was empty'); +} +for (const [state, reason] of states) { + if (!/^(loaded|available|deferred|missing)$/.test(state)) { + throw new Error(`an adapter row reported the state "${state}"`); + } + if (!reason) { + throw new Error(`the adapter row "${state}" gave no reason`); + } +} +observed.adapters = `${states.length} kinds: ${states.map(([state]) => state).join(', ')}`; + await page.screenshot({ path: path.join(outputDir, 'ely-eye-dashboard.png'), fullPage: true }); const status = await page.locator('.kpi-grid').innerText(); await browser.close(); -if (consoleErrors.length > 0) { - throw new Error(consoleErrors.join('\n')); +const unexpected = failedRequests.filter( + (line) => !expectedFailures.some((allowed) => line === allowed || line.startsWith(allowed)), +); +if (unexpected.length > 0 || consoleErrors.length > 0) { + throw new Error( + [`${unexpected.length} unexpected failed request(s): ${unexpected.join(', ')}`, ...consoleErrors].join('\n'), + ); } -console.log(JSON.stringify({ screenshot: path.join(outputDir, 'ely-eye-dashboard.png'), status }, null, 2)); +console.log( + JSON.stringify( + { + screenshot: path.join(outputDir, 'ely-eye-dashboard.png'), + mount: observed.mount, + adapters: observed.adapters, + expected_failures: expectedFailures.filter((line) => failedRequests.includes(line)), + status, + }, + null, + 2, + ), +); diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index a70ea15..a4b2aa3 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -1,96 +1,37 @@ -import { - Activity, - Archive, - Braces, - CheckCircle, - Cpu, - Database, - FileSearch, - Gauge, - GitBranch, - Image as ImageIcon, - Play, - RefreshCw, - Send, - Server, - ShieldCheck, - Upload, -} from 'lucide-react'; -import { FormEvent, ReactNode, useEffect, useMemo, useState } from 'react'; -import { - askQuestion, - getAdapters, - getLatestProofSuite, - getMemoryMap, - getStatus, - ingestPath, - runHashHop, - runProofSuite, - uploadFiles, -} from './api'; -import type { - AdapterStatus, - AnswerResponse, - HashHopProof, - IngestResponse, - MemoryMap, - ProofSuiteReport, - RuntimeProfile, - SystemStatus, -} from './types'; +import { Activity, Archive, Database, Gauge, RefreshCw, Server } from 'lucide-react'; +import { useEffect, useState } from 'react'; -const profiles: Array<{ id: RuntimeProfile; label: string }> = [ - { id: 'live_demo', label: 'Live Demo' }, - { id: 'extreme_context', label: 'Extreme' }, - { id: 'library_100m', label: '100M Library' }, - { id: 'research_theater', label: 'Research' }, -]; +import { getAdapters, getStatus } from './api'; +import { AdapterPanel } from './components/AdapterPanel'; +import { CartridgePanel } from './components/CartridgePanel'; +import { CompilerPanel } from './components/CompilerPanel'; +import { IngestPanel } from './components/IngestPanel'; +import { MemoryPanel } from './components/MemoryPanel'; +import { ProofPanel } from './components/ProofPanel'; +import { SensesPanel } from './components/SensesPanel'; +import { Kpi, Panel, Quiet } from './components/ui'; +import { errorMessage, formatBytes, formatNumber, formatRuntimeDetail, formatVram } from './lib/format'; +import type { AdapterStatus, CacheLayerStatus, SystemStatus } from './types'; export function App() { const [status, setStatus] = useState(null); - const [memoryMap, setMemoryMap] = useState(null); const [adapters, setAdapters] = useState([]); - const [profile, setProfile] = useState('library_100m'); - const [path, setPath] = useState(''); - const [cartridgeName, setCartridgeName] = useState('ely-eye-cartridge'); - const [files, setFiles] = useState([]); - const [question, setQuestion] = useState(''); - const [answer, setAnswer] = useState(null); - const [hashHop, setHashHop] = useState(null); - const [proofSuite, setProofSuite] = useState(null); - const [lastIngest, setLastIngest] = useState(null); - const [busy, setBusy] = useState(null); const [error, setError] = useState(null); - const refresh = async () => { - setError(null); - const [nextStatus, nextMap, nextAdapters, nextProofSuite] = await Promise.all([ - getStatus(), - getMemoryMap(), - getAdapters(), - getLatestProofSuite().catch(() => null), - ]); + async function refresh() { + const [nextStatus, nextAdapters] = await Promise.all([getStatus(), getAdapters()]); setStatus(nextStatus); - setMemoryMap(nextMap); setAdapters(nextAdapters); - setProofSuite(nextProofSuite); - }; + } useEffect(() => { let cancelled = false; async function loadInitial() { try { - const [nextStatus, nextMap, nextAdapters, nextProofSuite] = await Promise.all([ - getStatus(), - getMemoryMap(), - getAdapters(), - getLatestProofSuite().catch(() => null), - ]); + const [nextStatus, nextAdapters] = await Promise.all([getStatus(), getAdapters()]); if (!cancelled) { setStatus(nextStatus); - setMemoryMap(nextMap); setAdapters(nextAdapters); - setProofSuite(nextProofSuite); } } catch (err) { if (!cancelled) { @@ -104,86 +45,16 @@ export function App() { }; }, []); - const memoryLayers = useMemo(() => status?.memory.layers ?? [], [status]); - const boundAdapters = useMemo(() => adapters.filter((adapter) => adapter.cartridge_bound).length, [adapters]); - const passedProofChecks = useMemo( - () => proofSuite?.checks.filter((check) => check.status === 'passed').length ?? 0, - [proofSuite], - ); - - async function submitPath(event: FormEvent) { - event.preventDefault(); - setBusy('ingest-path'); - setError(null); + async function refreshQuietly() { try { - const result = await ingestPath(path, cartridgeName); - setLastIngest(result); await refresh(); } catch (err) { setError(errorMessage(err)); - } finally { - setBusy(null); } } - async function submitUpload(event: FormEvent) { - event.preventDefault(); - setBusy('upload'); - setError(null); - try { - const result = await uploadFiles(files, cartridgeName); - setLastIngest(result); - await refresh(); - } catch (err) { - setError(errorMessage(err)); - } finally { - setBusy(null); - } - } - - async function submitQuestion(event: FormEvent) { - event.preventDefault(); - setBusy('chat'); - setError(null); - try { - const result = await askQuestion(question, profile); - setAnswer(result); - await refresh(); - } catch (err) { - setError(errorMessage(err)); - } finally { - setBusy(null); - } - } - - async function submitHashHop(kind: 'text' | 'visual') { - setBusy(`hashhop-${kind}`); - setError(null); - try { - const result = await runHashHop(kind, kind === 'visual' ? 2 : 2, kind === 'visual' ? 262144 : 1010000); - setHashHop(result); - await refresh(); - } catch (err) { - setError(errorMessage(err)); - } finally { - setBusy(null); - } - } - - async function submitProofSuite() { - setBusy('proof-suite'); - setError(null); - try { - const cartridgeId = memoryMap?.cartridges[0]?.cartridge_id; - const result = await runProofSuite(cartridgeId); - setProofSuite(result); - await refresh(); - } catch (err) { - setError(errorMessage(err)); - } finally { - setBusy(null); - } - } + const layers: CacheLayerStatus[] = status?.memory.layers ?? []; + const boundAdapters = adapters.filter((adapter) => adapter.cartridge_bound).length; return (
@@ -192,7 +63,11 @@ export function App() {

Ely-Eye v4.0

Local VLM Context OS

- @@ -200,318 +75,57 @@ export function App() { {error &&
{error}
}
- } label="Runtime" value={status?.runtime.available ? 'ready' : 'offline'} detail={formatRuntimeDetail(status)} /> - } label="GPU" value={status?.runtime.gpu_name ?? 'unknown'} detail={formatVram(status)} /> - } label="Library" value={formatNumber(status?.memory.library_token_equivalent ?? 0)} detail="token eq" /> - } label="Cartridges" value={String(status?.memory.cartridges ?? 0)} detail={`${boundAdapters} adapters bound`} /> + } + label="Runtime" + value={status?.runtime.available ? 'ready' : 'offline'} + detail={formatRuntimeDetail(status)} + /> + } + label="GPU" + value={status?.runtime.gpu_name ?? 'unknown'} + detail={formatVram(status)} + /> + } + label="Library" + value={formatNumber(status?.memory.library_token_equivalent ?? 0)} + detail={`${formatNumber(status?.memory.active_token_equivalent ?? 0)} token eq open`} + /> + } + label="Cartridges" + value={String(status?.memory.cartridges ?? 0)} + detail={`${boundAdapters} adapters bound`} + />
- }> -
- - - -
-
- - -
- {lastIngest && ( -
- {lastIngest.source_count} sources - {lastIngest.atom_count} atoms - {formatNumber(lastIngest.token_equivalent)} tokens -
- )} - - - }> -
- {profiles.map((item) => ( - - ))} -
-
-