ZacharyZhang-NY
cd9851a12c
The dense leg used to be a pair of files next to the database: a full .npy re-encoded from every atom, re-normalized on every query, and stale the moment anything was ingested. It is now a vec0 table inside the same SQLite file, written in the same transaction as the atom it describes, so a memory is queryable the instant it is remembered. - vector_index.py maintains the table; the writer inserts, the reader never rebuilds. A repair scan (ely-eye build-dense-index) catches what a crash hid, and it is not on the search path. - Ranking stays in the engine; every boundary (closed, retired, not yet true, another holder's private memory, the mounted cartridge) is checked once against the atoms table afterwards. A visibility subquery inside the KNN cost 48.9s on 153,857 vectors; the same check over 200 candidates costs 0.2ms. When the boundaries reject enough candidates the ask widens once, never twice. - The Go mirror and the web panel carry the new status fields. - bench/dense_scale.py and bench/quant_probe.py measure the leg at corpus size; bench/scale_check.py now works on a copy so the frozen baseline stays frozen. Measured on the frozen 165,973-atom corpus. bench/reports/DENSE.md, QUANT.md and SCALE.md carry the numbers and the machine they were taken on. Gates: 285 pytest, ruff clean, pyright 0 on the touched files, go build/vet/test and gofmt clean, frontend build and eslint clean. mypy is 25 findings against the parent's 24: the new one is "Skipping analyzing sqlite_vec: missing library stubs", the same class as the two existing qwen_vl_utils lines, and the repo keeps no type: ignore anywhere.
78 lines
3.1 KiB
Python
78 lines
3.1 KiB
Python
"""The senses: what this machine can see and hear, and the commands that exercise them.
|
|
|
|
Each command here depends on a model snapshot the operator supplies. The report says which
|
|
snapshots are present so a claim about this install can be checked instead of believed.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from collections.abc import Callable
|
|
from dataclasses import asdict
|
|
from pathlib import Path
|
|
|
|
import typer
|
|
|
|
from .asr import DEFERRED as SPEECH_DEFERRED
|
|
from .asr import SpeechTranscriber
|
|
from .cli_common import json_dump
|
|
from .config import Settings
|
|
from .db import Database
|
|
from .describe import DEFERRED, VisionDescriber
|
|
from .vector_index import VectorIndex
|
|
from .visual_index import VisualIndex
|
|
|
|
|
|
def senses_report(settings: Settings, db: Database) -> dict[str, dict[str, object]]:
|
|
"""What this machine can see, and what it has to pretend about.
|
|
|
|
A missing snapshot is reported, not hidden: an unrun capability is a plan, and the status
|
|
output is where an operator learns which claims about this install are real today.
|
|
"""
|
|
visual = VisualIndex(settings, db)
|
|
describer = VisionDescriber(settings)
|
|
transcriber = SpeechTranscriber(settings)
|
|
index_present = visual.index_path.exists() and visual.map_path.exists()
|
|
return {
|
|
"picture_search": {
|
|
"model_dir": str(visual.model_dir() or ""),
|
|
"model_ready": visual.available(),
|
|
"index": "present" if index_present else "absent",
|
|
},
|
|
"picture_description": {
|
|
"model_dir": str(describer.model_dir() or ""),
|
|
"model_ready": describer.available(),
|
|
"note": "" if describer.available() else DEFERRED,
|
|
},
|
|
"speech": {
|
|
"model_dir": str(transcriber.model_dir() or ""),
|
|
"model_ready": transcriber.available(),
|
|
"note": "" if transcriber.available() else SPEECH_DEFERRED,
|
|
},
|
|
}
|
|
|
|
|
|
def register(app: typer.Typer, settings: Callable[[], Settings]) -> None:
|
|
"""Add the media commands. The settings thunk lets one patch in a test redirect every command."""
|
|
|
|
@app.command()
|
|
def transcribe(path: Path) -> None:
|
|
"""Words with the instant each was spoken; the snapshot stays the operator's to supply."""
|
|
segments = SpeechTranscriber(settings()).transcribe(path)
|
|
typer.echo(json_dump([asdict(segment) for segment in segments]))
|
|
|
|
@app.command("describe-image")
|
|
def describe_image(path: Path) -> None:
|
|
"""Say what a picture shows; names the setup step when no vision model is installed."""
|
|
typer.echo(json_dump({"image": str(path), "caption": VisionDescriber(settings()).describe(path)}))
|
|
|
|
@app.command("index-visual")
|
|
def index_visual() -> None:
|
|
"""Encode every image atom so a picture or a sentence can find it."""
|
|
database = Database(settings())
|
|
typer.echo(json_dump(VisualIndex(settings(), database).build()))
|
|
|
|
@app.command("build-dense-index")
|
|
def build_dense_index() -> None:
|
|
"""Encode the memories the dense index is missing; its vectors live beside the atoms."""
|
|
typer.echo(json_dump(VectorIndex(settings()).sync()))
|