Add models.dev metadata-enrichment pipeline (providers P0c-2)

Provider /models listings that return bare ids (OpenAI-style) get context
windows, thinking levels, image support, and display names from models.dev:
kigi-models owns the transform (parse_api_json — ONE field interpretation
for the bundled snapshot AND runtime refreshes), enrich_wire_model fills
gaps with wire values always winning and model availability strictly
wire-truth. Spec rows gained models_dev_id + wire_serves_metadata; all
three current platforms are wire-served, so this pipeline is provably
inert for them (byte-identical catalogs, zero egress, zero ~/.kigi writes
— adversarially verified).

Shell side: enrichment_fetch with a 24h disk cache guarded by binary
version + keep-set + future-stamp sanity (a registry change or downgrade
refetches instead of serving a catalog missing new providers), refresh of
https://models.dev/api.json filtered to registry ids, KIGI_MODELS_DEV_URL
override with case/whitespace-tolerant kill switch, fallback chain
fresh-cache > refresh > stale-cache > bundled (each step logged). The
fast path returns an empty catalog without forcing the bundled parse.

From the review: blast-radius-confined parsing (one drifted provider on
models.dev warn-skips instead of failing the whole refresh), registry-
coverage and field-coverage tests guarding script/parser drift, a path-
injectable core with 8 state-machine tests (one of which caught a guard
patch that had failed to apply), _meta provenance stamp in the snapshot,
and models.dev (MIT) attribution in NOTICE. Snapshot: 29 providers, 1124
models, 246KB, regenerated by scripts/gen_enrichment_snapshot.py (pure
filter, no transform).
This commit is contained in:
2026-07-21 03:36:51 -04:00
parent 83e6935189
commit fdf9b956f5
11 changed files with 953 additions and 8 deletions
+15 -3
View File
@@ -18,9 +18,12 @@ import) or any `KIMI_*` env var.
- **Zero egress**: outbound connections are limited to
`auth.kimi.com`, `api.kimi.com`, `api.moonshot.cn`, `api.moonshot.ai`,
GitHub Releases domains, and user-configured MCP servers. No telemetry,
no analytics, ever. `crates/codegen/kigi-env` is the single home of
first-party endpoints.
GitHub Releases domains, user-configured MCP servers, the endpoints of
provider platforms the user has credentialed, and `models.dev` (model
metadata refresh — reached ONLY when an enabled platform's `/models` wire
lacks metadata, `wire_serves_metadata=false`; Kimi/Moonshot never trigger
it; `KIGI_MODELS_DEV_URL=0` disables). No telemetry, no analytics, ever.
`crates/codegen/kigi-env` is the single home of first-party endpoints.
- **Toolchain**: Rust 1.97.0 (rust-toolchain.toml), edition 2024.
- **Gates** (all must stay green):
`cargo check --workspace --all-targets`,
@@ -165,6 +168,15 @@ edges stay deterministic Rust. The harness appends a terminal
are built generically from advertised methods (`AuthMethodKind::
ApiKeyPlatform`), so new registry rows appear in the picker with no TUI
changes.
- Model metadata (context window, thinking levels) comes from the provider
wire when served; metadata-poor listings are enriched from models.dev
(`kigi-models/src/enrichment.rs` — bundled raw snapshot regenerated by
`scripts/gen_enrichment_snapshot.py`, single Rust transform
`parse_api_json` for bundled + runtime refresh; 24h cache
`~/.kigi/models_dev_cache.json`). Wire values always win; enrichment
never invents model availability. Canonical reasoning efforts:
none/minimal/low/medium/high/xhigh/max (`max` split from `xhigh`
2026-07; Kimi wire spells its top tier `max`, kimi_compat renames).
## Milestones (PRD §8.3)