Kigi-CLI/crates/codegen/kigi-test-support

kigi-test-support

Shared test infrastructure for the kigi crates: mock inference server, SSE wire-format generators, ACP stdio clients, headless runner, and sandboxed process env. Consumed by kigi-shell integration tests, kigi-pager-pty-harness (ContentController), and kigi-sampler tests.

Freshness rule: update this README in the same PR that changes src/ — reviewers should treat a src/ diff without a README diff as incomplete.

How-to-test discovery lives with the pager PTY harness crate (kigi-pager-pty-harness). This file is the API reference for the shared test-support surface.

Module map

Module	What it provides
mock_server	MockInferenceServer — /v1/chat/completions, /v1/responses, /v1/messages, /v1/models, /v1/settings, /v1/user on 127.0.0.1:0. /v1/models entries are MockModelEntry (re-exported as MockModel for PTY tests): new(id) / with_agent_type(id, ty) plus chainable with_api_backend, with_supports_backend_search(bool) → supportsBackendSearch, with_supports_reasoning_effort(bool) → supportsReasoningEffort, with_reasoning_effort(&str) → reasoningEffort, with_reasoning_efforts(Vec<Value>) → reasoningEfforts (raw option tables/bare strings), all emitted top-level as parse_remote_model_value reads them. Three response modes for the inference endpoints, precedence scripted > required-auth > mode: (1) echo (default) streams Echo: <last user message>, whitespace-collapsing; (2) fixed via set_response(text), byte-exact delta reconstruction (newlines preserved — fenced code blocks survive); (3) scripted via enqueue_response. Constructors (start, start_with_models, start_with_required_auth) return anyhow::Result. Settings are 404-until-set (set_settings(impl Serialize), preset_allow_access() for the {"allow_access": true} gate); scripted /v1/settings one-shots (enqueue_response) take precedence over the steady-state value (stale-snapshot tests). /v1/user serves a minimal UserInfo whose subscriptionTier is controlled by set_user_subscription_tier(Option<&str>) (None = free); its log entries keep the query string (e.g. /v1/user?include=subscription) so subscription-check cadence is countable. Request log: requests() (LogEntry with body, authorization, full POST headers + header(name) accessor), request_bodies(), request_count(), has_chat_completion_request() / has_responses_request() (exact, per endpoint), messages_request_count(), last_system_prompt(), request_log_summary(). Storage: POST /v1/storage with flippable 401 (set_storage_unauthorized); accepted uploads via storage_uploads() → StorageUpload { path, size, body, authorization } (body retained up to 256 KiB, empty above; authorization is the raw header). Runtime knobs: set_models, set_messages_stop_reason. Shuts down on drop.
scripted	Data-only scripting (no axum types in the surface): SseEvent { event, data } (::data, ::with_event), ScriptedBody::{Json, Sse, Raw} (Raw = byte-controllable malformed SSE), ScriptedResponse { status, headers, body } (::sse, ::json, ::text). enqueue_response(path, response) queues FIFO per path; consumed by the three inference endpoints, falling back to the active mode when empty. Scripted SSE bodies honor the server's set_chunk_delay pacing, same as the echo/fixed modes. Validation is eager — bad status/header panics at the enqueue call site. Seed of the phase-2 scriptable-mock format.
sse	The three wire formats as event-list builders: chat_completion_events / responses_api_events / messages_api_events(text, model, stop_reason) (echo-style, whitespace-collapsing) plus byte-exact variants chat_completion_events_exact / responses_api_events_exact (messages is single-delta, byte-exact by construction). The exact/echo split is load-bearing — see the in-module byte-exactness tests. Also the scripted-scenario builders returning SseEvents (for ScriptedResponse::sse): responses_api_reasoning_only_events(reasoning, model) — reasoning summary deltas completing with a reasoning item but no message/output-text, so the shell collector classifies the turn EmptyReason::ReasoningOnly (the model-doomloop trigger); responses_api_reasoning_and_text_events(reasoning, text, model) — reasoning deltas then a normal text answer (the ordinary reasoning-model turn); responses_api_reasoning_then_tool_call_events(reasoning, call_id, name, arguments, model) + its Chat Completions twin chat_completions_reasoning_then_tool_call_events(...) — reasoning deltas then one tool call (the think-then-call turn whose tool call finishes the thought and keeps the turn non-empty); the doom-loop check trio: responses_api_doom_loop_check_events(triggers, reasoning, model) — a doomed reasoning-only turn with NAMED response.doom_loop_check frames re-sent per cumulative prefix of triggers plus the terminal doom_loop_check.triggers copy on response.completed, responses_api_doom_loop_terminal_only_events(triggers, reasoning, text, model) — a normal answer whose terminal response alone carries the field, and responses_api_with_doom_loop_frame(check_frame_data, reasoning, text, model) — splices one named check frame with a caller-supplied payload (byte-exact kigi_sampling_types::doom_loop::SAMPLE_CHECK_EVENT_DATA{,_CUMULATIVE} fixtures or malformed variants) into an ordinary turn.
acp_client	KigiStdioClient — drives kigi agent stdio over real pipes through agent-client-protocol: spawn variants (spawn, spawn_with_home, spawn_with_home_and_env, spawn_with_home_env_and_args), initialize/authenticate, session create/load, prompt, *_with_timeout wrappers, captured text + stderr. RawStdioClient — raw-wire sibling for bytes the typed ClientSideConnection can never produce (escaped-slash methods "session\/prompt", string UUID ids — the Xcode/Foundation shape): send_line writes a line verbatim; response_for_id matches the response by exact string id (the match IS the id-echo assertion), skips notifications, auto-refuses agent→client requests with -32601, and panics on timeout with skipped-traffic diagnostics (count + last lines; 0 other messages = true silence). Both spawn through one hermetic spawn_agent_process (sandbox env + debug-log kill-list exists once) atop process::spawn_piped_with_stderr_capture (crate-internal process module: pipes, kill_on_drop, stderr drain — also used by leader::LeaderStdioClient).
headless	run_headless(server, args, cwd) / run_headless_with_cmd(cmd) → HeadlessResult { status, stdout, stderr, timed_out } (60s cap), assert_headless_success, assert_no_crashes (panic/SIGSEGV/linker patterns), stderr_tail.
env	kigi_binary() (KIGI_BINARY env → CARGO_BIN_EXE → local debug build of kigi-tui), git_workdir() (temp git repo, forces full libgit2 init), test_env_cmd_tokio(cmd, mock_url, home) (sandboxed HOME and KIGI_SHARE_DIR — Windows resolves ~ via USERPROFILE, so HOME alone doesn't sandbox — + mock endpoints + telemetry kill-switches).
leader	Unix-only LeaderStdioClient (kigi agent --leader stdio, env_clear-hermetic, sandboxed KIGI_LEADER_SOCKET; spawn_with_binary runs an explicit binary for version-skew lanes, per-role resolution via leader_binary() / client_binary() honoring KIGI_BINARY_LEADER / KIGI_BINARY_CLIENT) + lock-file helpers: leader_lock_path, read_leader_pid, pid_alive, wait_for_live_leader, wait_for_new_leader, wait_for_replay_notifications, leader_log.
uds_proxy	Unix-only UdsProxy — frame-aware (4-byte BE length prefix) man-in-the-middle for leader IPC sockets. UdsProxy::spawn(proxy_path, upstream_path, FaultPlan); FaultPlan { direction, drop_frame, sever_mid_frame, delay, duplicate_frame } (1-based frame index, per connection per direction); runtime FaultHandle::sever_now() + forwarded(direction) counters; frame bodies capped at 64 MiB (leader-transport parity — corrupt lengths error instead of allocating). Zero production changes: point LeaderClient::connect / KIGI_LEADER_SOCKET at the proxy path.

Consumer matrix

Consumer	Uses	Notes
kigi-shell tests/*.rs	Everything	Direct imports (use kigi_test_support::* or module paths); no local shim.
kigi-pager-pty-harness src/content.rs	MockInferenceServer, MockModelEntry (re-exported as MockModel)	ContentController wraps the server and keeps the HOME-sandbox TempDir + env_for_pager() harness-side; presets allow_access + a fixed default response at construction.
kigi-sampler tests/test_actor.rs	sse generators	Happy-path payloads only; the actor keeps its own router for stall/conditional fixtures.

Adding a capability

A response mode (mock_server.rs): extend the private ResponseMode enum

• add the setter; wire the new arm into all three inference handlers (the match in each route); scripted responses must still win. Extend the in-crate tests: an HTTP round-trip for the new mode plus a leg in scripted_responses_serve_fifo_per_path_then_fall_back proving fallback reaches it. The echo pinning test (echo_mode_echoes_last_user_message) must pass unmodified — echo bytes are frozen.

A wire format (sse.rs): add the echo-style builder and, if clients reconstruct text byte-for-byte, an _exact variant built on a delta fn; then add the serving arm in mock_server (all modes) and a route if it is a new endpoint. Extend the byte-exactness pins (deltas_reconstruct_multiline_response_byte_for_byte, deltas_preserve_runs_of_whitespace) — they are the contract that fenced code blocks (mermaid) survive streaming. A scripted-scenario builder (one that models a specific completion the echo/fixed modes can't express, e.g. responses_api_reasoning_only_events) instead returns SseEvents for ScriptedResponse::sse, needs no mock_server mode wiring, and ships with an in-module shape test asserting its event shape.

A scripted-body kind (scripted.rs): new ScriptedBody variant + render arm in into_response_paced + eager checks in validate if the data can be invalid. Add an in-crate test asserting client-visible bytes (the Raw byte-exactness test is the template) and keep scripted_response_takes_precedence_over_required_auth green — precedence is part of the contract.