F9: smoke checklist + performance budgets in CI

- docs/SMOKE.md: the F9 capability checklist — every carried-over harness
  capability mapped to at least one automated case (test target) or a
  manual probe with its observable, per the PRD acceptance rule.
- scripts/bench.sh: enforces both PRD performance budgets and fails CI on
  a miss — `kigi --version` p95 <= 50ms via hyperfine, and TUI first
  frame <= 300ms measured through the pty harness (real vt100 emulator
  answering terminal queries) via the new scripts/first-frame.scenario.json.
  Local run: p95 10.1ms, first frame 134ms.
- pty harness: StepOutcome gains elapsed_ms (stamped per step by the
  scripted runner) — a leading wait_for_text step's elapsed IS the
  spawn-to-first-paint latency the budget reads.
- CI: new `perf` job (macOS + Linux) building the release binaries and
  running scripts/bench.sh.
This commit is contained in:
2026-07-17 20:50:54 -04:00
parent 913caed6d3
commit 28050e8e75
5 changed files with 141 additions and 1 deletions
@@ -585,6 +585,7 @@ impl ScriptedScenarioRunner {
for (index, step) in scenario.steps.iter().enumerate() {
let step_number = index + 1;
let step_started = std::time::Instant::now();
match run_step(
&mut harness,
&content,
@@ -593,7 +594,8 @@ impl ScriptedScenarioRunner {
step_number,
step,
) {
Ok(outcome) => {
Ok(mut outcome) => {
outcome.elapsed_ms = step_started.elapsed().as_millis() as u64;
report.artifacts.extend(outcome.artifacts.clone());
report.steps.push(outcome);
}
@@ -1189,6 +1191,11 @@ pub struct StepOutcome {
pub action: String,
pub status: StepStatus,
pub message: Option<String>,
/// Wall-clock time this step took. For a leading `WaitForText` step this
/// IS the pager's first-frame latency (spawn → text painted), which the
/// CI perf budget reads (scripts/bench.sh).
#[serde(default)]
pub elapsed_ms: u64,
#[serde(default)]
pub artifacts: Vec<VisualArtifact>,
}
@@ -1200,6 +1207,7 @@ impl StepOutcome {
action: action_name(action).to_owned(),
status: StepStatus::Passed,
message: None,
elapsed_ms: 0,
artifacts: Vec::new(),
}
}
@@ -1221,6 +1229,7 @@ impl StepOutcome {
action: action_name(action).to_owned(),
status: StepStatus::Failed,
message: Some(message),
elapsed_ms: 0,
artifacts: Vec::new(),
}
}