F9: smoke checklist + performance budgets in CI
- docs/SMOKE.md: the F9 capability checklist — every carried-over harness capability mapped to at least one automated case (test target) or a manual probe with its observable, per the PRD acceptance rule. - scripts/bench.sh: enforces both PRD performance budgets and fails CI on a miss — `kigi --version` p95 <= 50ms via hyperfine, and TUI first frame <= 300ms measured through the pty harness (real vt100 emulator answering terminal queries) via the new scripts/first-frame.scenario.json. Local run: p95 10.1ms, first frame 134ms. - pty harness: StepOutcome gains elapsed_ms (stamped per step by the scripted runner) — a leading wait_for_text step's elapsed IS the spawn-to-first-paint latency the budget reads. - CI: new `perf` job (macOS + Linux) building the release binaries and running scripts/bench.sh.
This commit is contained in:
@@ -585,6 +585,7 @@ impl ScriptedScenarioRunner {
|
||||
|
||||
for (index, step) in scenario.steps.iter().enumerate() {
|
||||
let step_number = index + 1;
|
||||
let step_started = std::time::Instant::now();
|
||||
match run_step(
|
||||
&mut harness,
|
||||
&content,
|
||||
@@ -593,7 +594,8 @@ impl ScriptedScenarioRunner {
|
||||
step_number,
|
||||
step,
|
||||
) {
|
||||
Ok(outcome) => {
|
||||
Ok(mut outcome) => {
|
||||
outcome.elapsed_ms = step_started.elapsed().as_millis() as u64;
|
||||
report.artifacts.extend(outcome.artifacts.clone());
|
||||
report.steps.push(outcome);
|
||||
}
|
||||
@@ -1189,6 +1191,11 @@ pub struct StepOutcome {
|
||||
pub action: String,
|
||||
pub status: StepStatus,
|
||||
pub message: Option<String>,
|
||||
/// Wall-clock time this step took. For a leading `WaitForText` step this
|
||||
/// IS the pager's first-frame latency (spawn → text painted), which the
|
||||
/// CI perf budget reads (scripts/bench.sh).
|
||||
#[serde(default)]
|
||||
pub elapsed_ms: u64,
|
||||
#[serde(default)]
|
||||
pub artifacts: Vec<VisualArtifact>,
|
||||
}
|
||||
@@ -1200,6 +1207,7 @@ impl StepOutcome {
|
||||
action: action_name(action).to_owned(),
|
||||
status: StepStatus::Passed,
|
||||
message: None,
|
||||
elapsed_ms: 0,
|
||||
artifacts: Vec::new(),
|
||||
}
|
||||
}
|
||||
@@ -1221,6 +1229,7 @@ impl StepOutcome {
|
||||
action: action_name(action).to_owned(),
|
||||
status: StepStatus::Failed,
|
||||
message: Some(message),
|
||||
elapsed_ms: 0,
|
||||
artifacts: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user