F8: distribution and GitHub-Releases self-update

- .github/workflows/release.yml: on tag v* build all 5 targets (macOS
  arm64/x86_64, Linux arm64/x86_64 incl. free arm runners, Windows
  x86_64) with the release-dist profile, archive kigi-<version>-<triple>
  with LICENSE/NOTICE/THIRD-PARTY-NOTICES, generate SHA256SUMS, publish
  the release (prerelease for tags containing '-'), with a tag↔workspace
  version guard.
- install.sh / install.ps1 (repo root): platform detection, latest or
  --version download from GitHub Releases, SHA-256 verification against
  SHA256SUMS, install into the kigi home's downloads/ + bin/kigi symlink
  (the same layout the self-updater manages), smoke test, PATH guidance.
- kigi-update rewritten onto the GitHub Releases API (documented wire
  shape; stable=/latest, alpha=semver-max across the list, pinned=/tags):
  SHA-256 gate before any binary swap, tar.gz/zip extraction per
  platform, atomic bin/kigi symlink swap, channel/rollback semantics and
  the KIGI_AUTO_UPDATE gate preserved verbatim; every x.ai/GCS/npm
  endpoint deleted, npm/gh-release installers removed, legacy grok/agent
  links retired on install. kigi-env owns the update base URL with a
  KIGI_UPDATE_BASE_URL override (this is what the test artifact server
  injects).
- .cargo/config.toml: removed the non-portable neoverse-v2 CPU pin on
  Linux arm64 (fleet-specific); RELRO/NX hardening link-args now apply
  to the gnu targets too, matching the release-dist profile's contract.
- THIRD-PARTY-NOTICES regenerated via cargo-about (about.toml +
  template); the M0 hand-built file is dropped and README points at the
  generated one. docs/RELEASE.md carries the release checklist.
- Deleted xAI-era leftovers: kigi-tui/scripts/install*.{sh,ps1} (x.ai
  CDN) and the @xai-official/grok npm skeleton (PRD F8: no npm).

Gates: fmt clean; workspace check/clippy 0/0 (--locked, -D warnings);
kigi-update 58 lib + 86 integration tests green; deny ok;
release-dist build of kigi-bin succeeds and reports 'kigi 0.1.0'.
This commit is contained in:
2026-07-18 00:54:53 -04:00
parent 5e4e24db99
commit 86e3724310
57 changed files with 27431 additions and 27787 deletions
@@ -1,27 +1,44 @@
//! Controllable raw HTTP/1.1 artifact server shared by the blitz
//! download/install tests and the concurrent-update convergence tests.
//! Controllable raw HTTP/1.1 GitHub-Releases-shaped server shared by the
//! blitz download/install tests and the concurrent-update convergence tests.
//!
//! Serves a real executable artifact and can truncate the body, close the
//! connection early, serve a right-length-but-garbage body, or hang
//! mid-transfer — for both the parallel byte-range path and the
//! single-connection path. It also counts body-serving GETs (HEAD probes are
//! excluded) so tests can assert how many downloads actually happened, and
//! supports a "slow" mode that widens the race window so concurrent
//! installers genuinely overlap in flight.
//! Serves the three routes the updater consumes:
//!
//! - `GET /releases/latest` and `GET /releases/tags/v{version}` — release
//! JSON in the real GitHub wire shape
//! (<https://docs.github.com/en/rest/releases/releases>):
//! `{"tag_name":"v0.1.0","assets":[{"name":"...","browser_download_url":"..."}]}`
//! - `GET /dl/{version}/SHA256SUMS` — checksum manifest for the archive
//! - `GET /dl/{version}/kigi-{version}-{triple}.tar.gz` — the archive itself
//!
//! The archive route can serve the real archive, truncate the body, serve a
//! right-length-but-garbage body (defeated by the SHA-256 gate), serve a
//! correctly-checksummed archive whose binary fails to run (defeated by the
//! smoke test), or hang mid-transfer — for both the parallel byte-range path
//! and the single-connection path. It also counts archive-serving GETs (HEAD
//! probes and metadata routes are excluded) so tests can assert how many
//! downloads actually happened, and supports a "slow" mode that widens the
//! race window so concurrent installers genuinely overlap in flight.
use std::collections::HashMap;
use std::io::{Read, Write};
use std::net::{TcpListener, TcpStream};
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;
/// How the server corrupts (or doesn't) the next download.
use super::{archive_name, make_release_archive, sha256_hex};
/// How the server corrupts (or doesn't) the next archive download.
#[derive(Clone, Copy, Debug)]
pub enum Mode {
/// Serve the real artifact correctly.
/// Serve the real archive correctly.
Full,
/// Serve a right-length body that exits non-zero (fails the smoke-test).
/// Serve a right-length body of wrong bytes — SHA256SUMS still lists the
/// GOOD archive's hash, so the checksum gate must reject it.
Garbage,
/// Serve a correctly-checksummed archive whose `kigi` binary exits
/// non-zero — only the smoke test can reject it.
BadBinary,
/// Advertise the full length but send only `k` bytes then close the socket
/// (silent truncation: premature EOF / short range chunk).
Truncate(usize),
@@ -29,11 +46,41 @@ pub enum Mode {
Hang(usize),
}
/// Precomputed per-version fixtures.
struct VersionFixture {
/// Real archive: tar.gz containing `kigi` = the good binary body.
good_archive: Arc<Vec<u8>>,
/// Same-shape archive whose `kigi` exits 1; its hash is served in
/// SHA256SUMS while `Mode::BadBinary` is active.
bad_archive: Arc<Vec<u8>>,
}
struct ServerState {
body: Arc<Vec<u8>>,
versions: HashMap<String, VersionFixture>,
/// The good binary body used to synthesize fixtures for versions
/// requested but not yet registered.
default_binary: Vec<u8>,
latest: String,
mode: Mode,
}
impl ServerState {
fn fixture(&mut self, version: &str) -> &VersionFixture {
if !self.versions.contains_key(version) {
let good = make_release_archive(&self.default_binary);
let bad = make_release_archive(b"#!/bin/sh\nexit 1\n");
self.versions.insert(
version.to_string(),
VersionFixture {
good_archive: Arc::new(good),
bad_archive: Arc::new(bad),
},
);
}
&self.versions[version]
}
}
pub struct ArtifactServer {
addr: std::net::SocketAddr,
state: Arc<Mutex<ServerState>>,
@@ -43,12 +90,17 @@ pub struct ArtifactServer {
}
impl ArtifactServer {
pub fn start(body: Vec<u8>) -> Self {
/// Start a server whose release archives contain `binary_body` as the
/// `kigi` binary. `latest` starts as `0.0.0`; set it with
/// [`ArtifactServer::set_latest`] before exercising latest-based flows.
pub fn start(binary_body: Vec<u8>) -> Self {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
listener.set_nonblocking(true).unwrap();
let addr = listener.local_addr().unwrap();
let state = Arc::new(Mutex::new(ServerState {
body: Arc::new(body),
versions: HashMap::new(),
default_binary: binary_body,
latest: "0.0.0".to_string(),
mode: Mode::Full,
}));
let shutdown = Arc::new(AtomicBool::new(false));
@@ -86,30 +138,49 @@ impl ArtifactServer {
}
}
pub fn uri(&self) -> String {
format!("http://{}", self.addr)
/// Base URL to hand the updater (`…/releases`, mirroring the production
/// `https://api.github.com/repos/{owner}/{repo}/releases`).
pub fn base(&self) -> String {
format!("http://{}/releases", self.addr)
}
/// Version served by `GET /releases/latest`.
pub fn set_latest(&self, version: &str) {
self.state.lock().unwrap().latest = version.to_string();
}
pub fn set_mode(&self, mode: Mode) {
self.state.lock().unwrap().mode = mode;
}
/// Number of body-serving GET requests handled so far (HEAD probes from
/// the parallel-download path are excluded). Tests use this to assert
/// how many downloads actually happened — e.g. that a sequential updater
/// converged onto an already-installed binary without re-downloading.
/// One download may span multiple GETs when the parallel byte-range path
/// splits it, so tests asserting exact counts use a small artifact
/// (single-connection path, 1 GET per download).
/// Length of the (good) archive for `version` — corruption offsets for
/// [`Mode::Truncate`]/[`Mode::Hang`] are positions within this body.
pub fn archive_len(&self, version: &str) -> usize {
self.state
.lock()
.unwrap()
.fixture(version)
.good_archive
.len()
}
/// Number of archive-serving GET requests handled so far (HEAD probes
/// from the parallel-download path and metadata/SHA256SUMS routes are
/// excluded). Tests use this to assert how many downloads actually
/// happened — e.g. that a sequential updater converged onto an
/// already-installed binary without re-downloading. One download may span
/// multiple GETs when the parallel byte-range path splits it, so tests
/// asserting exact counts use a small artifact (single-connection path,
/// 1 GET per download).
pub fn request_count(&self) -> usize {
self.gets.load(Ordering::Relaxed)
}
/// When enabled, hold each Full/Garbage response open ~500ms before
/// sending the body. This keeps an installer in flight long enough for
/// concurrent installers to genuinely overlap even on a heavily loaded
/// CI host — a too-short hold would let race tests run the installers
/// back-to-back and never exercise the concurrent window.
/// When enabled, hold each archive response open ~500ms before sending
/// the body. This keeps an installer in flight long enough for concurrent
/// installers to genuinely overlap even on a heavily loaded CI host — a
/// too-short hold would let race tests run the installers back-to-back
/// and never exercise the concurrent window.
pub fn set_slow(&self, slow: bool) {
self.slow.store(slow, Ordering::Relaxed);
}
@@ -134,6 +205,39 @@ fn parse_range(request: &str) -> Option<(usize, usize)> {
None
}
/// The request path, without query string.
fn parse_path(request: &str) -> String {
request
.lines()
.next()
.and_then(|l| l.split_whitespace().nth(1))
.unwrap_or("/")
.split('?')
.next()
.unwrap_or("/")
.to_string()
}
/// Release JSON for `version` with asset URLs rooted at this server.
fn release_json(addr: &std::net::SocketAddr, version: &str) -> String {
let name = archive_name(version);
format!(
r#"{{"tag_name":"v{version}","draft":false,"prerelease":false,"assets":[{{"name":"{name}","browser_download_url":"http://{addr}/dl/{version}/{name}"}},{{"name":"SHA256SUMS","browser_download_url":"http://{addr}/dl/{version}/SHA256SUMS"}}]}}"#
)
}
fn write_simple_response(stream: &mut TcpStream, status: &str, body: &[u8], is_head: bool) {
let head = format!(
"HTTP/1.1 {status}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n",
body.len()
);
let _ = stream.write_all(head.as_bytes());
if !is_head {
let _ = stream.write_all(body);
}
let _ = stream.flush();
}
fn handle_connection(
mut stream: TcpStream,
state: Arc<Mutex<ServerState>>,
@@ -169,18 +273,74 @@ fn handle_connection(
}
let request = String::from_utf8_lossy(&buf).to_string();
let is_head = request.starts_with("HEAD");
let path = parse_path(&request);
let range = parse_range(&request);
let addr = stream.local_addr().unwrap();
// ── Metadata routes ─────────────────────────────────────────────────────
if path == "/releases/latest" {
let latest = state.lock().unwrap().latest.clone();
let body = release_json(&addr, &latest);
write_simple_response(&mut stream, "200 OK", body.as_bytes(), is_head);
return;
}
if let Some(tag) = path.strip_prefix("/releases/tags/v") {
let body = release_json(&addr, tag);
write_simple_response(&mut stream, "200 OK", body.as_bytes(), is_head);
return;
}
// ── Asset routes: /dl/{version}/{name} ──────────────────────────────────
let Some(rest) = path.strip_prefix("/dl/") else {
write_simple_response(&mut stream, "404 Not Found", b"not found", is_head);
return;
};
let Some((version, asset)) = rest.split_once('/') else {
write_simple_response(&mut stream, "404 Not Found", b"not found", is_head);
return;
};
let (good, bad, mode) = {
let mut st = state.lock().unwrap();
let mode = st.mode;
let fixture = st.fixture(version);
(
fixture.good_archive.clone(),
fixture.bad_archive.clone(),
mode,
)
};
if asset == "SHA256SUMS" {
// BadBinary serves the bad archive WITH its correct hash (a release
// whose binary is broken but whose checksums are fine); every other
// mode lists the good archive's hash so in-transit corruption is
// caught by the checksum gate.
let hashed: &[u8] = match mode {
Mode::BadBinary => &bad,
_ => &good,
};
let body = format!("{} {}\n", sha256_hex(hashed), archive_name(version));
write_simple_response(&mut stream, "200 OK", body.as_bytes(), is_head);
return;
}
if asset != archive_name(version) {
write_simple_response(&mut stream, "404 Not Found", b"not found", is_head);
return;
}
// ── Archive body with corruption modes ──────────────────────────────────
// Count only body-serving GETs; the parallel path's HEAD probe is excluded.
if !is_head {
gets.fetch_add(1, Ordering::Relaxed);
}
let range = parse_range(&request);
let (body, mode) = {
let st = state.lock().unwrap();
(st.body.clone(), st.mode)
let body: &[u8] = match mode {
Mode::BadBinary => &bad,
_ => &good,
};
let total = body.len();
let body: &[u8] = &body;
// Determine the byte slice this request is for, plus the length we will
// claim in Content-Length.
@@ -190,7 +350,7 @@ fn handle_connection(
};
let claimed_len = slice_end_excl - slice_start;
// For truncation/hang, `k` is a GLOBAL cutoff across the whole artifact:
// For truncation/hang, `k` is a GLOBAL cutoff across the whole archive:
// a slice that reaches past byte `k` is sent short, so the parallel path's
// later chunk (or the single-connection body) is the one truncated.
let send_end = match mode {
@@ -201,9 +361,9 @@ fn handle_connection(
// truncated modes it may be shorter than the advertised `claimed_len`.
let payload: Vec<u8> = match mode {
Mode::Garbage => {
let mut bad = b"#!/bin/sh\nexit 1\n".to_vec();
bad.resize(claimed_len, b'\n');
bad
let mut bad_bytes = b"not the archive you checksummed".to_vec();
bad_bytes.resize(claimed_len, b'\n');
bad_bytes
}
_ => body[slice_start..send_end].to_vec(),
};
@@ -236,7 +396,7 @@ fn handle_connection(
}
match mode {
Mode::Full | Mode::Garbage => {
Mode::Full | Mode::Garbage | Mode::BadBinary => {
// Hold the connection open longer so concurrent installers
// genuinely overlap mid-download (see `set_slow`).
if slow.load(Ordering::Relaxed) {