M0: compilable skeleton — Kigi 0.1.0 fork surgery

Hard fork of xai-org/grok-build (Apache-2.0) re-targeted as Kigi, an
unofficial Kimi Code CLI community build.

Rename & identity
- 72 xai-*/xai-grok-* crates -> kigi-* (explicit: xai-grok-pager-bin ->
  kigi-bin [binary `kigi`], xai-grok-pager -> kigi-tui; rest mechanical);
  ptyctl, ptyctl-cli, third_party/ unchanged; proto package
  xai.grok.tools.v1 -> kigi.tools.v1
- Config home ~/.kigi (KIGI_SHARE_DIR override), env prefix GROK_* ->
  KIGI_*, `kigi --version` carries the unofficial-community-build notice
- clap identity, help text, startup banner, prompt templates rebranded
  (templates re-encrypted)

Deletions (PRD removal list #5/#6/#7/#9/#10)
- voice input (xai-grok-voice) and all TUI wiring
- telemetry: Mixpanel client, external OTel stream, Sentry, OTLP layers,
  trace/GCS/S3 upload queues (kigi-file-utils halved), workspace upload
  module & dc_log, heap-profile uploader, auth-diagnostics uploader,
  session-analytics halves of feedback; local zero-egress observability
  preserved in new kigi-log crate (unified log, --debug firehose,
  subsystem file logs, opt-in instrumentation)
- announcements (crate, remote-settings fields, TUI surfaces)
- plugin marketplace (crate, sources/browse/CTA/extensions-modal tab);
  direct plugin install/uninstall/update via kigi-agent git_install kept
- relay/gateway/assets endpoints and features (agent relay, headless
  relay transport, gateway bridge, LeaderEnvUrls); leader IPC socket now
  ~/.kigi/leader.sock + KIGI_LEADER_SOCKET, no ws-url derivation
- functional types rehomed instead of deleted: PermissionMode ->
  kigi-config-types, McpInitStrategy -> kigi-mcp, PrCreationSource ->
  session signals, TerminalDiagnostics -> kigi-pager-render, agent_id ->
  shell util

Endpoints
- kigi-env rewritten: single production KigiEndpoints {coding_api_base_url
  https://api.kimi.com/coding/v1 (KIGI_CODE_BASE_URL), oauth_host
  https://auth.kimi.com (KIGI_OAUTH_HOST), update_base_url (GitHub
  Releases API), upgrade_page_url}; GrokBuildEnvironment enum deleted

Toolchain & workspace hygiene
- Rust 1.97.0 pinned; edition 2024; full cargo update; git2 hoisted to
  workspace at 0.21 (Option->Result API migration), quick-xml 0.41
- Root Cargo.toml hand-maintained (PRD §8.1): version 0.1.0 inherited by
  all members, members sorted, unused deps pruned
- cargo-deny advisories gate (deny.toml with documented transitive
  exceptions); CI workflow (check/clippy/fmt/deny/test, macOS+Linux)
- cross-crate test seams re-gated behind `test-support` cargo feature;
  insta snapshot baselines renamed to the kigi_tui prefix
- clippy --workspace --all-targets: zero warnings; fmt clean

Fixes surfaced by the port
- updater probe/installer divergence (bin/kigi vs bin/grok symlink set)
- idle model-metadata refresh dead under KIGI_CODE_BASE_URL override
  (new is_effective_coding_endpoint_url, loopback+override aware)
- macOS symlinked-TMPDIR fixture canonicalization (foreign_sessions,
  fast-worktree); RSS measurement tests serialized via serial_test

Docs & legal (Apache §4)
- NOTICE added (upstream attribution + change statement); THIRD-PARTY
  notices sustained; kigi-tools ported-code notices extended; README,
  CONTRIBUTING, SECURITY, AGENTS.md rewritten

Out of scope for M0 (tracked): Kimi auth/inference (M1), search/fetch,
command parity, config import (M2), Computer Hub excision & final
brand-token sweep (M2), distribution & self-update rewrite (M3).
This commit is contained in:
ZacharyZhang-NY committed 2026-07-17 05:31:01 -04:00
commit d6c20fc13f
2612 files changed
+1353757

No files matched your search

+279
View File
@@ -0,0 +1,279 @@
//! Turn selection for compaction.
//!
//! Walks the item list backward to find a split point: keep the newest items
//! whose cumulative token count fits the target budget, compact everything
//! older.
//!
//! The split point must respect a critical invariant: an assistant item with
//! tool requests and the subsequent tool-result items that satisfy those
//! requests must stay together. Splitting between them would produce orphan
//! tool results in the next prompt, which the model API rejects with a 400.
//!
//! This is the harness-agnostic core: it operates over any slice of
//! [`CompactionItem`], so both Grok chat (`GrokTurn`) and grok-build
//! (`ConversationItem`) share one implementation.
use crate::item::CompactionItem;
/// Output of [`select_turns_to_compact`].
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SplitPlan {
/// Compact items at indices `0..split_idx`. Keep `split_idx..total`.
pub split_idx: usize,
/// Sum of `item_token_counts[..split_idx]`.
pub tokens_to_compact: u32,
}
/// Decide where to split the items for compaction.
///
/// Algorithm:
/// 1. Walk backward from the newest item, accumulating "keep" tokens.
/// 2. The candidate split index is the first one where adding more would
/// exceed `target_tokens`.
/// 3. **Snap forward** to a safe boundary: if the split would orphan tool
/// results, walk forward until past the matching tool-result items.
/// 4. Return `None` if the resulting compactable region's token count is
/// below `min_compactable` — not worth running the LLM.
///
/// # Tool-pair boundary safety
///
/// `items` is the agent's running state. A typical sequence:
///
/// ```text
/// [Assistant(tool_request_A, tool_request_B),
/// Tool(A_result),
/// Tool(B_result),
/// Assistant(response_text),
/// Assistant(tool_request_C),
/// Tool(C_result),
/// ...]
/// ```
///
/// A safe split point is one where everything **before** the split is
/// self-contained (no dangling tool requests waiting for results that live
/// after the split).
///
/// The rule we enforce: the split index must not fall in the middle of a
/// `[Assistant-with-tool-requests, Tool, Tool, ...]` run. If the candidate
/// split lands on a tool-result item, walk it forward until we pass the last
/// tool-result item following the most recent assistant-with-tool-requests.
pub fn select_turns_to_compact<T: CompactionItem>(
item_token_counts: &[u32],
items: &[T],
target_tokens: u32,
min_compactable: u32,
) -> Option<SplitPlan> {
debug_assert_eq!(
item_token_counts.len(),
items.len(),
"token counts and items must have the same length"
);
let total = items.len();
if total == 0 {
return None;
}
// Step 1: Walk backward, sum "keep" tokens until target is reached.
// Find the highest split_idx such that sum(item_token_counts[split_idx..]) ≤ target_tokens.
let mut kept = 0u32;
let mut split_idx = total; // start with "compact nothing", will move down
for i in (0..total).rev() {
let count = item_token_counts[i];
if kept.saturating_add(count) > target_tokens {
// Adding this item would exceed the budget — split here.
split_idx = i + 1;
break;
}
kept = kept.saturating_add(count);
split_idx = i;
}
// If the whole list fits within the budget, nothing to compact.
if split_idx == 0 {
return None;
}
// Step 2: Snap the split forward to a safe boundary.
let safe_split_idx = snap_to_safe_boundary(items, split_idx);
// After snapping forward we might have eaten everything.
if safe_split_idx >= total {
return None;
}
// Step 3: Compute tokens to compact and check the minimum.
let tokens_to_compact: u32 = item_token_counts[..safe_split_idx]
.iter()
.copied()
.fold(0u32, u32::saturating_add);
if tokens_to_compact < min_compactable {
return None;
}
Some(SplitPlan {
split_idx: safe_split_idx,
tokens_to_compact,
})
}
/// If `candidate` lands on a tool-result item, advance forward past all
/// tool-result items in the same tool-pair run. The "run" is delimited by the
/// previous assistant item (with tool requests) and the next non-tool item.
///
/// In effect: ensure the split lands either right before an assistant, user,
/// system, or developer item — never between an assistant-with-tool-requests
/// and its tool results.
fn snap_to_safe_boundary<T: CompactionItem>(items: &[T], candidate: usize) -> usize {
let total = items.len();
if candidate >= total {
return total;
}
// If candidate is not a tool-result item, no snap needed.
if !items[candidate].is_tool_result() {
return candidate;
}
// Candidate is a tool-result item. Find the run of contiguous tool-result
// items (starting from the assistant-with-tool-requests that preceded
// them) and advance to just past the last one in that run.
let mut idx = candidate;
while idx < total && items[idx].is_tool_result() {
idx += 1;
}
idx
}
#[cfg(test)]
mod tests {
use super::*;
use crate::item::CompactionRole;
/// Minimal mock implementing [`CompactionItem`] for selection tests.
struct MockItem {
role: CompactionRole,
}
impl MockItem {
fn user() -> Self {
Self {
role: CompactionRole::User,
}
}
fn assistant() -> Self {
Self {
role: CompactionRole::Assistant,
}
}
fn tool() -> Self {
Self {
role: CompactionRole::Tool,
}
}
}
impl CompactionItem for MockItem {
fn role(&self) -> CompactionRole {
self.role
}
fn text(&self) -> Option<String> {
None
}
fn has_tool_requests(&self) -> bool {
false
}
fn is_compaction_summary(&self) -> bool {
false
}
fn attachment_refs(&self) -> Vec<crate::item::CompactionFileRef> {
Vec::new()
}
}
#[test]
fn empty_returns_none() {
let items: Vec<MockItem> = vec![];
assert!(select_turns_to_compact(&[], &items, 100, 10).is_none());
}
#[test]
fn all_fits_in_budget_returns_none() {
let items = vec![MockItem::user(), MockItem::assistant()];
let counts = vec![10, 20];
assert!(select_turns_to_compact(&counts, &items, 1000, 5).is_none());
}
#[test]
fn splits_at_correct_index() {
// Total 100; target 30 → keep last few that fit in 30.
let items = vec![
MockItem::user(),
MockItem::assistant(),
MockItem::user(),
MockItem::assistant(),
];
let counts = vec![40, 30, 20, 10]; // keep last two (sum 30)
let plan = select_turns_to_compact(&counts, &items, 30, 5).expect("should split");
assert_eq!(plan.split_idx, 2);
assert_eq!(plan.tokens_to_compact, 70);
}
#[test]
fn below_min_compactable_returns_none() {
let items = vec![MockItem::user(), MockItem::assistant()];
let counts = vec![5, 100];
// Would split after index 0, but 5 < min_compactable (10).
assert!(select_turns_to_compact(&counts, &items, 50, 10).is_none());
}
#[test]
fn snaps_past_tool_results() {
// Layout: [User, Assistant-text, Assistant-with-tools, Tool, Tool, Assistant-text]
// If the naïve split lands on a Tool, snap forward past all Tools.
let items = vec![
MockItem::user(),
MockItem::assistant(),
MockItem::assistant(), // pretend this had tool_requests
MockItem::tool(),
MockItem::tool(),
MockItem::assistant(),
];
let counts = vec![10, 10, 10, 50, 50, 10];
// Target 60 → walking back: keep 10 (idx 5), keep 50 (idx 4)
// → 60 used. Adding idx 3 (50) overflows.
// Naïve split = 4. But items[4] is Tool → snap forward.
// Walk forward: items[4]=Tool, items[5]=Assistant → snap to 5.
let plan = select_turns_to_compact(&counts, &items, 60, 5).expect("should split");
assert_eq!(plan.split_idx, 5);
assert_eq!(plan.tokens_to_compact, 10 + 10 + 10 + 50 + 50);
}
#[test]
fn snap_does_not_advance_when_already_safe() {
let items = vec![
MockItem::user(),
MockItem::assistant(),
MockItem::user(), // safe split here
MockItem::assistant(),
];
let counts = vec![50, 50, 10, 10];
// Target 30 → keep last two (sum 20).
// Naïve split = 2. items[2] = User → safe, no snap needed.
let plan = select_turns_to_compact(&counts, &items, 30, 5).expect("should split");
assert_eq!(plan.split_idx, 2);
}
#[test]
fn snap_walks_to_end_returns_none() {
// Pathological: split would need to snap past all items.
let items = vec![MockItem::assistant(), MockItem::tool(), MockItem::tool()];
let counts = vec![10, 50, 50];
// Target 0 → naïve split = 1 (items[1] is Tool).
// Snap forward: items[1]=Tool, items[2]=Tool, idx=3=total.
// Return None — nothing left to keep.
assert!(select_turns_to_compact(&counts, &items, 0, 5).is_none());
}
}