M0: compilable skeleton — Kigi 0.1.0 fork surgery

Hard fork of xai-org/grok-build (Apache-2.0) re-targeted as Kigi, an
unofficial Kimi Code CLI community build.

Rename & identity
- 72 xai-*/xai-grok-* crates -> kigi-* (explicit: xai-grok-pager-bin ->
  kigi-bin [binary `kigi`], xai-grok-pager -> kigi-tui; rest mechanical);
  ptyctl, ptyctl-cli, third_party/ unchanged; proto package
  xai.grok.tools.v1 -> kigi.tools.v1
- Config home ~/.kigi (KIGI_SHARE_DIR override), env prefix GROK_* ->
  KIGI_*, `kigi --version` carries the unofficial-community-build notice
- clap identity, help text, startup banner, prompt templates rebranded
  (templates re-encrypted)

Deletions (PRD removal list #5/#6/#7/#9/#10)
- voice input (xai-grok-voice) and all TUI wiring
- telemetry: Mixpanel client, external OTel stream, Sentry, OTLP layers,
  trace/GCS/S3 upload queues (kigi-file-utils halved), workspace upload
  module & dc_log, heap-profile uploader, auth-diagnostics uploader,
  session-analytics halves of feedback; local zero-egress observability
  preserved in new kigi-log crate (unified log, --debug firehose,
  subsystem file logs, opt-in instrumentation)
- announcements (crate, remote-settings fields, TUI surfaces)
- plugin marketplace (crate, sources/browse/CTA/extensions-modal tab);
  direct plugin install/uninstall/update via kigi-agent git_install kept
- relay/gateway/assets endpoints and features (agent relay, headless
  relay transport, gateway bridge, LeaderEnvUrls); leader IPC socket now
  ~/.kigi/leader.sock + KIGI_LEADER_SOCKET, no ws-url derivation
- functional types rehomed instead of deleted: PermissionMode ->
  kigi-config-types, McpInitStrategy -> kigi-mcp, PrCreationSource ->
  session signals, TerminalDiagnostics -> kigi-pager-render, agent_id ->
  shell util

Endpoints
- kigi-env rewritten: single production KigiEndpoints {coding_api_base_url
  https://api.kimi.com/coding/v1 (KIGI_CODE_BASE_URL), oauth_host
  https://auth.kimi.com (KIGI_OAUTH_HOST), update_base_url (GitHub
  Releases API), upgrade_page_url}; GrokBuildEnvironment enum deleted

Toolchain & workspace hygiene
- Rust 1.97.0 pinned; edition 2024; full cargo update; git2 hoisted to
  workspace at 0.21 (Option->Result API migration), quick-xml 0.41
- Root Cargo.toml hand-maintained (PRD §8.1): version 0.1.0 inherited by
  all members, members sorted, unused deps pruned
- cargo-deny advisories gate (deny.toml with documented transitive
  exceptions); CI workflow (check/clippy/fmt/deny/test, macOS+Linux)
- cross-crate test seams re-gated behind `test-support` cargo feature;
  insta snapshot baselines renamed to the kigi_tui prefix
- clippy --workspace --all-targets: zero warnings; fmt clean

Fixes surfaced by the port
- updater probe/installer divergence (bin/kigi vs bin/grok symlink set)
- idle model-metadata refresh dead under KIGI_CODE_BASE_URL override
  (new is_effective_coding_endpoint_url, loopback+override aware)
- macOS symlinked-TMPDIR fixture canonicalization (foreign_sessions,
  fast-worktree); RSS measurement tests serialized via serial_test

Docs & legal (Apache §4)
- NOTICE added (upstream attribution + change statement); THIRD-PARTY
  notices sustained; kigi-tools ported-code notices extended; README,
  CONTRIBUTING, SECURITY, AGENTS.md rewritten

Out of scope for M0 (tracked): Kimi auth/inference (M1), search/fetch,
command parity, config import (M2), Computer Hub excision & final
brand-token sweep (M2), distribution & self-update rewrite (M3).
This commit is contained in:
2026-07-17 05:31:01 -04:00
commit d6c20fc13f
2612 changed files with 1353757 additions and 0 deletions
@@ -0,0 +1,142 @@
use crate::sampling::Client as OaiCompatClient;
use crate::sampling::types::ChatRequestMessage;
use crate::sampling::{ConversationItem, ConversationRequest, Role};
use anyhow::Result;
pub fn build_transcript(messages: &[ConversationItem]) -> String {
const MAX_CONTENT_BYTES: usize = 2000;
if messages.is_empty() {
return String::new();
}
// Pre-allocate with estimated capacity to avoid reallocations
let estimated_capacity: usize = messages
.iter()
.map(|m| m.text_content().len().min(MAX_CONTENT_BYTES) + 16)
.sum();
let mut result = String::with_capacity(estimated_capacity);
for m in messages {
let role = match m.role() {
Role::System => "system",
Role::User => "user",
Role::Assistant => "assistant",
Role::Tool => "tool",
};
let text_content = m.text_content();
let content = text_content.trim();
result.push('[');
result.push_str(role);
result.push_str("] ");
if content.len() > MAX_CONTENT_BYTES {
let end = floor_char_boundary(content, MAX_CONTENT_BYTES);
result.push_str(&content[..end]);
result.push_str("...");
} else {
result.push_str(content);
}
result.push_str("\n\n");
}
// Remove trailing newline to match original join behavior
result.pop();
result
}
/// Returns the largest valid UTF-8 character boundary index at or before `index`.
#[inline]
pub(super) fn floor_char_boundary(s: &str, index: usize) -> usize {
if index >= s.len() {
s.len()
} else if s.is_char_boundary(index) {
index
} else {
// UTF-8 characters are at most 4 bytes, back up at most 3 bytes
let mut i = index;
while i > 0 && !s.is_char_boundary(i) {
i -= 1;
}
i
}
}
pub fn truncate_middle_words(s: &str, max_words: usize) -> (String, Option<usize>) {
let words: Vec<&str> = s.split_whitespace().collect();
let total = words.len();
if total <= max_words || max_words == 0 {
return (s.to_string(), None);
}
let left = max_words / 2;
let right = max_words - left;
let prefix = words[..left].join(" ");
let suffix = words[total.saturating_sub(right)..].join(" ");
let truncated_count = total.saturating_sub(left + right);
let marker = format!("{} words truncated…", truncated_count);
(format!("{}\n{}\n{}", prefix, marker, suffix), Some(total))
}
pub async fn text_completion(
sampling_client: &OaiCompatClient,
system: ChatRequestMessage,
user: ChatRequestMessage,
temperature: Option<f32>,
max_tokens: Option<u32>,
) -> Result<String> {
let mut request = ConversationRequest::from_items(vec![
ConversationItem::from(system),
ConversationItem::from(user),
]);
request.temperature = temperature;
request.max_output_tokens = max_tokens;
let response = sampling_client.conversation_collect(request).await?;
let text = response
.assistant()
.map(|a| a.content.as_ref().to_owned())
.unwrap_or_default();
let trimmed = text.trim();
if trimmed.is_empty() {
anyhow::bail!("empty response");
}
Ok(trimmed.to_string())
}
/// Build a prompt string from a template by injecting a truncated transcript and additional variables.
/// - Replaces `{{transcript}}` with a truncated transcript derived from `conversation`
/// - Applies each `(needle, value)` replacement from `extras` sequentially
pub fn build_prompt_from_template(
conversation: &[ConversationItem],
template: &str,
word_budget: usize,
extras: &[(&str, &str)],
) -> String {
let transcript = build_transcript(conversation);
let (transcript_text, _words) = truncate_middle_words(&transcript, word_budget);
let mut prompt = template.replace("{{transcript}}", &transcript_text);
for (needle, value) in extras {
prompt = prompt.replace(needle, value);
}
prompt
}
/// Convenience helper to complete text from a template + system string using the common pattern.
/// Returns only the model text (already trimmed).
pub async fn template_completion(
sampling_client: &OaiCompatClient,
system_text: &str,
conversation: &[ConversationItem],
template: &str,
word_budget: usize,
extras: &[(&str, &str)],
temperature: Option<f32>,
max_tokens: Option<u32>,
) -> Result<String> {
let prompt = build_prompt_from_template(conversation, template, word_budget, extras);
let system = ChatRequestMessage::system(system_text);
let user = ChatRequestMessage::user(prompt);
text_completion(sampling_client, system, user, temperature, max_tokens).await
}
@@ -0,0 +1,450 @@
//! Rendering helpers for [`CompactionStateContext`] that depend on
//! shell-specific types (`kigi_tools::MemoryBackend`, memory context).
//!
//! The core [`CompactionStateContext`] struct and its builder live in
//! `kigi_chat_state::compaction_utils`. This module adds system-reminder
//! rendering that requires dependencies not available in `kigi-chat-state`.
//!
//! The three **common** active-agent sections (background tasks, TODO list,
//! running subagents) are formatted by
//! [`kigi_compaction::reminder`] so grok-chat and grok-build stay in lockstep.
//! Harness-only sections (edited files, AGENTS.md, skills, MCP, memory) stay here.
use std::path::PathBuf;
pub use kigi_chat_state::compaction_utils::{
BackgroundTaskSummary, CompactionInputs, CompactionServerSummary, CompactionStateContext,
RunningSubagentSummary, TodoSummary, TodoSummaryStatus, extract_last_user_query,
extract_messages_since_last_user, extract_user_query,
};
use kigi_compaction::reminder::{
self, ActiveAgentReminderState, BackgroundTask, RunningSubagent, TodoItem, TodoStatus,
};
/// Resolved model-facing tool names for the MCP usage hint in compaction
/// reminders.
///
/// Resolved at runtime via `TemplateRenderer` from `ToolKind::SearchTool`
/// and `ToolKind::UseTool`. Never hard-code tool names -- they can be
/// renamed by the client.
pub struct McpToolNames {
/// Model-facing name of the search/discover tool (e.g. "search_tool").
pub search: String,
/// Model-facing name of the dispatch/call tool (e.g. "use_tool").
pub call: String,
}
/// Resolved model-facing tool names for the subagent reminder section.
///
/// Both names are resolved at runtime via `TemplateRenderer` from
/// `ToolKind::BackgroundTaskAction` and `ToolKind::KillTaskAction`.
/// Never hard-code tool names — they can be renamed by the client.
pub struct SubagentToolNames {
/// Model-facing name of the poll/status tool (e.g. "get_task_output").
pub poll: String,
/// Model-facing name of the cancel/kill tool (e.g. "kill_task").
pub cancel: String,
}
/// Format state info as system reminder, without memory search.
///
/// Use this from sync contexts (e.g., `build_compacted_history`) where
/// memory re-injection is handled separately by the session actor.
pub fn to_system_reminder_sync(
ctx: &CompactionStateContext,
discovered_agents_md: &[PathBuf],
skills: &[kigi_tools::implementations::skills::types::SkillInfo],
subagent_tool_names: Option<&SubagentToolNames>,
mcp_tool_names: Option<&McpToolNames>,
) -> Option<String> {
to_system_reminder_inner(
ctx,
discovered_agents_md,
skills,
&[],
subagent_tool_names,
mcp_tool_names,
)
}
/// Format state info as system reminder for injection into chat.
///
/// When a `memory_backend` is provided, searches memory for relevant
/// context from past sessions (post-compaction recovery).
pub async fn to_system_reminder(
ctx: &CompactionStateContext,
discovered_agents_md: &[PathBuf],
skills: &[kigi_tools::implementations::skills::types::SkillInfo],
memory_backend: Option<&dyn kigi_tools::types::memory_backend::MemoryBackend>,
subagent_tool_names: Option<&SubagentToolNames>,
mcp_tool_names: Option<&McpToolNames>,
) -> Option<String> {
// Fetch memory results first (async), then pass to sync inner method
let mut memory_results = Vec::new();
if let Some(memory) = memory_backend {
let query = ctx.last_user_query.as_deref().unwrap_or("project context");
if let Ok(results) = memory.search(query, 3, 0.0).await {
tracing::debug!(
target: kigi_log::memory_log::TARGET,
results = results.len(),
"recovered memory context after compaction"
);
memory_results = results;
}
}
to_system_reminder_inner(
ctx,
discovered_agents_md,
skills,
&memory_results,
subagent_tool_names,
mcp_tool_names,
)
}
/// Shared implementation for both sync and async variants.
fn to_system_reminder_inner(
ctx: &CompactionStateContext,
discovered_agents_md: &[PathBuf],
skills: &[kigi_tools::implementations::skills::types::SkillInfo],
memory_results: &[kigi_tools::types::memory_backend::MemorySearchResult],
subagent_tool_names: Option<&SubagentToolNames>,
mcp_tool_names: Option<&McpToolNames>,
) -> Option<String> {
let mut sections = Vec::new();
// Agent-edited files (shell-only)
if !ctx.agent_edited_paths.is_empty() {
let files = ctx
.agent_edited_paths
.iter()
.map(|f| format!("- {}", f))
.collect::<Vec<_>>()
.join("\n");
sections.push(format!(
"## Files Edited This Session\n\
These files were modified by you during this session:\n{}",
files
));
}
// Discovered AGENTS.md files (runtime, not in initial system prompt; shell-only)
if !discovered_agents_md.is_empty() {
let paths = discovered_agents_md
.iter()
.map(|p| format!("- {}", p.display()))
.collect::<Vec<_>>()
.join("\n");
sections.push(format!(
"## Discovered Project Instruction Files\n\
These project instruction files were found during the session \
and may contain relevant coding conventions:\n{}",
paths
));
}
// Available skills (startup + dynamically discovered, from SkillManager).
// Reuse the standard listing renderer so the post-compaction listing matches
// the startup `<system-reminder>` (no hard-coded tool name, includes
// `Use when:` triggers and `Absolute path:`).
if let Some(listing) =
kigi_tools::types::skill_discovery_tracker::format_compaction_skill_listing(skills)
{
sections.push(format!("## Available Skills\n{listing}"));
}
// Common sections (BG → TODO → subagents) via shared formatter. Borrow
// long fields from `ctx` rather than cloning them into an owned DTO.
let commands: Vec<_> = ctx
.running_tasks
.iter()
.map(|t| BackgroundTask {
task_id: &t.task_id,
command: &t.command,
status: &t.status,
tool_name: t.tool_name.as_deref(),
})
.collect();
let todos: Vec<_> = ctx
.todos
.iter()
.map(|t| TodoItem {
id: &t.id,
content: &t.content,
status: match t.status {
TodoSummaryStatus::Pending => TodoStatus::Pending,
TodoSummaryStatus::InProgress => TodoStatus::InProgress,
TodoSummaryStatus::Completed => TodoStatus::Completed,
TodoSummaryStatus::Cancelled => TodoStatus::Cancelled,
},
})
.collect();
let subagents: Vec<_> = ctx
.running_subagents
.iter()
.map(|s| RunningSubagent {
subagent_id: &s.subagent_id,
subagent_type: Some(&s.subagent_type),
description: Some(&s.description),
elapsed_secs: s.elapsed_ms / 1000,
})
.collect();
sections.extend(reminder::format_active_agent_sections(
&ActiveAgentReminderState {
running_commands: &commands,
todos: &todos,
running_subagents: &subagents,
},
subagent_tool_names
.map(|t| reminder::SubagentToolNames {
poll: &t.poll,
cancel: &t.cancel,
})
.as_ref(),
));
// Connected MCP servers (shell-only)
if !ctx.connected_mcp_servers.is_empty() {
use kigi_tools::implementations::search_tool::format_compaction_server_line;
let servers: String = ctx
.connected_mcp_servers
.iter()
.map(|s| format_compaction_server_line(&s.name, s.tool_count, &s.description))
.collect();
let hint = if let Some(names) = mcp_tool_names {
format!(
"\nTo use MCP tools, you MUST call `{}` first to retrieve the tool's input schema before calling `{}`. NEVER guess parameter names — always use the exact schema returned by `{}`.",
names.search, names.call, names.search
)
} else {
String::new()
};
sections.push(format!(
"## Connected MCP Servers\n{}{}",
servers.trim_end(),
hint
));
}
// Relevant memory from past sessions (post-compaction recovery; shell-only)
if !memory_results.is_empty()
&& let Some(reminder) = super::memory_context::format_memory_reminder(memory_results)
{
sections.push(reminder);
}
reminder::wrap_system_reminder(sections)
}
#[cfg(test)]
mod tests {
use super::*;
fn ctx_with_running_subagents() -> CompactionStateContext {
CompactionStateContext {
running_subagents: vec![RunningSubagentSummary {
subagent_id: "sub-1".into(),
subagent_type: "explore".into(),
description: "find files".into(),
elapsed_ms: 5000,
}],
recent_messages: vec![],
last_user_query: None,
agent_edited_paths: vec![],
running_tasks: vec![],
connected_mcp_servers: vec![],
todos: vec![],
}
}
#[test]
fn system_reminder_includes_subagent_section_when_tool_names_present() {
let ctx = ctx_with_running_subagents();
let names = SubagentToolNames {
poll: "get_command_or_subagent_output".into(),
cancel: "kill_command_or_subagent".into(),
};
let result = to_system_reminder_sync(&ctx, &[], &[], Some(&names), None);
let text = result.expect("should produce a reminder");
assert!(
text.contains("Running Subagents"),
"missing subagent section"
);
assert!(text.contains("get_command_or_subagent_output"));
assert!(text.contains("kill_command_or_subagent"));
assert!(text.contains("sub-1"));
}
#[test]
fn system_reminder_includes_mcp_server_section() {
let ctx = CompactionStateContext {
connected_mcp_servers: vec![
CompactionServerSummary {
name: "grafana".into(),
tool_count: 28,
description: Some("Observability platform".into()),
},
CompactionServerSummary {
name: "linear".into(),
tool_count: 12,
description: None,
},
],
recent_messages: vec![],
last_user_query: None,
agent_edited_paths: vec![],
running_tasks: vec![],
running_subagents: vec![],
todos: vec![],
};
let result = to_system_reminder_sync(&ctx, &[], &[], None, None);
let text = result.expect("should produce a reminder");
let expected = "\
<system-reminder>
## Connected MCP Servers
- grafana (28 tools): Observability platform
- linear (12 tools)
</system-reminder>";
assert_eq!(text, expected, "got:\n{text}");
}
/// Regression: task IDs in the post-compaction reminder must be rendered
/// verbatim. A fabricated `task-` prefix produces an ID that does not
/// exist in the task registry, so the model's follow-up
/// `get_task_output(task_id="task-<uuid>")` calls fail.
#[test]
fn running_task_ids_render_verbatim() {
let ctx = CompactionStateContext {
running_tasks: vec![BackgroundTaskSummary {
task_id: "019ea7f0-cb66-7aa2-9a09-488a3a795795".into(),
command: "cargo test".into(),
status: "running".into(),
tool_name: Some("run_terminal_command".into()),
}],
recent_messages: vec![],
last_user_query: None,
agent_edited_paths: vec![],
running_subagents: vec![],
connected_mcp_servers: vec![],
todos: vec![],
};
let text =
to_system_reminder_sync(&ctx, &[], &[], None, None).expect("should produce a reminder");
assert!(
text.contains("- \"019ea7f0-cb66-7aa2-9a09-488a3a795795\": `cargo test`"),
"task ID must be quoted verbatim: {text}"
);
assert!(
!text.contains("task-019ea7f0"),
"task ID must not be decorated with a task- prefix: {text}"
);
}
#[test]
fn system_reminder_skips_subagent_section_when_tool_names_none() {
let ctx = ctx_with_running_subagents();
let result = to_system_reminder_sync(&ctx, &[], &[], None, None);
if let Some(text) = result {
assert!(
!text.contains("Running Subagents"),
"subagent section should be omitted when tool names are None"
);
}
}
fn ctx_with_todos(todos: Vec<TodoSummary>) -> CompactionStateContext {
CompactionStateContext {
todos,
recent_messages: vec![],
last_user_query: None,
agent_edited_paths: vec![],
running_tasks: vec![],
running_subagents: vec![],
connected_mcp_servers: vec![],
}
}
fn todo(id: &str, status: TodoSummaryStatus, content: &str) -> TodoSummary {
TodoSummary {
id: id.into(),
content: content.into(),
status,
}
}
/// Active todos are re-surfaced post-compaction: pending/in_progress items
/// render verbatim with id + status; completed/cancelled collapse to counts.
#[test]
fn system_reminder_includes_active_todos() {
let ctx = ctx_with_todos(vec![
todo("1", TodoSummaryStatus::InProgress, "wire up auth"),
todo("2", TodoSummaryStatus::Pending, "add tests"),
todo("3", TodoSummaryStatus::Completed, "read the code"),
todo("4", TodoSummaryStatus::Cancelled, "abandoned idea"),
]);
let text =
to_system_reminder_sync(&ctx, &[], &[], None, None).expect("should produce a reminder");
assert!(
text.contains("## TODO List"),
"missing TODO section: {text}"
);
assert!(
text.contains("- [in_progress] 1: wire up auth"),
"got:\n{text}"
);
assert!(text.contains("- [pending] 2: add tests"), "got:\n{text}");
// Done/cancelled items are summarized, not listed verbatim.
assert!(text.contains("(1 completed, 1 cancelled)"), "got:\n{text}");
assert!(
!text.contains("read the code"),
"completed item must not be listed verbatim: {text}"
);
assert!(
!text.contains("abandoned idea"),
"cancelled item must not be listed verbatim: {text}"
);
}
/// The TODO List section is rendered directly below Running Background Tasks.
#[test]
fn system_reminder_places_todos_below_background_tasks() {
let mut ctx = ctx_with_todos(vec![todo(
"1",
TodoSummaryStatus::InProgress,
"wire up auth",
)]);
ctx.running_tasks = vec![BackgroundTaskSummary {
task_id: "t1".into(),
command: "cargo test".into(),
status: "running".into(),
tool_name: Some("run_terminal_command".into()),
}];
let text =
to_system_reminder_sync(&ctx, &[], &[], None, None).expect("should produce a reminder");
let tasks_pos = text
.find("## Running Background Tasks")
.expect("tasks section");
let todo_pos = text.find("## TODO List").expect("todo section");
assert!(
tasks_pos < todo_pos,
"TODO List must appear below Running Background Tasks:\n{text}"
);
}
/// No actionable items (all completed/cancelled) → no TODO section.
#[test]
fn system_reminder_omits_todos_when_none_active() {
let ctx = ctx_with_todos(vec![
todo("1", TodoSummaryStatus::Completed, "done"),
todo("2", TodoSummaryStatus::Cancelled, "scrapped"),
]);
let result = to_system_reminder_sync(&ctx, &[], &[], None, None);
if let Some(text) = result {
assert!(
!text.contains("## TODO List"),
"TODO section should be omitted when nothing is active: {text}"
);
}
}
}
@@ -0,0 +1,399 @@
//! grok-build's L5 wiring onto the shared full-replace engine
//! (`kigi_compaction::code_compaction`).
//!
//! The shared engine drives the sample → retry → degenerate/failure
//! classification loop via [`sample_full_replace_summary`](kigi_compaction::sample_full_replace_summary);
//! this module adapts grok-build's transport and telemetry to its two seams:
//!
//! - [`ShellCompactionSampler`] wraps
//! [`generate_session_compact`](crate::session::helpers::session_compact::generate_session_compact)
//! as the shared [`CompactionSampler`]. It also stashes the full
//! [`CompactOutput`] of the last successful call so the L5 loop can still
//! record the streaming telemetry (TTFT / stream span / stop reason) that
//! the shared [`LlmCompactionOutput`] doesn't model.
//! - [`ShellFullReplaceObserver`] collects the per-attempt
//! [`CompactionAttempt`] rows, rejection counters, and emits the
//! `CompactionRetryDegraded` event — preserving the pre-migration telemetry.
//!
//! The verbatim → fitted → lossy **input ladder** and auto-compaction
//! suppression stay in L5 (`compaction.rs`), driven by the
//! `context_overflow` / `deterministic` flags on
//! [`FullReplaceError`](kigi_compaction::FullReplaceError).
use std::sync::Mutex;
use std::time::Duration;
use agent_client_protocol as acp;
use async_trait::async_trait;
use kigi_compaction::{
CompactionPrompt, CompactionSampleError, CompactionSampler, FullReplaceAttemptOutcome,
FullReplaceObserver, LlmCompactionOutput,
};
use kigi_sampler::SamplerConfig as SamplingConfig;
use kigi_sampling_types::{ConversationItem, HostedTool, ToolSpec};
use kigi_chat_state::compaction_utils::{
CompactionAttempt, MAX_CAPTURED_SUMMARY_CHARS, bound_captured_output,
};
use crate::sampling::Client as OaiCompatClient;
use crate::session::helpers::session_compact::{
CompactFailure, CompactOutput, build_compaction_chat_history, generate_session_compact,
};
/// Wraps `generate_session_compact` as the shared engine's
/// [`CompactionSampler`] for grok-build's full-replace pass.
///
/// Holds the per-call request context the seam does not carry (tools, client,
/// session, config) and stashes the last successful [`CompactOutput`] so the
/// caller can recover the streaming telemetry not modeled by
/// [`LlmCompactionOutput`].
///
/// The summarization prompt is selected here by `use_short_prompt` (the
/// short-prompt harness uses the short self-summarization prompt; everyone
/// else the structured grok-build prompt), so the shared `CompactionPrompt`
/// the engine passes is ignored — the engine builds the grok-build prompt,
/// which equals what `build_compaction_chat_history(.., false)` appends, and
/// the short-prompt harness needs its own variant the engine can't produce.
pub(crate) struct ShellCompactionSampler {
use_short_prompt: bool,
user_context: Option<String>,
tools: Vec<ToolSpec>,
hosted_tools: Vec<HostedTool>,
client: OaiCompatClient,
session_id: acp::SessionId,
sampling_config: SamplingConfig,
/// Per-chunk idle timeout forwarded to `generate_session_compact`: a stalled
/// summarizer stream (no model-output chunk for this long) fails instead of
/// hanging.
idle_timeout: Duration,
/// Wall-clock budget (secs) forwarded to `generate_session_compact` as the
/// reasoning-runaway backstop; `0` disables it.
wall_clock_budget_secs: u64,
/// Full output of the most recent successful sample (for L5 telemetry).
last_success: Mutex<Option<CompactOutput>>,
}
impl ShellCompactionSampler {
#[allow(clippy::too_many_arguments)]
pub(crate) fn new(
use_short_prompt: bool,
user_context: Option<String>,
tools: Vec<ToolSpec>,
hosted_tools: Vec<HostedTool>,
client: OaiCompatClient,
session_id: acp::SessionId,
sampling_config: SamplingConfig,
idle_timeout: Duration,
wall_clock_budget_secs: u64,
) -> Self {
Self {
use_short_prompt,
user_context,
tools,
hosted_tools,
client,
session_id,
sampling_config,
idle_timeout,
wall_clock_budget_secs,
last_success: Mutex::new(None),
}
}
/// Take the [`CompactOutput`] of the most recent successful sample, if any.
pub(crate) fn take_last_success(&self) -> Option<CompactOutput> {
self.last_success.lock().unwrap().take()
}
}
#[async_trait]
impl CompactionSampler for ShellCompactionSampler {
type Item = ConversationItem;
async fn sample_compaction(
&self,
turns: &[ConversationItem],
_prompt: &CompactionPrompt,
_timeout: Duration,
) -> Result<LlmCompactionOutput, CompactionSampleError> {
// Append the harness-selected summarization prompt as the final user
// message (compat short vs structured grok-build), ignoring the shared
// engine's `_prompt` (see the struct doc).
let chat_history = build_compaction_chat_history(
turns.to_vec(),
self.user_context.as_deref(),
self.use_short_prompt,
);
match generate_session_compact(
chat_history,
self.tools.clone(),
self.hosted_tools.clone(),
self.client.clone(),
self.session_id.clone(),
&self.sampling_config,
self.idle_timeout,
self.wall_clock_budget_secs,
)
.await
{
Ok(output) => {
let response = output.content.clone();
*self.last_success.lock().unwrap() = Some(output);
Ok(LlmCompactionOutput {
response,
thinking: String::new(),
})
}
Err(failure) => Err(compact_failure_to_sample_error(failure)),
}
}
}
/// Map grok-build's [`CompactFailure`] onto the shared engine's
/// [`CompactionSampleError`] so the shared retry loop classifies it the same
/// way the in-shell loop did:
///
/// - `Deterministic` → [`CompactionSampleError::Build`] (whose
/// `is_deterministic()` is `true`); a context-length overflow keeps its
/// message text so the engine's `is_context_length_error` check fires and
/// sets `context_overflow`.
/// - `Transient` → [`CompactionSampleError::Other`] (`is_deterministic()` is
/// `false`), so the engine retries it.
fn compact_failure_to_sample_error(failure: CompactFailure) -> CompactionSampleError {
let (deterministic, err) = match failure {
CompactFailure::Deterministic(err) => (true, err),
CompactFailure::Transient(err) => (false, err),
};
let message = acp_error_message(&err);
if deterministic {
CompactionSampleError::Build(message)
} else {
CompactionSampleError::Other(anyhow::anyhow!(message))
}
}
/// Render the human-readable detail an `acp::Error` carries in its `data`
/// field (where `classify_*` stash `"compact failed: <upstream>"`).
fn acp_error_message(err: &acp::Error) -> String {
err.data
.as_ref()
.and_then(|d| d.as_str())
.unwrap_or("<no data>")
.to_string()
}
/// Collected telemetry from a full-replace pass, drained by the L5 loop after
/// the shared engine returns.
pub(crate) struct FullReplaceTelemetry {
pub attempts: u32,
pub attempt_details: Vec<CompactionAttempt>,
pub degenerate_rejections: u32,
pub transient_rejections: u32,
pub deterministic_rejections: u32,
/// Raw text of the last degenerate (rejected) summary, for the artifact.
pub last_rejected_summary: Option<String>,
}
#[derive(Default)]
struct ObserverState {
attempts: u32,
attempt_details: Vec<CompactionAttempt>,
degenerate_rejections: u32,
transient_rejections: u32,
deterministic_rejections: u32,
last_rejected_summary: Option<String>,
last_error_msg: Option<String>,
}
/// [`FullReplaceObserver`] that reproduces grok-build's per-attempt telemetry:
/// `CompactionAttempt` rows, rejection counters, the `CompactionRetryDegraded`
/// event, and the warn/error tracing — without the shared engine depending on
/// a telemetry backend.
pub(crate) struct ShellFullReplaceObserver {
session_id: String,
estimated_input_tokens: u64,
retry_delay_secs: u64,
state: Mutex<ObserverState>,
}
impl ShellFullReplaceObserver {
pub(crate) fn new(
session_id: String,
estimated_input_tokens: u64,
retry_delay_secs: u64,
) -> Self {
Self {
session_id,
estimated_input_tokens,
retry_delay_secs,
state: Mutex::new(ObserverState::default()),
}
}
/// Cumulative number of attempts so far (across all input-ladder stages).
/// Read mid-loop to label the `input_overflow` retry event.
pub(crate) fn attempt_count(&self) -> u32 {
self.state.lock().unwrap().attempts
}
/// Whether any attempt so far produced a degenerate summary — lets the L5
/// loop distinguish degenerate-exhausted from empty-exhausted.
pub(crate) fn degenerate_seen(&self) -> bool {
self.state.lock().unwrap().degenerate_rejections > 0
}
/// The most recent rendered error/diagnostic detail, for `last_error`.
pub(crate) fn last_error_message(&self) -> Option<String> {
self.state.lock().unwrap().last_error_msg.clone()
}
/// Drain the collected telemetry. The cumulative attempt count spans all
/// input-ladder stages because the same observer instance is shared across
/// every per-stage call.
pub(crate) fn into_telemetry(self) -> FullReplaceTelemetry {
let s = self.state.into_inner().unwrap();
FullReplaceTelemetry {
attempts: s.attempts,
attempt_details: s.attempt_details,
degenerate_rejections: s.degenerate_rejections,
transient_rejections: s.transient_rejections,
deterministic_rejections: s.deterministic_rejections,
last_rejected_summary: s.last_rejected_summary,
}
}
}
impl FullReplaceObserver for ShellFullReplaceObserver {
fn on_attempt(&self, _attempt: u32, outcome: &FullReplaceAttemptOutcome<'_>) {
let mut s = self.state.lock().unwrap();
// The shared `attempt` resets per ladder stage; keep a cumulative count
// so artifact rows match the pre-migration numbering.
s.attempts += 1;
let attempt = s.attempts;
match outcome {
FullReplaceAttemptOutcome::Success { summary } => {
s.attempt_details.push(CompactionAttempt {
attempt,
outcome: "success".to_string(),
summary_chars: summary.chars().count() as u64,
summary: None,
error: None,
});
}
FullReplaceAttemptOutcome::Degenerate {
summary,
will_retry,
} => {
s.degenerate_rejections += 1;
let summary_chars = summary.chars().count();
s.attempt_details.push(CompactionAttempt {
attempt,
outcome: "degenerate".to_string(),
summary_chars: summary_chars as u64,
summary: Some(bound_captured_output(summary, MAX_CAPTURED_SUMMARY_CHARS)),
error: None,
});
s.last_rejected_summary = Some((*summary).to_string());
s.last_error_msg = Some(format!(
"compact failed: degenerate summary \
({summary_chars} chars for ~{} input tokens)",
self.estimated_input_tokens
));
if *will_retry {
tracing::warn!(
session_id = %self.session_id,
attempt,
summary_chars,
estimated_input_tokens = self.estimated_input_tokens,
retry_delay_secs = self.retry_delay_secs,
"Compaction produced a degenerate summary, retrying in {} seconds...",
self.retry_delay_secs
);
} else {
tracing::error!(
session_id = %self.session_id,
attempt,
summary_chars,
estimated_input_tokens = self.estimated_input_tokens,
"Compaction produced only degenerate summaries after max retries"
);
}
}
FullReplaceAttemptOutcome::EmptyResponse { .. } => {
// The shell surfaces an empty response as a transient error
// (`generate_session_compact` returns `Transient`), so it never
// reaches the shared `Ok("")` branch; handle defensively.
s.transient_rejections += 1;
let msg = "compact failed: model returned empty response".to_string();
s.attempt_details.push(CompactionAttempt {
attempt,
outcome: "transient".to_string(),
summary_chars: 0,
summary: None,
error: Some(msg.clone()),
});
s.last_error_msg = Some(msg);
}
FullReplaceAttemptOutcome::Failure {
message,
deterministic,
context_overflow,
will_retry,
} => {
// A context overflow is recorded as a `deterministic` attempt
// (matching the pre-migration row) but does NOT count toward
// `deterministic_rejections` — the L5 ladder steps down on it
// and tracks its own `input_overflow_rejections`.
if *deterministic {
if !*context_overflow {
s.deterministic_rejections += 1;
tracing::error!(
session_id = %self.session_id,
attempt,
error = %message,
"Compaction failed (deterministic error class, no further retries)"
);
}
s.attempt_details.push(CompactionAttempt {
attempt,
outcome: "deterministic".to_string(),
summary_chars: 0,
summary: None,
error: Some((*message).to_string()),
});
} else {
s.transient_rejections += 1;
s.attempt_details.push(CompactionAttempt {
attempt,
outcome: "transient".to_string(),
summary_chars: 0,
summary: None,
error: Some((*message).to_string()),
});
if *will_retry {
tracing::warn!(
session_id = %self.session_id,
attempt,
retry_delay_secs = self.retry_delay_secs,
error = %message,
"Compaction attempt {} failed, retrying in {} seconds...",
attempt,
self.retry_delay_secs
);
} else {
tracing::error!(
session_id = %self.session_id,
attempt,
error = %message,
"Compaction failed after max retries"
);
}
}
s.last_error_msg = Some((*message).to_string());
}
}
}
}
@@ -0,0 +1,357 @@
//! Format memory search results as `<system-reminder>` content.
//!
//! Used for:
//! - Session start: inject relevant past context on the first turn
//! - Post-compaction: recover relevant memory after context is lost
use kigi_chat_state::{MEMORY_CONTEXT_CLOSE_TAG, MEMORY_CONTEXT_OPEN_TAG};
use kigi_sampling_types::ConversationItem;
use kigi_tools::types::memory_backend::{MemorySearchResult, format_staleness_note};
/// Maximum characters to include per snippet in the injection.
const SNIPPET_MAX_CHARS: usize = 500;
/// Returns `true` if a memory-context block is already persisted in the
/// leading system message. Callers reuse a persisted block verbatim instead
/// of re-searching: a re-scored block would mutate the system-prompt prefix
/// and bust the KV cache for the whole downstream conversation.
pub fn conversation_has_memory_context(items: &[ConversationItem]) -> bool {
matches!(
items.first(),
Some(ConversationItem::System(sys)) if sys.content.contains(MEMORY_CONTEXT_OPEN_TAG)
)
}
/// Format memory search results as a markdown section for system-reminder injection.
///
/// Each result is formatted with score, source, file path, line range,
/// and the snippet in a fenced code block (preserving newlines/markdown).
/// This matches the output format of the `memory_search` tool for consistency.
///
/// Returns `None` if results are empty.
pub fn format_memory_reminder(results: &[MemorySearchResult]) -> Option<String> {
if results.is_empty() {
return None;
}
let mut section =
format!("{MEMORY_CONTEXT_OPEN_TAG}\n## Relevant Memory from Past Sessions\n\n");
for (i, r) in results.iter().enumerate() {
let truncated = r.snippet.chars().count() > SNIPPET_MAX_CHARS;
let mut snippet: String = r.snippet.chars().take(SNIPPET_MAX_CHARS).collect();
if truncated {
snippet.push_str("...");
}
let staleness = format_staleness_note(&r.source, r.created_at);
section.push_str(&format!(
"### Result {} (score: {:.2}, source: {})\n\
**File:** {} (lines {}-{})\n\
{}```\n{}\n```\n\n",
i + 1,
r.score,
r.source,
r.path,
r.start_line,
r.end_line,
staleness,
snippet,
));
}
section.push_str(MEMORY_CONTEXT_CLOSE_TAG);
Some(section)
}
/// Check if a message looks like a greeting or generic opener.
///
/// Used to detect vague first messages that won't produce useful memory
/// search results, so we can fall back to a broader project-context query.
pub fn is_greeting(text: &str) -> bool {
const GREETINGS: &[&str] = &[
"hi",
"hey",
"hello",
"howdy",
"continue",
"start",
"begin",
"go",
"good morning",
"good afternoon",
"good evening",
"what's up",
"whats up",
"sup",
];
let lowered = text.to_lowercase();
let trimmed = lowered.trim().trim_end_matches(['.', '!', '?', ',']);
GREETINGS.contains(&trimmed)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_format_empty() {
assert_eq!(format_memory_reminder(&[]), None);
}
#[test]
fn test_format_single_result() {
let results = vec![MemorySearchResult {
chunk_id: "test:0".to_string(),
path: "MEMORY.md".to_string(),
start_line: 0,
end_line: 5,
score: 0.9,
snippet: "Use tracing for logging, never println!".to_string(),
source: "workspace".to_string(),
created_at: None,
}];
let output = format_memory_reminder(&results).unwrap();
assert!(output.contains("<memory-context>"));
assert!(output.contains("### Result 1"));
assert!(output.contains("score: 0.90"));
assert!(output.contains("**File:** MEMORY.md (lines 0-5)"));
assert!(output.contains("```\nUse tracing for logging"));
}
#[test]
fn test_format_preserves_newlines() {
let results = vec![MemorySearchResult {
chunk_id: "test:0".to_string(),
path: "MEMORY.md".to_string(),
start_line: 0,
end_line: 3,
score: 0.85,
snippet: "## Conventions\n\n- Use Rust\n- No clones".to_string(),
source: "workspace".to_string(),
created_at: None,
}];
let output = format_memory_reminder(&results).unwrap();
assert!(
output.contains("## Conventions\n\n- Use Rust\n- No clones"),
"newlines in snippet should be preserved, not collapsed"
);
}
#[test]
fn test_format_truncates_long_snippets() {
let results = vec![MemorySearchResult {
chunk_id: "test:0".to_string(),
path: "test.md".to_string(),
start_line: 0,
end_line: 5,
score: 0.8,
snippet: "x".repeat(1000),
source: "session".to_string(),
created_at: None,
}];
let output = format_memory_reminder(&results).unwrap();
// Snippet should be truncated to SNIPPET_MAX_CHARS (500) + "..."
assert!(!output.contains(&"x".repeat(501)));
assert!(output.contains(&format!("{}...", "x".repeat(500))));
}
#[test]
fn test_format_multiple_results() {
let results = vec![
MemorySearchResult {
chunk_id: "a:0".to_string(),
path: "MEMORY.md".to_string(),
start_line: 0,
end_line: 5,
score: 0.9,
snippet: "First result".to_string(),
source: "workspace".to_string(),
created_at: None,
},
MemorySearchResult {
chunk_id: "b:0".to_string(),
path: "session.md".to_string(),
start_line: 10,
end_line: 15,
score: 0.7,
snippet: "Second result".to_string(),
source: "session".to_string(),
created_at: None,
},
];
let output = format_memory_reminder(&results).unwrap();
assert!(output.contains("### Result 1"));
assert!(output.contains("### Result 2"));
assert!(output.contains("score: 0.90"));
assert!(output.contains("score: 0.70"));
}
// -----------------------------------------------------------------------
// conversation_has_memory_context (idempotency guard) tests
// -----------------------------------------------------------------------
fn sample_result() -> MemorySearchResult {
MemorySearchResult {
chunk_id: "test:0".into(),
path: "MEMORY.md".into(),
start_line: 0,
end_line: 5,
score: 0.9,
snippet: "Project uses Rust for backend services.".into(),
source: "workspace".into(),
created_at: None,
}
}
#[test]
fn test_detects_persisted_block_in_system_message() {
let block = format_memory_reminder(&[sample_result()]).unwrap();
let system_content = format!("You are a helpful assistant.\n\n{block}");
let conversation = vec![
ConversationItem::system(system_content),
ConversationItem::user("help me fix the auth bug"),
];
assert!(
conversation_has_memory_context(&conversation),
"an already-injected memory-context block must be detected so it is reused, not re-searched"
);
}
#[test]
fn test_no_block_when_system_lacks_marker() {
let conversation = vec![
ConversationItem::system("You are a helpful assistant."),
ConversationItem::user("hi"),
];
assert!(!conversation_has_memory_context(&conversation));
}
#[test]
fn test_no_block_when_no_leading_system_message() {
let conversation = vec![ConversationItem::user("hi")];
assert!(!conversation_has_memory_context(&conversation));
}
#[test]
fn test_no_block_for_empty_conversation() {
assert!(!conversation_has_memory_context(&[]));
}
// -----------------------------------------------------------------------
// staleness annotation tests
// -----------------------------------------------------------------------
#[test]
fn test_staleness_shown_for_old_session_result() {
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs() as i64;
let results = vec![MemorySearchResult {
chunk_id: "s:0".into(),
path: "session.md".into(),
start_line: 0,
end_line: 5,
score: 0.8,
snippet: "old info".into(),
source: "session".into(),
created_at: Some(now - 86400 * 10),
}];
let output = format_memory_reminder(&results).unwrap();
assert!(
output.contains("**Stale ("),
"10-day-old session result should show stale warning, got: {output}"
);
}
#[test]
fn test_no_staleness_for_workspace_result() {
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs() as i64;
let results = vec![MemorySearchResult {
chunk_id: "w:0".into(),
path: "MEMORY.md".into(),
start_line: 0,
end_line: 5,
score: 0.9,
snippet: "workspace data".into(),
source: "workspace".into(),
created_at: Some(now - 86400 * 30),
}];
let output = format_memory_reminder(&results).unwrap();
assert!(
!output.contains("**Stale (") && !output.contains("**Note ("),
"workspace result must not show staleness, got: {output}"
);
}
// -----------------------------------------------------------------------
// is_greeting tests
// -----------------------------------------------------------------------
#[test]
fn test_greeting_detection() {
assert!(is_greeting("hi"));
assert!(is_greeting("Hey!"));
assert!(is_greeting("Hello."));
assert!(is_greeting("good morning"));
assert!(is_greeting("continue"));
assert!(is_greeting(" HELLO "));
}
#[test]
fn test_non_greeting() {
assert!(!is_greeting("help me fix the auth bug"));
assert!(!is_greeting("implement feature X"));
assert!(!is_greeting("what does this function do"));
assert!(!is_greeting("hi there, can you help me with something"));
}
// -----------------------------------------------------------------------
// Injection counter semantics tests
// -----------------------------------------------------------------------
/// `format_memory_reminder` returns `None` for an empty result list.
///
/// This is the key invariant for the `memory_injection_count` contract:
/// the counter must only be incremented when `memory_reminder.is_some()`,
/// which is only true when `format_memory_reminder` returns `Some(_)`.
/// An empty result set must produce `None`, preventing the counter from
/// overcounting attempts where memory search found nothing to inject.
#[test]
fn test_format_memory_reminder_empty_results_is_none() {
use kigi_tools::types::memory_backend::MemorySearchResult;
let results: Vec<MemorySearchResult> = vec![];
let reminder = format_memory_reminder(&results);
assert!(
reminder.is_none(),
"empty results must produce None — injection_count must NOT increment"
);
}
/// `format_memory_reminder` returns `Some(_)` for a non-empty result list.
///
/// Confirms that `memory_injection_count` correctly increments when there
/// are actual results to inject.
#[test]
fn test_format_memory_reminder_with_results_is_some() {
use kigi_tools::types::memory_backend::MemorySearchResult;
let results = vec![MemorySearchResult {
chunk_id: "test:0".into(),
path: "/mem/MEMORY.md".into(),
start_line: 0,
end_line: 3,
score: 0.85,
snippet: "Project uses Rust for backend services.".into(),
source: "workspace".into(),
created_at: None,
}];
let reminder = format_memory_reminder(&results);
assert!(
reminder.is_some(),
"non-empty results must produce Some(_) — injection_count SHOULD increment"
);
}
}
@@ -0,0 +1,739 @@
//! Pre-compaction memory flush logic.
//!
//! Before compacting the conversation, the session actor can optionally run
//! a "flush turn" that asks the model to summarize important information
//! for storage in memory files.
//!
//! This module provides:
//! - `should_flush()` — threshold check for when to trigger the flush
//! - `FLUSH_SYSTEM_PROMPT` — the prompt sent to the model during flush
//! - `process_flush_response()` — quality controls on the model's output
//! - `is_semantically_duplicate()` — embedding-based dedup gate before writing
//!
//! The session actor orchestrates the flush by:
//! 1. Setting `is_flushing = true` (suppresses auto-compact)
//! 2. Sending `FLUSH_SYSTEM_PROMPT` to the model (no tools offered)
//! 3. Calling `process_flush_response()` on the result
//! 4. Calling `is_semantically_duplicate()` to skip near-duplicate content
//! 5. Writing to `MemoryStorage::write_daily_log()` if accepted and not duplicate
//! 6. Setting `is_flushing = false`
use crate::config::MemoryFlushConfig;
use crate::sampling::{ChatRequestMessage, Role};
// Pure text helpers moved into the memory subsystem (breaks the
// dream <-> memory_flush module cycle).
use crate::session::memory::text_utils::{has_markdown_headers, is_no_reply};
/// Memory log target — matches `kigi_log::memory_log::TARGET`.
const LOG: &str = "xai_memory";
/// Check whether a memory flush should run before the next compaction.
///
/// Returns `true` when ALL of the following are met:
/// - `flush_config.enabled` is `true`
/// - This flush hasn't already run for the current compaction cycle
/// (`last_flush_compaction != current_compaction_count`)
/// - Token usage has reached the flush threshold (compact threshold
/// minus `soft_threshold_tokens` headroom)
///
/// The flush threshold sits below the compact threshold so the flush
/// completes before the context window overflows.
pub fn should_flush(
total_tokens: u64,
context_window: u64,
compact_threshold_percent: u8,
flush_config: &MemoryFlushConfig,
last_flush_compaction: u64,
current_compaction_count: u64,
) -> bool {
if !flush_config.enabled {
tracing::debug!(target: LOG, "MEMORY_FLUSH_CHECK: disabled");
return false;
}
if last_flush_compaction == current_compaction_count {
tracing::debug!(target: LOG,
"MEMORY_FLUSH_CHECK: already flushed this cycle (cycle={current_compaction_count})");
return false;
}
let should = kigi_token_estimation::exceeds_threshold_with_headroom(
total_tokens,
context_window,
compact_threshold_percent,
flush_config.soft_threshold_tokens,
);
// Approximate threshold for log readability; the decision uses scaled
// arithmetic above and may differ by 1 token at non-round windows.
let flush_threshold = context_window
.saturating_mul(compact_threshold_percent as u64)
.saturating_sub(flush_config.soft_threshold_tokens.saturating_mul(100))
/ 100;
tracing::info!(target: LOG,
"MEMORY_FLUSH_CHECK: tokens={total_tokens} threshold={flush_threshold} \
window={context_window} pct={compact_threshold_percent} soft={soft} -> {result}",
soft = flush_config.soft_threshold_tokens,
result = if should { "FLUSH" } else { "skip" },
);
should
}
// ---------------------------------------------------------------------------
// Flush prompt and response processing
// ---------------------------------------------------------------------------
/// System prompt injected for the flush model call.
pub const FLUSH_SYSTEM_PROMPT: &str = "\
You are a memory assistant. Extract ALL useful information from this conversation \
that would help you be more effective in future sessions with this user. \
Write a concise markdown summary with ## headers covering:
- **Decisions & rationale** — what was chosen and why
- **Technical context** — architecture, APIs, patterns, tools, file paths discussed
- **Debugging techniques & tools** — external APIs, CLI commands, query patterns, \
investigation workflows, or services discovered or used during debugging
- **Problems & solutions** — bugs found, how they were fixed, workarounds
Omit any section where there is nothing substantive to report. \
Do NOT include user preferences like OS, shell, or editor — these belong in global memory. \
Do NOT include an ephemeral progress section — transient status is not useful for future sessions.
Respond with NO_REPLY if nothing genuinely useful was learned — a routine task \
that followed standard patterns, brief Q&A, or sessions with no novel decisions \
or discoveries are not worth persisting. Only write content that a future session \
would concretely benefit from.";
/// System prompt for incremental (delta) flushes after the first flush.
///
/// Used when `flush_count > 0` and previous flush content is available.
/// The caller appends the previous flush output after this prompt.
pub const FLUSH_DELTA_SYSTEM_PROMPT: &str = "\
You are a memory assistant performing an incremental update. The previous \
flush output for this session is shown below. Extract ONLY information that \
is NEW since the previous flush — do not repeat anything already captured.
Write a concise markdown summary with ## headers covering only NEW items in:
- **Decisions & rationale** — new decisions since last flush
- **Technical context** — new architecture, APIs, patterns discovered
- **Debugging techniques** — new techniques used since last flush
- **Problems & solutions** — new bugs found and fixes
Omit any section that has no new content. Do NOT include user preferences \
(OS, shell, paths) — these are captured in global memory.
Do NOT include 'Current state' — this is ephemeral and not useful for future sessions.
Respond with NO_REPLY if nothing genuinely new and useful has happened since \
the previous flush. Routine changes that follow standard patterns are not worth \
an incremental update.
--- Previous flush content ---
";
/// Result of processing the model's flush response.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum FlushResult {
/// Model indicated nothing to store (empty response or NO_REPLY).
NothingToStore,
/// Response was accepted after quality checks. Contains the content to write.
/// The caller should run [`is_semantically_duplicate()`] before writing.
Accepted(String),
/// Response was rejected by quality controls.
Rejected(String),
}
/// Process the model's flush response, applying quality controls.
///
/// Quality checks:
/// 1. Empty/whitespace-only → `NothingToStore`
/// 2. Matches `NO_REPLY` pattern → `NothingToStore`
/// 3. Exceeds `max_flush_write_chars` → truncated
/// 4. Must contain at least one markdown header (`##`) → `Rejected` if not
pub fn process_flush_response(response: &str, config: &MemoryFlushConfig) -> FlushResult {
let trimmed = response.trim();
let len = trimmed.len();
let preview: String = trimmed.chars().take(200).collect();
tracing::info!(target: LOG,
"MEMORY_FLUSH_RESPONSE: len={len} preview=\"{preview}\"");
// Check for empty
if trimmed.is_empty() {
tracing::info!(target: LOG,
"MEMORY_FLUSH_RESPONSE: empty → NothingToStore");
return FlushResult::NothingToStore;
}
// Check for NO_REPLY
if is_no_reply(trimmed) {
tracing::info!(target: LOG,
"MEMORY_FLUSH_RESPONSE: matches NO_REPLY pattern → NothingToStore");
return FlushResult::NothingToStore;
}
// Truncate if too long (use char count for consistency with .chars().take())
let content = if trimmed.chars().count() > config.max_flush_write_chars {
tracing::warn!(target: LOG,
"MEMORY_FLUSH_RESPONSE: truncated from {len} to {} chars",
config.max_flush_write_chars);
trimmed
.chars()
.take(config.max_flush_write_chars)
.collect::<String>()
} else {
trimmed.to_string()
};
// Must contain at least one markdown header for structure
if !has_markdown_headers(&content) {
tracing::info!(target: LOG,
"MEMORY_FLUSH_RESPONSE: no markdown headers → Rejected");
return FlushResult::Rejected(
"flush response lacks markdown structure (no ## headers)".to_string(),
);
}
tracing::info!(target: LOG,
"MEMORY_FLUSH_RESPONSE: accepted ({} chars, has headers)", content.len());
FlushResult::Accepted(content)
}
/// Check if content is substantially similar to any existing memory chunk.
///
/// Returns `true` if an exact blake3 hash match is found (should skip write).
/// Uses open-per-query to avoid `!Send` issues with `rusqlite::Connection`.
pub fn is_duplicate(content: &str, db_path: &std::path::Path) -> bool {
let content_hash = blake3::hash(content.as_bytes()).to_hex().to_string();
// Journal-mode-aware open: never mmap a legacy WAL -shm on network
// mounts (SIGBUS); see kigi_sqlite_journal::JournalMode::open_readonly.
let conn = match kigi_sqlite_journal::JournalMode::for_db_path(db_path).open_readonly(db_path) {
Ok(c) => c,
Err(_) => {
tracing::debug!(target: LOG, "MEMORY_FLUSH_DEDUP: can't open DB, allowing write");
return false;
}
};
let exact_match: bool = conn
.query_row(
"SELECT EXISTS(SELECT 1 FROM chunks WHERE hash = ?1)",
rusqlite::params![content_hash],
|r| r.get(0),
)
.unwrap_or(false);
tracing::info!(target: LOG,
"MEMORY_FLUSH_DEDUP: hash={hash} duplicate={exact_match}",
hash = &content_hash[..12],
);
exact_match
}
/// Cosine similarity threshold above which flush content is considered a
/// semantic duplicate of an existing memory chunk. A value of 0.92 is
/// conservative — it catches near-identical rephrasings while allowing
/// content that adds meaningful new information to pass through.
///
/// Used as the fallback when no config override is set.
pub(crate) const SEMANTIC_DEDUP_SIMILARITY_THRESHOLD: f64 = 0.92;
/// Maximum L2 distance between two unit-norm embedding vectors (used to
/// convert sqlite-vec L2 distances to cosine similarity).
const MAX_L2_DISTANCE: f64 = 2.0;
/// Number of nearest neighbors to check during semantic dedup.
const SEMANTIC_DEDUP_KNN_LIMIT: usize = 3;
/// Check if flush content is semantically similar to existing memory chunks.
///
/// Uses the embedding provider to embed the flush content, then runs a KNN
/// search against the memory index. If any result exceeds `threshold`,
/// considers the content a duplicate.
///
/// `threshold` is the cosine similarity cutoff (0.01.0). Pass
/// `SEMANTIC_DEDUP_SIMILARITY_THRESHOLD` for the compiled-in default, or
/// a value from config for remote/local overrides.
///
/// Falls back gracefully: returns `false` (allow write) if embeddings are
/// unavailable, the index has no vector support, or any step fails.
///
/// Structured as sync/async/sync phases so `&MemoryIndex` (which contains
/// `!Send` `rusqlite::Connection`) is never held across `.await` boundaries,
/// matching the pattern used in `search.rs` and `backend.rs`.
pub async fn is_semantically_duplicate(
content: &str,
index: &crate::session::memory::MemoryIndex,
embedding_provider: Option<&dyn crate::session::memory::embedding::EmbeddingProvider>,
threshold: f64,
) -> bool {
// Phase 1 (sync): check prerequisites — borrows index, no .await
let provider = match embedding_provider {
Some(p) => p,
None => {
tracing::debug!(target: LOG,
"MEMORY_FLUSH_SEMANTIC_DEDUP: no embedding provider, skipping");
return false;
}
};
if !index.vec_available() {
tracing::debug!(target: LOG,
"MEMORY_FLUSH_SEMANTIC_DEDUP: sqlite-vec not available, skipping");
return false;
}
// Phase 2 (async): embed — no &index borrow across this .await
let embedding = match provider.embed_batch(&[content]).await {
Ok(mut vecs) if !vecs.is_empty() => vecs.swap_remove(0),
Ok(_) => {
tracing::warn!(target: LOG,
"MEMORY_FLUSH_SEMANTIC_DEDUP: embedding returned empty result");
return false;
}
Err(e) => {
tracing::warn!(target: LOG,
"MEMORY_FLUSH_SEMANTIC_DEDUP: embedding failed: {e}");
return false;
}
};
// Phase 3 (sync): vector search + threshold check — borrows index, no .await
let neighbors = match index.vector_search(&embedding, SEMANTIC_DEDUP_KNN_LIMIT) {
Ok(n) => n,
Err(e) => {
tracing::warn!(target: LOG,
"MEMORY_FLUSH_SEMANTIC_DEDUP: vector search failed: {e}");
return false;
}
};
let mut max_sim = 0.0_f64;
for (chunk_id, distance) in &neighbors {
let similarity = (1.0 - (*distance as f64 / MAX_L2_DISTANCE)).clamp(0.0, 1.0);
max_sim = max_sim.max(similarity);
if similarity > threshold {
tracing::info!(target: LOG,
"MEMORY_FLUSH_SEMANTIC_DEDUP: duplicate detected \
(chunk={chunk_id}, similarity={similarity:.4}, \
threshold={threshold})");
return true;
}
}
tracing::info!(target: LOG,
"MEMORY_FLUSH_SEMANTIC_DEDUP: no duplicate \
(checked={}, max_similarity={max_sim:.4}, \
threshold={threshold})",
neighbors.len());
false
}
/// Select a recent window from simplified chat messages for the flush model.
///
/// Starts with the last `recent_message_count` messages, then expands backward
/// to the nearest `User` message so the window always starts on a user
/// boundary. The returned window may be larger than `recent_message_count`.
/// System messages are excluded since the flush adds its own system prompt.
pub fn select_flush_window(
messages: Vec<ChatRequestMessage>,
recent_message_count: usize,
) -> Vec<ChatRequestMessage> {
let messages: Vec<_> = messages
.into_iter()
.filter(|m| m.role != Role::System)
.collect();
let total = messages.len();
let mut start = total.saturating_sub(recent_message_count);
while start > 0 && messages[start].role != Role::User {
start -= 1;
}
messages.into_iter().skip(start).collect()
}
#[cfg(test)]
mod tests {
use super::*;
fn default_flush_config() -> MemoryFlushConfig {
MemoryFlushConfig::default()
}
#[test]
fn test_should_flush_disabled() {
let config = MemoryFlushConfig {
enabled: false,
..default_flush_config()
};
assert!(!should_flush(90_000, 100_000, 85, &config, 0, 1));
}
#[test]
fn test_should_flush_already_flushed_this_cycle() {
let config = default_flush_config();
// same compaction count → already flushed
assert!(!should_flush(90_000, 100_000, 85, &config, 1, 1));
}
#[test]
fn test_should_flush_below_threshold() {
let config = default_flush_config();
// 100K context, 85% compact = 85K, flush at 85K - 4K = 81K
// 50K tokens → below threshold
assert!(!should_flush(50_000, 100_000, 85, &config, 0, 1));
}
#[test]
fn test_should_flush_at_threshold() {
let config = default_flush_config();
// 100K context, 85% compact = 85K, flush at 85K - 4K = 81K
// 81K tokens → at threshold → should flush
assert!(should_flush(81_000, 100_000, 85, &config, 0, 1));
}
#[test]
fn test_should_flush_above_threshold() {
let config = default_flush_config();
assert!(should_flush(83_000, 100_000, 85, &config, 0, 1));
}
#[test]
fn test_should_flush_custom_soft_threshold() {
let config = MemoryFlushConfig {
soft_threshold_tokens: 10_000,
..default_flush_config()
};
// 100K context, 85% compact = 85K, flush at 85K - 10K = 75K
assert!(!should_flush(74_000, 100_000, 85, &config, 0, 1));
assert!(should_flush(75_000, 100_000, 85, &config, 0, 1));
}
#[test]
fn test_should_flush_different_compaction_cycles() {
let config = default_flush_config();
// First cycle: should flush (counter is pre-incremented to 1 in run_compact)
assert!(should_flush(82_000, 100_000, 85, &config, 0, 1));
// After flush (same cycle): should not flush again
assert!(!should_flush(82_000, 100_000, 85, &config, 1, 1));
// New cycle: should flush again
assert!(should_flush(82_000, 100_000, 85, &config, 1, 2));
}
#[test]
fn test_should_flush_non_round_window() {
// cw=10_001, pct=85, soft=4_000. Scaled boundary:
// used*100 >= 10_001*85 - 4_000*100 = 850_085 - 400_000 = 450_085
// -> false at used=4500, true at used=4501.
let config = MemoryFlushConfig {
soft_threshold_tokens: 4_000,
..default_flush_config()
};
assert!(!should_flush(4_499, 10_001, 85, &config, 0, 1));
assert!(!should_flush(4_500, 10_001, 85, &config, 0, 1));
assert!(should_flush(4_501, 10_001, 85, &config, 0, 1));
}
#[test]
fn test_should_flush_same_counter_values_blocks() {
let config = default_flush_config();
// Equal counters → "already flushed this cycle" guard fires.
// Both starting at 0 is the initial state; pre-increment in
// maybe_pre_compaction_flush() prevents this from blocking the first flush.
assert!(!should_flush(82_000, 100_000, 85, &config, 0, 0));
assert!(!should_flush(82_000, 100_000, 85, &config, 5, 5));
}
// -----------------------------------------------------------------------
// process_flush_response tests
// -----------------------------------------------------------------------
#[test]
fn test_flush_response_empty() {
let config = default_flush_config();
assert_eq!(
process_flush_response("", &config),
FlushResult::NothingToStore
);
assert_eq!(
process_flush_response(" ", &config),
FlushResult::NothingToStore
);
assert_eq!(
process_flush_response("\n\n", &config),
FlushResult::NothingToStore
);
}
#[test]
fn test_flush_response_no_reply_variants() {
let config = default_flush_config();
assert_eq!(
process_flush_response("NO_REPLY", &config),
FlushResult::NothingToStore
);
assert_eq!(
process_flush_response("no reply", &config),
FlushResult::NothingToStore
);
assert_eq!(
process_flush_response("No-Reply", &config),
FlushResult::NothingToStore
);
assert_eq!(
process_flush_response("noreply", &config),
FlushResult::NothingToStore
);
assert_eq!(
process_flush_response(" NO_REPLY ", &config),
FlushResult::NothingToStore
);
}
#[test]
fn test_flush_response_accepted() {
let config = default_flush_config();
let content = "## Key Decisions\n\nWe chose Rust for performance.";
assert_eq!(
process_flush_response(content, &config),
FlushResult::Accepted(content.to_string())
);
}
#[test]
fn test_flush_response_rejected_no_headers() {
let config = default_flush_config();
let content = "Just some plain text without any markdown headers at all.";
assert!(matches!(
process_flush_response(content, &config),
FlushResult::Rejected(_)
));
}
#[test]
fn test_flush_response_truncated() {
let config = MemoryFlushConfig {
max_flush_write_chars: 50,
..default_flush_config()
};
let content = "# Title\n\n".to_string() + &"x".repeat(100);
let result = process_flush_response(&content, &config);
if let FlushResult::Accepted(text) = result {
assert!(text.chars().count() <= 50);
} else {
panic!("expected Accepted, got {result:?}");
}
}
#[test]
fn test_flush_response_h1_header_accepted() {
let config = default_flush_config();
let content = "# Top Level\n\nSome content.";
assert!(matches!(
process_flush_response(content, &config),
FlushResult::Accepted(_)
));
}
#[test]
fn test_flush_system_prompt_content() {
assert!(FLUSH_SYSTEM_PROMPT.contains("markdown"));
assert!(FLUSH_SYSTEM_PROMPT.contains("NO_REPLY"));
assert!(
!FLUSH_SYSTEM_PROMPT.contains("Always write something"),
"old bias toward always writing should be removed"
);
assert!(
FLUSH_SYSTEM_PROMPT.contains("genuinely useful"),
"prompt should bias toward NO_REPLY for low-value sessions"
);
assert!(!FLUSH_SYSTEM_PROMPT.contains("User preferences"));
}
#[test]
fn test_delta_system_prompt_content() {
assert!(FLUSH_DELTA_SYSTEM_PROMPT.contains("incremental update"));
assert!(FLUSH_DELTA_SYSTEM_PROMPT.contains("NO_REPLY"));
assert!(FLUSH_DELTA_SYSTEM_PROMPT.contains("Previous flush content"));
assert!(!FLUSH_DELTA_SYSTEM_PROMPT.contains("User preferences"));
assert!(
FLUSH_DELTA_SYSTEM_PROMPT.contains("genuinely new and useful"),
"delta prompt should use same selectivity standard as primary prompt"
);
}
#[test]
fn test_select_flush_window_expands_to_user_boundary() {
let mut messages = vec![ChatRequestMessage::user("early question")];
for i in 0..20 {
messages.push(ChatRequestMessage::assistant(
format!("response {i}"),
"",
None,
));
}
let window = select_flush_window(messages, 20);
assert_eq!(window.len(), 21);
assert_eq!(window[0].role, Role::User);
}
#[test]
fn test_select_flush_window_filters_system_messages() {
let messages = vec![
ChatRequestMessage::system("you are helpful"),
ChatRequestMessage::user("hi"),
ChatRequestMessage::assistant("hello", "", None),
];
let window = select_flush_window(messages, 20);
assert!(window.iter().all(|m| m.role != Role::System));
assert_eq!(window.len(), 2);
}
#[test]
fn test_select_flush_window_short_conversation() {
let messages = vec![
ChatRequestMessage::user("hi"),
ChatRequestMessage::assistant("hello", "", None),
];
let window = select_flush_window(messages, 20);
assert_eq!(window.len(), 2);
assert_eq!(window[0].role, Role::User);
}
// -----------------------------------------------------------------------
// is_semantically_duplicate tests
// -----------------------------------------------------------------------
#[tokio::test]
async fn test_semantic_dedup_no_provider_allows_write() {
use crate::session::memory::{MemoryIndex, MemoryStorage, index::init_sqlite_vec};
use tempfile::TempDir;
init_sqlite_vec();
let tmp = TempDir::new().unwrap();
let db_path = tmp.path().join("test.sqlite");
let storage =
MemoryStorage::with_paths(tmp.path().join("global"), tmp.path().join("workspace"));
let index = MemoryIndex::open_or_create(&db_path, storage, Default::default(), 4).unwrap();
// No embedding provider → always returns false (allow write).
let result = is_semantically_duplicate(
"## Test\n\nSome content.",
&index,
None,
SEMANTIC_DEDUP_SIMILARITY_THRESHOLD,
)
.await;
assert!(!result, "should allow write when no embedding provider");
}
#[tokio::test]
async fn test_semantic_dedup_no_similar_content() {
use crate::session::memory::embedding::MockEmbeddingProvider;
use crate::session::memory::{MemoryIndex, MemoryStorage, index::init_sqlite_vec};
use tempfile::TempDir;
init_sqlite_vec();
let tmp = TempDir::new().unwrap();
let db_path = tmp.path().join("test.sqlite");
let storage =
MemoryStorage::with_paths(tmp.path().join("global"), tmp.path().join("workspace"));
let index = MemoryIndex::open_or_create(&db_path, storage, Default::default(), 4).unwrap();
let provider = MockEmbeddingProvider { dimensions: 4 };
// Empty index → no neighbors → not a duplicate.
let result = is_semantically_duplicate(
"## New Content\n\nFresh ideas here.",
&index,
Some(&provider),
SEMANTIC_DEDUP_SIMILARITY_THRESHOLD,
)
.await;
assert!(!result, "should not be duplicate against empty index");
}
#[tokio::test]
async fn test_semantic_dedup_detects_identical_content() {
use crate::session::memory::embedding::{EmbeddingProvider, MockEmbeddingProvider};
use crate::session::memory::{MemoryIndex, MemoryStorage, index::init_sqlite_vec};
use tempfile::TempDir;
init_sqlite_vec();
let tmp = TempDir::new().unwrap();
let db_path = tmp.path().join("test.sqlite");
let storage =
MemoryStorage::with_paths(tmp.path().join("global"), tmp.path().join("workspace"));
let mut index =
MemoryIndex::open_or_create(&db_path, storage, Default::default(), 4).unwrap();
let provider = MockEmbeddingProvider { dimensions: 4 };
let content = "## Decisions\n\nWe chose Rust for memory safety.";
// Index a file containing the same content.
let file_path = tmp.path().join("existing.md");
std::fs::write(&file_path, content).unwrap();
index.reindex_file(&file_path, "session").unwrap();
// Embed the existing chunk.
let existing_embedding = provider.embed_batch(&[content]).await.unwrap();
let chunk_id = format!("{}:0", file_path.to_string_lossy());
index
.upsert_embedding(&chunk_id, &existing_embedding[0])
.unwrap();
// Same content → identical embedding → distance 0 → similarity 1.0 → duplicate.
let result = is_semantically_duplicate(
content,
&index,
Some(&provider),
SEMANTIC_DEDUP_SIMILARITY_THRESHOLD,
)
.await;
assert!(result, "identical content should be detected as duplicate");
}
#[tokio::test]
async fn test_semantic_dedup_allows_different_content() {
use crate::session::memory::embedding::{EmbeddingProvider, MockEmbeddingProvider};
use crate::session::memory::{MemoryIndex, MemoryStorage, index::init_sqlite_vec};
use tempfile::TempDir;
init_sqlite_vec();
let tmp = TempDir::new().unwrap();
let db_path = tmp.path().join("test.sqlite");
let storage =
MemoryStorage::with_paths(tmp.path().join("global"), tmp.path().join("workspace"));
let mut index =
MemoryIndex::open_or_create(&db_path, storage, Default::default(), 4).unwrap();
let provider = MockEmbeddingProvider { dimensions: 4 };
let existing = "## Decisions\n\nWe chose Rust for memory safety.";
// Index and embed existing content.
let file_path = tmp.path().join("existing.md");
std::fs::write(&file_path, existing).unwrap();
index.reindex_file(&file_path, "session").unwrap();
let emb = provider.embed_batch(&[existing]).await.unwrap();
let chunk_id = format!("{}:0", file_path.to_string_lossy());
index.upsert_embedding(&chunk_id, &emb[0]).unwrap();
// Different content should not be flagged as duplicate.
let novel = "## Architecture\n\nThe API uses Python FastAPI with async handlers.";
let result = is_semantically_duplicate(
novel,
&index,
Some(&provider),
SEMANTIC_DEDUP_SIMILARITY_THRESHOLD,
)
.await;
assert!(
!result,
"different content should not be flagged as duplicate"
);
}
}
@@ -0,0 +1,13 @@
pub mod chat;
pub mod compaction_context;
pub mod full_replace_compaction;
pub mod memory_context;
pub mod memory_flush;
pub mod prompt_suggest;
pub mod replay;
pub mod session_compact;
pub mod session_recap;
pub mod session_summary;
pub mod tool_input_parsing;
pub use compaction_context::CompactionStateContext;
@@ -0,0 +1,585 @@
//! Next-prompt prediction helpers (tab autocomplete ghost text).
//!
//! After a turn completes, the client asks the session to predict what the
//! user is likely to type next. The prediction renders as dim ghost text in
//! the empty prompt input; Tab accepts it. Modelled on common coding-agent
//! prompt suggestion features, but instead of replaying the full conversation prefix
//! it sends a *compact text-only transcript* — the call always routes to a
//! small dedicated model (configurable, [`DEFAULT_SUGGEST_MODEL`] by
//! default, never the session model — see [`effective_suggest_model`]),
//! where the parent session's prompt cache would not apply anyway, so a
//! small request wins on both cost and latency.
//!
//! The pure helpers here build the request items and filter the model output;
//! the actual model call lives on the `SessionActor`
//! (`handle_suggest_prompt`).
use crate::config::PromptSuggestModelPin;
use crate::sampling::ConversationItem;
use crate::session::helpers::chat::floor_char_boundary;
/// Model used for suggestion calls when nothing pins one (no env /
/// `[models] prompt_suggestion` / remote setting / client hint — see
/// [`effective_suggest_model`]). Suggestion requests must stay on a small,
/// fast model: falling back to the session model would multiply the per-turn
/// cost of the feature and add reasoning-model latency for a throwaway
/// prediction.
pub(crate) const DEFAULT_SUGGEST_MODEL: &str = "grok-build-0.1";
/// Resolve the model for one suggestion request, or `None` to skip the
/// request entirely (controlled disable).
///
/// Precedence: env pin > config.toml/remote pin > client hint (the request's
/// `model` param) > [`DEFAULT_SUGGEST_MODEL`]. Every tier except the env pin
/// is catalog-guarded via `in_catalog`: [`DEFAULT_SUGGEST_MODEL`]
/// (`grok-build-0.1`) is API-key-only and excluded from OAuth catalogs, so
/// firing it (or any unavailable pin) would send a doomed per-turn request
/// that can never render ghost text. Skipping keeps the per-turn cost at
/// zero; deliberately NOT a session-model fallback — a per-turn background
/// call must stay on a small cheap model. The env pin bypasses the guard so
/// `KIGI_PROMPT_SUGGESTIONS_MODEL` keeps working for models the catalog does
/// not list (mirrors the pager, which forwards the env value unchecked).
pub(crate) fn effective_suggest_model(
pin: &PromptSuggestModelPin,
client_hint: Option<&str>,
in_catalog: impl Fn(&str) -> bool,
) -> Option<String> {
let client_hint = client_hint.map(str::trim).filter(|s| !s.is_empty());
let (model, catalog_guarded) = match pin {
PromptSuggestModelPin::Env(m) => (m.as_str(), false),
PromptSuggestModelPin::Pinned(m) => (m.as_str(), true),
PromptSuggestModelPin::Unpinned => (client_hint.unwrap_or(DEFAULT_SUGGEST_MODEL), true),
};
if catalog_guarded && !in_catalog(model) {
return None;
}
Some(model.to_owned())
}
/// Total character budget for the compact transcript (~6k tokens at the
/// bytes/4 estimate). Keeps the per-turn cost of the feature trivial even on
/// long sessions.
const TRANSCRIPT_BUDGET_CHARS: usize = 24_000;
/// Per-message character cap inside the transcript. Long messages (pasted
/// logs, big diffs) carry little signal for next-prompt prediction.
const MESSAGE_CAP_CHARS: usize = 1_500;
/// Reject suggestions longer than this — a prompt suggestion should be a
/// short, obvious next step, not an essay.
const SUGGESTION_MAX_CHARS: usize = 120;
/// Reject suggestions with more words than this (mirrors common
/// "2-12 words" guidance with a little slack).
const SUGGESTION_MAX_WORDS: usize = 16;
/// Short replies that are useful suggestions despite being a single word.
const ONE_WORD_ALLOWLIST: &[&str] = &[
"yes", "yeah", "yep", "no", "ok", "okay", "continue", "proceed", "push", "commit", "deploy",
"stop", "check", "retry", "undo", "merge",
];
/// System prompt for the suggestion call. The model sees a compact transcript
/// and must reply with ONLY the predicted next user message (or nothing).
pub(crate) const SUGGEST_PROMPT_SYSTEM: &str = "You predict what the USER will type next into their coding agent CLI.\n\
You are shown a transcript of the conversation so far. The agent's latest reply ends the transcript.\n\n\
FIRST: look at the user's recent messages and original request.\n\
Your job is to predict what THEY would type next — not what you think they should do.\n\
THE TEST: would they think \"I was just about to type that\"?\n\n\
EXAMPLES:\n\
- User asked \"fix the bug and run tests\", bug is fixed -> \"run the tests\"\n\
- After code was written -> \"try it out\"\n\
- Agent offers options -> the option the user would likely pick, based on the conversation\n\
- Agent ends by asking a yes/no question (continue? delete it? print it?) -> the user's likely answer: \"yes\" or \"no\"\n\
- Task complete with an obvious follow-up -> \"commit this\" or \"push it\"\n\
- After an error or a misunderstanding -> NONE (let them assess)\n\n\
Be specific: \"run the tests\" beats \"continue\".\n\
When the agent's reply ends with a question, a suggestion almost always exists — predict the answer.\n\n\
NEVER SUGGEST:\n\
- A message the user already sent, or a rephrasing of one — the transcript is history, not a menu. \
Once a request was handled, predict the step AFTER it, never the request again \
(short confirmations like \"yes\" or \"continue\" are the only acceptable repeats)\n\
- Evaluative filler (\"looks good\", \"thanks\")\n\
- Questions back to the agent (\"what about...?\")\n\
- Agent-voice phrasing (\"Let me...\", \"I'll...\", \"Here's...\")\n\
- New ideas the user never asked about\n\
- Multiple sentences\n\n\
Stay silent if the next step is not obvious from what the user said: reply with the single word NONE.\n\n\
Format: 2-12 words, matching the user's own style and casing.\n\
Reply with ONLY the suggestion text (or NONE) — no quotes, no markdown, no explanation.";
/// One transcript line: role label + flattened text content.
fn transcript_line(role: &str, text: &str) -> Option<String> {
let text = text.trim();
if text.is_empty() {
return None;
}
let mut text = text;
if text.len() > MESSAGE_CAP_CHARS {
let cut = floor_char_boundary(text, MESSAGE_CAP_CHARS);
text = &text[..cut];
}
Some(format!("{role}: {text}"))
}
/// Build the compact transcript from a conversation snapshot.
///
/// Keeps genuine `User` messages (skipping runtime-synthesized ones) and
/// `Assistant` text, newest-last, walking backwards until the character
/// budget is exhausted. Tool calls/results, reasoning, and the system prompt
/// are dropped — the user/assistant dialogue carries the signal for "what
/// will the user type next", and dropping the rest keeps the request cheap.
///
/// Returns `None` when the conversation has no assistant reply yet (nothing
/// to predict from).
pub(crate) fn build_transcript(conversation: &[ConversationItem]) -> Option<String> {
let mut lines: Vec<String> = Vec::new();
let mut used = 0usize;
let mut saw_assistant = false;
for item in conversation.iter().rev() {
let line = match item {
ConversationItem::User(u) => {
if u.synthetic_reason.is_some() {
continue;
}
transcript_line("User", &item.text_content())
}
ConversationItem::Assistant(_) => {
let line = transcript_line("Agent", &item.text_content());
if line.is_some() {
saw_assistant = true;
}
line
}
_ => continue,
};
let Some(line) = line else { continue };
if used + line.len() > TRANSCRIPT_BUDGET_CHARS && !lines.is_empty() {
break;
}
used += line.len();
lines.push(line);
}
if !saw_assistant || lines.is_empty() {
return None;
}
lines.reverse();
Some(lines.join("\n\n"))
}
/// Build the user message for the suggestion request.
pub(crate) fn suggest_prompt_user_message(transcript: &str, cwd: &str) -> String {
format!(
"CWD: {cwd}\n\nTranscript:\n\n{transcript}\n\n\
Predict the user's next message. Reply with ONLY the suggestion text."
)
}
/// Minimum word count for the deterministic repeat filter. Short
/// command-like replies ("yes", "run tests", "try again") legitimately
/// recur across a session; a repeated multi-word task prompt is the
/// "it suggested my old prompt back to me" failure mode.
const REPEAT_MIN_WORDS: usize = 4;
/// Case- and whitespace-insensitive form used for repeat comparison, with
/// trailing sentence punctuation dropped.
fn normalize_for_repeat(text: &str) -> String {
text.split_whitespace()
.collect::<Vec<_>>()
.join(" ")
.trim_end_matches(['.', '!', '?'])
.to_ascii_lowercase()
}
/// Whether a sanitized suggestion merely repeats a message the user already
/// sent. Deterministic backstop behind the system prompt's anti-repeat rule:
/// prompt guidance reduces repeats, this guarantees an exact (normalized)
/// re-suggestion of a past multi-word prompt never renders as ghost text.
/// Short suggestions (< [`REPEAT_MIN_WORDS`] words) are exempt — repeating
/// "yes" or "run tests" is often exactly what the user is about to type.
pub(crate) fn is_repeat_of_user_message(
suggestion: &str,
conversation: &[ConversationItem],
) -> bool {
if suggestion.split_whitespace().count() < REPEAT_MIN_WORDS {
return false;
}
let needle = normalize_for_repeat(suggestion);
conversation.iter().any(|item| match item {
ConversationItem::User(u) if u.synthetic_reason.is_none() => {
normalize_for_repeat(&item.text_content()) == needle
}
_ => false,
})
}
/// Filter/normalize the raw model output into a usable suggestion.
///
/// Returns `None` for anything that should not be shown as ghost text:
/// meta/no-op replies, agent-voice phrasing, multi-sentence or multi-line
/// output, markdown, or over-long text. Mirrors typical coding-agent suggestion
/// filters, adapted to the compact-transcript prompt above.
pub(crate) fn sanitize_suggestion(raw: &str) -> Option<String> {
// First line only; the prompt asks for a single line but models drift.
let line = raw.trim().lines().next()?.trim();
// Strip common wrappers the prompt forbids but models still emit.
let line = line
.trim_start_matches(['"', '\'', '`', '“', ''])
.trim_end_matches(['"', '\'', '`', '”', ''])
.trim();
if line.is_empty() || line.len() >= SUGGESTION_MAX_CHARS {
return None;
}
// Meta / "no suggestion" replies.
let lowered = line.to_ascii_lowercase();
let meta = [
"none",
"n/a",
"no suggestion",
"nothing",
"(silence)",
"silence",
"null",
];
if meta
.iter()
.any(|m| lowered == *m || lowered.starts_with(&format!("{m}.")))
{
return None;
}
// Markdown / formatting — ghost text renders on a single styled line.
if line.contains('*') || line.contains("```") || line.starts_with('#') || line.starts_with('-')
{
return None;
}
// Agent-voice phrasing — the suggestion must be in the USER's voice.
let agent_voice = [
"i'll ",
"i will ",
"let me ",
"here's ",
"here is ",
"i'm going to ",
];
if agent_voice.iter().any(|p| lowered.starts_with(p)) {
return None;
}
// Parenthetical/bracketed meta replies like "(no suggestion)".
if (line.starts_with('(') && line.ends_with(')'))
|| (line.starts_with('[') && line.ends_with(']'))
{
return None;
}
// Label prefixes like "Suggestion: ..." / "User: ...".
if let Some((head, _)) = line.split_once(':')
&& !head.contains(' ')
&& head.chars().all(|c| c.is_ascii_alphabetic())
{
return None;
}
// Multiple sentences read as agent prose, not a prompt.
let multi_sentence = line
.as_bytes()
.windows(3)
.any(|w| matches!(w[0], b'.' | b'!' | b'?') && w[1] == b' ' && w[2].is_ascii_uppercase());
if multi_sentence {
return None;
}
// Word-count bounds: 1 word only from the allowlist, and never a wall of text.
let words = line.split_whitespace().count();
if words > SUGGESTION_MAX_WORDS {
return None;
}
if words == 1 {
let bare = lowered.trim_end_matches(['.', '!']);
if !ONE_WORD_ALLOWLIST.contains(&bare) && !bare.starts_with('/') {
return None;
}
}
Some(line.to_owned())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::config::PromptSuggestModelPin as Pin;
// -- effective_suggest_model ---------------------------------------------
#[test]
fn effective_model_default_requires_catalog() {
// No pin, no hint: the built-in default fires only when this shell's
// catalog can sample it.
assert_eq!(
effective_suggest_model(&Pin::Unpinned, None, |m| m == DEFAULT_SUGGEST_MODEL)
.as_deref(),
Some(DEFAULT_SUGGEST_MODEL)
);
// OAuth catalogs exclude grok-build-0.1 → skip the request entirely,
// never a doomed call (and never the session model).
assert_eq!(
effective_suggest_model(&Pin::Unpinned, None, |_| false),
None
);
}
#[test]
fn effective_model_client_hint_beats_default_and_is_guarded() {
assert_eq!(
effective_suggest_model(&Pin::Unpinned, Some("hinted"), |m| m == "hinted").as_deref(),
Some("hinted")
);
// A hint the shell can't sample skips — no silent fall-through.
assert_eq!(
effective_suggest_model(&Pin::Unpinned, Some("hinted"), |_| false),
None
);
// Blank hints are ignored: the default tier applies.
assert_eq!(
effective_suggest_model(&Pin::Unpinned, Some(" "), |m| m == DEFAULT_SUGGEST_MODEL)
.as_deref(),
Some(DEFAULT_SUGGEST_MODEL)
);
}
#[test]
fn effective_model_pin_beats_client_hint_and_is_guarded() {
assert_eq!(
effective_suggest_model(&Pin::Pinned("pinned".into()), Some("hinted"), |m| m
== "pinned"
|| m == "hinted")
.as_deref(),
Some("pinned")
);
// A pinned-but-unavailable model skips — the pin is an explicit
// choice, not a preference list; no fall-through to hint or default.
assert_eq!(
effective_suggest_model(&Pin::Pinned("pinned".into()), Some("hinted"), |m| m
== "hinted"),
None
);
}
#[test]
fn effective_model_env_pin_bypasses_catalog_guard() {
// KIGI_PROMPT_SUGGESTIONS_MODEL is the explicit escape hatch: used
// verbatim even when the catalog does not list the model (mirrors
// the pager, which forwards the env value unchecked).
assert_eq!(
effective_suggest_model(&Pin::Env("custom-model".into()), Some("hinted"), |_| false)
.as_deref(),
Some("custom-model")
);
}
// -- sanitize_suggestion ------------------------------------------------
#[test]
fn sanitize_accepts_short_imperative() {
assert_eq!(
sanitize_suggestion("run the tests").as_deref(),
Some("run the tests")
);
}
#[test]
fn sanitize_strips_quotes_and_backticks() {
assert_eq!(
sanitize_suggestion("\"commit this\"").as_deref(),
Some("commit this")
);
assert_eq!(sanitize_suggestion("`push it`").as_deref(), Some("push it"));
}
#[test]
fn sanitize_takes_first_line_only() {
assert_eq!(
sanitize_suggestion("run the tests\nthen commit").as_deref(),
Some("run the tests")
);
}
#[test]
fn sanitize_rejects_none_and_meta() {
for s in ["NONE", "none", "n/a", "no suggestion", "(silence)", ""] {
assert_eq!(sanitize_suggestion(s), None, "should reject {s:?}");
}
}
#[test]
fn sanitize_rejects_agent_voice() {
for s in [
"I'll run the tests",
"Let me check the output",
"Here's what to do next",
] {
assert_eq!(sanitize_suggestion(s), None, "should reject {s:?}");
}
}
#[test]
fn sanitize_rejects_markdown_and_labels() {
for s in [
"**run tests**",
"- run tests",
"# next",
"Suggestion: run tests",
"```run```",
] {
assert_eq!(sanitize_suggestion(s), None, "should reject {s:?}");
}
}
#[test]
fn sanitize_rejects_multi_sentence_and_overlong() {
assert_eq!(
sanitize_suggestion("Run the tests. Then commit the changes."),
None
);
let long = "word ".repeat(20);
assert_eq!(sanitize_suggestion(&long), None);
let chars = "x".repeat(200);
assert_eq!(sanitize_suggestion(&chars), None);
}
#[test]
fn sanitize_one_word_allowlist() {
assert_eq!(sanitize_suggestion("yes").as_deref(), Some("yes"));
assert_eq!(sanitize_suggestion("commit").as_deref(), Some("commit"));
// Bare one-word verbs outside the allowlist are too ambiguous.
assert_eq!(sanitize_suggestion("refactor"), None);
// Slash commands are fine.
assert_eq!(sanitize_suggestion("/review").as_deref(), Some("/review"));
}
#[test]
fn sanitize_allows_colon_after_multiword_head() {
// Only single-word alphabetic label heads are rejected.
assert_eq!(
sanitize_suggestion("fix the parse error: line 42").as_deref(),
Some("fix the parse error: line 42")
);
}
// -- build_transcript ---------------------------------------------------
fn user(text: &str) -> ConversationItem {
ConversationItem::user(text.to_owned())
}
fn assistant(text: &str) -> ConversationItem {
ConversationItem::assistant(text.to_owned())
}
// -- is_repeat_of_user_message -------------------------------------------
#[test]
fn repeat_filter_rejects_verbatim_past_prompt() {
let conv = vec![user("fix the flaky auth test"), assistant("Fixed it")];
assert!(is_repeat_of_user_message("fix the flaky auth test", &conv));
}
#[test]
fn repeat_filter_is_case_whitespace_and_punctuation_insensitive() {
let conv = vec![user("Fix the flaky\nauth test."), assistant("Fixed it")];
assert!(is_repeat_of_user_message("fix the flaky auth test!", &conv));
}
#[test]
fn repeat_filter_exempts_short_suggestions() {
let conv = vec![user("run the tests"), assistant("3 failures")];
// 3 words — legitimately recurs after new changes.
assert!(!is_repeat_of_user_message("run the tests", &conv));
assert!(!is_repeat_of_user_message("yes", &conv));
}
#[test]
fn repeat_filter_allows_novel_suggestions() {
let conv = vec![user("fix the flaky auth test"), assistant("Fixed it")];
assert!(!is_repeat_of_user_message("commit and push the fix", &conv));
}
#[test]
fn repeat_filter_ignores_synthetic_user_messages() {
let mut synthetic = user("please review the changes now");
if let ConversationItem::User(u) = &mut synthetic {
u.synthetic_reason = Some(crate::sampling::SyntheticReason::SystemReminder);
}
let conv = vec![synthetic, assistant("done")];
assert!(!is_repeat_of_user_message(
"please review the changes now",
&conv
));
}
#[test]
fn transcript_keeps_user_and_assistant_in_order() {
let conv = vec![
ConversationItem::system("sys".to_owned()),
user("fix the bug"),
assistant("Fixed it in foo.rs"),
];
let t = build_transcript(&conv).unwrap();
assert_eq!(t, "User: fix the bug\n\nAgent: Fixed it in foo.rs");
}
#[test]
fn transcript_requires_an_assistant_reply() {
let conv = vec![ConversationItem::system("sys".to_owned()), user("hello")];
assert!(build_transcript(&conv).is_none());
assert!(build_transcript(&[]).is_none());
}
#[test]
fn transcript_skips_synthetic_user_messages() {
let mut synthetic = ConversationItem::user("synthetic reminder".to_owned());
if let ConversationItem::User(u) = &mut synthetic {
u.synthetic_reason = Some(crate::sampling::SyntheticReason::SystemReminder);
}
let conv = vec![user("real question"), synthetic, assistant("answer")];
let t = build_transcript(&conv).unwrap();
assert!(!t.contains("synthetic reminder"));
assert!(t.contains("User: real question"));
}
#[test]
fn transcript_caps_long_messages() {
let long = "a".repeat(10_000);
let conv = vec![user(&long), assistant("ok")];
let t = build_transcript(&conv).unwrap();
assert!(
t.len() < 2_000,
"long message must be truncated: {}",
t.len()
);
}
#[test]
fn transcript_budget_keeps_newest_messages() {
let filler = "b".repeat(MESSAGE_CAP_CHARS);
let mut conv = Vec::new();
for _ in 0..40 {
conv.push(user(&filler));
conv.push(assistant(&filler));
}
conv.push(user("newest question"));
conv.push(assistant("newest answer"));
let t = build_transcript(&conv).unwrap();
assert!(t.len() <= TRANSCRIPT_BUDGET_CHARS + MESSAGE_CAP_CHARS + 64);
assert!(t.contains("newest question"));
assert!(t.ends_with("Agent: newest answer"));
}
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,846 @@
//! Session recap generation helpers.
//!
//! A *recap* is a short "where was I" summary of the session so far, modelled
//! on common coding-agent `/recap` + automatic session-recap features. Unlike
//! compaction, a recap never mutates the conversation: it is generated from a
//! read-only snapshot and surfaced to the client for display only.
//!
//! Generation reuses the parent session's conversation prefix verbatim (so the
//! provider prompt cache stays warm) and appends a single instruction turn that
//! asks for the recap. The pure helpers here build that request and tidy the
//! model's output; the actual model call lives on the `SessionActor`
//! (`handle_recap`).
use crate::sampling::ConversationItem;
use crate::session::helpers::chat::floor_char_boundary;
use kigi_chat_state::{compaction_utils, estimate_conversation_tokens, estimate_item_tokens};
/// Hard cap on the recap text length (characters). Generous headroom: the recap
/// instruction targets ~2540 words (≈240 chars at the top end), so this only
/// guards against runaway model output and never cuts a normal recap.
const RECAP_MAX_CHARS: usize = 1200;
/// Build the instruction turn appended to the conversation snapshot.
///
/// All recap directions live in this single user message (wrapped in a
/// `<system-reminder>`) rather than a separate system prompt, so the
/// conversation prefix — including the agent's real system prompt at
/// `conversation[0]` — is reused verbatim and the prompt cache stays warm.
///
/// `tag` is the reminder tag for the active harness (`"system-reminder"`, or
/// template-specific tags).
///
/// Body text only — the pager adds `Recap —` on render (manual and auto).
///
/// Keep in sync with the recap prompt eval harness (hillclimb there first).
/// Few-shots must stay synthetic — never embed real eval/session content.
pub(crate) fn recap_instruction(tag: &str) -> String {
format!(
"<{tag}>Write ONE sentence recap body for a user returning from idle. \
Output ONLY the body (the UI adds the \"Recap —\" label).\n\n\
Lead with agency:\n\
- \"You asked …\" if the session was mainly questions, walkthroughs, or review with no landed change.\n\
- \"We <past-tense verb> …\" if the agent implemented, fixed, merged, or changed code/config/docs \
(e.g. \"We fixed …\", \"We merged …\", \"We wired …\" — not \"We did fix\" / \"We did merge\").\n\
- If almost nothing happened: \"You had just begun this session.\"\n\n\
Shape: <lead>: <concrete specifics — crate/file/flag/behavior/endpoint>. ~2540 words.\n\n\
Synthetic examples (style only — adapt to THIS session, do not copy):\n\n\
You asked how retries work in the payment client: exponential backoff in `billing/retry.rs`, max 5 attempts, 429s only.\n\n\
You asked for a walkthrough of the auth middleware change: warn-only mode in the API layer, no hard fail on missing claims.\n\n\
We fixed the flaky integration test: race in `queue_worker` shutdown by awaiting the drain channel before exit.\n\n\
We merged the feature branch: kept the new telemetry hooks, dropped the obsolete feature flag in `config/flags.toml`.\n\n\
Bad (never):\n\
- Start with Recap / Session recap / extra labels\n\
- Quote or restate this reminder or any system prompt\n\
- Bullets, markdown, code fences, extra sentences\n\
- Invent work not reflected in the session</{tag}>"
)
}
/// Prepare the conversation snapshot for a recap request.
///
/// 1. Optionally strips reasoning/thinking blocks (`strip_reasoning`). This is
/// only needed on the Anthropic Messages backend, which rejects thinking
/// blocks sent without a top-level `thinking` config. Every other backend
/// (grok/SGLang via ChatCompletions/Responses) keeps reasoning VERBATIM so
/// the conversation prefix is byte-identical to the last turn and the
/// provider's prefix KV cache stays warm — which is the whole reason we
/// append the instruction after the prefix. Mirrors compaction's
/// `summary_strips_reasoning`.
/// 2. Truncates a trailing incomplete assistant/tool-result run — a recap can
/// fire mid-turn, and the Anthropic Messages API rejects `tool_use` ids without a
/// matching `tool_result`.
/// 3. Appends the recap instruction as a final user turn.
pub(crate) fn build_recap_items(
conversation: Vec<ConversationItem>,
tag: &str,
strip_reasoning: bool,
) -> Vec<ConversationItem> {
let mut items = if strip_reasoning {
kigi_chat_state::compaction_utils::strip_reasoning_blocks(conversation)
} else {
conversation
};
pop_trailing_tool_run(&mut items);
items.push(ConversationItem::user(recap_instruction(tag)));
items
}
/// Cap on the effective context window for recap budgeting: the verified
/// `max_prompt_length` for current `grok-build` / `grok-4.5` product backends
/// (`500000`). Applied via `min(window, CAP)`, so a smaller real window still
/// wins (e.g. a 256k legacy model or a debug override).
const RECAP_CONTEXT_WINDOW_CAP: u64 = 500_000;
/// Fraction of the (conservative) window a recap may occupy — the DEFAULT
/// auto-compact threshold. Fixed rather than the remote-settings-resolved value (which
/// can exceed 85), so recap stays at least as conservative as the turn path.
const RECAP_BUDGET_THRESHOLD_PERCENT: u64 = 85;
/// Estimator/serialization slack (mirrors memory-flush's soft-threshold pad). The
/// appended instruction is reserved SEPARATELY via `snapshot_budget`, so it is not
/// double-counted here. (`max_prompt_length` is input-length, so output doesn't count.)
const RECAP_BUDGET_HEADROOM_TOKENS: u64 = 4_000;
/// Budget-aware variant of [`build_recap_items`]. Best-effort: returns a
/// structurally-valid, non-empty request trimmed to the estimated prompt budget
/// (the same bytes/4 estimator compaction triggers on) to prevent
/// `ic_400_prompt_too_long` on long sessions. Not an absolute guarantee — a
/// degenerate tiny window, an oversized retained `System` prefix, or estimator
/// optimism can still exceed the real limit (the 85% + headroom + 500k cap make
/// that unlikely for normal grok-build sessions).
///
/// * Fast path — if the whole snapshot already fits, returns
/// `build_recap_items(...)` verbatim (keeps the grok prefix KV cache warm;
/// honors the caller's `strip_reasoning`).
/// * Over budget — strip reasoning (the prefix cache is lost once we trim),
/// normalize the trailing boundary ([`pop_trailing_tool_run`]),
/// front-trim to fit via `fit_conversation_to_budget` (System kept, most-recent
/// turn truncated in place, never emptied), then append the instruction.
///
/// `context_window` MUST be the window of the model the recap is actually sent to
/// (today the session model).
pub(crate) fn budget_recap_items(
conversation: Vec<ConversationItem>,
tag: &str,
strip_reasoning: bool,
context_window: u64,
) -> Vec<ConversationItem> {
let effective_window = context_window.min(RECAP_CONTEXT_WINDOW_CAP);
let prompt_budget = (effective_window.saturating_mul(RECAP_BUDGET_THRESHOLD_PERCENT) / 100)
.saturating_sub(RECAP_BUDGET_HEADROOM_TOKENS);
let instruction = ConversationItem::user(recap_instruction(tag));
let snapshot_budget = prompt_budget.saturating_sub(estimate_item_tokens(&instruction));
// Un-stripped estimate is a safe upper bound (stripping only shrinks); the
// verbatim path keeps the grok prefix cache warm.
let pre_tokens = estimate_conversation_tokens(&conversation);
if pre_tokens <= snapshot_budget {
return build_recap_items(conversation, tag, strip_reasoning);
}
// Normalize the trailing boundary BEFORE trimming (ordering matters — see doc).
let mut snapshot =
compaction_utils::prepare_conversation_for_verbatim_summarization(conversation, true);
pop_trailing_tool_run(&mut snapshot);
let mut items = compaction_utils::fit_conversation_to_budget(snapshot, snapshot_budget);
let post_tokens = estimate_conversation_tokens(&items);
tracing::debug!(
context_window,
effective_window,
prompt_budget,
snapshot_budget,
pre_tokens,
post_tokens,
"recap over budget: trimmed conversation to fit"
);
items.push(instruction);
items
}
/// Trailing normalization shared by [`build_recap_items`] and
/// [`budget_recap_items`]: pop a trailing tool run — trailing `ToolResult`s and
/// any trailing `Assistant` with `tool_calls` (complete runs included) — so it ends on
/// a clean boundary and the appended `User` instruction never follows a
/// `tool_use`/`tool_result`.
fn pop_trailing_tool_run(items: &mut Vec<ConversationItem>) {
while let Some(last) = items.last() {
match last {
ConversationItem::Assistant(a) if !a.tool_calls.is_empty() => {
items.pop();
}
ConversationItem::ToolResult(_) => {
items.pop();
}
_ => break,
}
}
}
/// Minimum main turns before an automatic return-from-away recap (manual exempt).
pub(crate) const MIN_TURNS_FOR_AUTO_RECAP: usize = 3;
/// Real user prompts (`synthetic_reason.is_none()`), not assistant/tool items.
pub(crate) fn main_turn_count(conversation: &[ConversationItem]) -> usize {
conversation
.iter()
.filter(|item| {
matches!(
item,
ConversationItem::User(u) if u.synthetic_reason.is_none()
)
})
.count()
}
/// Manual: any `main_turns > 0`. Auto: new turn since `last`, min turns, idle.
pub(crate) fn recap_gate(
main_turns: usize,
last: usize,
auto: bool,
idle_ok: bool,
) -> Result<(), &'static str> {
if main_turns == 0 {
return Err("no main turns yet");
}
if auto {
if main_turns <= last {
return Err("no new main turn since last recap");
}
if main_turns < MIN_TURNS_FOR_AUTO_RECAP {
return Err("fewer than min turns for auto recap");
}
if !idle_ok {
return Err("idle threshold not met");
}
}
Ok(())
}
/// Auto recaps longer than this (raw bytes) are saved but not shown.
pub(crate) const RECAP_AUTO_RAW_DISPLAY_MAX: usize = 500;
/// Auto only: long-tail output — persist artifact, do not display.
pub(crate) fn should_suppress_auto_recap_display(raw: &str, summary: &str) -> bool {
if raw.len() > RECAP_AUTO_RAW_DISPLAY_MAX {
return true;
}
summary.ends_with('\u{2026}') && summary.len() >= RECAP_MAX_CHARS
}
/// Clean the model's raw recap output into a readable one-liner body.
///
/// Normalizes whitespace, strips a stray leading label/quotes if the model
/// added one anyway, and caps length at [`RECAP_MAX_CHARS`] as a safety net
/// against runaway output (the cap is generous, so a normal recap is never
/// cut). Does not prepend `Recap —` — the pager always prefixes with that
/// label on render.
pub(crate) fn clean_recap_text(raw: &str) -> String {
// Collapse runs of whitespace/newlines into single spaces (one scrollback line).
let mut out: String = raw.split_whitespace().collect::<Vec<_>>().join(" ");
// Strip a stray leading label if the model added one anyway.
for label in [
"Recap —",
"Recap—",
"Recap -",
"Recap:",
"recap:",
"Session recap:",
"Summary:",
] {
if let Some(rest) = out.strip_prefix(label) {
out = rest.trim_start().to_string();
break;
}
}
// Strip symmetric wrapping quotes around the whole string.
if out.len() >= 2 {
let bytes = out.as_bytes();
let first = bytes[0];
let last = bytes[bytes.len() - 1];
if (first == b'"' && last == b'"') || (first == b'\'' && last == b'\'') {
out = out[1..out.len() - 1].trim().to_string();
}
}
if out.len() > RECAP_MAX_CHARS {
let cut = floor_char_boundary(&out, RECAP_MAX_CHARS);
out.truncate(cut);
out = out.trim_end().to_string();
out.push('\u{2026}'); // …
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use crate::sampling::ConversationItem;
#[test]
fn clean_collapses_whitespace_and_newlines() {
let raw = "Refactored the\n\nparser\tand added tests.";
assert_eq!(
clean_recap_text(raw),
"Refactored the parser and added tests."
);
}
#[test]
fn clean_strips_leading_label() {
assert_eq!(
clean_recap_text("Recap: fixed the auth bug"),
"fixed the auth bug"
);
assert_eq!(
clean_recap_text("Session recap: wired up the API"),
"wired up the API"
);
}
#[test]
fn clean_strips_wrapping_quotes() {
assert_eq!(clean_recap_text("\"did the thing\""), "did the thing");
assert_eq!(clean_recap_text("'did the thing'"), "did the thing");
}
#[test]
fn clean_caps_length_on_char_boundary() {
// Far past the cap → truncated on a char boundary with an ellipsis.
let long = "word ".repeat(RECAP_MAX_CHARS);
let out = clean_recap_text(&long);
assert!(out.len() <= RECAP_MAX_CHARS + 4, "len was {}", out.len());
assert!(out.ends_with('\u{2026}'));
}
#[test]
fn clean_cap_is_utf8_safe() {
// 3-byte chars straddling the byte cap must not panic.
let big = "".repeat(RECAP_MAX_CHARS);
let out = clean_recap_text(&big);
assert!(out.ends_with('\u{2026}'));
}
#[test]
fn clean_keeps_normal_recap_in_full() {
// A normal multi-sentence recap is well under the generous cap, so it is
// returned verbatim — never cut mid-sentence.
let recap = "We fixed the flaky integration test by awaiting the drain \
channel before exit, added a regression test for the shutdown \
path, and updated the runbook with the new sequence.";
let out = clean_recap_text(recap);
assert!(out.len() < RECAP_MAX_CHARS);
assert!(!out.ends_with('\u{2026}'));
assert!(out.ends_with("with the new sequence."));
}
#[test]
fn build_appends_instruction_user_turn() {
let conv = vec![
ConversationItem::system("sys".to_string()),
ConversationItem::user("hello".to_string()),
ConversationItem::assistant("hi".to_string()),
];
let items = build_recap_items(conv, "system-reminder", true);
assert!(matches!(items.last(), Some(ConversationItem::User(_))));
// System prompt prefix is preserved verbatim for cache reuse.
assert!(matches!(items.first(), Some(ConversationItem::System(_))));
}
#[test]
fn build_truncates_trailing_tool_result() {
let conv = vec![
ConversationItem::system("sys".to_string()),
ConversationItem::user("hello".to_string()),
ConversationItem::tool_result("call-1".to_string(), "output".to_string()),
];
let items = build_recap_items(conv, "system-reminder", false);
// The dangling ToolResult is dropped; only system + user + instruction remain.
assert_eq!(items.len(), 3);
assert!(matches!(items.last(), Some(ConversationItem::User(_))));
assert!(
!items
.iter()
.any(|i| matches!(i, ConversationItem::ToolResult(_)))
);
}
#[test]
fn main_turn_count_counts_real_users_only() {
use kigi_sampling_types::{ContentPart, SyntheticReason, ToolCall, UserItem};
use std::sync::Arc;
let conv = vec![
ConversationItem::system("sys".to_string()),
ConversationItem::user("hi".to_string()),
ConversationItem::assistant("hello".to_string()),
ConversationItem::user("again".to_string()),
ConversationItem::User(UserItem {
content: vec![ContentPart::Text {
text: Arc::from("injected"),
}],
synthetic_reason: Some(SyntheticReason::SystemReminder),
..Default::default()
}),
];
assert_eq!(main_turn_count(&conv), 2);
let empty: Vec<ConversationItem> = vec![ConversationItem::system("sys".to_string())];
assert_eq!(main_turn_count(&empty), 0);
let tool_loop = vec![
ConversationItem::user("fix it".to_string()),
ConversationItem::assistant_tool_calls(vec![ToolCall {
id: Arc::from("c1"),
name: "read_file".into(),
arguments: Arc::from("{}"),
}]),
ConversationItem::tool_result("c1".to_string(), "ok".to_string()),
ConversationItem::assistant("done".to_string()),
];
assert_eq!(main_turn_count(&tool_loop), 1);
}
#[test]
fn gate_allows_first_recap_when_last_is_zero() {
assert!(recap_gate(1, 0, false, false).is_ok());
}
#[test]
fn gate_manual_allows_re_recap_on_same_main_turn() {
assert!(recap_gate(2, 2, false, true).is_ok());
assert!(recap_gate(3, 3, false, false).is_ok());
}
#[test]
fn gate_auto_denies_second_recap_on_same_main_turn() {
assert_eq!(
recap_gate(3, 3, true, true),
Err("no new main turn since last recap")
);
}
#[test]
fn gate_allows_after_new_main_turn() {
assert!(recap_gate(3, 2, false, false).is_ok());
assert!(recap_gate(3, 2, true, true).is_ok());
}
#[test]
fn gate_auto_requires_min_turns_and_idle() {
assert_eq!(
recap_gate(2, 0, true, true),
Err("fewer than min turns for auto recap")
);
assert_eq!(recap_gate(3, 0, true, false), Err("idle threshold not met"));
assert!(recap_gate(3, 0, true, true).is_ok());
}
#[test]
fn gate_denies_zero_main_turns() {
assert_eq!(recap_gate(0, 0, false, true), Err("no main turns yet"));
}
#[test]
fn gate_allows_after_compaction_heal_watermark() {
assert!(recap_gate(3, 2, false, false).is_ok());
assert!(recap_gate(3, 3, false, false).is_ok());
assert_eq!(
recap_gate(3, 3, true, false),
Err("no new main turn since last recap")
);
}
#[test]
fn suppress_auto_long_tail_raw_over_display_max() {
let normal = "We fixed gitignored project Claude commands loading as slash skills.";
assert!(!should_suppress_auto_recap_display(
normal,
&clean_recap_text(normal)
));
let long_raw = "Creating the PR from the worktree. ".repeat(20);
assert!(long_raw.len() > RECAP_AUTO_RAW_DISPLAY_MAX);
assert!(should_suppress_auto_recap_display(
&long_raw,
&clean_recap_text(&long_raw)
));
}
#[test]
fn suppress_auto_when_clean_hits_hard_cap_ellipsis() {
let huge = "word ".repeat(RECAP_MAX_CHARS);
let summary = clean_recap_text(&huge);
assert!(summary.ends_with('\u{2026}'));
assert!(should_suppress_auto_recap_display(&huge, &summary));
}
#[test]
fn normal_auto_recap_not_suppressed() {
let raw = "We fixed the flaky integration test: race in queue_worker shutdown.";
assert!(raw.len() < RECAP_AUTO_RAW_DISPLAY_MAX);
assert!(!should_suppress_auto_recap_display(
raw,
&clean_recap_text(raw)
));
}
#[test]
fn instruction_uses_provided_tag() {
assert!(recap_instruction("system_reminder").contains("<system_reminder>"));
assert!(recap_instruction("system-reminder").contains("</system-reminder>"));
}
#[test]
fn instruction_asks_for_one_sentence_body() {
let text = recap_instruction("system-reminder");
assert!(text.contains("Output ONLY the body"));
assert!(text.contains("You asked"));
assert!(text.contains("We fixed"));
assert!(text.contains("We merged"));
assert!(text.contains("billing/retry.rs"));
assert!(text.contains("queue_worker"));
assert!(text.contains("We fixed the flaky"));
assert!(text.contains("We merged the feature"));
assert!(!text.contains("217584"));
assert!(!text.contains("lead with \"Recap"));
}
#[test]
fn clean_returns_body_without_recap_prefix() {
assert_eq!(
clean_recap_text("Recap: You fixed auth in foo.rs."),
"You fixed auth in foo.rs."
);
assert_eq!(
clean_recap_text("You fixed auth in foo.rs."),
"You fixed auth in foo.rs."
);
}
// ---- budget_recap_items ------------------------------------------------
/// Recompute the prompt budget the helper uses, for end-state assertions.
fn recap_prompt_budget(context_window: u64) -> u64 {
(context_window.min(RECAP_CONTEXT_WINDOW_CAP) * RECAP_BUDGET_THRESHOLD_PERCENT / 100)
.saturating_sub(RECAP_BUDGET_HEADROOM_TOKENS)
}
fn mk_reasoning(id: &str) -> ConversationItem {
use crate::sampling::rs;
ConversationItem::Reasoning(rs::ReasoningItem {
id: id.to_string(),
summary: vec![rs::SummaryPart::SummaryText(rs::SummaryTextContent {
text: format!("secret thinking {id}"),
})],
content: None,
encrypted_content: None,
status: None,
})
}
fn mk_tool_call(id: &str, args: &str) -> kigi_sampling_types::ToolCall {
use std::sync::Arc;
kigi_sampling_types::ToolCall {
id: Arc::from(id),
name: "read_file".into(),
arguments: Arc::from(args),
}
}
#[test]
fn budget_fast_path_matches_build_recap_items() {
// Include a reasoning block so `strip_reasoning=true` actually exercises
// stripping on the fits path (not just a no-op).
let conv = vec![
ConversationItem::system("sys"),
ConversationItem::user("hello"),
mk_reasoning("r1"),
ConversationItem::assistant("hi"),
];
let budgeted = budget_recap_items(conv.clone(), "system-reminder", true, 256_000);
let built = build_recap_items(conv, "system-reminder", true);
assert_eq!(
serde_json::to_string(&budgeted).unwrap(),
serde_json::to_string(&built).unwrap(),
"under-budget snapshot must match build_recap_items verbatim"
);
assert!(
!budgeted
.iter()
.any(|i| matches!(i, ConversationItem::Reasoning(_))),
"strip_reasoning=true must strip reasoning on the fast path too"
);
assert!(matches!(
budgeted.first(),
Some(ConversationItem::System(_))
));
assert!(matches!(budgeted.last(), Some(ConversationItem::User(_))));
}
#[test]
fn budget_over_budget_trims_within_budget() {
let conv = vec![
ConversationItem::system("sys"),
ConversationItem::user("q1"),
ConversationItem::assistant("a1"),
ConversationItem::user("d".repeat(40_000)), // ~10k est tokens
ConversationItem::assistant("a2"),
ConversationItem::user("recent q"),
];
let snapshot_plus_instruction = conv.len() + 1;
let out = budget_recap_items(conv, "system-reminder", false, 8_000);
assert!(
estimate_conversation_tokens(&out) <= recap_prompt_budget(8_000),
"trimmed recap must fit the prompt budget"
);
assert!(
out.len() < snapshot_plus_instruction,
"over-budget output must be strictly smaller than the untrimmed snapshot + instruction"
);
assert!(matches!(out.first(), Some(ConversationItem::System(_))));
assert!(matches!(out.last(), Some(ConversationItem::User(_))));
}
#[test]
fn budget_over_budget_drops_orphan_tool_result_at_front() {
// The Assistant(tool_use) is heavy and gets excluded by the front-trim;
// its ToolResult would then be a leading orphan — which must be dropped.
let conv = vec![
ConversationItem::system("sys"),
ConversationItem::assistant_tool_calls(vec![mk_tool_call("c1", &"b".repeat(40_000))]),
ConversationItem::tool_result("c1", "small result"),
ConversationItem::user("recent"),
];
let out = budget_recap_items(conv, "system-reminder", false, 8_000);
let first_non_system = out
.iter()
.find(|i| !matches!(i, ConversationItem::System(_)));
assert!(
!matches!(first_non_system, Some(ConversationItem::ToolResult(_))),
"retained tail must not begin with an orphan ToolResult"
);
}
#[test]
fn budget_over_budget_no_trailing_tool_run_and_keeps_recent_user() {
// Regression guard locking the "normalize trailing boundary BEFORE
// fit_conversation_to_budget" ordering. The trailing ToolResult is sized
// LARGER than the budget on purpose: with the WRONG order (fit-then-pop),
// `fit` sees the lone giant tool tail, truncates it in place, and drops
// the most-recent real user turn — so assertion (a) fails. With the
// correct order (pop-then-fit) the trailing tool run is removed first, so
// the recent user turn is what survives the front-trim.
let conv = vec![
ConversationItem::system("sys"),
ConversationItem::user("c".repeat(40_000)), // oldest real user, dropped
ConversationItem::user("what changed in the parser?"), // most-recent real user
ConversationItem::assistant_tool_calls(vec![mk_tool_call("c9", "{}")]),
ConversationItem::tool_result("c9", "t".repeat(40_000)), // trailing run, > budget
];
let out = budget_recap_items(conv, "system-reminder", false, 8_000);
// (b) No trailing tool run before the appended instruction.
assert!(matches!(out.last(), Some(ConversationItem::User(_))));
let before = &out[out.len() - 2];
assert!(
!matches!(before, ConversationItem::ToolResult(_)),
"no tool_result immediately before the appended instruction"
);
assert!(
!matches!(before, ConversationItem::Assistant(a) if !a.tool_calls.is_empty()),
"no dangling assistant tool_use immediately before the appended instruction"
);
// (a) The most-recent real user turn survives (FAILS under fit-then-pop,
// which would instead keep a truncated lone tool tail).
assert!(
out.iter().any(|i| matches!(
i,
ConversationItem::User(u) if u.content.iter().any(|p| matches!(
p,
kigi_sampling_types::ContentPart::Text { text }
if text.contains("what changed in the parser?")
))
)),
"most-recent real user turn must survive the trim (locks normalize-before-fit)"
);
}
#[test]
fn budget_giant_single_turn_truncated_in_place() {
// A single turn larger than the whole budget must be kept, truncated in
// place — never dropped to an empty request.
let conv = vec![ConversationItem::user("y".repeat(200_000))];
let out = budget_recap_items(conv, "system-reminder", false, 8_000);
assert!(
out.len() >= 2,
"giant turn must be truncated in place, not dropped"
);
assert!(matches!(out.last(), Some(ConversationItem::User(_))));
assert!(estimate_conversation_tokens(&out) <= recap_prompt_budget(8_000));
let serialized = serde_json::to_string(&out).unwrap();
assert!(
serialized.contains("truncated"),
"retained turn must carry the in-place truncation marker"
);
}
#[test]
fn budget_over_budget_strips_reasoning_even_on_grok() {
let conv = vec![
mk_reasoning("r1"),
ConversationItem::assistant("did stuff"),
ConversationItem::user("z".repeat(40_000)),
];
// grok backend => strip_reasoning=false, but the over-budget branch must
// strip reasoning anyway (the prefix cache is already lost once trimmed).
let out = budget_recap_items(conv, "system-reminder", false, 8_000);
assert!(
!out.iter()
.any(|i| matches!(i, ConversationItem::Reasoning(_))),
"over-budget branch must strip reasoning even when strip_reasoning=false"
);
}
#[test]
fn budget_fast_path_keeps_reasoning_on_grok() {
let conv = vec![
mk_reasoning("r1"),
ConversationItem::assistant("did stuff"),
ConversationItem::user("small"),
];
// Fits under a large window on grok (strip_reasoning=false) => verbatim,
// reasoning kept so the prefix KV cache stays warm.
let out = budget_recap_items(conv, "system-reminder", false, 256_000);
assert!(
out.iter()
.any(|i| matches!(i, ConversationItem::Reasoning(_))),
"fits path on grok must keep reasoning verbatim"
);
}
#[test]
fn budget_1m_clamps_to_floor_and_256k_shrinks() {
// One giant user turn larger than any window's budget: fit truncates it in
// place to exactly snapshot_budget, so the output size is a direct readout
// of the budget the helper used.
let giant = || {
vec![
ConversationItem::system("sys"),
ConversationItem::user("x".repeat(2_400_000)),
]
};
let out_500 = budget_recap_items(giant(), "system-reminder", false, 500_000);
let out_1m = budget_recap_items(giant(), "system-reminder", false, 1_000_000);
let out_256 = budget_recap_items(giant(), "system-reminder", false, 256_000);
// 1M advertises a larger window but clamps to the 500k floor => identical.
assert_eq!(
estimate_conversation_tokens(&out_1m),
estimate_conversation_tokens(&out_500),
"1M must clamp to the 500k floor (identical budget)"
);
assert!(estimate_conversation_tokens(&out_500) <= recap_prompt_budget(500_000));
// 256k is below the floor => strictly smaller budget and output.
assert!(
estimate_conversation_tokens(&out_256) < estimate_conversation_tokens(&out_500),
"256k must produce a smaller budget than the 500k floor"
);
assert!(estimate_conversation_tokens(&out_256) <= recap_prompt_budget(256_000));
}
#[test]
fn pop_trailing_removes_tool_run_keeps_clean_tail() {
let mut items = vec![
ConversationItem::user("hi"),
ConversationItem::assistant_tool_calls(vec![mk_tool_call("c1", "{}")]),
ConversationItem::tool_result("c1", "out"),
];
pop_trailing_tool_run(&mut items);
assert_eq!(items.len(), 1);
assert!(matches!(items[0], ConversationItem::User(_)));
let mut clean = vec![
ConversationItem::user("hi"),
ConversationItem::assistant("done"),
];
pop_trailing_tool_run(&mut clean);
assert_eq!(clean.len(), 2, "a clean (non-tool) tail is left untouched");
}
#[test]
fn budget_empty_conversation_returns_only_instruction() {
// The helper is directly reachable (the handler gates `vec![]` upstream);
// an empty snapshot must return just the appended instruction, no panic.
let out = budget_recap_items(Vec::new(), "system-reminder", false, 256_000);
assert_eq!(out.len(), 1, "empty input yields only the instruction turn");
assert!(matches!(out.last(), Some(ConversationItem::User(_))));
}
#[test]
fn budget_threshold_boundary_selects_fast_vs_over_budget() {
// Lock the `<=` fits-vs-over-budget comparison. At exactly snapshot_budget
// the fast path is taken (reasoning kept, `strip_reasoning=false`); one
// token over takes the over-budget path (reasoning stripped). The
// reasoning item's presence is the observable branch discriminator.
let tag = "system-reminder";
let instruction_tokens =
estimate_item_tokens(&ConversationItem::user(recap_instruction(tag)));
let snapshot_budget = recap_prompt_budget(8_000).saturating_sub(instruction_tokens);
assert!(snapshot_budget > 8, "window must leave room for the probe");
let reasoning_tokens = estimate_item_tokens(&mk_reasoning("r"));
let filler_tokens = snapshot_budget - reasoning_tokens;
// Exactly at budget => fast path (`<=`) keeps reasoning verbatim.
let at = vec![
mk_reasoning("r"),
ConversationItem::user("a".repeat((filler_tokens * 4) as usize)),
];
assert_eq!(estimate_conversation_tokens(&at), snapshot_budget);
let out_at = budget_recap_items(at, tag, false, 8_000);
assert!(
out_at
.iter()
.any(|i| matches!(i, ConversationItem::Reasoning(_))),
"exactly-at-budget must take the fast path (reasoning kept), locking `<=`"
);
// One token over => over-budget path strips reasoning.
let over = vec![
mk_reasoning("r"),
ConversationItem::user("a".repeat((filler_tokens * 4 + 4) as usize)),
];
assert!(estimate_conversation_tokens(&over) > snapshot_budget);
let out_over = budget_recap_items(over, tag, false, 8_000);
assert!(
!out_over
.iter()
.any(|i| matches!(i, ConversationItem::Reasoning(_))),
"one token over budget must take the over-budget path (reasoning stripped)"
);
}
#[test]
fn budget_degenerate_tiny_window_stays_valid_and_nonempty() {
// Window below the headroom => prompt_budget saturates to 0. The
// instruction is still appended, so the output necessarily exceeds the
// computed 0 budget but stays tiny and structurally valid (cannot cause a
// 400). Asserts graceful degradation — NOT `est <= budget` (documents the
// informational degenerate-window behavior).
let conv = vec![
ConversationItem::system("sys"),
ConversationItem::user("w".repeat(40_000)),
];
let out = budget_recap_items(conv, "system-reminder", false, 1_000);
assert!(
!out.is_empty(),
"a degenerate tiny window must still return a valid, non-empty request"
);
assert!(matches!(out.last(), Some(ConversationItem::User(_))));
}
}
@@ -0,0 +1,252 @@
//! Session title generation via LLM tool call.
use crate::sampling::{
Client as OaiCompatClient, ConversationItem, ConversationRequest, ConversationToolChoice,
ToolSpec,
};
use crate::session::helpers::chat::floor_char_boundary;
/// Upper bound on the user text that feeds title generation; titles only need
/// the opening, and this keeps the request well under the model prompt limit.
const TITLE_SOURCE_MAX_BYTES: usize = 8_000;
#[derive(serde::Deserialize)]
struct SessionTitle {
session_title: String,
}
/// Remove `<system-reminder>…</system-reminder>` blocks from `text` — they are
/// system-injected context (e.g. the `/goal` setup reminder), not the user's
/// words, so they must not drive the session title.
fn strip_system_reminder_blocks(text: &str) -> String {
const OPEN: &str = "<system-reminder>";
const CLOSE: &str = "</system-reminder>";
let mut out = String::with_capacity(text.len());
let mut rest = text;
while let Some(start) = rest.find(OPEN) {
out.push_str(&rest[..start]);
let after_open = &rest[start + OPEN.len()..];
// An unterminated reminder drops the remainder — it is system text.
let Some(end) = after_open.find(CLOSE) else {
return out.trim().to_string();
};
rest = &after_open[end + CLOSE.len()..];
}
out.push_str(rest);
out.trim().to_string()
}
/// Text the session title is derived from: strip system reminders and skill XML
/// markup, then cap to the first few KB. Stripping runs before the cap so a
/// leading reminder larger than the cap is still removed.
fn title_source_text(user_message: &str) -> String {
let without_reminders = strip_system_reminder_blocks(user_message);
let base = if without_reminders.is_empty() {
user_message
} else {
&without_reminders
};
let mut display = kigi_tools::implementations::skills::skill::extract_skill_display_text(base)
.unwrap_or_else(|| base.to_string());
display.truncate(floor_char_boundary(&display, TITLE_SOURCE_MAX_BYTES));
display
}
pub(crate) fn title_fallback_from_user_text(user_message: &str) -> String {
let text = title_source_text(user_message);
let s = text
.split_whitespace()
.take(10)
.collect::<Vec<_>>()
.join(" ");
if s.is_empty() {
"New session".to_string()
} else {
s
}
}
/// Generates a title for the session by looking at the first user message
/// We do not generate more of it on next user message unless its very important
///
/// Ideally we should be updating it as the session continues, but ... skipping that for now
pub async fn generate_session_summary(
user_message: String,
client: OaiCompatClient,
model: &str,
) -> String {
let clean_message = title_source_text(&user_message);
let request = ConversationRequest::from_items(vec![
ConversationItem::system(
r#"You are tasked with generating the session title. The user is asking almost always software engineering related questions on their codebase.
We describe the session title below
# Session Title
A short and distinctive 5-10 word descriptive title for the session. Super info dense, no filler.
You will be given the user query below encapsulated in <user_query></user_query>.
Just generate the session_title and nothing else"#,
),
ConversationItem::user(format!(
r#"<user_query>
{}
</user_query>"#,
clean_message
)),
])
.with_model(model)
.with_tools(vec![ToolSpec {
name: "session_title".to_owned(),
description: Some("Generate the session_title which we use for the user_message".to_owned()),
parameters: serde_json::json!({
"type": "object",
"required": ["session_title"],
"properties": {
"session_title": {
"type": "string",
"description": "Final session title, just 5-10 word descriptive title for the session. Super info dense, no filler."
}
},
"additionalProperties": false
}),
}])
.with_max_output_tokens(100)
.with_temperature(1.0)
.with_tool_choice(ConversationToolChoice::Function("session_title".to_owned()));
match client.conversation_collect(request).await {
Ok(response) => {
if let Some(a) = response.assistant()
&& let Some(tool_call) = a.tool_calls.first()
&& let Ok(result) = serde_json::from_str::<SessionTitle>(&tool_call.arguments)
{
return result.session_title;
}
tracing::debug!(
model = %model,
"session title generation: response did not contain a session_title tool call"
);
}
Err(e) => {
tracing::warn!(
model = %model,
error = %e,
"session title generation failed, falling back to truncated user text"
);
}
}
title_fallback_from_user_text(&clean_message)
}
#[cfg(test)]
mod tests {
use super::{
TITLE_SOURCE_MAX_BYTES, strip_system_reminder_blocks, title_fallback_from_user_text,
title_source_text,
};
#[test]
fn title_source_text_caps_oversized_input() {
let big = "word ".repeat(10_000);
let out = title_source_text(&big);
assert!(!out.is_empty() && out.len() <= TITLE_SOURCE_MAX_BYTES);
}
#[test]
fn title_source_text_cap_is_utf8_safe() {
// 3-byte chars straddle the byte cap; must truncate on a boundary, not panic.
let big = "".repeat(10_000);
let out = title_source_text(&big);
assert!(!out.is_empty() && out.len() <= TITLE_SOURCE_MAX_BYTES);
}
#[test]
fn title_source_text_strips_leading_reminder_larger_than_cap() {
// A leading reminder bigger than the cap must still be stripped, so the
// title derives from the objective rather than reminder text.
let reminder = "x".repeat(TITLE_SOURCE_MAX_BYTES * 2);
let input =
format!("<system-reminder>\n{reminder}\n</system-reminder>\n\nbuild a mario game");
let out = title_source_text(&input);
assert_eq!(out, "build a mario game");
}
#[test]
fn strip_removes_goal_setup_reminder_leaving_objective() {
let input = "<system-reminder>\nA goal has been set: do stuff\nlots of rules\nStart \
now.\n</system-reminder>\n\nbuild a mario platformer game";
assert_eq!(
strip_system_reminder_blocks(input),
"build a mario platformer game"
);
}
#[test]
fn strip_handles_unterminated_reminder() {
assert_eq!(
strip_system_reminder_blocks("<system-reminder>\nrules with no close tag"),
""
);
}
#[test]
fn strip_no_reminder_is_identity() {
assert_eq!(
strip_system_reminder_blocks("fix the auth bug"),
"fix the auth bug"
);
}
/// Regression: a `/goal <objective>` first turn must title off the
/// objective, not the injected `<system-reminder>` setup block.
#[test]
fn fallback_titles_off_goal_objective_not_reminder() {
let input = "<system-reminder>\nA goal has been set: do stuff\nStart \
now.\n</system-reminder>\n\nbuild a mario platformer game in html";
assert_eq!(
title_fallback_from_user_text(input),
"build a mario platformer game in html"
);
}
#[test]
fn fallback_trims_to_words() {
assert_eq!(
title_fallback_from_user_text(
"one two three four five six seven eight nine ten eleven"
),
"one two three four five six seven eight nine ten"
);
}
#[test]
fn fallback_new_session_when_whitespace_only() {
assert_eq!(title_fallback_from_user_text(" \n\t"), "New session");
}
#[test]
fn fallback_strips_skill_xml_with_args() {
let input = "<command-name>implement</command-name>\n\
<command-message>/implement</command-message>\n\
<command-args>fix the rendering bug</command-args>";
assert_eq!(
title_fallback_from_user_text(input),
"/implement fix the rendering bug",
);
}
#[test]
fn fallback_strips_skill_xml_no_args() {
let input = "<command-name>deploy</command-name>\n\
<command-message>/deploy</command-message>";
assert_eq!(title_fallback_from_user_text(input), "/deploy");
}
#[test]
fn fallback_plain_text_unaffected() {
assert_eq!(
title_fallback_from_user_text("fix the auth bug in login.rs"),
"fix the auth bug in login.rs",
);
}
}
@@ -0,0 +1,174 @@
pub fn try_extract_concatenated_json_objects(arguments: &str) -> Option<Vec<serde_json::Value>> {
let trimmed = arguments.trim();
// Quick check: must start with '{'.
if !trimmed.starts_with('{') {
return None;
}
// If it parses as valid JSON already, no recovery needed.
if serde_json::from_str::<serde_json::Value>(trimmed).is_ok() {
return None;
}
// Use serde_json::StreamDeserializer to parse concatenated JSON objects.
// This handles nested braces correctly (unlike naive string splitting on "}{").
let stream = serde_json::Deserializer::from_str(trimmed).into_iter::<serde_json::Value>();
let mut objects = Vec::new();
for result in stream {
match result {
Ok(value) if value.is_object() => objects.push(value),
_ => break,
}
}
// Need at least 2 objects for this to be concatenated JSON.
if objects.len() >= 2 {
Some(objects)
} else {
None
}
}
/// Normalize empty tool call arguments to `"{}"`.
///
/// Zero-arg MCP tools (e.g. `get_me`) sometimes receive `""` from the model
/// instead of `"{}"`, which fails JSON parsing. This normalizes empty/whitespace
/// strings to `"{}"` so downstream parsing succeeds.
pub fn normalize_empty_arguments(arguments: &str) -> &str {
if arguments.trim().is_empty() {
"{}"
} else {
arguments
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_extract_objects() {
let args = r#"{"target_file": "a.java"}{"target_file": "b.java"}{"target_file": "c.java"}"#;
let objects = try_extract_concatenated_json_objects(args).unwrap();
assert_eq!(objects.len(), 3);
assert_eq!(objects[0]["target_file"], "a.java");
}
#[test]
fn test_no_extract_for_valid_single_object() {
assert!(
try_extract_concatenated_json_objects(r#"{"target_file": "src/main.rs"}"#).is_none()
);
}
#[test]
fn test_no_extract_for_valid_object_with_braces_in_value() {
assert!(
try_extract_concatenated_json_objects(r#"{"command": "echo '}{' && ls"}"#).is_none()
);
}
#[test]
fn test_no_extract_for_array() {
assert!(
try_extract_concatenated_json_objects(
r#"[{"target_file": "a.java"}, {"target_file": "b.java"}]"#
)
.is_none()
);
}
#[test]
fn test_no_extract_for_empty_or_non_json() {
assert!(try_extract_concatenated_json_objects("").is_none());
assert!(try_extract_concatenated_json_objects("not json").is_none());
}
#[test]
fn test_extract_with_nested_braces() {
let args = r#"{"file": "a.rs", "opts": {"line": 1}}{"file": "b.rs", "opts": {"line": 2}}"#;
let objects = try_extract_concatenated_json_objects(args).unwrap();
assert_eq!(objects.len(), 2);
assert_eq!(objects[0]["opts"]["line"], 1);
}
#[test]
fn test_extract_with_whitespace_between_objects() {
let objects = try_extract_concatenated_json_objects(r#"{"a": 1} {"b": 2}"#).unwrap();
assert_eq!(objects.len(), 2);
}
#[test]
fn test_extract_real_world_20_files() {
let mut args = String::new();
for i in 0..20 {
args.push_str(&format!(r#"{{"target_file": "src/File{i}.java"}}"#));
}
let objects = try_extract_concatenated_json_objects(&args).unwrap();
assert_eq!(objects.len(), 20);
}
#[test]
fn test_no_extract_for_truncated_json() {
assert!(try_extract_concatenated_json_objects(r#"{"a": 1} garbage"#).is_none());
}
/// Parse after normalizing — mirrors the production pattern in handle_tool_call.
fn normalize_and_parse(arguments: &str) -> serde_json::Value {
let normalized = normalize_empty_arguments(arguments);
serde_json::from_str(normalized).unwrap_or_else(|_| serde_json::json!({"raw": arguments}))
}
#[test]
fn empty_string_becomes_empty_object() {
assert_eq!(normalize_and_parse(""), serde_json::json!({}));
}
#[test]
fn whitespace_only_becomes_empty_object() {
assert_eq!(normalize_and_parse(" "), serde_json::json!({}));
assert_eq!(normalize_and_parse("\n\t"), serde_json::json!({}));
}
#[test]
fn valid_json_unchanged() {
assert_eq!(
normalize_and_parse(r#"{"query": "test"}"#),
serde_json::json!({"query": "test"})
);
}
#[test]
fn empty_object_string_unchanged() {
assert_eq!(normalize_and_parse("{}"), serde_json::json!({}));
}
#[test]
fn invalid_json_falls_back_to_raw() {
let result = normalize_and_parse("not json");
assert_eq!(result["raw"], "not json");
}
#[test]
fn complex_args_with_arrays_unchanged() {
let args = r#"{"pages": [{"title": "Test"}], "limit": 10}"#;
let result = normalize_and_parse(args);
assert!(result["pages"].is_array());
assert_eq!(result["limit"], 10);
}
#[test]
fn normalize_empty_returns_braces() {
assert_eq!(normalize_empty_arguments(""), "{}");
assert_eq!(normalize_empty_arguments(" "), "{}");
assert_eq!(normalize_empty_arguments("\n\t"), "{}");
}
#[test]
fn normalize_non_empty_passthrough() {
assert_eq!(normalize_empty_arguments(r#"{"a":1}"#), r#"{"a":1}"#);
assert_eq!(normalize_empty_arguments("not json"), "not json");
}
}