M0: compilable skeleton — Kigi 0.1.0 fork surgery

Hard fork of xai-org/grok-build (Apache-2.0) re-targeted as Kigi, an
unofficial Kimi Code CLI community build.

Rename & identity
- 72 xai-*/xai-grok-* crates -> kigi-* (explicit: xai-grok-pager-bin ->
  kigi-bin [binary `kigi`], xai-grok-pager -> kigi-tui; rest mechanical);
  ptyctl, ptyctl-cli, third_party/ unchanged; proto package
  xai.grok.tools.v1 -> kigi.tools.v1
- Config home ~/.kigi (KIGI_SHARE_DIR override), env prefix GROK_* ->
  KIGI_*, `kigi --version` carries the unofficial-community-build notice
- clap identity, help text, startup banner, prompt templates rebranded
  (templates re-encrypted)

Deletions (PRD removal list #5/#6/#7/#9/#10)
- voice input (xai-grok-voice) and all TUI wiring
- telemetry: Mixpanel client, external OTel stream, Sentry, OTLP layers,
  trace/GCS/S3 upload queues (kigi-file-utils halved), workspace upload
  module & dc_log, heap-profile uploader, auth-diagnostics uploader,
  session-analytics halves of feedback; local zero-egress observability
  preserved in new kigi-log crate (unified log, --debug firehose,
  subsystem file logs, opt-in instrumentation)
- announcements (crate, remote-settings fields, TUI surfaces)
- plugin marketplace (crate, sources/browse/CTA/extensions-modal tab);
  direct plugin install/uninstall/update via kigi-agent git_install kept
- relay/gateway/assets endpoints and features (agent relay, headless
  relay transport, gateway bridge, LeaderEnvUrls); leader IPC socket now
  ~/.kigi/leader.sock + KIGI_LEADER_SOCKET, no ws-url derivation
- functional types rehomed instead of deleted: PermissionMode ->
  kigi-config-types, McpInitStrategy -> kigi-mcp, PrCreationSource ->
  session signals, TerminalDiagnostics -> kigi-pager-render, agent_id ->
  shell util

Endpoints
- kigi-env rewritten: single production KigiEndpoints {coding_api_base_url
  https://api.kimi.com/coding/v1 (KIGI_CODE_BASE_URL), oauth_host
  https://auth.kimi.com (KIGI_OAUTH_HOST), update_base_url (GitHub
  Releases API), upgrade_page_url}; GrokBuildEnvironment enum deleted

Toolchain & workspace hygiene
- Rust 1.97.0 pinned; edition 2024; full cargo update; git2 hoisted to
  workspace at 0.21 (Option->Result API migration), quick-xml 0.41
- Root Cargo.toml hand-maintained (PRD §8.1): version 0.1.0 inherited by
  all members, members sorted, unused deps pruned
- cargo-deny advisories gate (deny.toml with documented transitive
  exceptions); CI workflow (check/clippy/fmt/deny/test, macOS+Linux)
- cross-crate test seams re-gated behind `test-support` cargo feature;
  insta snapshot baselines renamed to the kigi_tui prefix
- clippy --workspace --all-targets: zero warnings; fmt clean

Fixes surfaced by the port
- updater probe/installer divergence (bin/kigi vs bin/grok symlink set)
- idle model-metadata refresh dead under KIGI_CODE_BASE_URL override
  (new is_effective_coding_endpoint_url, loopback+override aware)
- macOS symlinked-TMPDIR fixture canonicalization (foreign_sessions,
  fast-worktree); RSS measurement tests serialized via serial_test

Docs & legal (Apache §4)
- NOTICE added (upstream attribution + change statement); THIRD-PARTY
  notices sustained; kigi-tools ported-code notices extended; README,
  CONTRIBUTING, SECURITY, AGENTS.md rewritten

Out of scope for M0 (tracked): Kimi auth/inference (M1), search/fetch,
command parity, config import (M2), Computer Hub excision & final
brand-token sweep (M2), distribution & self-update rewrite (M3).
This commit is contained in:
ZacharyZhang-NY committed 2026-07-17 05:31:01 -04:00
commit d6c20fc13f
2612 files changed
+1353757

No files matched your search

@@ -0,0 +1,103 @@
//! 401 attribution: callback hook + shared helpers for tool HTTP clients.
use std::sync::Arc;
/// Bearer prefix length shared across crate boundaries.
pub const SENT_BEARER_PREFIX_LEN: usize = 12;
/// Which tool endpoint produced the 401.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ToolConsumer {
ImageGen,
VideoGenStart,
VideoGenPoll,
WebSearch,
}
impl ToolConsumer {
pub fn as_str(self) -> &'static str {
match self {
Self::ImageGen => "ImageGen",
Self::VideoGenStart => "VideoGen.start",
Self::VideoGenPoll => "VideoGen.poll",
Self::WebSearch => "WebSearch",
}
}
}
/// 401 attribution callback. Shell wires this to emit telemetry.
pub trait Auth401AttributionCallback: Send + Sync + std::fmt::Debug {
/// `sent_bearer_prefix` is truncated to [`SENT_BEARER_PREFIX_LEN`]
/// before crossing this boundary. `None` = no bearer was sent.
fn record_401(&self, consumer: ToolConsumer, sent_bearer_prefix: Option<&str>);
}
/// Shared, cheap-to-clone alias for the attribution callback.
pub type SharedAttributionCallback = Arc<dyn Auth401AttributionCallback>;
/// Record a 401 attribution event if a callback is wired. Truncates
/// the bearer to [`SENT_BEARER_PREFIX_LEN`] before crossing the
/// trait boundary.
pub(crate) fn emit_401(
callback: Option<&SharedAttributionCallback>,
consumer: ToolConsumer,
sent_bearer: Option<&str>,
) {
if let Some(cb) = callback {
let prefix = sent_bearer.map(|s| truncate_to_prefix(s.to_string()));
cb.record_401(consumer, prefix.as_deref());
}
}
/// Truncate a bearer string to the first [`SENT_BEARER_PREFIX_LEN`]
/// characters. Used by tool clients before passing the bearer across
/// the [`Auth401AttributionCallback`] boundary.
///
/// Bearer tokens are ASCII (per the `Authorization` header grammar)
/// so the byte index is always a char boundary; this function uses
/// `String::truncate` which would otherwise panic on a non-boundary
/// cut.
pub(crate) fn truncate_to_prefix(mut bearer: String) -> String {
bearer.truncate(SENT_BEARER_PREFIX_LEN.min(bearer.len()));
bearer
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn truncate_to_prefix_long_string_cuts_at_12() {
assert_eq!(
truncate_to_prefix("xai-key-aaaaaaaaaaaaaaaaaaa".to_string()),
"xai-key-aaaa"
);
}
#[test]
fn truncate_to_prefix_short_string_unchanged() {
assert_eq!(truncate_to_prefix("abc".to_string()), "abc");
}
#[test]
fn truncate_to_prefix_exact_12_unchanged() {
assert_eq!(
truncate_to_prefix("123456789012".to_string()),
"123456789012"
);
assert_eq!(truncate_to_prefix("123456789012".to_string()).len(), 12);
}
#[test]
fn truncate_to_prefix_empty_unchanged() {
assert_eq!(truncate_to_prefix(String::new()), "");
}
#[test]
fn tool_consumer_as_str_stable_identifiers() {
assert_eq!(ToolConsumer::ImageGen.as_str(), "ImageGen");
assert_eq!(ToolConsumer::VideoGenStart.as_str(), "VideoGen.start");
assert_eq!(ToolConsumer::VideoGenPoll.as_str(), "VideoGen.poll");
assert_eq!(ToolConsumer::WebSearch.as_str(), "WebSearch");
}
}
+833
View File
@@ -0,0 +1,833 @@
//! ToolBridge: adapter that wraps `kigi-tools`'s `ToolRegistry` and
//! exposes it through a session layer.
//!
//! The bridge:
//! 1. Owns a `ToolRegistry` with all built-in tools registered
//! 2. Dispatches tool calls via `call_new_tool()`
//! 3. Manages tool definitions, enable/disable, name overrides
use std::sync::Arc;
use crate::computer::types::KillOutcome;
use crate::computer::types::TaskKind;
use crate::computer::types::TerminalBackend;
use crate::registry::types::{
FinalizedToolset, SessionContext, ToolRegistryBuilder, ToolServerConfig,
};
use crate::types::TaskSnapshot;
use crate::types::ToolInput;
use crate::types::agents_md_tracker::AgentsMdTracker;
use crate::types::definition::ToolDefinition;
use crate::types::output::{ToolOutput, ToolRunResult};
use crate::types::resources::{OwnerSessionId, State, Terminal};
use crate::types::template_renderer::TemplateRenderer;
use crate::types::tool::ToolKind;
/// Result of executing a tool through the bridge.
///
/// Carries all the data the session needs to:
/// 1. Send ACP notifications (from `output`)
/// 2. Build the model prompt (from `prompt_text`)
#[derive(Debug)]
pub struct ToolBridgeResult {
/// Clean tool output — for JSON serialization, ACP conversion, hunk tracking.
pub output: ToolOutput,
/// Prompt-ready text — with system reminders appended.
pub prompt_text: String,
}
impl From<ToolRunResult> for ToolBridgeResult {
fn from(result: ToolRunResult) -> Self {
Self {
output: result.output,
prompt_text: result.prompt_text,
}
}
}
/// Bridges the `ToolRegistry` into a session layer.
///
/// Owns the registry and dispatches tool calls via `call_new_tool()`.
/// All state lives in `Resources` on the registry — no separate `ToolState`.
///
/// # Cancellation Safety
///
/// The `terminal` field is stored separately from the registry lock to enable
/// cancellation during tool execution. When a bash command is running, the
/// registry lock is held by `call()`. If the user cancels, `kill_foreground_commands()`
/// needs to access the terminal without blocking on the lock.
#[derive(Clone)]
pub struct ToolBridge {
registry: Arc<FinalizedToolset>,
terminal: Option<Arc<dyn TerminalBackend>>,
}
impl ToolBridge {
pub fn get_builder() -> ToolRegistryBuilder {
ToolRegistryBuilder::new()
}
pub async fn finalize_builder(
builder: ToolRegistryBuilder,
config: ToolServerConfig,
ctx: SessionContext,
) -> Result<Self, kigi_tool_runtime::ToolError> {
let finalized_toolset = builder.finalize(config, ctx).map_err(|errs| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Requirements unsatisfied: {errs:?}"
))
})?;
let terminal;
{
terminal = finalized_toolset
.resources
.lock()
.await
.get::<Terminal>()
.map(|t| t.0.clone());
}
Ok(Self {
registry: Arc::new(finalized_toolset),
terminal,
})
}
pub async fn tool_definitions(&self) -> Vec<ToolDefinition> {
self.registry.tool_definitions()
}
/// Returns the client-facing name of the tool registered with the given
/// `ToolKind`, if any. Looks up the kind->name map populated by
/// `FinalizedToolset` from each tool's `kind()`. Useful for "does this
/// agent have a way to do X?" checks where the X is identified by kind
/// rather than by namespaced id.
///
/// Example: `tool_for_kind(ToolKind::BackgroundTaskAction)` returns
/// `Some("get_task_output")` for the grok_build agent and `None` for
/// agents that do not register a tool of that kind.
pub async fn tool_for_kind(&self, kind: ToolKind) -> Option<String> {
self.registry
.resources
.lock()
.await
.get::<TemplateRenderer>()
.and_then(|r| r.tool_for_kind(kind).map(str::to_string))
}
/// [`ToolKind`] for a registered tool by client-facing name, or
/// `None` for unknown names. Sync — uses the registry's
/// `RwLock::read`.
pub fn tool_kind(&self, tool_name: &str) -> Option<ToolKind> {
self.registry.get_tool_metadata(tool_name).map(|m| m.kind())
}
/// Get only built-in tool definitions (exclude MCP tools).
pub async fn tool_definitions_builtins_only(&self) -> Vec<ToolDefinition> {
self.registry.tool_definitions_builtins_only()
}
/// Render a prompt template through [`TemplateRenderer`] with extra
/// agent-specific context fields.
///
/// The template can use both `${{ tools.by_kind.* }}` (resolved from
/// the finalized tool registry) and caller-provided fields like
/// `${{ os_name }}`, `${{ memory_enabled }}`, etc.
///
/// Returns `None` if the renderer is not yet available.
pub async fn render_prompt(
&self,
template: &str,
placeholders: &serde_json::Value,
) -> Option<String> {
let registry = &*self.registry;
let result;
{
result = registry
.resources
.lock()
.await
.get::<TemplateRenderer>()
.and_then(|r| r.render_with_extra(template, placeholders).ok());
}
result
}
pub async fn register_mcp_tools<T>(
&self,
mcp_name: String,
tool: T,
input_schema: Option<serde_json::Value>,
) -> Result<(), kigi_tool_runtime::ToolError>
where
T: kigi_tool_runtime::Tool
+ crate::types::tool_metadata::ToolMetadata
+ std::fmt::Debug
+ Send
+ Sync
+ 'static,
T::Output: serde::Serialize,
{
self.registry.register_tool(mcp_name, tool, input_schema)?;
Ok(())
}
pub fn unregister_tools_by_prefix(&self, prefix: &str) -> usize {
self.registry.unregister_tools_by_prefix(prefix)
}
pub fn unregister_tool_by_name(&self, name: &str) -> bool {
self.registry.unregister_tool_by_name(name)
}
/// Access the underlying `FinalizedToolset`.
///
/// Used by `WorkspaceOps::bind_local_session` to install the agent's
/// toolset on the workspace session so local-mode tool calls dispatch
/// through the workspace.
pub fn toolset(&self) -> Arc<FinalizedToolset> {
Arc::clone(&self.registry)
}
pub async fn call(
&self,
client_function_name: &str,
client_params: serde_json::Value,
tool_call_id: &str,
) -> Result<ToolRunResult, kigi_tool_runtime::ToolError> {
self.registry
.call(client_function_name, client_params, tool_call_id, None)
.await
}
pub async fn try_parse(
&self,
client_function_name: &str,
client_params: serde_json::Value,
) -> Result<ToolInput, kigi_tool_runtime::ToolError> {
self.registry
.try_parse(client_function_name, &client_params)
.await
}
/// Seed the AGENTS.md tracker.
///
/// `compat` gates which rules dirs and agent filenames runtime discovery
/// scans. Defaults to all-on at the caller for historical behavior.
pub async fn seed_agents_md(
&self,
initial_paths: Vec<std::path::PathBuf>,
git_root: Option<std::path::PathBuf>,
initial_chain: Vec<std::path::PathBuf>,
gitignore: Option<ignore::gitignore::Gitignore>,
compat: crate::types::compat::CompatConfig,
) {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
if let Some(tracker) = res.get_mut::<AgentsMdTracker>() {
tracker.set_compat(compat);
tracker
.seed(initial_paths, git_root, initial_chain, gitignore)
.await;
}
}
/// Restore announced skill names from persisted state.
///
/// Must be called BEFORE `seed_skill_discovery()` so that `seed()`
/// sees non-empty `announced_names` and skips the BaselineChange pending.
pub async fn restore_announced_skill_names(&self, names: std::collections::HashSet<String>) {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
let tracker = res.get_or_default::<crate::types::skill_discovery_tracker::SkillManager>();
tracker.restore_announced_names(names);
}
/// Get the current set of announced skill names (for persistence).
pub async fn get_announced_skill_names(&self) -> std::collections::HashSet<String> {
let registry = &*self.registry;
let res = registry.resources.lock().await;
res.get::<crate::types::skill_discovery_tracker::SkillManager>()
.map(|t| t.announced_names().clone())
.unwrap_or_default()
}
/// The model-facing skill listing rendered from the current skill set,
/// for `/context` accounting. `None` when no skill qualifies.
pub async fn skill_listing_snapshot(
&self,
) -> Option<crate::types::skill_discovery_tracker::SkillListingSnapshot> {
let registry = &*self.registry;
let res = registry.resources.lock().await;
res.get::<crate::types::skill_discovery_tracker::SkillManager>()
.and_then(|t| t.listing_snapshot())
}
/// Seed the SkillDiscoveryTracker with session context and startup skills.
///
/// Must be called at session start so the `SkillDiscoveryReminder` can
/// discover skills in subdirectories.
/// `display_cwd`: If set (forked sessions), skill paths in
/// model-visible announcements are rewritten from real cwd to this
/// value. Runtime invocation uses the real path.
pub async fn seed_skill_discovery(
&self,
cwd: Option<std::path::PathBuf>,
git_root: Option<std::path::PathBuf>,
startup_skills: Vec<crate::implementations::skills::types::SkillInfo>,
display_cwd: Option<String>,
context_window_tokens: Option<u64>,
skill_budget_percent: Option<f64>,
compat: crate::types::compat::CompatConfig,
) {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
// Resolve client-facing tool names from the template renderer
// so listing headers and descriptions use the correct (possibly randomized) names.
let renderer = res.get::<TemplateRenderer>();
let skill_tool_name = renderer.and_then(|r| r.render("${{ tools.by_kind.skill }}").ok());
let read_tool_name = renderer.and_then(|r| r.render("${{ tools.by_kind.read }}").ok());
let tracker = res.get_or_default::<crate::types::skill_discovery_tracker::SkillManager>();
if let Some(name) = skill_tool_name {
tracker.set_skill_tool_name(name);
}
if let Some(name) = read_tool_name {
tracker.set_read_tool_name(name);
}
tracker.set_compat(compat);
tracker.seed(
cwd,
git_root,
startup_skills,
display_cwd,
context_window_tokens,
skill_budget_percent,
);
}
/// Enable XML formatting for mid-session skill announcements.
///
/// When set, `take_pending()` produces `<agent_skill>` XML rows instead of
/// markdown, matching the startup `<agent_skills>` preamble format.
pub async fn set_skill_listing_xml_format(&self, enabled: bool) {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
if let Some(tracker) = res.get_mut::<crate::types::skill_discovery_tracker::SkillManager>()
{
tracker.set_xml_format(enabled);
}
}
/// Seed the gitignore filter so `read_file` and `search_replace` refuse
/// to access gitignored paths (matching `list_dir`/`grep` behavior).
pub async fn seed_gitignore_filter(
&self,
gitignore: ignore::gitignore::Gitignore,
git_root: std::path::PathBuf,
) {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
res.insert(crate::types::resources::GitignoreFilter::new(
gitignore, git_root,
));
}
pub async fn on_agents_md_compaction(&self) {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
if let Some(tracker) = res.get_mut::<AgentsMdTracker>() {
tracker.on_compaction();
}
}
/// Clear `announced_names` and `checked_dirs` so skills get re-announced
/// and re-discovered after compaction.
pub async fn on_skill_discovery_compaction(&self) {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
if let Some(tracker) = res.get_mut::<crate::types::skill_discovery_tracker::SkillManager>()
{
tracker.on_compaction();
}
}
/// Full reset of skill discovery state for /clear.
/// Startup baseline is preserved; a pending reconciliation is queued.
pub async fn on_skill_discovery_clear(&self) {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
if let Some(tracker) = res.get_mut::<crate::types::skill_discovery_tracker::SkillManager>()
{
tracker.on_clear();
}
}
/// Replace the startup baseline (plugin reload).
/// Dynamic discoveries are preserved; a pending reconciliation is queued.
pub async fn update_skill_baseline(
&self,
new_skills: Vec<crate::implementations::skills::types::SkillInfo>,
) {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
if let Some(tracker) = res.get_mut::<crate::types::skill_discovery_tracker::SkillManager>()
{
tracker.update_startup_baseline(new_skills);
}
}
/// Apply any pending skill updates.
///
/// If the tracker has a pending change (discovery, baseline update, /clear),
/// this method:
/// 1. Computes runtime and display projections internally.
/// 2. Writes the runtime projection into `AvailableSkills` in Resources.
/// 3. Returns `SkillUpdateEffects` with conversation/UI side-effects
/// for the session to execute (system-reminder injection, slash
/// command refresh, prompt finalization).
///
/// Returns `None` if nothing changed.
pub async fn apply_pending_skill_update(
&self,
) -> Option<crate::types::skill_discovery_tracker::SkillUpdateEffects> {
let registry = &*self.registry;
let mut res = registry.resources.lock().await;
let tracker = res.get_mut::<crate::types::skill_discovery_tracker::SkillManager>()?;
let (runtime_skills, effects) = tracker.take_pending()?;
// Write the runtime projection directly -- the shell never sees this.
res.insert(crate::types::resources::AvailableSkills(runtime_skills));
Some(effects)
}
/// Get the current display-deduped skill list for slash commands.
///
/// Returns the combined (startup + discovered) list with canonical-path
/// and name dedup applied. This is the authoritative source for slash
/// command advertisement — PromptContext is NOT used.
pub async fn slash_skills(&self) -> Vec<crate::implementations::skills::types::SkillInfo> {
let registry = &*self.registry;
let res = registry.resources.lock().await;
res.get::<crate::types::skill_discovery_tracker::SkillManager>()
.map(|m| m.slash_skills())
.unwrap_or_default()
}
/// Get the paths that have been reminded about.
pub async fn agents_md_reminded_paths(&self) -> std::collections::HashSet<std::path::PathBuf> {
let registry = &*self.registry;
let result;
{
result = registry
.resources
.lock()
.await
.get::<AgentsMdTracker>()
.map(|t| t.reminded_paths().clone())
.unwrap_or_default();
}
result
}
/// Set the stable display path for forked sessions.
///
/// Inserts [`DisplayCwd`] into the tool registry's [`Resources`] so that
/// tools can use [`resolve_model_path`] and [`display_cwd_or_cwd`] to
/// rewrite model-provided paths and format output paths correctly.
pub async fn set_display_cwd(&self, display_cwd: std::path::PathBuf) {
let registry = &*self.registry;
registry
.resources
.lock()
.await
.insert(crate::types::resources::DisplayCwd(display_cwd));
}
/// List all known background tasks from the terminal backend.
/// Used for context compaction to include task state in summaries.
pub async fn list_background_tasks(&self) -> Vec<crate::computer::types::TaskSnapshot> {
if let Some(terminal) = &self.terminal {
terminal.list_tasks().await
} else {
vec![]
}
}
/// Kill all foreground terminal commands.
pub async fn kill_foreground_commands(&self) {
if let Some(terminal) = &self.terminal {
terminal.kill_foreground_commands().await;
}
}
/// Kill all running foreground processes owned by a specific session.
pub async fn kill_foreground_commands_by_owner(&self, owner_session_id: &str) {
if let Some(terminal) = &self.terminal {
terminal
.kill_foreground_commands_by_owner(owner_session_id)
.await;
}
}
/// Kill all running background tasks.
pub async fn kill_all_background_tasks(&self) {
if let Some(terminal) = &self.terminal {
terminal.kill_all_background_tasks().await;
}
}
/// Kill all running background tasks owned by a specific session.
/// Used during subagent teardown on a shared terminal backend.
pub async fn kill_all_background_tasks_by_owner(&self, owner_session_id: &str) {
if let Some(terminal) = &self.terminal {
terminal
.kill_all_background_tasks_by_owner(owner_session_id)
.await;
}
}
/// Reparent notification handles for tasks owned by `old_owner_session_id`.
pub async fn reparent_notifications(
&self,
old_owner_session_id: &str,
new_owner_session_id: &str,
new_handle: crate::notification::types::ToolNotificationHandle,
) {
if let Some(terminal) = &self.terminal {
// Weak anchored by this bridge's backend `Arc` (lives as long as the session).
let backend_weak = std::sync::Arc::downgrade(terminal);
terminal
.reparent_notifications(
old_owner_session_id,
new_owner_session_id,
new_handle,
backend_weak,
)
.await;
}
}
/// Read a typed resource from the registry.
///
/// Returns `None` if the resource type has never been inserted.
/// The resource is cloned so no lock is held after this returns.
pub async fn read_resource<T: Clone + Send + Sync + 'static>(&self) -> Option<T> {
self.registry.resources.lock().await.get::<T>().cloned()
}
/// Get the shared resources handle for direct access.
/// Used by the skill reconciliation helper which needs to update
/// `AvailableSkills` in the Resources directly.
pub async fn shared_resources(&self) -> crate::types::resources::SharedResources {
self.registry.resources.clone()
}
/// Insert a typed resource into the registry's `Resources`.
/// Used by the host session to inject `ToolIndex` for search_tool.
pub async fn update_resource<T: Send + Sync + 'static>(&self, resource: T) {
let _ = self.registry.update_resource(resource).await;
}
/// Kill any background task
pub async fn kill_background_task(
&self,
task_id: &str,
) -> Result<KillOutcome, kigi_tool_runtime::ToolError> {
if let Some(terminal) = &self.terminal {
Ok(terminal.kill_task(task_id).await)
} else {
Err(kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Missing Task Id: {task_id}"
)))
}
}
pub async fn delete_scheduled_task(
&self,
task_id: &str,
) -> Result<bool, kigi_tool_runtime::ToolError> {
use crate::implementations::grok_build::scheduler::types::{
SchedulerCommand, SchedulerHandle,
};
let sender = {
let res = self.registry.resources.lock().await;
res.get::<SchedulerHandle>()
.ok_or_else(|| {
kigi_tool_runtime::ToolError::custom("missing_resource", "SchedulerHandle")
})?
.0
.clone()
};
let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
sender
.send(SchedulerCommand::Delete {
id: task_id.to_owned(),
reply: reply_tx,
})
.map_err(|_| {
kigi_tool_runtime::ToolError::custom("process_manager", "Scheduler actor stopped")
})?;
reply_rx.await.map_err(|_| {
kigi_tool_runtime::ToolError::custom("process_manager", "Scheduler actor dropped reply")
})
}
/// Move a foreground command to background by tool_call_id.
/// Returns `true` if a matching foreground process was found and unblocked.
pub async fn background_foreground_command(&self, tool_call_id: &str) -> bool {
if let Some(terminal) = &self.terminal {
terminal.background_foreground_command(tool_call_id).await
} else {
false
}
}
/// Gives the output of all terminal tasks which are managed by the tool bridge
pub async fn list_tasks(&self) -> Option<Vec<TaskSnapshot>> {
if let Some(terminal) = &self.terminal {
Some(terminal.list_tasks().await)
} else {
None
}
}
/// Drain newly-completed bash background tasks not yet reported.
/// Marks returned tasks in [`ReportedTaskCompletions`] to prevent
/// duplicate reminders from [`TaskCompletionReminder`].
pub async fn drain_between_turn_bash_completions(&self) -> Vec<TaskSnapshot> {
let tasks = match self.list_tasks().await {
Some(t) => t,
None => return Vec::new(),
};
let completed: Vec<TaskSnapshot> = tasks
.into_iter()
.filter(|t| t.completed && t.kind != TaskKind::Monitor)
.collect();
if completed.is_empty() {
return Vec::new();
}
use crate::reminders::task_completion::{ReportedTaskCompletions, task_owned_by_session};
let mut res = self.registry.resources.lock().await;
// Subagents share the parent's terminal backend, so `list_tasks()`
// returns tasks owned by other sessions. Scope the between-turn
// "While you were idle, … background task completed" drain to tasks
// this session owns, mirroring the per-tool-call
// `TaskCompletionReminder` filter — otherwise a parent (or sibling)
// bash task that finished mid-subagent-turn leaks its completion
// `<system-reminder>` into the subagent's conversation. The owner
// filter runs before `mark_reported` so the owning session still
// reports the task on its own next turn.
let my_owner = res.get::<OwnerSessionId>().map(|o| o.0.clone());
let state = res.get_or_default::<State<ReportedTaskCompletions>>();
completed
.into_iter()
.filter(|t| task_owned_by_session(t, my_owner.as_deref()))
.filter(|t| state.mark_reported(&t.task_id))
.collect()
}
/// Construct a minimal bridge for tests. Has no tools registered.
///
/// Bypasses `ToolRegistryBuilder::finalize()` entirely so this can
/// be called from sync `#[test]` functions that lack a tokio runtime.
/// (`finalize()` spawns background tasks via `tokio::spawn`.)
pub fn for_test() -> Self {
let toolset = FinalizedToolset::empty_for_test();
Self {
registry: Arc::new(toolset),
terminal: None,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::computer::types::{BackgroundHandle, TerminalRunRequest, TerminalRunResult};
use crate::reminders::task_completion::ReportedTaskCompletions;
use std::time::Duration;
#[derive(Debug)]
struct KindFixture {
kind: ToolKind,
id: &'static str,
}
impl crate::types::tool_metadata::ToolMetadata for KindFixture {
fn kind(&self) -> ToolKind {
self.kind
}
fn tool_namespace(&self) -> crate::types::tool::ToolNamespace {
crate::types::tool::ToolNamespace::MCP
}
fn description_template(&self) -> &str {
"kind fixture"
}
}
impl kigi_tool_runtime::Tool for KindFixture {
type Args = serde_json::Value;
type Output = String;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new(self.id).expect("valid id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(self.id, "kind fixture")
}
async fn run(
&self,
_ctx: kigi_tool_runtime::ToolCallContext,
_input: serde_json::Value,
) -> Result<String, kigi_tool_runtime::ToolError> {
Ok("ok".into())
}
}
fn register_fixture(toolset: &FinalizedToolset, name: &str, kind: ToolKind, id: &'static str) {
toolset
.register_tool(
name.into(),
KindFixture { kind, id },
Some(serde_json::json!({"type": "object", "properties": {}})),
)
.unwrap();
}
#[test]
fn tool_kind_returns_registered_kind_per_namespace_and_none_for_unknown() {
let bridge = ToolBridge::for_test();
let toolset = bridge.toolset();
// PascalCase + grok_build's snake_case in one registry
// to exercise the lookup on the literal name strings each
// namespace ships.
register_fixture(&toolset, "Write", ToolKind::Write, "fixture_write");
register_fixture(
&toolset,
"StrReplace",
ToolKind::Edit,
"fixture_str_replace",
);
register_fixture(&toolset, "Delete", ToolKind::Delete, "fixture_delete");
register_fixture(
&toolset,
"run_terminal_cmd",
ToolKind::Execute,
"fixture_run_terminal_cmd",
);
assert_eq!(bridge.tool_kind("Write"), Some(ToolKind::Write));
assert_eq!(bridge.tool_kind("StrReplace"), Some(ToolKind::Edit));
assert_eq!(bridge.tool_kind("Delete"), Some(ToolKind::Delete));
assert_eq!(
bridge.tool_kind("run_terminal_cmd"),
Some(ToolKind::Execute)
);
assert_eq!(bridge.tool_kind("not_a_registered_tool"), None);
// Exact client-name lookup is case-sensitive.
assert_eq!(bridge.tool_kind("write"), None);
}
// ── drain_between_turn_bash_completions owner scoping (the "While you
// were idle, … background task completed" path) ──
#[derive(Debug)]
struct MockTerminal {
tasks: Vec<TaskSnapshot>,
}
#[async_trait::async_trait]
impl TerminalBackend for MockTerminal {
async fn run(
&self,
_: TerminalRunRequest,
) -> Result<TerminalRunResult, crate::computer::types::ComputerError> {
unimplemented!()
}
async fn run_background(
&self,
_: TerminalRunRequest,
) -> Result<BackgroundHandle, crate::computer::types::ComputerError> {
unimplemented!()
}
async fn kill_task(&self, _: &str) -> KillOutcome {
KillOutcome::NotFound
}
async fn get_task(&self, _: &str) -> Option<TaskSnapshot> {
None
}
async fn wait_for_completion(&self, _: &str, _: Option<Duration>) -> Option<TaskSnapshot> {
None
}
async fn list_tasks(&self) -> Vec<TaskSnapshot> {
self.tasks.clone()
}
}
fn completed_task(id: &str, owner: Option<&str>) -> TaskSnapshot {
TaskSnapshot {
task_id: id.into(),
command: "echo test".into(),
display_command: None,
cwd: String::new(),
start_time: std::time::SystemTime::now(),
end_time: Some(std::time::SystemTime::now()),
output: String::new(),
output_file: std::path::PathBuf::new(),
truncated: false,
exit_code: Some(0),
signal: None,
completed: true,
kind: Default::default(),
block_waited: false,
explicitly_killed: false,
owner_session_id: owner.map(|s| s.to_string()),
}
}
/// Regression: subagents share the parent's terminal backend, so the
/// between-turn drain must not surface another session's completed bash
/// task (which leaked as a "While you were idle, 1 background task
/// completed" `<system-reminder>` into the subagent's conversation).
#[tokio::test]
async fn between_turn_bash_completions_scoped_to_owning_session() {
let toolset = FinalizedToolset::empty_for_test();
{
let mut res = toolset.resources.lock().await;
res.insert(OwnerSessionId("subagent-1".into()));
res.register_state::<ReportedTaskCompletions>();
}
let backend: Arc<dyn TerminalBackend> = Arc::new(MockTerminal {
tasks: vec![
completed_task("mine-task", Some("subagent-1")),
completed_task("parent-task", Some("parent-0")),
completed_task("unowned-task", None),
],
});
let bridge = ToolBridge {
registry: Arc::new(toolset),
terminal: Some(backend),
};
let drained = bridge.drain_between_turn_bash_completions().await;
let ids: Vec<&str> = drained.iter().map(|t| t.task_id.as_str()).collect();
assert!(ids.contains(&"mine-task"), "own task must drain: {ids:?}");
assert!(
ids.contains(&"unowned-task"),
"unowned task must drain (backwards compat): {ids:?}"
);
assert!(
!ids.contains(&"parent-task"),
"another session's task must NOT leak into this session: {ids:?}"
);
}
}
@@ -0,0 +1,538 @@
//! Cgroup v2 memory-high monitor for graceful OOM handling.
//!
//! Approach:
//!
//! 1. On startup we create a child cgroup under the current process's cgroup and
//! configure:
//! - `memory.high` = soft limit (the "desired" ceiling)
//! - `memory.max` = soft limit + headroom (hard OOM kill boundary)
//!
//! 2. Before each spawned command, we write the child PID into `cgroup.procs` to
//! move it (and its entire process group) into the cgroup.
//!
//! 3. A background `MemoryHighMonitor` task watches `memory.events` via inotify.
//! When the kernel increments the `high` counter (meaning a process touched
//! `memory.high`), the monitor reads `memory.current` and — if RSS is still
//! above 90 % of `memory.high` — sends a `MemoryHighEvent` through a
//! `tokio::sync::watch` channel.
//!
//! 4. The terminal actor polls `monitor.try_recv()` on every tick. When it
//! receives an event it kills the offending process group with SIGKILL and
//! reports exit-code **137** (128 + SIGKILL) with signal `"oom"`.
//!
//! The kigi-tools process itself is **never** inside this cgroup — only spawned
//! child commands are. After the child exits the cgroup is empty until the next
//! command.
//!
//! ## Platform
//!
//! Everything compiles on all platforms, but the actual cgroup + inotify logic is
//! gated behind `#[cfg(target_os = "linux")]`. On macOS / Windows the public
//! constructors return `None` / no-op stubs so callers don't need `#[cfg]`.
/// Exit code for processes killed due to memory pressure.
/// Matches the POSIX convention: 128 + signal-number (SIGKILL = 9).
pub const PROCESS_OOM_EXIT_CODE: i32 = 137;
// ============================================================================
// Public types (cross-platform)
// ============================================================================
/// Event emitted when `memory.high` is breached and RSS is still above
/// the 90 % buffer threshold.
#[derive(Debug, Clone)]
pub struct MemoryHighEvent {
/// Current cgroup memory usage in bytes when the event fired.
pub memory_current: u64,
/// The configured `memory.high` threshold in bytes.
pub memory_high_threshold: u64,
}
/// Configuration for cgroup memory limits.
#[derive(Debug, Clone)]
pub struct CgroupMemoryConfig {
/// Soft memory limit (`memory.high`). When a process inside the cgroup
/// exceeds this, the monitor fires.
pub memory_high_bytes: u64,
/// Extra headroom above `memory.high` before the kernel hard-kills
/// (`memory.max = memory_high_bytes + headroom_bytes`).
/// A reasonable default is 256 MiB.
pub headroom_bytes: u64,
}
impl CgroupMemoryConfig {
/// memory.max = memory.high + headroom
#[cfg(target_os = "linux")]
fn memory_max(&self) -> u64 {
self.memory_high_bytes.saturating_add(self.headroom_bytes)
}
}
// ============================================================================
// Linux implementation
// ============================================================================
#[cfg(target_os = "linux")]
mod linux {
use super::*;
use std::os::fd::{AsRawFd, FromRawFd, OwnedFd};
use std::os::unix::ffi::OsStrExt;
use std::path::PathBuf;
use tokio::io::Interest;
use tokio::io::unix::AsyncFd;
use tokio::sync::watch;
// ── inotify FFI ──────────────────────────────────────────────────────
unsafe fn inotify_init1(flags: libc::c_int) -> std::io::Result<i32> {
#[allow(clippy::cast_possible_truncation)]
let ret = unsafe { libc::syscall(libc::SYS_inotify_init1, flags) as libc::c_int };
if ret < 0 {
return Err(std::io::Error::last_os_error());
}
Ok(ret)
}
unsafe fn inotify_add_watch(
fd: libc::c_int,
pathname: *const libc::c_char,
mask: u32,
) -> std::io::Result<i32> {
#[allow(clippy::cast_possible_truncation)]
let ret = unsafe {
libc::syscall(libc::SYS_inotify_add_watch, fd, pathname, mask) as libc::c_int
};
if ret < 0 {
return Err(std::io::Error::last_os_error());
}
Ok(ret)
}
const IN_MODIFY: u32 = 0x0000_0002;
// ── Inotify wrapper ──────────────────────────────────────────────────
struct Inotify {
fd: AsyncFd<OwnedFd>,
}
impl Inotify {
fn new() -> std::io::Result<Self> {
let raw = unsafe { inotify_init1(libc::O_NONBLOCK | libc::O_CLOEXEC) }?;
let owned = unsafe { OwnedFd::from_raw_fd(raw) };
let fd = AsyncFd::with_interest(owned, Interest::READABLE)?;
Ok(Inotify { fd })
}
fn add_watch(&self, path: &std::path::Path) -> std::io::Result<i32> {
let mut bytes = path.as_os_str().as_bytes().to_vec();
bytes.push(0); // NUL-terminate
let wd = unsafe {
inotify_add_watch(
self.fd.get_ref().as_raw_fd(),
bytes.as_ptr().cast(),
IN_MODIFY,
)
}?;
Ok(wd)
}
async fn wait_and_drain(&self) -> std::io::Result<()> {
let mut guard = self.fd.readable().await?;
// Drain all pending inotify events
let mut buf = [0u8; 4096];
loop {
let n = unsafe {
libc::read(
guard.get_inner().as_raw_fd(),
buf.as_mut_ptr().cast(),
buf.len(),
)
};
if n > 0 {
continue;
}
break;
}
guard.clear_ready();
Ok(())
}
}
// ── Cgroup helpers ───────────────────────────────────────────────────
/// Read `/proc/self/cgroup` to find our own cgroup path (cgroup v2 unified).
fn read_self_cgroup() -> std::io::Result<String> {
let contents = std::fs::read_to_string("/proc/self/cgroup")?;
// In cgroupv2 unified hierarchy, the line is "0::<path>"
for line in contents.lines() {
if let Some(rest) = line.strip_prefix("0::") {
return Ok(rest.to_owned());
}
}
Err(std::io::Error::new(
std::io::ErrorKind::NotFound,
"Could not find cgroupv2 entry in /proc/self/cgroup",
))
}
/// Parse the `high <N>` counter from `memory.events` contents.
fn parse_memory_events_high(contents: &str) -> Option<u64> {
for line in contents.lines() {
if let Some(value) = line.strip_prefix("high ") {
return value.trim().parse::<u64>().ok();
}
}
None
}
// ── CgroupHandle ─────────────────────────────────────────────────────
/// Owns the lifecycle of a child cgroup directory.
pub(crate) struct CgroupHandle {
fs_path: PathBuf,
}
impl CgroupHandle {
/// Create a child cgroup under the current process's cgroup and
/// configure memory limits.
pub(crate) async fn create(config: &CgroupMemoryConfig) -> std::io::Result<Self> {
let self_cgroup = read_self_cgroup()?;
let name = format!("kigi-tools-{}", uuid::Uuid::now_v7());
let fs_path = PathBuf::from(format!("/sys/fs/cgroup{}/{}", self_cgroup, name));
tokio::fs::create_dir_all(&fs_path).await?;
// Enable memory + cpu controllers in the child cgroup's parent
// (the parent's subtree_control must list the controllers).
let parent = fs_path.parent().unwrap();
let subtree_ctl = parent.join("cgroup.subtree_control");
// Best-effort; may already be enabled or not permitted.
let _ = tokio::fs::write(&subtree_ctl, "+memory +cpu").await;
// Configure memory.high (soft limit)
let memory_high_path = fs_path.join("memory.high");
tokio::fs::write(&memory_high_path, config.memory_high_bytes.to_string()).await?;
// Configure memory.max (hard limit = high + headroom)
let memory_max_path = fs_path.join("memory.max");
tokio::fs::write(&memory_max_path, config.memory_max().to_string()).await?;
tracing::info!(
cgroup = %fs_path.display(),
memory_high = config.memory_high_bytes,
memory_max = config.memory_max(),
"Created cgroup with memory limits"
);
Ok(CgroupHandle { fs_path })
}
/// Move a process (by PID) into this cgroup.
pub(crate) async fn add_process(&self, pid: u32) -> std::io::Result<()> {
let procs_path = self.fs_path.join("cgroup.procs");
tokio::fs::write(&procs_path, pid.to_string()).await
}
/// Read `memory.current` from this cgroup.
#[allow(dead_code)]
pub(crate) async fn memory_current(&self) -> std::io::Result<u64> {
let s: String = tokio::fs::read_to_string(self.fs_path.join("memory.current")).await?;
s.trim()
.parse::<u64>()
.map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))
}
/// Filesystem path to this cgroup.
pub(crate) fn path(&self) -> &std::path::Path {
&self.fs_path
}
}
impl Drop for CgroupHandle {
fn drop(&mut self) {
let path = self.fs_path.clone();
// Always use tokio::spawn: the cleanup future is Send and Drop
// can fire after the LocalSet has shut down, making spawn_local
// unsafe here.
tokio::spawn(async move {
let kill_path = path.join("cgroup.kill");
let _ = tokio::fs::write(&kill_path, "1").await;
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
if let Err(e) = tokio::fs::remove_dir(&path).await {
tracing::debug!(
cgroup = %path.display(),
"Failed to remove cgroup dir on drop (may already be gone): {e}"
);
}
});
}
}
// ── MemoryHighMonitor ────────────────────────────────────────────────
/// Watches `memory.events` via inotify and signals when the `high`
/// counter increments while RSS is still above 90% of the threshold.
pub(crate) struct MemoryHighMonitor {
rx: watch::Receiver<Option<MemoryHighEvent>>,
/// Dropping this aborts the background task.
_handle: tokio::task::JoinHandle<()>,
}
impl MemoryHighMonitor {
/// Start monitoring the given cgroup for memory.high events.
pub(crate) async fn start(
cgroup_path: PathBuf,
memory_high_threshold: u64,
use_spawn_local: bool,
) -> std::io::Result<Self> {
let (tx, rx) = watch::channel(None);
let inotify = Inotify::new()?;
let events_path = cgroup_path.join("memory.events");
inotify.add_watch(&events_path)?;
let monitor_fut = Self::monitor_loop(inotify, cgroup_path, memory_high_threshold, tx);
let handle = if use_spawn_local {
tokio::task::spawn_local(monitor_fut)
} else {
tokio::spawn(monitor_fut)
};
Ok(MemoryHighMonitor {
rx,
_handle: handle,
})
}
/// Non-blocking check: returns `Some(event)` if memory.high was
/// breached since the last call, `None` otherwise.
pub(crate) fn try_recv(&mut self) -> Option<MemoryHighEvent> {
if self.rx.has_changed().unwrap_or(false) {
self.rx.borrow_and_update().clone()
} else {
None
}
}
async fn monitor_loop(
inotify: Inotify,
cgroup_path: PathBuf,
memory_high_threshold: u64,
tx: watch::Sender<Option<MemoryHighEvent>>,
) {
let events_path = cgroup_path.join("memory.events");
let current_path = cgroup_path.join("memory.current");
// Read baseline high counter
let mut last_high_count = Self::read_high_counter(&events_path).await.unwrap_or(0);
loop {
// Block until inotify fires (memory.events was modified)
if inotify.wait_and_drain().await.is_err() {
break;
}
// Read new high counter
let current_high = Self::read_high_counter(&events_path).await.unwrap_or(0);
if current_high <= last_high_count {
continue;
}
last_high_count = current_high;
// Read current memory usage
let memory_current = match tokio::fs::read_to_string(&current_path).await {
Ok(s) => {
let s: String = s;
s.trim().parse::<u64>().unwrap_or(0)
}
Err(_) => continue,
};
// Only fire if still above 90 % of threshold (avoids false
// positives from transient spikes the kernel already handled).
let buffer_threshold = memory_high_threshold.saturating_mul(9) / 10;
if memory_current >= buffer_threshold {
let event = MemoryHighEvent {
memory_current,
memory_high_threshold,
};
if tx.send(Some(event)).is_err() {
break; // receiver dropped
}
}
}
}
async fn read_high_counter(events_path: &std::path::Path) -> Option<u64> {
let contents = tokio::fs::read_to_string(events_path).await.ok()?;
parse_memory_events_high(&contents)
}
}
impl Drop for MemoryHighMonitor {
fn drop(&mut self) {
self._handle.abort();
}
}
}
// ============================================================================
// Cross-platform re-exports
// ============================================================================
/// Cgroup handle — owns the child cgroup's lifecycle.
///
/// On Linux, this creates a real cgroupv2 directory with memory limits.
/// On other platforms, this is a no-op.
pub struct CgroupGuard {
#[cfg(target_os = "linux")]
inner: Option<linux::CgroupHandle>,
}
impl CgroupGuard {
/// Try to create a cgroup with the given memory config.
/// Returns a guard that cleans up the cgroup on drop.
///
/// On non-Linux platforms this always returns a no-op guard.
/// On Linux, if cgroup creation fails (e.g., not running as root,
/// cgroupv2 not available), it logs a warning and returns a no-op guard.
pub async fn try_create(config: &CgroupMemoryConfig) -> Self {
#[cfg(target_os = "linux")]
{
match linux::CgroupHandle::create(config).await {
Ok(handle) => CgroupGuard {
inner: Some(handle),
},
Err(e) => {
tracing::warn!("Failed to create cgroup (falling back to no limits): {e}");
CgroupGuard { inner: None }
}
}
}
#[cfg(not(target_os = "linux"))]
{
let _ = config;
CgroupGuard {}
}
}
/// No-op guard with no backing cgroup.
pub fn noop() -> Self {
#[cfg(target_os = "linux")]
{
CgroupGuard { inner: None }
}
#[cfg(not(target_os = "linux"))]
{
CgroupGuard {}
}
}
/// Move a process into this cgroup by PID.
/// No-op if cgroup was not created.
pub async fn add_process(&self, _pid: u32) -> std::io::Result<()> {
#[cfg(target_os = "linux")]
{
if let Some(ref handle) = self.inner {
return handle.add_process(_pid).await;
}
}
Ok(())
}
/// Returns the cgroup filesystem path, if available.
#[allow(dead_code)]
pub fn path(&self) -> Option<&std::path::Path> {
#[cfg(target_os = "linux")]
{
if let Some(ref handle) = self.inner {
return Some(handle.path());
}
}
None
}
/// Returns true if this guard has a real cgroup backing it.
pub fn is_active(&self) -> bool {
#[cfg(target_os = "linux")]
{
self.inner.is_some()
}
#[cfg(not(target_os = "linux"))]
{
false
}
}
}
/// Memory-high monitor — watches for memory pressure events.
///
/// On Linux, uses inotify on `memory.events`.
/// On other platforms, this is a no-op that never fires.
pub struct MemoryMonitor {
#[cfg(target_os = "linux")]
inner: Option<linux::MemoryHighMonitor>,
}
impl MemoryMonitor {
/// Start monitoring the given cgroup guard for memory.high events.
/// Returns a no-op monitor if the guard has no backing cgroup.
pub async fn start(
guard: &CgroupGuard,
config: &CgroupMemoryConfig,
use_spawn_local: bool,
) -> Self {
#[cfg(target_os = "linux")]
{
if let Some(ref handle) = guard.inner {
match linux::MemoryHighMonitor::start(
handle.path().to_path_buf(),
config.memory_high_bytes,
use_spawn_local,
)
.await
{
Ok(monitor) => {
return MemoryMonitor {
inner: Some(monitor),
};
}
Err(e) => {
tracing::warn!("Failed to start memory monitor: {e}");
}
}
}
MemoryMonitor { inner: None }
}
#[cfg(not(target_os = "linux"))]
{
let _ = (guard, config, use_spawn_local);
MemoryMonitor {}
}
}
/// No-op monitor that never fires.
pub fn noop() -> Self {
#[cfg(target_os = "linux")]
{
MemoryMonitor { inner: None }
}
#[cfg(not(target_os = "linux"))]
{
MemoryMonitor {}
}
}
/// Non-blocking poll: returns `Some(event)` if memory.high was breached
/// since the last call, `None` otherwise.
pub fn try_recv(&mut self) -> Option<MemoryHighEvent> {
#[cfg(target_os = "linux")]
{
if let Some(ref mut monitor) = self.inner {
return monitor.try_recv();
}
}
None
}
}
@@ -0,0 +1,805 @@
//! Shadow `find`→`bfs` and `grep`→`ugrep` when those binaries resolve.
//!
//! Per-tool enable state (default on) is resolved by the host via the shared
//! config helper `kigi-shell::util::config::resolve_search_tools_enabled`
//! (requirements > env `KIGI_TOOLS_FIND_BFS` / `KIGI_TOOLS_GREP_UGREP` (+
//! `KIGI_FIND_BFS` / `KIGI_GREP_UGREP` aliases, `DISABLE_EMBEDDED_SEARCH_TOOLS`
//! master) > `[toolset.bash]` config.toml > managed > default), baked into the
//! `LocalTerminalBackend` as a [`SearchShadowConfig`] and passed to
//! [`search_injection`] per command. The enable state lives on the backend (not
//! a process-global): a subagent that reuses the parent's backend inherits the
//! parent's shadows instead of clobbering a shared static. This module no longer
//! parses the flags itself.
//!
//! Resolve (host side, memoized): env override if a regular file → bundled binary
//! (release builds, self-extracted to `~/.kigi/vendor/<name>-<ver>-<target>`) →
//! `~/.kigi/vendor/{name}` if a regular file → `which` on the agent `$PATH`.
//! Env/vendor only require `is_file()` as a lenient hint (no `--version` probe).
//! This memoized path is only a *hint*: the injected shadow re-resolves at
//! **call time** — it uses the hint when it's still *executable* (`[ -x ]`), else
//! `command -v {bin}` on the live shell `PATH` (which includes login/rc additions
//! the agent process may lack), else falls back to the OS `{name}`. So a removed
//! or non-executable binary self-heals to OS `find`/`grep`, and a binary
//! reachable only through the login shell is still found.
//!
//! Inject is **always** non-empty on Unix callers: either install a shadow
//! function (which tags itself with a `__grok_shadow_{name}` marker) or a
//! marker-gated `unalias`+`unset -f` that drops *only* a prior harness shadow —
//! never a user-defined `find`/`grep` function replayed from the snapshot.
use super::SearchShadowConfig;
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
/// BRE + ignore-files; recursive flags are no-ops on stdin (computer parity).
const UGREP_DEFAULT_ARGS: &[&str] = &[
"-G",
"--ignore-files",
"--hidden",
"-I",
"--exclude-dir=.git",
"--exclude-dir=.svn",
"--exclude-dir=.hg",
"--exclude-dir=.bzr",
"--exclude-dir=.jj",
"--exclude-dir=.sl",
];
// Binaries embedded by build.rs when `KIGI_TOOLS_BUNDLE_{BFS,UGREP}_PATH` is set
// (release pipeline). Self-extracted to `~/.kigi/vendor` on first use, mirroring
// the ripgrep bundling in `grok_build::grep::ripgrep`.
#[cfg(bundle_bfs)]
const BFS_BYTES: &[u8] = include_bytes!(concat!(
env!("OUT_DIR"),
"/bundle-bfs/bfs-",
env!("KIGI_TOOLS_BFS_VER"),
"-",
env!("KIGI_TOOLS_BFS_TARGET"),
".bin"
));
#[cfg(bundle_ugrep)]
const UGREP_BYTES: &[u8] = include_bytes!(concat!(
env!("OUT_DIR"),
"/bundle-ugrep/ugrep-",
env!("KIGI_TOOLS_UGREP_VER"),
"-",
env!("KIGI_TOOLS_UGREP_TARGET"),
".bin"
));
/// Oneline inject for shell wrappers; always ends with `"; "`.
///
/// `cfg` is the backend's resolved per-tool enable state (see module docs); it
/// is passed in per command rather than read from a process-global so subagents
/// sharing a backend can't clobber each other's shadows.
pub fn search_injection(cfg: SearchShadowConfig) -> String {
build_injection(cfg.find_bfs, cfg.grep_ugrep, resolved_tools())
}
/// Compose the inject from per-tool enable flags + resolved binaries. An enabled
/// tool installs a self-resolving shadow (the memoized `tools` path is only a
/// fast-path hint; the shadow re-resolves at call time and falls back to the OS
/// binary — see [`shell_function`]). A disabled tool emits a marker-gated
/// `restore` that drops only a prior harness shadow. Kept pure (flags/tools
/// passed in) so tests need no process-global env mutation — that is UB against
/// the `shell_state` integration tests that read env / spawn children
/// concurrently.
fn build_injection(find_on: bool, grep_on: bool, tools: &ResolvedTools) -> String {
let find = if find_on {
shell_function("find", "bfs", tools.bfs.as_deref(), &[])
} else {
restore_command("find")
};
let grep = if grep_on {
shell_function("grep", "ugrep", tools.ugrep.as_deref(), UGREP_DEFAULT_ARGS)
} else {
restore_command("grep")
};
format!("{find}; {grep}; ")
}
/// Drop a *previously installed harness* shadow so command-word `{name}` uses the
/// OS binary again. Gated on the `__grok_shadow_{name}` marker that
/// [`shell_function`] sets, so a user-defined `{name}` function replayed from the
/// shell snapshot is left intact — only the harness's own shadow is removed.
/// `set -u`/`set -e` safe and idempotent (`unset -f` is bash + zsh).
fn restore_command(name: &str) -> String {
format!(
"if [ -n \"${{__grok_shadow_{name}-}}\" ]; then \
unalias {name} 2>/dev/null || true; \
unset -f {name} 2>/dev/null || true; \
unset __grok_shadow_{name} 2>/dev/null || true; \
fi"
)
}
struct ResolvedTools {
bfs: Option<PathBuf>,
ugrep: Option<PathBuf>,
}
fn resolved_tools() -> &'static ResolvedTools {
static TOOLS: OnceLock<ResolvedTools> = OnceLock::new();
TOOLS.get_or_init(|| ResolvedTools {
bfs: resolve_tool("bfs", "KIGI_TOOLS_BFS_PATH", bundled_bfs()),
ugrep: resolve_tool("ugrep", "KIGI_TOOLS_UGREP_PATH", bundled_ugrep()),
})
}
/// Write embedded `bytes` to `~/.kigi/vendor/<versioned_name>` (chmod 755) on
/// first use and return the path; reused on later runs. Versioned so bumping the
/// bundled version writes a fresh file instead of reusing a stale one.
#[cfg(any(bundle_bfs, bundle_ugrep))]
fn extract_bundled(versioned_name: &str, bytes: &[u8]) -> std::io::Result<PathBuf> {
use std::os::unix::fs::PermissionsExt;
let dir = crate::util::kigi_home().join("vendor");
let dest = dir.join(versioned_name);
if !dest.exists() {
std::fs::create_dir_all(&dir)?;
// Write to a unique temp then atomically rename, so a concurrent first
// use (or an interrupted write) can't leave a half-written binary that
// gets cached and exec'd.
let tmp = dir.join(format!(
"{versioned_name}.tmp.{}.{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
std::fs::write(&tmp, bytes)?;
let mut perms = std::fs::metadata(&tmp)?.permissions();
perms.set_mode(0o755);
std::fs::set_permissions(&tmp, perms)?;
// Rename is atomic on the same filesystem. If another process won the
// race, `dest` already exists and is correct — drop our temp copy.
if let Err(e) = std::fs::rename(&tmp, &dest) {
let _ = std::fs::remove_file(&tmp);
if !dest.exists() {
return Err(e);
}
}
}
Ok(dest)
}
/// Path to the bundled `bfs` (extracted on first use), or `None` when not bundled.
fn bundled_bfs() -> Option<PathBuf> {
#[cfg(bundle_bfs)]
{
extract_bundled(
concat!(
"bfs-",
env!("KIGI_TOOLS_BFS_VER"),
"-",
env!("KIGI_TOOLS_BFS_TARGET")
),
BFS_BYTES,
)
.ok()
}
#[cfg(not(bundle_bfs))]
{
None
}
}
/// Path to the bundled `ugrep` (extracted on first use), or `None` when not bundled.
fn bundled_ugrep() -> Option<PathBuf> {
#[cfg(bundle_ugrep)]
{
extract_bundled(
concat!(
"ugrep-",
env!("KIGI_TOOLS_UGREP_VER"),
"-",
env!("KIGI_TOOLS_UGREP_TARGET")
),
UGREP_BYTES,
)
.ok()
}
#[cfg(not(bundle_ugrep))]
{
None
}
}
fn resolve_tool(bin_name: &str, env_override: &str, bundled: Option<PathBuf>) -> Option<PathBuf> {
resolve_tool_from(
std::env::var_os(env_override).map(PathBuf::from),
bundled,
crate::util::kigi_home().join("vendor").join(bin_name),
bin_name,
)
}
/// Resolution order: explicit env path → bundled (self-extracted) →
/// `~/.kigi/vendor/<bin>` → `which`. Env and vendor only require `is_file()` here
/// (a lenient hint, no `+x` probe) so an odd-permission copy still resolves; the
/// injected shadow gates on `[ -x ]` at call time and falls back to the OS binary
/// if the hint isn't executable, so a non-exec path can't hard-fail `find`/`grep`.
fn resolve_tool_from(
env_path: Option<PathBuf>,
bundled: Option<PathBuf>,
vendor: PathBuf,
bin_name: &str,
) -> Option<PathBuf> {
if let Some(path) = env_path
&& path.is_file()
{
return Some(path);
}
if let Some(path) = bundled {
return Some(path);
}
if vendor.is_file() {
return Some(vendor);
}
which::which(bin_name).ok()
}
fn bash_safe_quote(s: &str) -> String {
if s.chars().all(|c| {
c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '/' | '=' | ':' | '@' | '+' | '-')
}) {
return s.to_string();
}
format!("'{}'", s.replace('\'', "'\\''"))
}
/// Oneline `name() { … }` for `-c` inject — a *self-resolving* shadow.
///
/// At call time it picks the binary: the host-resolved `preferred` path
/// (bundled/env/vendor/which) when that file still exists, else `command -v
/// {bin_name}` on the live shell `PATH` (which carries login/rc additions the
/// agent process may not have), else it falls back to the OS `{name}`. This keeps
/// the fast hard-coded path for the common case while self-healing when the
/// binary was removed (revalidation) or is only reachable through the shell's
/// richer `PATH`.
///
/// `exec -a` runs inside a subshell so a top-level call can't replace the wrapper
/// shell (it must survive to dump state); a call already inside a subshell
/// (`BASH_SUBSHELL > 0`, bash) execs directly to skip a fork. `${ZSH_VERSION-}`
/// keeps the probe `set -u`-safe (a bare `$ZSH_VERSION` aborts bash under
/// nounset). `exec -a` gives the binary the `find`/`grep` argv0 (ps display +
/// ugrep grep-personality) in both bash and zsh. The trailing
/// `__grok_shadow_{name}=1` marks this as a harness shadow so `restore_command`
/// only ever removes our own function — never a user's.
fn shell_function(
name: &str,
bin_name: &str,
preferred: Option<&Path>,
prepend_args: &[&str],
) -> String {
let qpref = preferred
.map(|p| bash_safe_quote(&p.to_string_lossy()))
.unwrap_or_else(|| "''".to_string());
let prepend = {
let qargs: Vec<String> = prepend_args.iter().map(|a| bash_safe_quote(a)).collect();
if qargs.is_empty() {
String::new()
} else {
format!("{} ", qargs.join(" "))
}
};
// `local __grok_bin` is re-resolved every call. The host hint is trusted
// only when it's *executable* (`[ -x ]`, not just `[ -f ]`): the resolver
// accepts any regular file as a hint, but `exec` needs `+x`, so a non-exec
// hint must fall through rather than hard-fail with no OS fallback. Then
// `command -v` on the live shell PATH (returns an executable), else the OS
// binary. `|| __grok_bin=''` keeps the lookup `set -e`-safe (a failed
// `command -v` would otherwise abort the function under errexit). The OS
// fallback uses `command {name}` to bypass this function. `{prepend}` is
// empty for find, the ugrep default flags for grep (and is omitted from the
// OS fallback, which gets the original args).
format!(
"unalias {name} 2>/dev/null || true; \
{name}() {{ \
local __grok_bin={qpref}; \
[ -x \"$__grok_bin\" ] || __grok_bin=$(command -v {bin_name} 2>/dev/null) || __grok_bin=''; \
if [ -z \"$__grok_bin\" ]; then command {name} \"$@\"; return; fi; \
if [[ -z ${{ZSH_VERSION-}} ]] && (( BASH_SUBSHELL > 0 )); then \
exec -a {name} \"$__grok_bin\" {prepend}\"$@\"; \
else \
(exec -a {name} \"$__grok_bin\" {prepend}\"$@\"); \
fi; \
}}; \
__grok_shadow_{name}=1"
)
}
#[cfg(test)]
mod tests {
use super::*;
/// Both binaries resolved, for `build_injection` shape tests.
fn both_tools() -> ResolvedTools {
ResolvedTools {
bfs: Some(PathBuf::from("/tmp/bfs")),
ugrep: Some(PathBuf::from("/tmp/ugrep")),
}
}
#[test]
fn shell_function_shape() {
let fn_body = shell_function("find", "bfs", Some(Path::new("/tmp/bfs")), &[]);
assert!(fn_body.contains("unalias find"));
// Preferred path is the fast-path hint; the shadow execs `$__grok_bin`.
assert!(fn_body.contains("local __grok_bin=/tmp/bfs"));
assert!(fn_body.contains("exec -a find \"$__grok_bin\" \"$@\""));
// Hint is trusted only when executable (`[ -x ]`, not `[ -f ]`), so a
// non-exec hint falls through instead of hard-failing exec.
assert!(fn_body.contains("[ -x \"$__grok_bin\" ]"));
assert!(!fn_body.contains("[ -f \"$__grok_bin\" ]"));
// Self-heal: live-PATH lookup + OS fallback.
assert!(fn_body.contains("command -v bfs"));
assert!(fn_body.contains("command find \"$@\""));
assert!(fn_body.contains("BASH_SUBSHELL > 0"));
assert!(fn_body.contains("(exec -a find"));
// Marker so `restore_command` only removes our own shadow.
assert!(fn_body.contains("__grok_shadow_find=1"));
// set -u-safe zsh probe (a bare $ZSH_VERSION aborts bash under nounset).
assert!(fn_body.contains("${ZSH_VERSION-}"));
assert!(!fn_body.contains("[[ -n $ZSH_VERSION ]]"));
}
#[test]
fn shell_function_unresolved_uses_empty_hint() {
// No host-resolved path → empty hint, relies on live-PATH `command -v`.
let fn_body = shell_function("find", "bfs", None, &[]);
assert!(fn_body.contains("local __grok_bin=''"));
assert!(fn_body.contains("command -v bfs"));
assert!(fn_body.contains("command find \"$@\""));
}
#[test]
fn grep_prepends_ugrep_defaults() {
let fn_body = shell_function(
"grep",
"ugrep",
Some(Path::new("/tmp/ugrep")),
UGREP_DEFAULT_ARGS,
);
assert!(fn_body.contains("local __grok_bin=/tmp/ugrep"));
assert!(fn_body.contains("\"$__grok_bin\" -G --ignore-files --hidden -I"));
assert!(fn_body.contains("--exclude-dir=.git"));
assert!(fn_body.contains("command -v ugrep"));
}
#[test]
fn bash_safe_quote_escapes_metacharacters() {
assert_eq!(bash_safe_quote("/usr/bin/bfs"), "/usr/bin/bfs");
assert_eq!(bash_safe_quote("/tmp/my bfs"), "'/tmp/my bfs'");
let body = shell_function("find", "bfs", Some(Path::new("/tmp/evil$(id)")), &[]);
assert!(body.contains("local __grok_bin='/tmp/evil$(id)'"), "{body}");
assert!(!body.contains("=/tmp/evil$(id)"));
}
#[test]
fn restore_command_is_marker_gated() {
let r = restore_command("find");
// Only removes the harness shadow when our marker is set.
assert!(r.contains("if [ -n \"${__grok_shadow_find-}\" ]"));
assert!(r.contains("unalias find"));
assert!(r.contains("unset -f find"));
assert!(r.contains("unset __grok_shadow_find"));
}
#[test]
fn config_default_is_both_on() {
// Standalone/no-host backends default to shadowing both tools.
let cfg = SearchShadowConfig::default();
assert!(cfg.find_bfs);
assert!(cfg.grep_ugrep);
}
#[test]
fn build_injection_off_emits_marker_gated_restore_not_function() {
// Disabled tools emit a marker-gated restore so a stale harness shadow
// from a prior snapshot is dropped, but a user function is left intact.
let inject = build_injection(false, false, &both_tools());
assert!(inject.ends_with("; "));
assert!(inject.contains("if [ -n \"${__grok_shadow_find-}\" ]"));
assert!(inject.contains("if [ -n \"${__grok_shadow_grep-}\" ]"));
assert!(inject.contains("unset -f find"));
assert!(inject.contains("unset -f grep"));
assert!(!inject.contains("find()"));
assert!(!inject.contains("grep()"));
}
#[test]
fn build_injection_on_shadows_resolved_tools() {
let inject = build_injection(true, true, &both_tools());
assert!(inject.contains("find()"));
assert!(inject.contains("grep()"));
assert!(inject.contains("-G --ignore-files"));
assert!(inject.contains("__grok_shadow_find=1"));
}
#[test]
fn build_injection_enabled_unresolved_still_self_heals() {
// Enabled but no host-resolved path → still install a self-resolving
// shadow (live-PATH `command -v` + OS fallback), never a bare restore.
let tools = ResolvedTools {
bfs: None,
ugrep: None,
};
let inject = build_injection(true, true, &tools);
assert!(inject.contains("find()"));
assert!(inject.contains("grep()"));
assert!(inject.contains("command -v bfs"));
assert!(inject.contains("command -v ugrep"));
// OS fallback present; not a marker-gated restore.
assert!(inject.contains("command find \"$@\""));
assert!(!inject.contains("if [ -n \"${__grok_shadow_find-}\" ]"));
}
#[test]
fn injection_always_nonempty_and_trailing_sep() {
// Structural invariants that hold regardless of host flags/binaries:
// never empty (so a stale snapshot shadow is always overwritten) and a
// trailing `"; "` separator before the user command.
for cfg in [
SearchShadowConfig {
find_bfs: true,
grep_ugrep: true,
},
SearchShadowConfig {
find_bfs: false,
grep_ugrep: false,
},
] {
let inject = search_injection(cfg);
assert!(!inject.is_empty());
assert!(inject.ends_with("; "));
assert!(!inject.contains('\n'));
}
}
#[test]
fn env_override_accepts_regular_file_without_exec_bit() {
let bin = std::env::temp_dir().join(format!(
"grok-bfs-noexec-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
std::fs::write(&bin, b"#!/bin/sh\necho ok\n").unwrap();
// Mode 0o644 — no execute bit; Nix-like / restricted copies.
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let mut perms = std::fs::metadata(&bin).unwrap().permissions();
perms.set_mode(0o644);
std::fs::set_permissions(&bin, perms).unwrap();
}
// Bundled + vendor intentionally absent so the env override is what wins.
let got = resolve_tool_from(
Some(bin.clone()),
None,
PathBuf::from("/nonexistent/vendor/bfs"),
"bfs",
);
let _ = std::fs::remove_file(&bin);
assert_eq!(got.as_deref(), Some(bin.as_path()));
}
#[test]
fn resolve_tool_precedence() {
// Real temp files so the is_file() checks pass.
let dir = std::env::temp_dir().join(format!("grok-resolve-{}-{:?}", std::process::id(), {
use std::time::{SystemTime, UNIX_EPOCH};
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_nanos()
}));
std::fs::create_dir_all(&dir).unwrap();
let envp = dir.join("env");
let bundled = dir.join("bundled");
let vendor = dir.join("vendor");
for p in [&envp, &bundled, &vendor] {
std::fs::write(p, b"x").unwrap();
}
// env override beats everything.
assert_eq!(
resolve_tool_from(
Some(envp.clone()),
Some(bundled.clone()),
vendor.clone(),
"bfs"
)
.as_deref(),
Some(envp.as_path())
);
// bundled beats the manual vendor copy.
assert_eq!(
resolve_tool_from(None, Some(bundled.clone()), vendor.clone(), "bfs").as_deref(),
Some(bundled.as_path())
);
// vendor used when neither env nor bundled is present.
assert_eq!(
resolve_tool_from(None, None, vendor.clone(), "bfs").as_deref(),
Some(vendor.as_path())
);
// A non-file env override is ignored and falls through to vendor.
assert_eq!(
resolve_tool_from(Some(dir.join("missing")), None, vendor.clone(), "bfs").as_deref(),
Some(vendor.as_path())
);
let _ = std::fs::remove_dir_all(&dir);
}
/// Only compiled when the binaries are actually bundled (release pipeline, or
/// `KIGI_TOOLS_BUNDLE_{BFS,UGREP}_PATH` at build time). Verifies the embedded
/// bytes self-extract under `~/.kigi/vendor` and the extracted `bfs` runs.
#[cfg(all(bundle_bfs, bundle_ugrep))]
#[test]
fn bundled_binaries_extract_and_run() {
let vendor = crate::util::kigi_home().join("vendor");
let bfs = bundled_bfs().expect("bfs should be bundled");
let ugrep = bundled_ugrep().expect("ugrep should be bundled");
assert!(bfs.is_file() && bfs.starts_with(&vendor), "bfs at {bfs:?}");
assert!(
ugrep.is_file() && ugrep.starts_with(&vendor),
"ugrep at {ugrep:?}"
);
let v = std::process::Command::new(&bfs)
.arg("--version")
.output()
.unwrap();
assert!(
String::from_utf8_lossy(&v.stdout)
.to_lowercase()
.contains("bfs"),
"bfs --version: {}",
String::from_utf8_lossy(&v.stdout)
);
}
/// Regression: a bare `$ZSH_VERSION` probe aborted the shadow under `set -u`
/// ("unbound variable"), killing find/grep and dropping the state dump. Runs
/// the generated function with `/bin/echo` standing in for the binary.
#[test]
fn shadow_runs_under_nounset_bash() {
let Ok(bash) = which::which("bash") else {
return;
};
let inject = shell_function("find", "bfs", Some(Path::new("/bin/echo")), &[]);
let script = format!("set -euo pipefail; {inject}; find hello world");
let out = std::process::Command::new(&bash)
.args(["-c", &script])
.output()
.unwrap();
assert!(
out.status.success(),
"find shadow aborted under set -u: {:?}\nstderr: {}",
out.status,
String::from_utf8_lossy(&out.stderr)
);
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "hello world");
}
/// Command substitution runs with `BASH_SUBSHELL > 0`, taking the direct
/// `exec -a` branch; it must still forward args and succeed under `set -u`,
/// and the outer shell must survive (so its state dump can run).
#[test]
fn shadow_execs_directly_in_subshell_bash() {
let Ok(bash) = which::which("bash") else {
return;
};
let inject = shell_function("find", "bfs", Some(Path::new("/bin/echo")), &[]);
let script =
format!("set -euo pipefail; {inject}; printf '[%s]' \"$(find sub shell)\"; echo ALIVE");
let out = std::process::Command::new(&bash)
.args(["-c", &script])
.output()
.unwrap();
assert!(
out.status.success(),
"stderr: {}",
String::from_utf8_lossy(&out.stderr)
);
assert_eq!(String::from_utf8_lossy(&out.stdout), "[sub shell]ALIVE\n");
}
/// zsh takes the subshell `exec -a` branch (consistent argv0 with bash); it
/// must run under nounset without referencing the bash-only `BASH_SUBSHELL`.
#[test]
fn shadow_runs_under_nounset_zsh() {
let Ok(zsh) = which::which("zsh") else {
return;
};
let inject = shell_function("grep", "ugrep", Some(Path::new("/bin/echo")), &[]);
let script = format!("setopt nounset errexit pipefail; {inject}; grep hi there");
let out = std::process::Command::new(&zsh)
.args(["-c", &script])
.output()
.unwrap();
assert!(
out.status.success(),
"grep shadow aborted in zsh: {:?}\nstderr: {}",
out.status,
String::from_utf8_lossy(&out.stderr)
);
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "hi there");
}
/// #3 regression: a user-defined `find` must survive a disabled-tool restore.
/// Without the marker gate, the unconditional `unset -f find` dropped it.
#[test]
fn restore_preserves_user_function_without_marker() {
let Ok(bash) = which::which("bash") else {
return;
};
let restore = restore_command("find");
let script =
format!("set -euo pipefail; find() {{ echo USERFIND; }}; {restore}; find anything");
let out = std::process::Command::new(&bash)
.args(["-c", &script])
.output()
.unwrap();
assert!(
out.status.success(),
"stderr: {}",
String::from_utf8_lossy(&out.stderr)
);
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "USERFIND");
}
/// The marker-gated restore *does* drop a prior harness shadow (which set the
/// marker), so a flipped-off tool falls back to the OS binary.
#[test]
fn restore_removes_harness_shadow_with_marker() {
let Ok(bash) = which::which("bash") else {
return;
};
let shadow = shell_function("find", "bfs", Some(Path::new("/bin/echo")), &[]);
let restore = restore_command("find");
let script = format!("set -euo pipefail; {shadow}; {restore}; type -t find || echo NOFUNC");
let out = std::process::Command::new(&bash)
.args(["-c", &script])
.output()
.unwrap();
assert!(
out.status.success(),
"stderr: {}",
String::from_utf8_lossy(&out.stderr)
);
let stdout = String::from_utf8_lossy(&out.stdout);
assert!(
!stdout.lines().any(|l| l.trim() == "function"),
"find should no longer be a function after restore: {stdout:?}"
);
}
/// #1 + #4: the host hint points at a missing file, but the binary is on the
/// live shell `PATH` — the shadow re-resolves via `command -v` at call time.
#[test]
fn shadow_self_heals_via_path_lookup() {
let Ok(bash) = which::which("bash") else {
return;
};
let dir = std::env::temp_dir().join(format!(
"grok-selfheal-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
std::fs::create_dir_all(&dir).unwrap();
let bin = dir.join("myfind");
std::fs::write(&bin, "#!/bin/sh\necho SELFHEAL \"$@\"\n").unwrap();
{
use std::os::unix::fs::PermissionsExt;
let mut perms = std::fs::metadata(&bin).unwrap().permissions();
perms.set_mode(0o755);
std::fs::set_permissions(&bin, perms).unwrap();
}
// Hint is a nonexistent path; `myfind` is only reachable via PATH.
let shadow = shell_function("find", "myfind", Some(Path::new("/nonexistent/bfs")), &[]);
let script = format!(
"set -euo pipefail; export PATH={}:\"$PATH\"; {shadow}; find X",
bash_safe_quote(&dir.to_string_lossy())
);
let out = std::process::Command::new(&bash)
.args(["-c", &script])
.output()
.unwrap();
let _ = std::fs::remove_dir_all(&dir);
assert!(
out.status.success(),
"stderr: {}",
String::from_utf8_lossy(&out.stderr)
);
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "SELFHEAL X");
}
/// #1 fallback: neither the hint nor `command -v` resolves → the OS binary
/// runs (via `command find`), so the shadow never breaks `find`.
#[test]
fn shadow_falls_back_to_os_when_binary_absent() {
let Ok(bash) = which::which("bash") else {
return;
};
let shadow = shell_function(
"find",
"grok_no_such_search_bin_xyz",
Some(Path::new("/nonexistent/bfs")),
&[],
);
// `find /dev/null` prints the path on every find implementation.
let script = format!("set -euo pipefail; {shadow}; find /dev/null");
let out = std::process::Command::new(&bash)
.args(["-c", &script])
.output()
.unwrap();
assert!(
out.status.success(),
"stderr: {}",
String::from_utf8_lossy(&out.stderr)
);
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "/dev/null");
}
/// #1 regression: a host hint that exists but is **not executable** (e.g. a
/// mode-0644 `KIGI_TOOLS_*_PATH` / vendor copy) must fall through to the OS
/// binary rather than hard-fail `exec` with EACCES. The `[ -x ]` guard (not
/// `[ -f ]`) is what makes this work.
#[test]
fn shadow_falls_back_when_hint_not_executable() {
let Ok(bash) = which::which("bash") else {
return;
};
let dir = std::env::temp_dir().join(format!(
"grok-noexec-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
std::fs::create_dir_all(&dir).unwrap();
let hint = dir.join("bfs");
std::fs::write(&hint, "#!/bin/sh\necho SHOULD_NOT_RUN\n").unwrap();
{
// Mode 0644 — exists but not executable by anyone.
use std::os::unix::fs::PermissionsExt;
let mut perms = std::fs::metadata(&hint).unwrap().permissions();
perms.set_mode(0o644);
std::fs::set_permissions(&hint, perms).unwrap();
}
// Hint is the non-exec file; bin_name isn't on PATH → must reach OS find.
let shadow = shell_function(
"find",
"grok_no_such_search_bin_xyz",
Some(hint.as_path()),
&[],
);
let script = format!("set -euo pipefail; {shadow}; find /dev/null");
let out = std::process::Command::new(&bash)
.args(["-c", &script])
.output()
.unwrap();
let _ = std::fs::remove_dir_all(&dir);
assert!(
out.status.success(),
"non-exec hint should fall back to OS find, not fail: {:?}\nstderr: {}",
out.status,
String::from_utf8_lossy(&out.stderr)
);
let stdout = String::from_utf8_lossy(&out.stdout);
assert_eq!(stdout.trim(), "/dev/null");
assert!(
!stdout.contains("SHOULD_NOT_RUN"),
"non-exec hint was run: {stdout:?}"
);
}
}
@@ -0,0 +1,344 @@
use std::{future::Future, io, path::Path, time::Duration};
use tokio::{fs, time::sleep};
use crate::computer::types::{AsyncFileSystem, ComputerError};
/// Creates a local FS access which allows writing and reading from the local files
pub struct LocalFs;
// Keep the window short: these retries absorb brief Windows editor/indexer/AV
// races without hiding persistent locks, ACL failures, or sandbox denials.
const WRITE_RETRY_DELAYS: &[Duration] = &[
Duration::from_millis(25),
Duration::from_millis(50),
Duration::from_millis(100),
Duration::from_millis(200),
Duration::from_millis(400),
];
#[cfg(any(windows, test))]
const WINDOWS_ERROR_SHARING_VIOLATION: i32 = 32;
#[cfg(any(windows, test))]
const WINDOWS_ERROR_LOCK_VIOLATION: i32 = 33;
/// Check if an IO error is a permission denial (EACCES or EPERM),
/// which indicates a sandbox violation.
fn is_permission_error(e: &io::Error) -> bool {
matches!(e.kind(), io::ErrorKind::PermissionDenied)
}
#[cfg(any(windows, test))]
fn is_windows_transient_write_lock_raw_os_error(raw_os_error: Option<i32>) -> bool {
matches!(
raw_os_error,
Some(WINDOWS_ERROR_SHARING_VIOLATION | WINDOWS_ERROR_LOCK_VIOLATION)
)
}
fn is_transient_write_lock_error(e: &io::Error) -> bool {
#[cfg(windows)]
{
is_windows_transient_write_lock_raw_os_error(e.raw_os_error())
}
#[cfg(not(windows))]
{
let _ = e;
false
}
}
#[cfg(test)]
fn is_test_transient_write_lock_error(e: &io::Error) -> bool {
is_windows_transient_write_lock_raw_os_error(e.raw_os_error())
}
async fn write_file_with_transient_lock_retries(path: &Path, data: &[u8]) -> io::Result<()> {
write_file_with_retry_hooks(
|| fs::write(path, data),
|delay| sleep(delay),
|retry_count| {
tracing::debug!(
path = %path.display(),
retry_count,
"file write succeeded after transient lock retries"
);
},
|error, retry_count, delay| {
tracing::debug!(
path = %path.display(),
error = %error,
retry_count,
delay_ms = delay.as_millis(),
"file write hit transient lock; retrying"
);
},
|error, retry_count| {
tracing::debug!(
path = %path.display(),
error = %error,
retry_count,
"file write exhausted transient lock retries"
);
},
is_transient_write_lock_error,
)
.await
}
async fn write_file_with_retry_hooks<W, WFut, S, SFut, Success, Retry, Exhausted, IsRetryable>(
mut write: W,
mut sleep_for: S,
mut on_retry_success: Success,
mut on_retry: Retry,
mut on_exhausted: Exhausted,
is_retryable: IsRetryable,
) -> io::Result<()>
where
W: FnMut() -> WFut,
WFut: Future<Output = io::Result<()>>,
S: FnMut(Duration) -> SFut,
SFut: Future<Output = ()>,
Success: FnMut(usize),
Retry: FnMut(&io::Error, usize, Duration),
Exhausted: FnMut(&io::Error, usize),
IsRetryable: Fn(&io::Error) -> bool,
{
let mut retry_count = 0usize;
loop {
match write().await {
Ok(()) => {
if retry_count > 0 {
on_retry_success(retry_count);
}
return Ok(());
}
Err(e) if is_retryable(&e) => {
if retry_count >= WRITE_RETRY_DELAYS.len() {
on_exhausted(&e, retry_count);
return Err(e);
}
let delay = WRITE_RETRY_DELAYS[retry_count];
retry_count += 1;
on_retry(&e, retry_count, delay);
sleep_for(delay).await;
}
Err(e) => return Err(e),
}
}
}
#[async_trait::async_trait]
impl AsyncFileSystem for LocalFs {
#[tracing::instrument(name = "fs.read_file", skip_all)]
async fn read_file(&self, path: &Path) -> Result<Vec<u8>, ComputerError> {
match fs::read(path).await {
Ok(data) => Ok(data),
Err(e) => {
if is_permission_error(&e) {
kigi_sandbox::log_violation(&path.display().to_string(), "read");
}
Err(e.into())
}
}
}
#[tracing::instrument(name = "fs.write_file", skip_all)]
async fn write_file(&self, path: &Path, data: &[u8]) -> Result<(), ComputerError> {
// implicitly creates the missing directories if any
if let Some(dir) = path.parent()
&& let Err(e) = fs::create_dir_all(dir).await
{
if is_permission_error(&e) {
kigi_sandbox::log_violation(&dir.display().to_string(), "mkdir");
}
return Err(e.into());
}
if let Err(e) = write_file_with_transient_lock_retries(path, data).await {
if is_permission_error(&e) {
kigi_sandbox::log_violation(&path.display().to_string(), "write");
}
return Err(e.into());
}
Ok(())
}
#[tracing::instrument(name = "fs.delete_file", skip_all)]
async fn delete_file(&self, path: &Path) -> Result<(), ComputerError> {
if let Err(e) = fs::remove_file(path).await {
if is_permission_error(&e) {
kigi_sandbox::log_violation(&path.display().to_string(), "delete");
}
return Err(e.into());
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::cell::Cell;
use std::rc::Rc;
#[test]
fn classifies_windows_transient_write_lock_errors() {
assert!(is_windows_transient_write_lock_raw_os_error(Some(32)));
assert!(is_windows_transient_write_lock_raw_os_error(Some(33)));
assert!(!is_windows_transient_write_lock_raw_os_error(Some(5)));
assert!(!is_windows_transient_write_lock_raw_os_error(None));
}
#[cfg(windows)]
#[test]
fn classifies_windows_transient_write_lock_io_errors() {
assert!(is_transient_write_lock_error(
&io::Error::from_raw_os_error(WINDOWS_ERROR_SHARING_VIOLATION,)
));
assert!(is_transient_write_lock_error(
&io::Error::from_raw_os_error(WINDOWS_ERROR_LOCK_VIOLATION,)
));
assert!(!is_transient_write_lock_error(
&io::Error::from_raw_os_error(5)
));
}
#[cfg(not(windows))]
#[test]
fn non_windows_does_not_retry_windows_raw_error_numbers() {
assert!(!is_transient_write_lock_error(
&io::Error::from_raw_os_error(WINDOWS_ERROR_SHARING_VIOLATION,)
));
}
#[tokio::test]
async fn first_attempt_success_does_not_fire_retry_callbacks() {
let result = write_file_with_retry_hooks(
|| async { Ok(()) },
|_| async { panic!("should not sleep") },
|_| panic!("should not fire on_retry_success"),
|_, _, _| panic!("should not fire on_retry"),
|_, _| panic!("should not fire on_exhausted"),
is_test_transient_write_lock_error,
)
.await;
result.unwrap();
}
#[tokio::test]
async fn transient_lock_errors_are_retried_until_success() {
let attempts = Rc::new(Cell::new(0usize));
let sleeps = Rc::new(Cell::new(0usize));
let retry_success_count = Rc::new(Cell::new(0usize));
let retry_log_count = Rc::new(Cell::new(0usize));
let result = write_file_with_retry_hooks(
{
let attempts = Rc::clone(&attempts);
move || {
let attempts = Rc::clone(&attempts);
async move {
let next = attempts.get() + 1;
attempts.set(next);
if next <= 2 {
Err(io::Error::from_raw_os_error(
WINDOWS_ERROR_SHARING_VIOLATION,
))
} else {
Ok(())
}
}
}
},
{
let sleeps = Rc::clone(&sleeps);
move |_| {
let sleeps = Rc::clone(&sleeps);
async move {
sleeps.set(sleeps.get() + 1);
}
}
},
|count| retry_success_count.set(count),
|_, _, _| retry_log_count.set(retry_log_count.get() + 1),
|_, _| panic!("retry budget should not be exhausted"),
is_test_transient_write_lock_error,
)
.await;
result.unwrap();
assert_eq!(attempts.get(), 3);
assert_eq!(sleeps.get(), 2);
assert_eq!(retry_log_count.get(), 2);
assert_eq!(retry_success_count.get(), 2);
}
#[tokio::test]
async fn non_transient_errors_are_not_retried() {
let attempts = Rc::new(Cell::new(0usize));
let result = write_file_with_retry_hooks(
{
let attempts = Rc::clone(&attempts);
move || {
let attempts = Rc::clone(&attempts);
async move {
attempts.set(attempts.get() + 1);
Err(io::Error::new(io::ErrorKind::NotFound, "missing"))
}
}
},
|_| async {},
|_| panic!("write did not succeed"),
|_, _, _| panic!("non-transient errors must not be retried"),
|_, _| panic!("non-transient errors must not exhaust retry budget"),
is_test_transient_write_lock_error,
)
.await;
assert_eq!(result.unwrap_err().kind(), io::ErrorKind::NotFound);
assert_eq!(attempts.get(), 1);
}
#[tokio::test]
async fn persistent_transient_lock_exhausts_retry_budget() {
let attempts = Rc::new(Cell::new(0usize));
let sleeps = Rc::new(Cell::new(0usize));
let exhausted_count = Rc::new(Cell::new(0usize));
let result = write_file_with_retry_hooks(
{
let attempts = Rc::clone(&attempts);
move || {
let attempts = Rc::clone(&attempts);
async move {
attempts.set(attempts.get() + 1);
Err(io::Error::from_raw_os_error(WINDOWS_ERROR_LOCK_VIOLATION))
}
}
},
{
let sleeps = Rc::clone(&sleeps);
move |_| {
let sleeps = Rc::clone(&sleeps);
async move {
sleeps.set(sleeps.get() + 1);
}
}
},
|_| panic!("write did not succeed"),
|_, _, _| {},
|_, count| exhausted_count.set(count),
is_test_transient_write_lock_error,
)
.await;
assert_eq!(
result.unwrap_err().raw_os_error(),
Some(WINDOWS_ERROR_LOCK_VIOLATION)
);
assert_eq!(attempts.get(), WRITE_RETRY_DELAYS.len() + 1);
assert_eq!(sleeps.get(), WRITE_RETRY_DELAYS.len());
assert_eq!(exhausted_count.get(), WRITE_RETRY_DELAYS.len());
}
}
@@ -0,0 +1,122 @@
//! Mock file system implementation for testing.
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use tokio::sync::RwLock;
use crate::computer::types::{AsyncFileSystem, ComputerError};
/// In-memory file system for testing.
/// Thread-safe and async-compatible.
pub struct MockFs {
files: Arc<RwLock<HashMap<PathBuf, Vec<u8>>>>,
}
impl Default for MockFs {
fn default() -> Self {
Self::new()
}
}
impl MockFs {
/// Create a new empty mock file system.
pub fn new() -> Self {
Self {
files: Arc::new(RwLock::new(HashMap::new())),
}
}
/// Set a file's contents directly (for test setup).
pub async fn set_file(&self, path: impl AsRef<Path>, content: &[u8]) {
self.files
.write()
.await
.insert(path.as_ref().to_path_buf(), content.to_vec());
}
/// Get a file's contents directly (for test assertions).
pub async fn get_file(&self, path: impl AsRef<Path>) -> Option<Vec<u8>> {
self.files.read().await.get(path.as_ref()).cloned()
}
/// Check if a file exists.
pub async fn exists(&self, path: impl AsRef<Path>) -> bool {
self.files.read().await.contains_key(path.as_ref())
}
/// List all files in the mock filesystem.
pub async fn list_files(&self) -> Vec<PathBuf> {
self.files.read().await.keys().cloned().collect()
}
}
#[async_trait::async_trait]
impl AsyncFileSystem for MockFs {
async fn read_file(&self, path: &Path) -> Result<Vec<u8>, ComputerError> {
self.files.read().await.get(path).cloned().ok_or_else(|| {
ComputerError::IOError(
format!("File not found: {}", path.display()),
Some(std::io::ErrorKind::NotFound),
)
})
}
async fn write_file(&self, path: &Path, data: &[u8]) -> Result<(), ComputerError> {
self.files
.write()
.await
.insert(path.to_path_buf(), data.to_vec());
Ok(())
}
async fn delete_file(&self, path: &Path) -> Result<(), ComputerError> {
self.files.write().await.remove(path);
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_mock_fs_read_write() {
let fs = MockFs::new();
// File doesn't exist initially
assert!(fs.read_file(Path::new("/test.txt")).await.is_err());
// Write a file
fs.write_file(Path::new("/test.txt"), b"hello world")
.await
.unwrap();
// Read it back
let content = fs.read_file(Path::new("/test.txt")).await.unwrap();
assert_eq!(content, b"hello world");
}
#[tokio::test]
async fn test_mock_fs_delete() {
let fs = MockFs::new();
fs.write_file(Path::new("/test.txt"), b"hello")
.await
.unwrap();
assert!(fs.exists(Path::new("/test.txt")).await);
fs.delete_file(Path::new("/test.txt")).await.unwrap();
assert!(!fs.exists(Path::new("/test.txt")).await);
}
#[tokio::test]
async fn test_mock_fs_set_file() {
let fs = MockFs::new();
fs.set_file("/preset.txt", b"preset content").await;
let content = fs.read_file(Path::new("/preset.txt")).await.unwrap();
assert_eq!(content, b"preset content");
}
}
@@ -0,0 +1,37 @@
pub mod cgroup;
#[cfg(unix)]
pub mod embedded_search_tools;
pub mod file_system;
pub mod mock_fs;
#[cfg(unix)]
pub mod shell_state;
pub mod terminal;
pub use cgroup::{CgroupMemoryConfig, PROCESS_OOM_EXIT_CODE};
pub use file_system::LocalFs;
pub use mock_fs::MockFs;
pub use terminal::{ExitStatus, LocalTerminalBackend};
/// Per-backend enable state for the bash-harness `find`→`bfs` / `grep`→`ugrep`
/// shadows.
///
/// Resolved once by the host (config.toml `[toolset.bash]` / env / requirements)
/// and baked into a [`LocalTerminalBackend`] at creation. Keeping it on the
/// backend instead of a process-global means a subagent that reuses the parent's
/// `LocalTerminalBackend` inherits the parent's shadows — it can't overwrite the
/// enable state for bash that later runs on the shared backend. Defaults to
/// both-on for standalone backends with no host wiring.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct SearchShadowConfig {
pub find_bfs: bool,
pub grep_ugrep: bool,
}
impl Default for SearchShadowConfig {
fn default() -> Self {
Self {
find_bfs: true,
grep_ugrep: true,
}
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,3 @@
pub mod local;
/// Contains the computer implementation
pub mod types;
@@ -0,0 +1,422 @@
use std::{
collections::HashMap,
path::{Path, PathBuf},
sync::Arc,
time::Duration,
};
use crate::notification::types::ToolNotificationHandle;
// ============================================================================
// Error types
// ============================================================================
#[derive(thiserror::Error, Debug, Clone)]
pub enum ComputerError {
#[error("IO Error: {0}")]
IOError(String, Option<std::io::ErrorKind>),
#[error("UnQuoted command")]
CommandNotQuoted,
}
impl ComputerError {
/// Create an IO error from a message string with no preserved error kind.
pub fn io(msg: impl Into<String>) -> Self {
Self::IOError(msg.into(), None)
}
pub fn io_with_kind(msg: impl Into<String>, kind: std::io::ErrorKind) -> Self {
Self::IOError(msg.into(), Some(kind))
}
/// Returns the underlying `io::ErrorKind` or `None`
pub fn io_error_kind(&self) -> Option<std::io::ErrorKind> {
match self {
Self::IOError(_, kind) => *kind,
_ => None,
}
}
}
impl From<std::io::Error> for ComputerError {
fn from(err: std::io::Error) -> Self {
Self::IOError(err.to_string(), Some(err.kind()))
}
}
// ============================================================================
// File system trait
// ============================================================================
#[async_trait::async_trait]
pub trait AsyncFileSystem: Send + Sync {
async fn read_file(&self, path: &Path) -> Result<Vec<u8>, ComputerError>;
async fn write_file(&self, path: &Path, data: &[u8]) -> Result<(), ComputerError>;
async fn delete_file(&self, path: &Path) -> Result<(), ComputerError>;
}
// ============================================================================
// Terminal types
// ============================================================================
pub struct TerminalRunRequest {
pub command: String,
pub working_directory: PathBuf,
pub env: HashMap<String, String>,
pub timeout: Duration,
pub output_byte_limit: usize,
/// File path to write output incrementally as it arrives.
/// This ensures full output is always available even after in-memory buffer is truncated.
/// For background tasks, this allows retrieval of output after the agent has moved on.
pub output_file: PathBuf,
/// Notification handle for streaming output chunks during execution.
/// The backend sends `BashOutputChunk` notifications every ~100ms.
/// Callers that don't need streaming pass `ToolNotificationHandle::noop()`
/// — messages are silently dropped. No `Option` wrapper needed.
pub notification_handle: ToolNotificationHandle,
/// Tool call ID for correlating notifications with the tool invocation.
/// Flows from `ToolContext::tool_call_id()` through the actor to
/// `BashOutputChunk.base.tool_call_id`.
pub tool_call_id: String,
/// Original user command before isolation wrapping.
///
/// When set, the terminal actor stores this on the `ProcessEntry` so
/// `get_task()` returns it in `TaskSnapshot.display_command`. This
/// ensures model-facing `get_task_output` shows the user's command
/// instead of the `unshare`/mount wrapper.
pub display_command: Option<String>,
/// Auto-background on timeout instead of killing (default `false`).
pub auto_background_on_timeout: bool,
/// When [`Self::auto_background_on_timeout`] is true, maximum time the
/// command may block the turn before being moved to the background (process
/// keeps running). Independent of [`Self::timeout`].
///
/// - `None` → use the terminal backend default (typically 15s).
/// - `Some(Duration::MAX)` → no short budget; auto-bg only when `timeout` elapses.
/// - `Some(d)` → auto-bg after `d` if still running.
pub foreground_block_budget: Option<Duration>,
/// Task kind for distinguishing monitor tasks from regular bash tasks.
pub kind: TaskKind,
/// Session that owns this process. Used to scope kill operations so
/// `kill_all_background_tasks_by_owner` only targets the requesting
/// session's processes — not the parent's or sibling's.
pub owner_session_id: Option<String>,
}
/// Distinguishes different types of background tasks.
#[derive(
Debug,
Clone,
Copy,
PartialEq,
Eq,
Default,
serde::Serialize,
serde::Deserialize,
schemars::JsonSchema,
)]
#[serde(rename_all = "snake_case")]
pub enum TaskKind {
/// Regular bash command.
#[default]
Bash,
/// Monitor tool — streams stdout events with rate limiting.
Monitor,
}
#[derive(Clone)]
pub struct TerminalRunResult {
pub combined_output: String,
pub exit_code: Option<i32>,
pub truncated: bool,
pub signal: Option<String>,
pub timed_out: bool,
/// Path to the output file where full output is stored.
/// Use read_file tool to retrieve full output when truncated.
pub output_file: PathBuf,
/// Total bytes of output (before truncation).
/// When truncated, combined_output contains the first and last portions up to output_byte_limit chars.
pub total_bytes: usize,
/// PID of the spawned shell process, when available. Set by the
/// local terminal backend at spawn time. Useful for foreground
/// commands that auto-background on timeout: the resulting
/// `BackgroundTaskStarted` can carry the real PID instead of a
/// placeholder. `None` for backends that cannot surface a local
/// PID (e.g. ACP/remote terminals) or when the process exited
/// before `child.id()` could be queried.
pub pid: Option<u32>,
}
/// Returned by `TerminalBackend::run_background` — gives the caller the task_id
/// to use for subsequent queries via `get_task`, `kill_task`, `wait_for_completion`.
pub struct BackgroundHandle {
pub task_id: String,
pub output_file: PathBuf,
/// PID of the spawned shell process, when available. `None` for
/// backends that do not surface a local PID (e.g. ACP/remote
/// gateways) or when the process exited before the PID could be
/// captured.
pub pid: Option<u32>,
}
/// Full snapshot of a task's state.
/// Used by both local and ACP backends.
#[derive(
Debug, Clone, Eq, PartialEq, serde::Serialize, serde::Deserialize, schemars::JsonSchema,
)]
pub struct TaskSnapshot {
pub task_id: String,
/// The actual command that was executed (may be isolation-wrapped).
pub command: String,
/// The original user command before isolation wrapping.
///
/// When set, model/user-facing output should prefer this over `command`
/// to avoid exposing internal isolation mechanics (unshare/mount wrapper).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub display_command: Option<String>,
pub cwd: String,
pub start_time: std::time::SystemTime,
pub end_time: Option<std::time::SystemTime>,
pub output: String,
pub output_file: PathBuf,
pub truncated: bool,
pub exit_code: Option<i32>,
pub signal: Option<String>,
pub completed: bool,
/// Task kind: bash (default) or monitor.
#[serde(default)]
pub kind: TaskKind,
/// Whether a block-waiter (`block=true`) consumed this task's result.
/// When set, the notification bridge skips auto-wake synthetic prompts
/// because the blocking caller already received the result directly.
#[serde(default)]
pub block_waited: bool,
/// Whether this task was explicitly killed via the `kill_command_or_subagent` tool.
/// When set, auto-wake synthetic prompts are suppressed because the model
/// already received the kill result via `KillTaskResult`.
/// Also set during `kill_all_background_tasks` teardown (e.g. subagent
/// cleanup), where auto-wake suppression is irrelevant since the session
/// is shutting down.
#[serde(default)]
pub explicitly_killed: bool,
/// Session that owns this task. Used for scoped kill operations so
/// subagent teardown only kills the subagent's own tasks, not
/// the parent's or sibling's.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub owner_session_id: Option<String>,
}
impl TaskSnapshot {
/// Calculate duration in seconds.
/// If task is still running, returns time since start.
pub fn duration_secs(&self) -> f64 {
let end = self.end_time.unwrap_or_else(std::time::SystemTime::now);
end.duration_since(self.start_time)
.map(|d| d.as_secs_f64())
.unwrap_or(0.0)
}
/// True iff the task has NOT yet completed — covers bash AND
/// monitor task kinds (the `kind` field doesn't change this
/// predicate; the runtime turn-end TodoGate counts both as
/// backing work).
pub fn is_outstanding(&self) -> bool {
!self.completed
}
}
/// Result of killing a terminal task.
///
/// Serialized over the wire in the `x.ai/task/kill` ext response
/// (`kigi-shell::extensions::task::KillTaskResponse`) and deserialized
/// by clients (kigi-tui), so it derives both serde directions.
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum KillOutcome {
Killed,
AlreadyExited,
NotFound,
}
// ============================================================================
// TerminalBackend trait
// ============================================================================
/// The single abstraction over terminal execution backends.
///
/// Implemented by:
/// - `LocalTerminalBackend` (in kigi-tools, spawns processes)
/// - `AcpTerminalBackend` (in kigi-shell, calls ACP protocol)
#[async_trait::async_trait]
pub trait TerminalBackend: Send + Sync {
/// Run a command. Blocks until completion or timeout.
async fn run(&self, request: TerminalRunRequest) -> Result<TerminalRunResult, ComputerError>;
/// Start a command in the background. Returns immediately with a handle ID.
/// The process continues running; use get_task/kill_task/wait_for_completion to manage it.
async fn run_background(
&self,
request: TerminalRunRequest,
) -> Result<BackgroundHandle, ComputerError>;
/// Get current snapshot of a background task.
async fn get_task(&self, task_id: &str) -> Option<TaskSnapshot>;
/// Kill a background task.
async fn kill_task(&self, task_id: &str) -> KillOutcome;
/// Kill all running foreground processes.
async fn kill_foreground_commands(&self) {}
/// Kill all running foreground processes owned by a specific session.
/// Used on a shared terminal backend so a subagent's cancel doesn't
/// kill the parent's foreground commands.
async fn kill_foreground_commands_by_owner(&self, _owner_session_id: &str) {}
/// Kill all running background tasks.
/// Used during subagent teardown to clean up orphaned processes.
async fn kill_all_background_tasks(&self) {}
/// Kill all running background tasks owned by a specific session.
/// Used during subagent teardown on a shared terminal backend so
/// only the subagent's own tasks are killed — not the parent's.
async fn kill_all_background_tasks_by_owner(&self, _owner_session_id: &str) {}
/// Fire-and-forget prewarm of the persistent login shell; default no-op for
/// backends without one (ACP/remote, non-persistent).
async fn warm_persistent_shell(&self, _cwd: &std::path::Path) {}
/// Reparent notification handles for all tasks owned by `old_owner_session_id`.
/// Swaps the dead child session's notification handle with the parent's
/// live handle so events from surviving processes route correctly.
/// Also re-spawns monitor pipelines on the caller's runtime so monitor
/// events continue streaming to the parent.
///
/// `backend_weak` is a [`Weak`](std::sync::Weak) to *this* backend (anchored
/// by the parent session's `Arc`); it drives re-spawned monitor pipelines
/// without keeping the backend alive. See `run_monitor_pipeline`.
async fn reparent_notifications(
&self,
_old_owner_session_id: &str,
_new_owner_session_id: &str,
_new_handle: crate::notification::types::ToolNotificationHandle,
_backend_weak: std::sync::Weak<dyn TerminalBackend>,
) {
}
/// Move a foreground command to background by tool_call_id.
/// The process keeps running but the foreground waiter is unblocked.
/// Returns `true` if a matching foreground process was found.
async fn background_foreground_command(&self, _tool_call_id: &str) -> bool {
false
}
/// Wait for a background task to complete, with optional timeout.
async fn wait_for_completion(
&self,
task_id: &str,
timeout: Option<Duration>,
) -> Option<TaskSnapshot>;
/// List all known background tasks (running and completed).
/// Used for context compaction to include task state in summaries.
async fn list_tasks(&self) -> Vec<TaskSnapshot>;
/// Return the persistent shell's current working directory, if persistent
/// shell state is enabled. Returns `None` when persistence is off or the
/// backend doesn't support it (e.g. ACP/remote).
async fn get_shell_cwd(&self) -> Option<std::path::PathBuf> {
None
}
}
// ============================================================================
// Computer struct
// ============================================================================
/// Contains the computer struct which provides access to both the terminal and the fs
pub struct Computer {
pub terminal: Arc<dyn TerminalBackend>,
pub file_system: Arc<dyn AsyncFileSystem>,
}
impl Computer {
pub fn new(terminal: Arc<dyn TerminalBackend>, file_system: Arc<dyn AsyncFileSystem>) -> Self {
Self {
terminal,
file_system,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn io_error_kind_preserved_through_from() {
let io_err = std::io::Error::new(std::io::ErrorKind::NotFound, "gone");
let ce = ComputerError::from(io_err);
assert_eq!(ce.io_error_kind(), Some(std::io::ErrorKind::NotFound));
}
#[test]
fn io_error_kind_is_a_directory() {
let io_err = std::io::Error::new(std::io::ErrorKind::IsADirectory, "it's a dir");
let ce = ComputerError::from(io_err);
assert_eq!(ce.io_error_kind(), Some(std::io::ErrorKind::IsADirectory));
}
#[test]
fn io_error_kind_none_for_string_constructor() {
let ce = ComputerError::io("something broke");
assert_eq!(ce.io_error_kind(), None);
}
#[test]
fn io_error_kind_none_for_non_io_variant() {
let ce = ComputerError::CommandNotQuoted;
assert_eq!(ce.io_error_kind(), None);
}
#[test]
fn io_with_kind_preserves_not_found() {
let ce = ComputerError::io_with_kind("Resource not found", std::io::ErrorKind::NotFound);
assert_eq!(ce.io_error_kind(), Some(std::io::ErrorKind::NotFound));
}
#[test]
fn io_with_kind_preserves_permission_denied() {
let ce = ComputerError::io_with_kind("access denied", std::io::ErrorKind::PermissionDenied);
assert_eq!(
ce.io_error_kind(),
Some(std::io::ErrorKind::PermissionDenied)
);
}
#[test]
fn io_with_kind_matches_local_fs_dispatch_for_not_found() {
// Simulate what LocalFs produces for a missing file
let local_err = ComputerError::from(std::io::Error::new(
std::io::ErrorKind::NotFound,
"No such file or directory (os error 2)",
));
// Simulate what AcpFsAdapter now produces for RESOURCE_NOT_FOUND
let acp_err =
ComputerError::io_with_kind("Resource not found", std::io::ErrorKind::NotFound);
// Both must produce the same io_error_kind so read_file dispatches
// to FileNotFound("Error: {path} does not exist.") in both cases.
assert_eq!(local_err.io_error_kind(), acp_err.io_error_kind());
}
}
+121
View File
@@ -0,0 +1,121 @@
//! Shared gitignore matching utility.
//!
//! Single source of truth for checking whether a path is ignored by
//! `.gitignore` rules. Used by both the initial AGENTS.md discovery
//! (`kigi-agent::prompt::ignore`) and the runtime tracker
//! (`AgentsMdTracker`).
use ignore::gitignore::Gitignore;
use std::path::Path;
/// Check if a path is ignored by the given gitignore rules.
///
/// Strips `git_root` prefix before matching — gitignore patterns are
/// repo-relative, so `/repo/build/out.o` becomes `build/out.o` when
/// `git_root` is `/repo`.
///
/// This is a pure function — no filesystem access, just `Gitignore::matched()`.
pub fn is_ignored(gitignore: &Gitignore, path: &Path, git_root: Option<&Path>) -> bool {
let check_path = match git_root {
Some(root) => match path.strip_prefix(root) {
Ok(relative) => relative,
// Outside the repo (e.g. ~/.kigi/Agents.md) → not ignored.
Err(_) => return false,
},
None => {
// Absolute path + no git root → can't strip to repo-relative;
// the `ignore` crate panics on absolute paths not under root.
if path.is_absolute() {
return false;
}
path
}
};
// matched_path_or_any_parents checks parent dirs too, so
// `build/AGENTS.md` correctly matches a `build/` pattern.
gitignore
.matched_path_or_any_parents(check_path, check_path.is_dir())
.is_ignore()
}
#[cfg(test)]
mod tests {
use super::*;
use ignore::gitignore::GitignoreBuilder;
fn build_gitignore(root: &Path, patterns: &[&str]) -> Gitignore {
let mut builder = GitignoreBuilder::new(root);
for pattern in patterns {
builder.add_line(None, pattern).unwrap();
}
builder.build().unwrap()
}
#[test]
fn is_ignored_matches_gitignored_paths() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
// Canonicalize to handle macOS /tmp → /private/tmp
let root = &dunce::canonicalize(root).unwrap();
let gi = build_gitignore(root, &["build/"]);
assert!(is_ignored(&gi, &root.join("build/out.o"), Some(root)));
assert!(is_ignored(&gi, &root.join("build/sub/file.rs"), Some(root)));
}
#[test]
fn is_ignored_does_not_match_non_gitignored_paths() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let root = &dunce::canonicalize(root).unwrap();
let gi = build_gitignore(root, &["build/"]);
assert!(!is_ignored(&gi, &root.join("src/main.rs"), Some(root)));
assert!(!is_ignored(&gi, &root.join("AGENTS.md"), Some(root)));
}
#[test]
fn is_ignored_strips_git_root_prefix() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let root = &dunce::canonicalize(root).unwrap();
let gi = build_gitignore(root, &["build/"]);
// With root: strips prefix, matches build/out.o
assert!(is_ignored(&gi, &root.join("build/out.o"), Some(root)));
// Without root: relative path still matches
assert!(is_ignored(
&gi,
&std::path::PathBuf::from("build/out.o"),
None
));
}
#[test]
fn is_ignored_returns_false_for_path_outside_git_root() {
let tmp = tempfile::tempdir().unwrap();
let root = tmp.path();
let root = &dunce::canonicalize(root).unwrap();
let gi = build_gitignore(root, &["build/", "*.md"]);
// A path completely outside the git root should not be checked
// against the repo's .gitignore (e.g., ~/.kigi/Agents.md).
let outside_path = std::path::PathBuf::from("/some/other/path/Agents.md");
assert!(!is_ignored(&gi, &outside_path, Some(root)));
}
/// Regression: running outside a git repo panicked with
/// "path is expected to be under the root" (ignore crate assert).
#[test]
fn regression_no_panic_on_absolute_path_without_git_root() {
let gi = build_gitignore(Path::new("."), &["node_modules/", "*.log"]);
let abs_path = Path::new("/Users/someone/home/AGENTS.md");
// Proves the raw crate panics with these inputs.
assert!(
std::panic::catch_unwind(|| {
gi.matched_path_or_any_parents(abs_path, false);
})
.is_err()
);
// Our wrapper guards against it.
assert!(!is_ignored(&gi, abs_path, None));
}
}
@@ -0,0 +1,319 @@
//! Pure (I/O-free) patch application logic.
//!
//! Ported from `codex-rs/apply-patch/src/lib.rs`, but refactored so that
//! **no function in this module touches the filesystem**. Every function
//! accepts `&str` content directly, making the logic trivially testable.
use std::path::Path;
use super::errors::ApplyPatchError;
use super::parser::UpdateFileChunk;
use super::seek_sequence::seek_sequence;
/// Given the original file content as a `&str` and the list of update chunks,
/// compute and return the new file contents as a `String`.
///
/// This is the main entry point for the apply logic. It does NOT read from or
/// write to the filesystem.
pub fn derive_new_contents(
original_content: &str,
path: &Path,
chunks: &[UpdateFileChunk],
) -> Result<String, ApplyPatchError> {
let mut original_lines: Vec<String> = original_content.split('\n').map(String::from).collect();
// Drop the trailing empty element that results from the final newline so
// that line counts match the behaviour of standard `diff`.
if original_lines.last().is_some_and(String::is_empty) {
original_lines.pop();
}
let replacements = compute_replacements(&original_lines, path, chunks)?;
let mut new_lines = apply_replacements(original_lines, &replacements);
// Ensure the file ends with a trailing newline.
if !new_lines.last().is_some_and(String::is_empty) {
new_lines.push(String::new());
}
Ok(new_lines.join("\n"))
}
/// Compute a list of replacements needed to transform `original_lines` into the
/// new lines, given the patch `chunks`. Each replacement is returned as
/// `(start_index, old_len, new_lines)`.
pub fn compute_replacements(
original_lines: &[String],
path: &Path,
chunks: &[UpdateFileChunk],
) -> Result<Vec<(usize, usize, Vec<String>)>, ApplyPatchError> {
let mut replacements: Vec<(usize, usize, Vec<String>)> = Vec::new();
let mut line_index: usize = 0;
for chunk in chunks {
// If a chunk has a `change_context`, use seek_sequence to find it,
// then adjust our `line_index` to continue from there.
if let Some(ctx_line) = &chunk.change_context {
if let Some(idx) = seek_sequence(
original_lines,
std::slice::from_ref(ctx_line),
line_index,
false,
) {
line_index = idx + 1;
} else {
return Err(ApplyPatchError::ComputeReplacements(format!(
"Failed to find context '{}' in {}",
ctx_line,
path.display()
)));
}
}
if chunk.old_lines.is_empty() {
// Pure addition (no old lines). Add at the end or just before the
// final empty line if one exists.
let insertion_idx = if original_lines.last().is_some_and(String::is_empty) {
original_lines.len() - 1
} else {
original_lines.len()
};
replacements.push((insertion_idx, 0, chunk.new_lines.clone()));
continue;
}
// Try to match the existing lines in the file with the old lines from
// the chunk. If the pattern ends with a trailing empty string (final
// newline), retry without it.
let mut pattern: &[String] = &chunk.old_lines;
let mut found = seek_sequence(original_lines, pattern, line_index, chunk.is_end_of_file);
let mut new_slice: &[String] = &chunk.new_lines;
if found.is_none() && pattern.last().is_some_and(String::is_empty) {
pattern = &pattern[..pattern.len() - 1];
if new_slice.last().is_some_and(String::is_empty) {
new_slice = &new_slice[..new_slice.len() - 1];
}
found = seek_sequence(original_lines, pattern, line_index, chunk.is_end_of_file);
}
if let Some(start_idx) = found {
replacements.push((start_idx, pattern.len(), new_slice.to_vec()));
line_index = start_idx + pattern.len();
} else {
return Err(ApplyPatchError::ComputeReplacements(format!(
"Failed to find expected lines in {}:\n{}",
path.display(),
chunk.old_lines.join("\n"),
)));
}
}
replacements.sort_by_key(|(lhs_idx, _, _)| *lhs_idx);
Ok(replacements)
}
/// Apply the `(start_index, old_len, new_lines)` replacements to
/// `original_lines`, returning the modified file contents as a vector of lines.
///
/// Replacements are applied in **reverse order** so that earlier replacements
/// don't shift the positions of later ones.
pub fn apply_replacements(
mut lines: Vec<String>,
replacements: &[(usize, usize, Vec<String>)],
) -> Vec<String> {
for (start_idx, old_len, new_segment) in replacements.iter().rev() {
let start_idx = *start_idx;
let old_len = *old_len;
// Remove old lines.
for _ in 0..old_len {
if start_idx < lines.len() {
lines.remove(start_idx);
}
}
// Insert new lines.
for (offset, new_line) in new_segment.iter().enumerate() {
lines.insert(start_idx + offset, new_line.clone());
}
}
lines
}
// ─── Tests ───────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use std::path::PathBuf;
use super::*;
use crate::implementations::codex::apply_patch::parser::{Hunk, parse_patch};
/// Helper to construct a patch string with the given body.
fn wrap_patch(body: &str) -> String {
format!("*** Begin Patch\n{body}\n*** End Patch")
}
#[test]
fn test_update_file_modifies_content() {
let original = "foo\nbar\n";
let path = PathBuf::from("update.txt");
let patch = wrap_patch(&format!(
"*** Update File: {}\n@@\n foo\n-bar\n+baz",
path.display()
));
let parsed = parse_patch(&patch).unwrap();
let chunks = match &parsed.hunks[0] {
Hunk::UpdateFile { chunks, .. } => chunks,
_ => panic!("expected UpdateFile"),
};
let result = derive_new_contents(original, &path, chunks).unwrap();
assert_eq!(result, "foo\nbaz\n");
}
#[test]
fn test_multiple_chunks_in_single_file() {
let original = "foo\nbar\nbaz\nqux\n";
let path = PathBuf::from("multi.txt");
let patch = wrap_patch(&format!(
"*** Update File: {}\n@@\n foo\n-bar\n+BAR\n@@\n baz\n-qux\n+QUX",
path.display()
));
let parsed = parse_patch(&patch).unwrap();
let chunks = match &parsed.hunks[0] {
Hunk::UpdateFile { chunks, .. } => chunks,
_ => panic!("expected UpdateFile"),
};
let result = derive_new_contents(original, &path, chunks).unwrap();
assert_eq!(result, "foo\nBAR\nbaz\nQUX\n");
}
#[test]
fn test_interleaved_changes() {
let original = "a\nb\nc\nd\ne\nf\n";
let path = PathBuf::from("interleaved.txt");
let patch = wrap_patch(&format!(
"*** Update File: {}\n\
@@\n a\n-b\n+B\n\
@@\n c\n d\n-e\n+E\n\
@@\n f\n+g\n*** End of File",
path.display()
));
let parsed = parse_patch(&patch).unwrap();
let chunks = match &parsed.hunks[0] {
Hunk::UpdateFile { chunks, .. } => chunks,
_ => panic!("expected UpdateFile"),
};
let result = derive_new_contents(original, &path, chunks).unwrap();
assert_eq!(result, "a\nB\nc\nd\nE\nf\ng\n");
}
#[test]
fn test_unicode_dash_matching() {
// Original line contains EN DASH (\u{2013}) and NON-BREAKING HYPHEN (\u{2011}).
let original = "import asyncio # local import \u{2013} avoids top\u{2011}level dep\n";
let path = PathBuf::from("unicode.py");
// Patch uses plain ASCII dash / hyphen.
let patch = wrap_patch(&format!(
"*** Update File: {}\n\
@@\n\
-import asyncio # local import - avoids top-level dep\n\
+import asyncio # HELLO",
path.display()
));
let parsed = parse_patch(&patch).unwrap();
let chunks = match &parsed.hunks[0] {
Hunk::UpdateFile { chunks, .. } => chunks,
_ => panic!("expected UpdateFile"),
};
let result = derive_new_contents(original, &path, chunks).unwrap();
assert_eq!(result, "import asyncio # HELLO\n");
}
#[test]
fn test_pure_addition_then_removal() {
let original = "line1\nline2\nline3\n";
let path = PathBuf::from("panic.txt");
let patch = wrap_patch(&format!(
"*** Update File: {}\n\
@@\n+after-context\n+second-line\n\
@@\n line1\n-line2\n-line3\n+line2-replacement",
path.display()
));
let parsed = parse_patch(&patch).unwrap();
let chunks = match &parsed.hunks[0] {
Hunk::UpdateFile { chunks, .. } => chunks,
_ => panic!("expected UpdateFile"),
};
let result = derive_new_contents(original, &path, chunks).unwrap();
assert_eq!(
result,
"line1\nline2-replacement\nafter-context\nsecond-line\n"
);
}
#[test]
fn test_context_not_found_returns_error() {
let original = "foo\nbar\n";
let path = PathBuf::from("missing.txt");
let patch = wrap_patch(&format!(
"*** Update File: {}\n@@ nonexistent_context\n-bar\n+baz",
path.display()
));
let parsed = parse_patch(&patch).unwrap();
let chunks = match &parsed.hunks[0] {
Hunk::UpdateFile { chunks, .. } => chunks,
_ => panic!("expected UpdateFile"),
};
let err = derive_new_contents(original, &path, chunks).unwrap_err();
match err {
ApplyPatchError::ComputeReplacements(msg) => {
assert!(msg.contains("Failed to find context"));
}
other => panic!("expected ComputeReplacements, got: {other:?}"),
}
}
#[test]
fn test_old_lines_not_found_returns_error() {
let original = "foo\nbar\n";
let path = PathBuf::from("mismatch.txt");
let patch = wrap_patch(&format!(
"*** Update File: {}\n@@\n-nonexistent\n+replacement",
path.display()
));
let parsed = parse_patch(&patch).unwrap();
let chunks = match &parsed.hunks[0] {
Hunk::UpdateFile { chunks, .. } => chunks,
_ => panic!("expected UpdateFile"),
};
let err = derive_new_contents(original, &path, chunks).unwrap_err();
match err {
ApplyPatchError::ComputeReplacements(msg) => {
assert!(msg.contains("Failed to find expected lines"));
}
other => panic!("expected ComputeReplacements, got: {other:?}"),
}
}
#[test]
fn test_end_of_file_marker_handling() {
let original = "foo\nbar\nbaz\n";
let path = PathBuf::from("eof.txt");
let patch = wrap_patch(&format!(
"*** Update File: {}\n@@\n+quux\n*** End of File",
path.display()
));
let parsed = parse_patch(&patch).unwrap();
let chunks = match &parsed.hunks[0] {
Hunk::UpdateFile { chunks, .. } => chunks,
_ => panic!("expected UpdateFile"),
};
let result = derive_new_contents(original, &path, chunks).unwrap();
assert_eq!(result, "foo\nbar\nbaz\nquux\n");
}
}
@@ -0,0 +1,58 @@
//! Error types for the codex apply-patch engine.
use std::path::PathBuf;
use thiserror::Error;
/// Errors encountered while parsing a patch.
#[derive(Debug, PartialEq, Clone, Error)]
pub enum ParseError {
#[error("invalid patch: {0}")]
InvalidPatchError(String),
#[error("invalid hunk at line {line_number}, {message}")]
InvalidHunkError { message: String, line_number: usize },
}
/// Errors encountered while applying a parsed patch to file contents.
#[derive(Debug, Error)]
pub enum ApplyPatchError {
/// The patch text could not be parsed.
#[error(transparent)]
Parse(#[from] ParseError),
/// A context line or old-lines block could not be located in the file.
#[error("{0}")]
ComputeReplacements(String),
/// An I/O error occurred while reading or writing a file.
/// Stored as a string so that the type remains `PartialEq`-friendly in
/// tests (std::io::Error is not PartialEq).
#[error("{context}: {message}")]
Io {
context: String,
message: String,
path: Option<PathBuf>,
},
}
impl PartialEq for ApplyPatchError {
fn eq(&self, other: &Self) -> bool {
match (self, other) {
(Self::Parse(a), Self::Parse(b)) => a == b,
(Self::ComputeReplacements(a), Self::ComputeReplacements(b)) => a == b,
(
Self::Io {
context: ca,
message: ma,
path: pa,
},
Self::Io {
context: cb,
message: mb,
path: pb,
},
) => ca == cb && ma == mb && pa == pb,
_ => false,
}
}
}
@@ -0,0 +1,25 @@
//! Codex `apply_patch` — core patch engine (pure library, no I/O).
//!
//! This module ports the codex patch parser, fuzzy matcher, and replacement
//! logic as pure functions with zero filesystem dependencies. All I/O
//! (reading/writing files) is handled by the tool layer in a later milestone.
//!
//! # Submodules
//!
//! - [`apply`] — `derive_new_contents()`, `compute_replacements()`,
//! `apply_replacements()` — all accept `&str` input.
//! - [`errors`] — `ApplyPatchError`, `ParseError`.
//! - [`parser`] — `parse_patch()`, `Hunk`, `UpdateFileChunk`.
//! - [`seek_sequence`] — 4-tier fuzzy line matcher.
pub mod apply;
pub mod errors;
pub mod parser;
pub mod seek_sequence;
pub mod tool;
// Re-exports for convenience.
pub use apply::derive_new_contents;
pub use errors::{ApplyPatchError, ParseError};
pub use parser::{Hunk, ParsedPatch, UpdateFileChunk, parse_patch};
pub use tool::{ApplyPatchInput, ApplyPatchTool};
@@ -0,0 +1,779 @@
//! Patch parser for the codex apply-patch format.
//!
//! Ported from the codex `apply-patch` crate (`codex-rs/apply-patch/src/parser.rs`).
//! Parses a custom patch format (not unified diff) with `*** Add File`,
//! `*** Delete File`, and `*** Update File` operations into a list of [`Hunk`]s.
//!
//! The official Lark grammar for the apply-patch format is:
//!
//! ```text
//! start: begin_patch hunk+ end_patch
//! begin_patch: "*** Begin Patch" LF
//! end_patch: "*** End Patch" LF?
//!
//! hunk: add_hunk | delete_hunk | update_hunk
//! add_hunk: "*** Add File: " filename LF add_line+
//! delete_hunk: "*** Delete File: " filename LF
//! update_hunk: "*** Update File: " filename LF change_move? change?
//! filename: /(.+)/
//! add_line: "+" /(.+)/ LF -> line
//!
//! change_move: "*** Move to: " filename LF
//! change: (change_context | change_line)+ eof_line?
//! change_context: ("@@" | "@@ " /(.+)/) LF
//! change_line: ("+" | "-" | " ") /(.+)/ LF
//! eof_line: "*** End of File" LF
//! ```
//!
//! The parser is slightly more lenient than the explicit spec and allows for
//! leading/trailing whitespace around patch markers.
use std::path::PathBuf;
use super::errors::ParseError;
use ParseError::*;
// ─── Marker constants ────────────────────────────────────────────────
const BEGIN_PATCH_MARKER: &str = "*** Begin Patch";
const END_PATCH_MARKER: &str = "*** End Patch";
const ADD_FILE_MARKER: &str = "*** Add File: ";
const DELETE_FILE_MARKER: &str = "*** Delete File: ";
const UPDATE_FILE_MARKER: &str = "*** Update File: ";
const MOVE_TO_MARKER: &str = "*** Move to: ";
const EOF_MARKER: &str = "*** End of File";
const CHANGE_CONTEXT_MARKER: &str = "@@ ";
const EMPTY_CHANGE_CONTEXT_MARKER: &str = "@@";
/// We always use lenient mode (matching the codex default).
const PARSE_IN_STRICT_MODE: bool = false;
// ─── Public types ────────────────────────────────────────────────────
/// A parsed patch: the list of hunks plus the normalised patch text.
#[derive(Debug, PartialEq)]
pub struct ParsedPatch {
pub hunks: Vec<Hunk>,
pub patch: String,
}
/// A single hunk within a parsed patch.
#[derive(Debug, PartialEq, Clone)]
#[allow(clippy::enum_variant_names)]
pub enum Hunk {
AddFile {
path: PathBuf,
contents: String,
},
DeleteFile {
path: PathBuf,
},
UpdateFile {
path: PathBuf,
move_path: Option<PathBuf>,
/// Chunks should be in order — the `change_context` of one chunk
/// should occur later in the file than the previous chunk.
chunks: Vec<UpdateFileChunk>,
},
}
/// A single contiguous edit within an `UpdateFile` hunk.
#[derive(Debug, PartialEq, Clone)]
pub struct UpdateFileChunk {
/// A single line of context used to narrow down the position of the chunk
/// (usually a class, method, or function definition).
pub change_context: Option<String>,
/// A contiguous block of lines that should be replaced with `new_lines`.
/// `old_lines` must occur strictly after `change_context`.
pub old_lines: Vec<String>,
pub new_lines: Vec<String>,
/// If set to true, `old_lines` must occur at the end of the source file.
pub is_end_of_file: bool,
}
// ─── Public entry point ──────────────────────────────────────────────
/// Parse a patch string into a [`ParsedPatch`].
pub fn parse_patch(patch: &str) -> Result<ParsedPatch, ParseError> {
let mode = if PARSE_IN_STRICT_MODE {
ParseMode::Strict
} else {
ParseMode::Lenient
};
parse_patch_text(patch, mode)
}
// ─── Internal helpers ────────────────────────────────────────────────
enum ParseMode {
/// Parse the patch text argument as-is.
Strict,
/// In lenient mode we strip heredoc wrappers (`<<EOF` / `<<'EOF'` /
/// `<<"EOF"`) before trying strict parsing.
Lenient,
}
fn parse_patch_text(patch: &str, mode: ParseMode) -> Result<ParsedPatch, ParseError> {
let lines: Vec<&str> = patch.trim().lines().collect();
let lines: &[&str] = match check_patch_boundaries_strict(&lines) {
Ok(()) => &lines,
Err(e) => match mode {
ParseMode::Strict => {
return Err(e);
}
ParseMode::Lenient => check_patch_boundaries_lenient(&lines, e)?,
},
};
let mut hunks: Vec<Hunk> = Vec::new();
// The boundary checks guarantee lines.len() >= 2.
let last_line_index = lines.len().saturating_sub(1);
let mut remaining_lines = &lines[1..last_line_index];
let mut line_number = 2;
while !remaining_lines.is_empty() {
let (hunk, hunk_lines) = parse_one_hunk(remaining_lines, line_number)?;
hunks.push(hunk);
line_number += hunk_lines;
remaining_lines = &remaining_lines[hunk_lines..];
}
let patch = lines.join("\n");
Ok(ParsedPatch { hunks, patch })
}
fn check_patch_boundaries_strict(lines: &[&str]) -> Result<(), ParseError> {
let (first_line, last_line) = match lines {
[] => (None, None),
[first] => (Some(first), Some(first)),
[first, .., last] => (Some(first), Some(last)),
};
check_start_and_end_lines_strict(first_line, last_line)
}
fn check_patch_boundaries_lenient<'a>(
original_lines: &'a [&'a str],
original_parse_error: ParseError,
) -> Result<&'a [&'a str], ParseError> {
match original_lines {
[first, .., last] => {
if (first == &"<<EOF" || first == &"<<'EOF'" || first == &"<<\"EOF\"")
&& last.ends_with("EOF")
&& original_lines.len() >= 4
{
let inner_lines = &original_lines[1..original_lines.len() - 1];
match check_patch_boundaries_strict(inner_lines) {
Ok(()) => Ok(inner_lines),
Err(e) => Err(e),
}
} else {
Err(original_parse_error)
}
}
_ => Err(original_parse_error),
}
}
fn check_start_and_end_lines_strict(
first_line: Option<&&str>,
last_line: Option<&&str>,
) -> Result<(), ParseError> {
let first_line = first_line.map(|line| line.trim());
let last_line = last_line.map(|line| line.trim());
match (first_line, last_line) {
(Some(first), Some(last)) if first == BEGIN_PATCH_MARKER && last == END_PATCH_MARKER => {
Ok(())
}
(Some(first), _) if first != BEGIN_PATCH_MARKER => Err(InvalidPatchError(String::from(
"The first line of the patch must be '*** Begin Patch'",
))),
_ => Err(InvalidPatchError(String::from(
"The last line of the patch must be '*** End Patch'",
))),
}
}
/// Parse a single hunk from the start of `lines`.
/// Returns the parsed hunk and the number of lines consumed.
fn parse_one_hunk(lines: &[&str], line_number: usize) -> Result<(Hunk, usize), ParseError> {
let first_line = lines[0].trim();
if let Some(path) = first_line.strip_prefix(ADD_FILE_MARKER) {
// ── Add File ─────────────────────────────────────────────
let mut contents = String::new();
let mut parsed_lines = 1;
for add_line in &lines[1..] {
if let Some(line_to_add) = add_line.strip_prefix('+') {
contents.push_str(line_to_add);
contents.push('\n');
parsed_lines += 1;
} else {
break;
}
}
return Ok((
Hunk::AddFile {
path: PathBuf::from(path),
contents,
},
parsed_lines,
));
} else if let Some(path) = first_line.strip_prefix(DELETE_FILE_MARKER) {
// ── Delete File ──────────────────────────────────────────
return Ok((
Hunk::DeleteFile {
path: PathBuf::from(path),
},
1,
));
} else if let Some(path) = first_line.strip_prefix(UPDATE_FILE_MARKER) {
// ── Update File ──────────────────────────────────────────
let mut remaining_lines = &lines[1..];
let mut parsed_lines = 1;
// Optional: move-to line.
let move_path = remaining_lines
.first()
.and_then(|x| x.strip_prefix(MOVE_TO_MARKER));
if move_path.is_some() {
remaining_lines = &remaining_lines[1..];
parsed_lines += 1;
}
let mut chunks = Vec::new();
while !remaining_lines.is_empty() {
// Skip blank lines between chunks.
if remaining_lines[0].trim().is_empty() {
parsed_lines += 1;
remaining_lines = &remaining_lines[1..];
continue;
}
// Stop at the next hunk header.
if remaining_lines[0].starts_with("***") {
break;
}
let (chunk, chunk_lines) = parse_update_file_chunk(
remaining_lines,
line_number + parsed_lines,
chunks.is_empty(),
)?;
chunks.push(chunk);
parsed_lines += chunk_lines;
remaining_lines = &remaining_lines[chunk_lines..];
}
if chunks.is_empty() {
return Err(InvalidHunkError {
message: format!("Update file hunk for path '{path}' is empty"),
line_number,
});
}
return Ok((
Hunk::UpdateFile {
path: PathBuf::from(path),
move_path: move_path.map(PathBuf::from),
chunks,
},
parsed_lines,
));
}
Err(InvalidHunkError {
message: format!(
"'{first_line}' is not a valid hunk header. Valid hunk headers: \
'*** Add File: {{path}}', '*** Delete File: {{path}}', '*** Update File: {{path}}'"
),
line_number,
})
}
fn parse_update_file_chunk(
lines: &[&str],
line_number: usize,
allow_missing_context: bool,
) -> Result<(UpdateFileChunk, usize), ParseError> {
if lines.is_empty() {
return Err(InvalidHunkError {
message: "Update hunk does not contain any lines".to_string(),
line_number,
});
}
// Check for explicit @@ context marker.
let (change_context, start_index) = if lines[0] == EMPTY_CHANGE_CONTEXT_MARKER {
(None, 1)
} else if let Some(context) = lines[0].strip_prefix(CHANGE_CONTEXT_MARKER) {
(Some(context.to_string()), 1)
} else {
if !allow_missing_context {
return Err(InvalidHunkError {
message: format!(
"Expected update hunk to start with a @@ context marker, got: '{}'",
lines[0]
),
line_number,
});
}
(None, 0)
};
if start_index >= lines.len() {
return Err(InvalidHunkError {
message: "Update hunk does not contain any lines".to_string(),
line_number: line_number + 1,
});
}
let mut chunk = UpdateFileChunk {
change_context,
old_lines: Vec::new(),
new_lines: Vec::new(),
is_end_of_file: false,
};
let mut parsed_lines = 0;
for line in &lines[start_index..] {
match *line {
EOF_MARKER => {
if parsed_lines == 0 {
return Err(InvalidHunkError {
message: "Update hunk does not contain any lines".to_string(),
line_number: line_number + 1,
});
}
chunk.is_end_of_file = true;
parsed_lines += 1;
break;
}
line_contents => match line_contents.chars().next() {
None => {
// Interpret empty line as a context line.
chunk.old_lines.push(String::new());
chunk.new_lines.push(String::new());
}
Some(' ') => {
chunk.old_lines.push(line_contents[1..].to_string());
chunk.new_lines.push(line_contents[1..].to_string());
}
Some('+') => {
chunk.new_lines.push(line_contents[1..].to_string());
}
Some('-') => {
chunk.old_lines.push(line_contents[1..].to_string());
}
_ => {
if parsed_lines == 0 {
return Err(InvalidHunkError {
message: format!(
"Unexpected line found in update hunk: '{line_contents}'. \
Every line should start with ' ' (context line), \
'+' (added line), or '-' (removed line)"
),
line_number: line_number + 1,
});
}
// Assume this is the start of the next hunk.
break;
}
},
}
parsed_lines += 1;
}
Ok((chunk, parsed_lines + start_index))
}
// ─── Tests ───────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_patch_bad_first_line() {
assert_eq!(
parse_patch_text("bad", ParseMode::Strict),
Err(InvalidPatchError(
"The first line of the patch must be '*** Begin Patch'".to_string()
))
);
}
#[test]
fn test_parse_patch_bad_last_line() {
assert_eq!(
parse_patch_text("*** Begin Patch\nbad", ParseMode::Strict),
Err(InvalidPatchError(
"The last line of the patch must be '*** End Patch'".to_string()
))
);
}
#[test]
fn test_parse_patch_add_file_with_whitespace_around_markers() {
assert_eq!(
parse_patch_text(
concat!(
"*** Begin Patch",
" ",
"\n*** Add File: foo\n+hi\n",
" ",
"*** End Patch"
),
ParseMode::Strict
)
.unwrap()
.hunks,
vec![Hunk::AddFile {
path: PathBuf::from("foo"),
contents: "hi\n".to_string()
}]
);
}
#[test]
fn test_parse_patch_empty_update_hunk() {
assert_eq!(
parse_patch_text(
"*** Begin Patch\n\
*** Update File: test.py\n\
*** End Patch",
ParseMode::Strict
),
Err(InvalidHunkError {
message: "Update file hunk for path 'test.py' is empty".to_string(),
line_number: 2,
})
);
}
#[test]
fn test_parse_patch_empty_hunks() {
assert_eq!(
parse_patch_text(
"*** Begin Patch\n\
*** End Patch",
ParseMode::Strict
)
.unwrap()
.hunks,
Vec::new()
);
}
#[test]
fn test_parse_patch_all_hunk_types() {
assert_eq!(
parse_patch_text(
"*** Begin Patch\n\
*** Add File: path/add.py\n\
+abc\n\
+def\n\
*** Delete File: path/delete.py\n\
*** Update File: path/update.py\n\
*** Move to: path/update2.py\n\
@@ def f():\n\
- pass\n\
+ return 123\n\
*** End Patch",
ParseMode::Strict
)
.unwrap()
.hunks,
vec![
Hunk::AddFile {
path: PathBuf::from("path/add.py"),
contents: "abc\ndef\n".to_string()
},
Hunk::DeleteFile {
path: PathBuf::from("path/delete.py")
},
Hunk::UpdateFile {
path: PathBuf::from("path/update.py"),
move_path: Some(PathBuf::from("path/update2.py")),
chunks: vec![UpdateFileChunk {
change_context: Some("def f():".to_string()),
old_lines: vec![" pass".to_string()],
new_lines: vec![" return 123".to_string()],
is_end_of_file: false
}]
}
]
);
}
#[test]
fn test_parse_patch_update_followed_by_add() {
assert_eq!(
parse_patch_text(
"*** Begin Patch\n\
*** Update File: file.py\n\
@@\n\
+line\n\
*** Add File: other.py\n\
+content\n\
*** End Patch",
ParseMode::Strict
)
.unwrap()
.hunks,
vec![
Hunk::UpdateFile {
path: PathBuf::from("file.py"),
move_path: None,
chunks: vec![UpdateFileChunk {
change_context: None,
old_lines: vec![],
new_lines: vec!["line".to_string()],
is_end_of_file: false
}],
},
Hunk::AddFile {
path: PathBuf::from("other.py"),
contents: "content\n".to_string()
}
]
);
}
#[test]
fn test_parse_patch_update_without_explicit_context_marker() {
assert_eq!(
parse_patch_text(
r#"*** Begin Patch
*** Update File: file2.py
import foo
+bar
*** End Patch"#,
ParseMode::Strict
)
.unwrap()
.hunks,
vec![Hunk::UpdateFile {
path: PathBuf::from("file2.py"),
move_path: None,
chunks: vec![UpdateFileChunk {
change_context: None,
old_lines: vec!["import foo".to_string()],
new_lines: vec!["import foo".to_string(), "bar".to_string()],
is_end_of_file: false,
}],
}]
);
}
#[test]
fn test_parse_patch_lenient_heredoc_variants() {
let patch_text = r#"*** Begin Patch
*** Update File: file2.py
import foo
+bar
*** End Patch"#;
let expected_hunks = vec![Hunk::UpdateFile {
path: PathBuf::from("file2.py"),
move_path: None,
chunks: vec![UpdateFileChunk {
change_context: None,
old_lines: vec!["import foo".to_string()],
new_lines: vec!["import foo".to_string(), "bar".to_string()],
is_end_of_file: false,
}],
}];
let expected_error =
InvalidPatchError("The first line of the patch must be '*** Begin Patch'".to_string());
// <<EOF variant
let patch_in_heredoc = format!("<<EOF\n{patch_text}\nEOF\n");
assert_eq!(
parse_patch_text(&patch_in_heredoc, ParseMode::Strict),
Err(expected_error.clone())
);
assert_eq!(
parse_patch_text(&patch_in_heredoc, ParseMode::Lenient)
.unwrap()
.hunks,
expected_hunks
);
// <<'EOF' variant
let patch_in_sq_heredoc = format!("<<'EOF'\n{patch_text}\nEOF\n");
assert_eq!(
parse_patch_text(&patch_in_sq_heredoc, ParseMode::Strict),
Err(expected_error.clone())
);
assert_eq!(
parse_patch_text(&patch_in_sq_heredoc, ParseMode::Lenient)
.unwrap()
.hunks,
expected_hunks
);
// <<"EOF" variant
let patch_in_dq_heredoc = format!("<<\"EOF\"\n{patch_text}\nEOF\n");
assert_eq!(
parse_patch_text(&patch_in_dq_heredoc, ParseMode::Strict),
Err(expected_error.clone())
);
assert_eq!(
parse_patch_text(&patch_in_dq_heredoc, ParseMode::Lenient)
.unwrap()
.hunks,
expected_hunks
);
// Mismatched quotes → fail even in lenient mode
let patch_in_mismatch = format!("<<\"EOF'\n{patch_text}\nEOF\n");
assert_eq!(
parse_patch_text(&patch_in_mismatch, ParseMode::Strict),
Err(expected_error.clone())
);
assert_eq!(
parse_patch_text(&patch_in_mismatch, ParseMode::Lenient),
Err(expected_error)
);
// Missing closing heredoc marker
let patch_missing_close =
"<<EOF\n*** Begin Patch\n*** Update File: file2.py\nEOF\n".to_string();
assert_eq!(
parse_patch_text(&patch_missing_close, ParseMode::Lenient),
Err(InvalidPatchError(
"The last line of the patch must be '*** End Patch'".to_string()
))
);
}
#[test]
fn test_parse_one_hunk_bad_header() {
assert_eq!(
parse_one_hunk(&["bad"], 234),
Err(InvalidHunkError {
message: "'bad' is not a valid hunk header. Valid hunk headers: \
'*** Add File: {path}', '*** Delete File: {path}', \
'*** Update File: {path}'"
.to_string(),
line_number: 234
})
);
}
#[test]
fn test_update_file_chunk_bad_start() {
assert_eq!(
parse_update_file_chunk(&["bad"], 123, false),
Err(InvalidHunkError {
message: "Expected update hunk to start with a @@ context marker, got: 'bad'"
.to_string(),
line_number: 123
})
);
}
#[test]
fn test_update_file_chunk_empty_after_context() {
assert_eq!(
parse_update_file_chunk(&["@@"], 123, false),
Err(InvalidHunkError {
message: "Update hunk does not contain any lines".to_string(),
line_number: 124
})
);
}
#[test]
fn test_update_file_chunk_bad_diff_line() {
assert_eq!(
parse_update_file_chunk(&["@@", "bad"], 123, false),
Err(InvalidHunkError {
message: "Unexpected line found in update hunk: 'bad'. \
Every line should start with ' ' (context line), \
'+' (added line), or '-' (removed line)"
.to_string(),
line_number: 124
})
);
}
#[test]
fn test_update_file_chunk_eof_with_no_lines() {
assert_eq!(
parse_update_file_chunk(&["@@", "*** End of File"], 123, false),
Err(InvalidHunkError {
message: "Update hunk does not contain any lines".to_string(),
line_number: 124
})
);
}
#[test]
fn test_update_file_chunk_with_context_and_diff_lines() {
assert_eq!(
parse_update_file_chunk(
&[
"@@ change_context",
"",
" context",
"-remove",
"+add",
" context2",
"*** End Patch",
],
123,
false
),
Ok((
UpdateFileChunk {
change_context: Some("change_context".to_string()),
old_lines: vec![
"".to_string(),
"context".to_string(),
"remove".to_string(),
"context2".to_string()
],
new_lines: vec![
"".to_string(),
"context".to_string(),
"add".to_string(),
"context2".to_string()
],
is_end_of_file: false
},
6
))
);
}
#[test]
fn test_update_file_chunk_with_eof_marker() {
assert_eq!(
parse_update_file_chunk(&["@@", "+line", "*** End of File"], 123, false),
Ok((
UpdateFileChunk {
change_context: None,
old_lines: vec![],
new_lines: vec!["line".to_string()],
is_end_of_file: true
},
3
))
);
}
#[test]
fn test_parse_patch_public_api() {
let patch = "*** Begin Patch\n\
*** Add File: hello.txt\n\
+hello world\n\
*** End Patch";
let result = parse_patch(patch).unwrap();
assert_eq!(
result.hunks,
vec![Hunk::AddFile {
path: PathBuf::from("hello.txt"),
contents: "hello world\n".to_string(),
}]
);
}
}
@@ -0,0 +1,186 @@
//! Fuzzy line-sequence matcher for the codex apply-patch engine.
//!
//! Ported verbatim from `codex-rs/apply-patch/src/seek_sequence.rs`.
//!
//! Attempts to find a sequence of `pattern` lines within `lines` beginning at
//! or after `start`. Matches are attempted with decreasing strictness:
//!
//! 1. **Exact match**
//! 2. **rstrip** — ignore trailing whitespace
//! 3. **trim** — ignore leading and trailing whitespace
//! 4. **Unicode normalise** — normalise common Unicode punctuation to ASCII
//! equivalents (typographic dashes → `-`, smart quotes → `'`/`"`, etc.)
/// Find `pattern` within `lines` starting at `start`.
///
/// When `eof` is `true`, the search begins at the end of the file (so that
/// patterns intended to match file endings are applied at the end), falling
/// back to searching from `start` if needed.
///
/// Returns the starting index of the match, or `None` if not found.
///
/// # Edge cases
///
/// - Empty `pattern` → returns `Some(start)` (no-op match).
/// - `pattern.len() > lines.len()` → returns `None`.
pub fn seek_sequence(
lines: &[String],
pattern: &[String],
start: usize,
eof: bool,
) -> Option<usize> {
if pattern.is_empty() {
return Some(start);
}
// When the pattern is longer than the available input there is no
// possible match.
if pattern.len() > lines.len() {
return None;
}
let search_start = if eof && lines.len() >= pattern.len() {
lines.len() - pattern.len()
} else {
start
};
// ── Pass 1: exact match ──────────────────────────────────────────
for i in search_start..=lines.len().saturating_sub(pattern.len()) {
if lines[i..i + pattern.len()] == *pattern {
return Some(i);
}
}
// ── Pass 2: rstrip match ─────────────────────────────────────────
for i in search_start..=lines.len().saturating_sub(pattern.len()) {
let mut ok = true;
for (p_idx, pat) in pattern.iter().enumerate() {
if lines[i + p_idx].trim_end() != pat.trim_end() {
ok = false;
break;
}
}
if ok {
return Some(i);
}
}
// ── Pass 3: trim both sides ──────────────────────────────────────
for i in search_start..=lines.len().saturating_sub(pattern.len()) {
let mut ok = true;
for (p_idx, pat) in pattern.iter().enumerate() {
if lines[i + p_idx].trim() != pat.trim() {
ok = false;
break;
}
}
if ok {
return Some(i);
}
}
// ── Pass 4: Unicode normalise ────────────────────────────────────
// Normalise common Unicode punctuation to ASCII equivalents so that
// diffs authored with plain ASCII characters can still be applied to
// source files that contain typographic dashes / quotes, etc.
fn normalise(s: &str) -> String {
s.trim()
.chars()
.map(|c| match c {
// Various dash / hyphen code-points → ASCII '-'
'\u{2010}' | '\u{2011}' | '\u{2012}' | '\u{2013}' | '\u{2014}' | '\u{2015}'
| '\u{2212}' => '-',
// Fancy single quotes → '\''
'\u{2018}' | '\u{2019}' | '\u{201A}' | '\u{201B}' => '\'',
// Fancy double quotes → '"'
'\u{201C}' | '\u{201D}' | '\u{201E}' | '\u{201F}' => '"',
// Non-breaking space and other odd spaces → normal space
'\u{00A0}' | '\u{2002}' | '\u{2003}' | '\u{2004}' | '\u{2005}' | '\u{2006}'
| '\u{2007}' | '\u{2008}' | '\u{2009}' | '\u{200A}' | '\u{202F}' | '\u{205F}'
| '\u{3000}' => ' ',
other => other,
})
.collect::<String>()
}
for i in search_start..=lines.len().saturating_sub(pattern.len()) {
let mut ok = true;
for (p_idx, pat) in pattern.iter().enumerate() {
if normalise(&lines[i + p_idx]) != normalise(pat) {
ok = false;
break;
}
}
if ok {
return Some(i);
}
}
None
}
// ─── Tests ───────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::seek_sequence;
fn to_vec(strings: &[&str]) -> Vec<String> {
strings.iter().map(ToString::to_string).collect()
}
#[test]
fn test_exact_match_finds_sequence() {
let lines = to_vec(&["foo", "bar", "baz"]);
let pattern = to_vec(&["bar", "baz"]);
assert_eq!(seek_sequence(&lines, &pattern, 0, false), Some(1));
}
#[test]
fn test_rstrip_match_ignores_trailing_whitespace() {
let lines = to_vec(&["foo ", "bar\t\t"]);
let pattern = to_vec(&["foo", "bar"]);
assert_eq!(seek_sequence(&lines, &pattern, 0, false), Some(0));
}
#[test]
fn test_trim_match_ignores_leading_and_trailing_whitespace() {
let lines = to_vec(&[" foo ", " bar\t"]);
let pattern = to_vec(&["foo", "bar"]);
assert_eq!(seek_sequence(&lines, &pattern, 0, false), Some(0));
}
#[test]
fn test_pattern_longer_than_input_returns_none() {
let lines = to_vec(&["just one line"]);
let pattern = to_vec(&["too", "many", "lines"]);
assert_eq!(seek_sequence(&lines, &pattern, 0, false), None);
}
#[test]
fn test_empty_pattern_returns_start() {
let lines = to_vec(&["foo", "bar"]);
let pattern: Vec<String> = vec![];
assert_eq!(seek_sequence(&lines, &pattern, 1, false), Some(1));
}
#[test]
fn test_eof_flag_searches_from_end() {
let lines = to_vec(&["a", "b", "c", "b", "c"]);
let pattern = to_vec(&["b", "c"]);
// Without eof, finds the first occurrence.
assert_eq!(seek_sequence(&lines, &pattern, 0, false), Some(1));
// With eof, prefers the last occurrence.
assert_eq!(seek_sequence(&lines, &pattern, 0, true), Some(3));
}
#[test]
fn test_unicode_normalise_matches_typographic_dashes() {
// Line contains EN DASH (\u{2013}).
let lines = to_vec(&["hello \u{2013} world"]);
// Pattern uses plain ASCII dash.
let pattern = to_vec(&["hello - world"]);
assert_eq!(seek_sequence(&lines, &pattern, 0, false), Some(0));
}
}
@@ -0,0 +1,750 @@
//! `ApplyPatchTool` — Tool trait implementation for the codex apply-patch format.
//!
//! Wires the pure-library patch engine (parser + apply) through `AsyncFileSystem`
//! for all I/O and emits `FileWritten` notifications.
use std::fmt::Write as _;
use std::path::PathBuf;
use std::sync::Arc;
use crate::computer::types::AsyncFileSystem;
use crate::notification::types::FileWritten;
use crate::types::output::{ApplyPatchFileResult, ApplyPatchOutput};
use crate::types::requirements::Expr;
#[allow(unused_imports)]
use crate::types::resources::{Cwd, FileSystem, NotificationHandle, SharedResources};
use crate::types::tool::{ToolKind, ToolNamespace};
use super::errors::ParseError;
use super::parser::{self, Hunk};
use super::{apply, errors::ApplyPatchError};
// ─── Description ─────────────────────────────────────────────────────
/// Tool description derived from the codex `apply_patch_tool_instructions.md`.
const DESCRIPTION: &str = r#"Use the `apply_patch` tool to edit files.
Your patch language is a stripped‑down, file‑oriented diff format designed to be easy to parse and safe to apply. You can think of it as a high‑level envelope:
*** Begin Patch
[ one or more file sections ]
*** End Patch
Within that envelope, you get a sequence of file operations.
You MUST include a header to specify the action you are taking.
Each operation starts with one of three headers:
*** Add File: <path> - create a new file. Every following line is a + line (the initial contents).
*** Delete File: <path> - remove an existing file. Nothing follows.
*** Update File: <path> - patch an existing file in place (optionally with a rename).
May be immediately followed by *** Move to: <new path> if you want to rename the file.
Then one or more “hunks”, each introduced by @@ (optionally followed by a hunk header).
Within a hunk each line starts with:
For instructions on [context_before] and [context_after]:
- By default, show 3 lines of code immediately above and 3 lines immediately below each change. If a change is within 3 lines of a previous change, do NOT duplicate the first change’s [context_after] lines in the second change’s [context_before] lines.
- If 3 lines of context is insufficient to uniquely identify the snippet of code within the file, use the @@ operator to indicate the class or function to which the snippet belongs. For instance, we might have:
@@ class BaseClass
[3 lines of pre-context]
- [old_code]
+ [new_code]
[3 lines of post-context]
- If a code block is repeated so many times in a class or function such that even a single `@@` statement and 3 lines of context cannot uniquely identify the snippet of code, you can use multiple `@@` statements to jump to the right context. For instance:
@@ class BaseClass
@@ def method():
[3 lines of pre-context]
- [old_code]
+ [new_code]
[3 lines of post-context]
The full grammar definition is below:
Patch := Begin { FileOp } End
Begin := "*** Begin Patch" NEWLINE
End := "*** End Patch" NEWLINE
FileOp := AddFile | DeleteFile | UpdateFile
AddFile := "*** Add File: " path NEWLINE { "+" line NEWLINE }
DeleteFile := "*** Delete File: " path NEWLINE
UpdateFile := "*** Update File: " path NEWLINE [ MoveTo ] { Hunk }
MoveTo := "*** Move to: " newPath NEWLINE
Hunk := "@@" [ header ] NEWLINE { HunkLine } [ "*** End of File" NEWLINE ]
HunkLine := (" " | "-" | "+") text NEWLINE
A full patch can combine several operations:
*** Begin Patch
*** Add File: hello.txt
+Hello world
*** Update File: src/app.py
*** Move to: src/main.py
@@ def greet():
-print("Hi")
+print("Hello, world!")
*** Delete File: obsolete.txt
*** End Patch
It is important to remember:
- You must include a header with your intended action (Add/Delete/Update)
- You must prefix new lines with `+` even when creating a new file
- File references can only be relative, NEVER ABSOLUTE.
"#;
// ─── Input ───────────────────────────────────────────────────────────
/// Input for the `apply_patch` tool.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct ApplyPatchInput {
/// The patch text in codex patch format.
pub patch: String,
}
// ─── Tool ────────────────────────────────────────────────────────────
/// ApplyPatch tool — applies multi-file patches in the codex patch format.
#[derive(Debug, Default)]
pub struct ApplyPatchTool;
// ─── Internal types ──────────────────────────────────────────────────
/// A computed file change — all content determined in-memory, ready to write.
enum FileChange {
Add {
path: PathBuf,
content: String,
},
Delete {
path: PathBuf,
original_content: String,
},
Update {
path: PathBuf,
original_content: String,
new_content: String,
},
Move {
source_path: PathBuf,
dest_path: PathBuf,
original_content: String,
new_content: String,
},
}
// ─── Helpers ─────────────────────────────────────────────────────────
/// Create parent directories for a file path if they don't exist.
async fn ensure_parent_dirs(path: &std::path::Path) -> Result<(), kigi_tool_runtime::ToolError> {
if let Some(parent) = path.parent()
&& !parent.as_os_str().is_empty()
{
tokio::fs::create_dir_all(parent).await.map_err(|e| {
kigi_tool_runtime::ToolError::execution(
kigi_tool_protocol::ToolId::new("apply_patch").expect("valid"),
e.to_string(),
)
})?;
}
Ok(())
}
/// Compute all file changes in memory without writing anything.
/// Returns an error string if any hunk can't be applied.
async fn compute_all_changes(
cwd: &std::path::Path,
fs: &Arc<dyn AsyncFileSystem>,
hunks: &[Hunk],
) -> Result<Vec<FileChange>, String> {
let mut changes = Vec::new();
for hunk in hunks {
match hunk {
Hunk::AddFile { path, contents } => {
let resolved = cwd.join(path);
changes.push(FileChange::Add {
path: resolved,
content: contents.clone(),
});
}
Hunk::DeleteFile { path } => {
let resolved = cwd.join(path);
let original_content = read_file_as_string(fs, &resolved)
.await
.map_err(|e| format!("Failed to read file: {}, {e}", resolved.display()))?;
changes.push(FileChange::Delete {
path: resolved,
original_content,
});
}
Hunk::UpdateFile {
path,
move_path,
chunks,
} => {
let resolved = cwd.join(path);
let original_content = read_file_as_string(fs, &resolved).await.map_err(|e| {
format!("Failed to read file to update: {}, {e}", resolved.display())
})?;
let new_content = apply::derive_new_contents(&original_content, &resolved, chunks)
.map_err(|e| match e {
ApplyPatchError::ComputeReplacements(msg) => msg,
other => other.to_string(),
})?;
if let Some(dest) = move_path {
let resolved_dest = cwd.join(dest);
changes.push(FileChange::Move {
source_path: resolved,
dest_path: resolved_dest,
original_content,
new_content,
});
} else {
changes.push(FileChange::Update {
path: resolved,
original_content,
new_content,
});
}
}
}
}
Ok(changes)
}
/// Read a file via AsyncFileSystem and convert to String.
async fn read_file_as_string(
fs: &Arc<dyn AsyncFileSystem>,
path: &std::path::Path,
) -> Result<String, String> {
let bytes = fs.read_file(path).await.map_err(|e| format!("{e}"))?;
Ok(String::from_utf8_lossy(&bytes).into_owned())
}
/// Build the codex-style summary string.
fn build_summary(results: &[ApplyPatchFileResult]) -> String {
let mut out = String::from("Success. Updated the following files:\n");
for r in results {
let prefix = match r.action.as_str() {
"added" => "A",
"deleted" => "D",
"moved" => "M",
_ => "M", // "modified"
};
let _ = writeln!(out, "{prefix} {}", r.path.display());
}
out
}
// ─── Tests ───────────────────────────────────────────────────────────
impl crate::types::tool_metadata::ToolMetadata for ApplyPatchTool {
fn kind(&self) -> ToolKind {
ToolKind::Edit
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::Codex
}
fn description_template(&self) -> &str {
DESCRIPTION
}
fn emitted_notifications(&self) -> &'static [&'static str] {
&["FileWritten"]
}
fn requires_expr(&self) -> Expr<crate::types::requirements::ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for ApplyPatchTool {
type Args = ApplyPatchInput;
type Output = ApplyPatchOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("apply_patch").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"apply_patch",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(name = "tool.apply_patch", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: ApplyPatchInput,
) -> Result<ApplyPatchOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let (cwd, fs, notification_handle);
{
cwd = crate::types::tool_metadata::resolve_cwd(&ctx, &resources).await?;
let res = resources.lock().await;
fs = res.require::<FileSystem>()?.0.clone();
notification_handle = res.require::<NotificationHandle>()?.0.clone();
}
let tool_call_id = ctx.call_id.as_str().to_owned();
// ── Phase 1: Parse ───────────────────────────────────────
let parsed = match parser::parse_patch(&input.patch) {
Ok(p) => p,
Err(e) => {
let msg = match &e {
ParseError::InvalidPatchError(m) => format!("Invalid patch: {m}"),
ParseError::InvalidHunkError {
message,
line_number,
} => format!("Invalid patch hunk on line {line_number}: {message}"),
};
return Ok(ApplyPatchOutput::ParseError(msg));
}
};
if parsed.hunks.is_empty() {
return Ok(ApplyPatchOutput::EmptyPatch(
"No files were modified.".to_string(),
));
}
// ── Phase 2: Compute all changes in memory (no writes yet) ───
let changes = match compute_all_changes(&cwd, &fs, &parsed.hunks).await {
Ok(c) => c,
Err(msg) => return Ok(ApplyPatchOutput::ApplicationError(msg)),
};
// ── Phase 3: Apply all changes (write to filesystem) ─────
let mut file_results = Vec::new();
for change in &changes {
match change {
FileChange::Add { path, content } => {
// Create parent directories if needed.
ensure_parent_dirs(path).await?;
fs.write_file(path, content.as_bytes()).await.map_err(|e| {
kigi_tool_runtime::ToolError::execution(
kigi_tool_protocol::ToolId::new("apply_patch").expect("valid"),
e.to_string(),
)
})?;
notification_handle.send_file_written(FileWritten {
tool_call_id: tool_call_id.clone(),
absolute_path: path.clone(),
content: content.clone(),
previous_content: None,
is_new_file: true,
});
file_results.push(ApplyPatchFileResult {
path: path.clone(),
action: "added".to_string(),
old_text: None,
new_text: content.clone(),
move_to: None,
});
}
FileChange::Delete {
path,
original_content,
} => {
fs.delete_file(path).await.map_err(|e| {
kigi_tool_runtime::ToolError::execution(
kigi_tool_protocol::ToolId::new("apply_patch").expect("valid"),
e.to_string(),
)
})?;
notification_handle.send_file_written(FileWritten {
tool_call_id: tool_call_id.clone(),
absolute_path: path.clone(),
content: String::new(),
previous_content: Some(original_content.clone()),
is_new_file: false,
});
file_results.push(ApplyPatchFileResult {
path: path.clone(),
action: "deleted".to_string(),
old_text: Some(original_content.clone()),
new_text: String::new(),
move_to: None,
});
}
FileChange::Update {
path,
original_content,
new_content,
} => {
fs.write_file(path, new_content.as_bytes())
.await
.map_err(|e| {
kigi_tool_runtime::ToolError::execution(
kigi_tool_protocol::ToolId::new("apply_patch").expect("valid"),
e.to_string(),
)
})?;
notification_handle.send_file_written(FileWritten {
tool_call_id: tool_call_id.clone(),
absolute_path: path.clone(),
content: new_content.clone(),
previous_content: Some(original_content.clone()),
is_new_file: false,
});
file_results.push(ApplyPatchFileResult {
path: path.clone(),
action: "modified".to_string(),
old_text: Some(original_content.clone()),
new_text: new_content.clone(),
move_to: None,
});
}
FileChange::Move {
source_path,
dest_path,
original_content,
new_content,
} => {
// Create parent dirs for destination.
ensure_parent_dirs(dest_path).await?;
fs.write_file(dest_path, new_content.as_bytes())
.await
.map_err(|e| {
kigi_tool_runtime::ToolError::execution(
kigi_tool_protocol::ToolId::new("apply_patch").expect("valid"),
e.to_string(),
)
})?;
fs.delete_file(source_path).await.map_err(|e| {
kigi_tool_runtime::ToolError::execution(
kigi_tool_protocol::ToolId::new("apply_patch").expect("valid"),
e.to_string(),
)
})?;
// Notify destination (new file at new location).
notification_handle.send_file_written(FileWritten {
tool_call_id: tool_call_id.clone(),
absolute_path: dest_path.clone(),
content: new_content.clone(),
previous_content: None,
is_new_file: true,
});
// Notify source (deleted).
notification_handle.send_file_written(FileWritten {
tool_call_id: tool_call_id.clone(),
absolute_path: source_path.clone(),
content: String::new(),
previous_content: Some(original_content.clone()),
is_new_file: false,
});
file_results.push(ApplyPatchFileResult {
path: source_path.clone(),
action: "moved".to_string(),
old_text: Some(original_content.clone()),
new_text: new_content.clone(),
move_to: Some(dest_path.clone()),
});
}
}
}
// ── Phase 4: Build summary ───────────────────────────────
let tool_output_for_prompt = build_summary(&file_results);
Ok(ApplyPatchOutput::Success {
files: file_results,
tool_output_for_prompt,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::computer::local::LocalFs;
use crate::notification::types::ToolNotificationHandle;
use crate::types::resources::Resources;
use crate::types::tool_metadata::test_ctx;
use tempfile::TempDir;
/// Set up Resources with real filesystem for tests.
fn test_resources(cwd: &std::path::Path) -> Resources {
let mut resources = Resources::new();
resources.insert(Cwd(cwd.to_path_buf()));
resources.insert(FileSystem(Arc::new(LocalFs)));
resources.insert(NotificationHandle(ToolNotificationHandle::noop()));
resources
}
/// Build a runtime `ToolCallContext` with the given shared resources.
fn make_input(patch: &str) -> ApplyPatchInput {
ApplyPatchInput {
patch: patch.to_string(),
}
}
fn wrap_patch(body: &str) -> String {
format!("*** Begin Patch\n{body}\n*** End Patch")
}
// ── Add file ─────────────────────────────────────────────────
#[tokio::test]
async fn add_file_creates_with_correct_content() {
let tmp = TempDir::new().unwrap();
let tool = ApplyPatchTool;
let resources = test_resources(tmp.path());
let shared = resources.into_shared();
let patch = wrap_patch("*** Add File: new.txt\n+hello\n+world");
let result =
kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), make_input(&patch))
.await
.unwrap();
match result {
ApplyPatchOutput::Success {
files,
tool_output_for_prompt,
} => {
assert_eq!(files.len(), 1);
assert_eq!(files[0].action, "added");
assert!(tool_output_for_prompt.contains("A "));
let content = std::fs::read_to_string(tmp.path().join("new.txt")).unwrap();
assert_eq!(content, "hello\nworld\n");
}
other => panic!("Expected Success, got: {other:?}"),
}
}
// ── Delete file ──────────────────────────────────────────────
#[tokio::test]
async fn delete_file_removes_file() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("del.txt"), "content").unwrap();
let tool = ApplyPatchTool;
let resources = test_resources(tmp.path());
let shared = resources.into_shared();
let patch = wrap_patch("*** Delete File: del.txt");
let result =
kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), make_input(&patch))
.await
.unwrap();
match result {
ApplyPatchOutput::Success {
files,
tool_output_for_prompt,
} => {
assert_eq!(files.len(), 1);
assert_eq!(files[0].action, "deleted");
assert!(tool_output_for_prompt.contains("D "));
assert!(!tmp.path().join("del.txt").exists());
}
other => panic!("Expected Success, got: {other:?}"),
}
}
// ── Update file ──────────────────────────────────────────────
#[tokio::test]
async fn update_file_modifies_content() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("update.txt"), "foo\nbar\n").unwrap();
let tool = ApplyPatchTool;
let resources = test_resources(tmp.path());
let shared = resources.into_shared();
let patch = wrap_patch("*** Update File: update.txt\n@@\n foo\n-bar\n+baz");
let result =
kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), make_input(&patch))
.await
.unwrap();
match result {
ApplyPatchOutput::Success {
files,
tool_output_for_prompt,
} => {
assert_eq!(files.len(), 1);
assert_eq!(files[0].action, "modified");
assert!(tool_output_for_prompt.contains("M "));
let content = std::fs::read_to_string(tmp.path().join("update.txt")).unwrap();
assert_eq!(content, "foo\nbaz\n");
}
other => panic!("Expected Success, got: {other:?}"),
}
}
// ── Move file ────────────────────────────────────────────────
#[tokio::test]
async fn move_file_renames_and_modifies() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("src.txt"), "line\n").unwrap();
let tool = ApplyPatchTool;
let resources = test_resources(tmp.path());
let shared = resources.into_shared();
let patch = wrap_patch("*** Update File: src.txt\n*** Move to: dst.txt\n@@\n-line\n+line2");
let result =
kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), make_input(&patch))
.await
.unwrap();
match result {
ApplyPatchOutput::Success { files, .. } => {
assert_eq!(files.len(), 1);
assert_eq!(files[0].action, "moved");
assert_eq!(files[0].move_to, Some(tmp.path().join("dst.txt")));
assert!(!tmp.path().join("src.txt").exists());
let content = std::fs::read_to_string(tmp.path().join("dst.txt")).unwrap();
assert_eq!(content, "line2\n");
}
other => panic!("Expected Success, got: {other:?}"),
}
}
// ── Multiple files in one patch ──────────────────────────────
#[tokio::test]
async fn multiple_files_in_one_patch() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("existing.txt"), "old\n").unwrap();
let tool = ApplyPatchTool;
let resources = test_resources(tmp.path());
let shared = resources.into_shared();
let patch = wrap_patch(
"*** Add File: a.txt\n+aaa\n\
*** Add File: b.txt\n+bbb",
);
let result =
kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), make_input(&patch))
.await
.unwrap();
match result {
ApplyPatchOutput::Success {
files,
tool_output_for_prompt,
} => {
assert_eq!(files.len(), 2);
assert!(tool_output_for_prompt.contains("A "));
assert_eq!(
std::fs::read_to_string(tmp.path().join("a.txt")).unwrap(),
"aaa\n"
);
assert_eq!(
std::fs::read_to_string(tmp.path().join("b.txt")).unwrap(),
"bbb\n"
);
}
other => panic!("Expected Success, got: {other:?}"),
}
}
// ── Parse error ──────────────────────────────────────────────
#[tokio::test]
async fn parse_error_returns_no_changes() {
let tmp = TempDir::new().unwrap();
let tool = ApplyPatchTool;
let resources = test_resources(tmp.path());
let shared = resources.into_shared();
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx(shared.clone()),
make_input("not a valid patch"),
)
.await
.unwrap();
match result {
ApplyPatchOutput::ParseError(msg) => {
assert!(msg.contains("Invalid patch"));
}
other => panic!("Expected ParseError, got: {other:?}"),
}
}
// ── Application error ────────────────────────────────────────
#[tokio::test]
async fn application_error_on_missing_lines() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("file.txt"), "actual\n").unwrap();
let tool = ApplyPatchTool;
let resources = test_resources(tmp.path());
let shared = resources.into_shared();
let patch = wrap_patch("*** Update File: file.txt\n@@\n-nonexistent\n+replacement");
let result =
kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), make_input(&patch))
.await
.unwrap();
match result {
ApplyPatchOutput::ApplicationError(msg) => {
assert!(msg.contains("Failed to find expected lines"));
}
other => panic!("Expected ApplicationError, got: {other:?}"),
}
}
// ── Empty patch ──────────────────────────────────────────────
#[tokio::test]
async fn empty_patch_returns_empty_patch_output() {
let tmp = TempDir::new().unwrap();
let tool = ApplyPatchTool;
let resources = test_resources(tmp.path());
let shared = resources.into_shared();
let patch = "*** Begin Patch\n*** End Patch";
let result =
kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), make_input(patch))
.await
.unwrap();
match result {
ApplyPatchOutput::EmptyPatch(msg) => {
assert!(msg.contains("No files were modified"));
}
other => panic!("Expected EmptyPatch, got: {other:?}"),
}
}
}
@@ -0,0 +1,7 @@
//! Codex `grep_files` — file-path-only regex search.
//!
//! Ported from `codex-rs/core/src/tools/handlers/grep_files.rs`.
pub mod tool;
pub use tool::{CodexGrepFilesInput, CodexGrepFilesTool};
@@ -0,0 +1,548 @@
//! `CodexGrepFilesTool` — file-path-only regex search via ripgrep.
//!
//! This is a faithful port of `codex-rs/core/src/tools/handlers/grep_files.rs`.
//! It returns **file paths only** (`--files-with-matches`), sorted by
//! modification time. See the plan document for the full diff vs the
//! grok-build `GrepTool`.
use std::path::{Path, PathBuf};
use std::time::Duration;
use serde::{Deserialize, Serialize};
use tokio::process::Command;
use tokio::time::timeout;
use crate::implementations::grok_build::grep::ripgrep::rg_path;
use crate::types::output::CodexGrepFilesOutput;
use crate::types::requirements::Expr;
#[allow(unused_imports)]
use crate::types::resources::Cwd;
use crate::types::tool::{ToolKind, ToolNamespace};
// ─── Constants ──────────────────────────────────────────────────────
const DEFAULT_LIMIT: usize = 100;
const MAX_LIMIT: usize = 2000;
const COMMAND_TIMEOUT: Duration = Duration::from_secs(30);
// ─── Description ────────────────────────────────────────────────────
const DESCRIPTION: &str =
"Finds files whose contents match the pattern and lists them by modification time.";
// ─── Input ──────────────────────────────────────────────────────────
fn default_limit() -> usize {
DEFAULT_LIMIT
}
/// Input for the codex `grep_files` tool.
#[derive(Debug, Clone, Serialize, Deserialize, schemars::JsonSchema)]
pub struct CodexGrepFilesInput {
/// Regular expression pattern to search for.
pub pattern: String,
/// Optional glob that limits which files are searched (e.g. "*.rs" or "*.{ts,tsx}").
#[serde(default)]
pub include: Option<String>,
/// Directory or file path to search. Defaults to the session's working directory.
#[serde(default)]
pub path: Option<String>,
/// Maximum number of file paths to return (defaults to 100).
#[serde(default = "default_limit")]
pub limit: usize,
}
// ─── Tool ───────────────────────────────────────────────────────────
/// Codex-namespace grep_files tool — file-path-only regex search.
///
/// Shares `ToolKind::Search` with the grok-build `GrepTool`. These tools are
/// namespace-exclusive — consumers enable either `GrokBuild` or `Codex` search,
/// never both simultaneously. This follows the same pattern as
/// `CodexListDirTool`/`ListDirTool` (`ToolKind::ListDir`) and
/// `CodexReadFileTool`/`ReadFileImpl` (`ToolKind::Read`).
#[derive(Debug, Default)]
pub struct CodexGrepFilesTool;
// ─── rg execution ───────────────────────────────────────────────────
/// Run `rg --files-with-matches` and return matching file paths.
///
/// Direct port from `codex-rs/core/src/tools/handlers/grep_files.rs`.
async fn run_rg_search(
pattern: &str,
include: Option<&str>,
search_path: &Path,
limit: usize,
cwd: &Path,
) -> Result<Vec<String>, String> {
let rg_exec = rg_path();
let mut command = Command::new(rg_exec);
command
.current_dir(cwd)
.arg("--files-with-matches")
.arg("--sortr=modified")
.arg("--regexp")
.arg(pattern)
.arg("--no-messages");
if let Some(glob) = include {
command.arg("--glob").arg(glob);
}
command.arg("--").arg(search_path);
crate::util::detach_command(&mut command);
command.stdin(std::process::Stdio::null());
let output = timeout(COMMAND_TIMEOUT, command.output())
.await
.map_err(|_| "rg timed out after 30 seconds".to_string())?
.map_err(|err| {
format!("failed to launch rg: {err}. Ensure ripgrep is installed and on PATH.")
})?;
match output.status.code() {
Some(0) => Ok(parse_results(&output.stdout, limit)),
Some(1) => Ok(Vec::new()),
_ => {
let stderr = String::from_utf8_lossy(&output.stderr);
Err(format!("rg failed: {stderr}"))
}
}
}
/// Parse newline-separated file paths from rg stdout.
///
/// Direct port from `codex-rs/core/src/tools/handlers/grep_files.rs`.
fn parse_results(stdout: &[u8], limit: usize) -> Vec<String> {
let mut results = Vec::new();
for line in stdout.split(|byte| *byte == b'\n') {
if line.is_empty() {
continue;
}
if let Ok(text) = std::str::from_utf8(line) {
if text.is_empty() {
continue;
}
results.push(text.to_string());
if results.len() == limit {
break;
}
}
}
results
}
// ─── Tests ──────────────────────────────────────────────────────────
impl crate::types::tool_metadata::ToolMetadata for CodexGrepFilesTool {
fn kind(&self) -> ToolKind {
ToolKind::Search
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::Codex
}
fn description_template(&self) -> &str {
DESCRIPTION
}
fn requires_expr(&self) -> Expr<crate::types::requirements::ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for CodexGrepFilesTool {
type Args = CodexGrepFilesInput;
type Output = CodexGrepFilesOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("grep_files").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"grep_files",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(name = "tool.codex_grep_files", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: CodexGrepFilesInput,
) -> Result<CodexGrepFilesOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let cwd = crate::types::tool_metadata::resolve_cwd(&ctx, &resources).await?;
// Validation (exact codex rules)
let pattern = input.pattern.trim().to_string();
if pattern.is_empty() {
return Ok(CodexGrepFilesOutput::Error(
"pattern must not be empty".to_string(),
));
}
if input.limit == 0 {
return Ok(CodexGrepFilesOutput::Error(
"limit must be greater than zero".to_string(),
));
}
let limit = input.limit.min(MAX_LIMIT);
// Resolve search path
let search_path = match &input.path {
Some(p) if !p.is_empty() => {
let p = PathBuf::from(p);
if p.is_absolute() { p } else { cwd.join(p) }
}
_ => cwd.clone(),
};
// Verify path exists
if let Err(err) = tokio::fs::metadata(&search_path).await {
return Ok(CodexGrepFilesOutput::Error(format!(
"unable to access `{}`: {err}",
search_path.display()
)));
}
// Clean up include glob
let include = input.include.as_deref().map(str::trim).and_then(|v| {
if v.is_empty() {
None
} else {
Some(v.to_string())
}
});
// Run rg
let results = run_rg_search(&pattern, include.as_deref(), &search_path, limit, &cwd).await;
match results {
Ok(files) if files.is_empty() => Ok(CodexGrepFilesOutput::NoMatches(
"No matches found.".to_string(),
)),
Ok(files) => {
let file_count = files.len();
Ok(CodexGrepFilesOutput::Matches {
content: files.join("\n"),
file_count,
})
}
Err(msg) => Ok(CodexGrepFilesOutput::Error(msg)),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::resources::Resources;
use std::process::Command as StdCommand;
use tempfile::TempDir;
/// Build a runtime `ToolCallContext` with the given resources.
fn test_ctx(cwd: &Path) -> kigi_tool_runtime::ToolCallContext {
let mut resources = Resources::new();
resources.insert(Cwd(cwd.to_path_buf()));
let mut ctx = kigi_tool_runtime::ToolCallContext::default();
ctx.extensions.insert(resources.into_shared());
ctx
}
fn rg_available() -> bool {
StdCommand::new("rg")
.arg("--version")
.output()
.map(|output| output.status.success())
.unwrap_or(false)
}
/// Build a runtime `ToolCallContext` with the given resources.
// ── Unit tests (parse_results) ──────────────────────────────
#[test]
fn parses_basic_results() {
let stdout = b"/tmp/file_a.rs\n/tmp/file_b.rs\n";
let parsed = parse_results(stdout, 10);
assert_eq!(
parsed,
vec!["/tmp/file_a.rs".to_string(), "/tmp/file_b.rs".to_string()]
);
}
#[test]
fn parse_truncates_after_limit() {
let stdout = b"/tmp/file_a.rs\n/tmp/file_b.rs\n/tmp/file_c.rs\n";
let parsed = parse_results(stdout, 2);
assert_eq!(
parsed,
vec!["/tmp/file_a.rs".to_string(), "/tmp/file_b.rs".to_string()]
);
}
#[test]
fn parse_skips_empty_lines() {
let stdout = b"/tmp/file_a.rs\n\n\n/tmp/file_b.rs\n";
let parsed = parse_results(stdout, 10);
assert_eq!(
parsed,
vec!["/tmp/file_a.rs".to_string(), "/tmp/file_b.rs".to_string()]
);
}
#[test]
fn parse_returns_empty_for_empty_input() {
let stdout = b"";
let parsed = parse_results(stdout, 10);
assert!(parsed.is_empty());
}
// ── Integration tests (run_rg_search) ───────────────────────
#[tokio::test]
async fn run_search_returns_results() {
if !rg_available() {
return;
}
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("match.rs"), "needle in haystack").unwrap();
std::fs::write(tmp.path().join("nomatch.rs"), "just hay").unwrap();
let results = run_rg_search("needle", None, tmp.path(), 100, tmp.path())
.await
.unwrap();
assert_eq!(results.len(), 1);
assert!(results[0].contains("match.rs"));
}
#[tokio::test]
async fn run_search_with_glob_filter() {
if !rg_available() {
return;
}
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("alpha.rs"), "needle").unwrap();
std::fs::write(tmp.path().join("beta.txt"), "needle").unwrap();
let results = run_rg_search("needle", Some("*.rs"), tmp.path(), 100, tmp.path())
.await
.unwrap();
assert_eq!(results.len(), 1);
assert!(results[0].contains("alpha.rs"));
}
#[tokio::test]
async fn run_search_respects_limit() {
if !rg_available() {
return;
}
let tmp = TempDir::new().unwrap();
for i in 0..5 {
std::fs::write(tmp.path().join(format!("file_{i}.rs")), "needle").unwrap();
}
let results = run_rg_search("needle", None, tmp.path(), 2, tmp.path())
.await
.unwrap();
assert_eq!(results.len(), 2);
}
#[tokio::test]
async fn run_search_handles_no_matches() {
if !rg_available() {
return;
}
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("file.rs"), "no match here").unwrap();
let results = run_rg_search("nonexistent_pattern_xyz", None, tmp.path(), 100, tmp.path())
.await
.unwrap();
assert!(results.is_empty());
}
// ── Tool-level tests ────────────────────────────────────────
#[tokio::test]
async fn tool_reports_empty_pattern_error() {
let tmp = TempDir::new().unwrap();
let tool = CodexGrepFilesTool;
let input = CodexGrepFilesInput {
pattern: " ".to_string(),
include: None,
path: None,
limit: 100,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(tmp.path()), input)
.await
.unwrap();
match result {
CodexGrepFilesOutput::Error(msg) => {
assert_eq!(msg, "pattern must not be empty");
}
other => panic!("Expected Error, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_reports_zero_limit_error() {
let tmp = TempDir::new().unwrap();
let tool = CodexGrepFilesTool;
let input = CodexGrepFilesInput {
pattern: "test".to_string(),
include: None,
path: None,
limit: 0,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(tmp.path()), input)
.await
.unwrap();
match result {
CodexGrepFilesOutput::Error(msg) => {
assert_eq!(msg, "limit must be greater than zero");
}
other => panic!("Expected Error, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_reports_no_matches() {
if !rg_available() {
return;
}
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("file.rs"), "nothing interesting").unwrap();
let tool = CodexGrepFilesTool;
let input = CodexGrepFilesInput {
pattern: "nonexistent_pattern_xyz".to_string(),
include: None,
path: None,
limit: 100,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(tmp.path()), input)
.await
.unwrap();
match result {
CodexGrepFilesOutput::NoMatches(msg) => {
assert_eq!(msg, "No matches found.");
}
other => panic!("Expected NoMatches, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_collects_matches() {
if !rg_available() {
return;
}
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("alpha.rs"), "needle here").unwrap();
std::fs::write(tmp.path().join("beta.rs"), "needle there").unwrap();
std::fs::write(tmp.path().join("gamma.txt"), "no match").unwrap();
let tool = CodexGrepFilesTool;
let input = CodexGrepFilesInput {
pattern: "needle".to_string(),
include: Some("*.rs".to_string()),
path: None,
limit: 100,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(tmp.path()), input)
.await
.unwrap();
match result {
CodexGrepFilesOutput::Matches {
file_count,
content,
} => {
assert_eq!(file_count, 2);
assert!(content.contains("alpha.rs"));
assert!(content.contains("beta.rs"));
assert!(!content.contains("gamma.txt"));
}
other => panic!("Expected Matches, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_reports_nonexistent_path_error() {
let tmp = TempDir::new().unwrap();
let tool = CodexGrepFilesTool;
let input = CodexGrepFilesInput {
pattern: "test".to_string(),
include: None,
path: Some("nonexistent_dir".to_string()),
limit: 100,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(tmp.path()), input)
.await
.unwrap();
match result {
CodexGrepFilesOutput::Error(msg) => {
assert!(
msg.contains("unable to access"),
"Expected path error, got: {msg}"
);
}
other => panic!("Expected Error, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_clamps_limit_to_max() {
if !rg_available() {
return;
}
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("file.rs"), "needle").unwrap();
let tool = CodexGrepFilesTool;
let input = CodexGrepFilesInput {
pattern: "needle".to_string(),
include: None,
path: None,
limit: 5000, // exceeds MAX_LIMIT (2000)
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(tmp.path()), input)
.await
.unwrap();
match result {
CodexGrepFilesOutput::Matches { file_count, .. } => {
assert_eq!(file_count, 1);
}
other => panic!("Expected Matches, got: {other:?}"),
}
}
}
@@ -0,0 +1,7 @@
//! Codex `list_dir` — paginated, depth-limited directory listing.
//!
//! Ported from `codex-rs/core/src/tools/handlers/list_dir.rs`.
pub mod tool;
pub use tool::{CodexListDirInput, CodexListDirTool};
@@ -0,0 +1,716 @@
//! `CodexListDirTool` — paginated, depth-limited, BFS directory listing.
//!
//! This is a faithful port of `codex-rs/core/src/tools/handlers/list_dir.rs`.
//! It does NOT respect `.gitignore`, does NOT exclude hidden files, and requires
//! absolute paths. See the plan document for the full diff vs the grok-build
//! `ListDirTool`.
use std::collections::VecDeque;
use std::path::{Path, PathBuf};
use crate::types::output::{ListDirContent, ListDirOutput};
use crate::types::requirements::Expr;
use crate::types::tool::{ToolKind, ToolNamespace};
// ─── Constants ──────────────────────────────────────────────────────
/// Maximum length (in bytes) for a single entry name before truncation.
const MAX_ENTRY_LENGTH: usize = 500;
/// Number of spaces per depth level for indentation.
const INDENTATION_SPACES: usize = 2;
// ─── Description ────────────────────────────────────────────────────
const DESCRIPTION: &str =
"Lists entries in a local directory with 1-indexed entry numbers and simple type labels.";
// ─── Input ──────────────────────────────────────────────────────────
fn default_offset() -> usize {
1
}
fn default_limit() -> usize {
25
}
fn default_depth() -> usize {
2
}
/// Input for the codex `list_dir` tool.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct CodexListDirInput {
/// Absolute path to the directory to list.
pub dir_path: String,
/// The entry number to start listing from. Must be 1 or greater.
#[serde(default = "default_offset")]
pub offset: usize,
/// The maximum number of entries to return.
#[serde(default = "default_limit")]
pub limit: usize,
/// The maximum directory depth to traverse. Must be 1 or greater.
#[serde(default = "default_depth")]
pub depth: usize,
}
// ─── Internal types ─────────────────────────────────────────────────
#[derive(Clone)]
struct DirEntry {
/// Full relative path from the listing root (used for sorting).
name: String,
/// Just the file/directory component name (used for display).
display_name: String,
/// Depth level (0 = root's direct children).
depth: usize,
/// Entry type.
kind: DirEntryKind,
}
#[derive(Clone, Copy, PartialEq, Eq)]
enum DirEntryKind {
Directory,
File,
Symlink,
Other,
}
impl From<&std::fs::FileType> for DirEntryKind {
fn from(ft: &std::fs::FileType) -> Self {
// Check is_symlink() FIRST — on Unix, a symlink to a directory has
// both is_symlink() and is_dir() true when the file_type is obtained
// via tokio::fs::DirEntry::file_type() (which follows symlinks).
// Codex checks symlink first so these are rendered with `@`, not `/`.
if ft.is_symlink() {
DirEntryKind::Symlink
} else if ft.is_dir() {
DirEntryKind::Directory
} else if ft.is_file() {
DirEntryKind::File
} else {
DirEntryKind::Other
}
}
}
// ─── Tool ───────────────────────────────────────────────────────────
/// Codex-namespace list_dir tool — paginated, depth-limited directory listing.
#[derive(Debug, Default)]
pub struct CodexListDirTool;
// ─── Core BFS logic ─────────────────────────────────────────────────
/// Orchestrator: collect entries via BFS → sort → paginate → format.
async fn list_dir_slice(
dir_path: &Path,
offset: usize,
limit: usize,
depth: usize,
) -> Result<Vec<String>, String> {
let mut entries = Vec::new();
collect_entries(dir_path, Path::new(""), 0, depth, &mut entries)
.await
.map_err(|e| format!("Failed to read directory: {e}"))?;
// Sort by full relative path (slash-normalized), case-sensitive.
entries.sort_unstable_by(|a, b| a.name.cmp(&b.name));
// Empty directory is a valid success case, not an error.
if entries.is_empty() {
return Ok(Vec::new());
}
let total = entries.len();
// offset is 1-indexed
let start_index = offset - 1;
if start_index >= total {
return Err("offset exceeds directory entry count".to_string());
}
// Compute end index, saturating to avoid overflow with large limits.
let end_index = start_index.saturating_add(limit).min(total);
let page = &entries[start_index..end_index];
let mut lines: Vec<String> = page.iter().map(format_entry_line).collect();
// Overflow message when more entries exist beyond the page.
// Use capped_limit (actual number of entries returned) to match codex behavior.
if end_index < total {
let capped_limit = end_index - start_index;
lines.push(format!("More than {} entries found", capped_limit));
}
Ok(lines)
}
/// BFS walker using `tokio::fs::read_dir`.
///
/// Collects entries breadth-first up to `max_depth` levels. Directories
/// beyond the depth limit are listed but not descended into.
///
/// Uses `PathBuf` for the relative prefix (matching codex's `prefix: &Path`
/// + `prefix.join(&file_name)`). The raw `PathBuf` is kept for recursion so
/// that subdirectory prefixes are never affected by `format_entry_name`
/// truncation or normalization.
async fn collect_entries(
dir_path: &Path,
relative_prefix: &Path,
current_depth: usize,
max_depth: usize,
entries: &mut Vec<DirEntry>,
) -> Result<(), std::io::Error> {
// Queue items: (absolute path, raw relative prefix, depth)
let mut queue: VecDeque<(PathBuf, PathBuf, usize)> = VecDeque::new();
queue.push_back((
dir_path.to_path_buf(),
relative_prefix.to_path_buf(),
current_depth,
));
while let Some((abs_path, rel_prefix, depth)) = queue.pop_front() {
let mut read_dir = tokio::fs::read_dir(&abs_path).await?;
// Collect children first so we can sort them.
// Each item: (DirEntry, absolute path, raw relative path for recursion)
let mut children: Vec<(DirEntry, PathBuf, PathBuf)> = Vec::new();
while let Some(entry) = read_dir.next_entry().await? {
let file_type = entry.file_type().await?;
let kind = DirEntryKind::from(&file_type);
let raw_name = entry.file_name();
let display_name = format_entry_component(&raw_name);
// Build the raw relative path using Path::join (matches codex).
let entry_relative_path = rel_prefix.join(&raw_name);
// The sort key is the formatted (slash-normalized, truncated) name.
let name = format_entry_name(&entry_relative_path.to_string_lossy());
children.push((
DirEntry {
name,
display_name,
depth,
kind,
},
entry.path(),
entry_relative_path,
));
}
// Sort children by relative path for deterministic ordering.
children.sort_unstable_by(|a, b| a.0.name.cmp(&b.0.name));
for (dir_entry, child_abs_path, child_raw_rel) in children {
let is_dir = dir_entry.kind == DirEntryKind::Directory;
entries.push(dir_entry);
// Descend into directories if we haven't reached max depth.
// Pass the raw relative path (not the formatted name) as the
// prefix for the next level, matching codex behavior.
if is_dir && depth + 1 < max_depth {
queue.push_back((child_abs_path, child_raw_rel, depth + 1));
}
}
}
Ok(())
}
/// Format a single entry line: `"{indent}{display_name}{suffix}"`.
fn format_entry_line(entry: &DirEntry) -> String {
let indent = " ".repeat(entry.depth * INDENTATION_SPACES);
let suffix = match entry.kind {
DirEntryKind::Directory => "/",
DirEntryKind::Symlink => "@",
DirEntryKind::Other => "?",
DirEntryKind::File => "",
};
format!("{}{}{}", indent, entry.display_name, suffix)
}
/// Normalize path separators (backslash → forward slash) and truncate.
fn format_entry_name(path: &str) -> String {
let normalized = path.replace('\\', "/");
take_at_char_boundary(&normalized, MAX_ENTRY_LENGTH).to_string()
}
/// Truncate an `OsStr` file name component at `MAX_ENTRY_LENGTH` bytes.
fn format_entry_component(name: &std::ffi::OsStr) -> String {
let s = name.to_string_lossy();
take_at_char_boundary(&s, MAX_ENTRY_LENGTH).to_string()
}
/// Truncate a string at a char boundary, returning at most `max_bytes` bytes.
fn take_at_char_boundary(s: &str, max_bytes: usize) -> &str {
let end = crate::util::floor_char_boundary(s, max_bytes);
&s[..end]
}
// ─── Tests ──────────────────────────────────────────────────────────
impl crate::types::tool_metadata::ToolMetadata for CodexListDirTool {
fn kind(&self) -> ToolKind {
ToolKind::ListDir
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::Codex
}
fn description_template(&self) -> &str {
DESCRIPTION
}
fn requires_expr(&self) -> Expr<crate::types::requirements::ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for CodexListDirTool {
type Args = CodexListDirInput;
type Output = ListDirOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("list_dir").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"list_dir",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.codex_list_dir",
skip_all,
fields(path = %input.dir_path)
)]
async fn run(
&self,
_ctx: kigi_tool_runtime::ToolCallContext,
input: CodexListDirInput,
) -> Result<ListDirOutput, kigi_tool_runtime::ToolError> {
let CodexListDirInput {
dir_path,
offset,
limit,
depth,
} = input;
if offset == 0 {
return Ok(ListDirOutput::Error(
"offset must be a 1-indexed entry number".to_string(),
));
}
if limit == 0 {
return Ok(ListDirOutput::Error(
"limit must be greater than zero".to_string(),
));
}
if depth == 0 {
return Ok(ListDirOutput::Error(
"depth must be greater than zero".to_string(),
));
}
let path = PathBuf::from(&dir_path);
if !path.is_absolute() {
return Ok(ListDirOutput::Error(
"dir_path must be an absolute path".to_string(),
));
}
let entries = match list_dir_slice(&path, offset, limit, depth).await {
Ok(entries) => entries,
Err(msg) => return Ok(ListDirOutput::Error(msg)),
};
let mut output = Vec::with_capacity(entries.len() + 1);
output.push(format!("Absolute path: {}", path.display()));
output.extend(entries);
let content = output.join("\n");
Ok(ListDirOutput::Content(ListDirContent {
content,
absolute_root_path: path,
}))
}
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::TempDir;
// ── Unit tests (core logic) ─────────────────────────────────
#[tokio::test]
async fn lists_directory_entries() {
let tmp = TempDir::new().unwrap();
// Create files and dirs.
std::fs::write(tmp.path().join("file.txt"), "content").unwrap();
std::fs::create_dir(tmp.path().join("subdir")).unwrap();
#[cfg(unix)]
std::os::unix::fs::symlink(tmp.path().join("file.txt"), tmp.path().join("link")).unwrap();
let result = list_dir_slice(tmp.path(), 1, 25, 2).await.unwrap();
// Should contain file.txt, subdir/, and on unix: link@
let joined = result.join("\n");
assert!(joined.contains("file.txt"), "missing file.txt in: {joined}");
assert!(joined.contains("subdir/"), "missing subdir/ in: {joined}");
#[cfg(unix)]
assert!(joined.contains("link@"), "missing link@ in: {joined}");
}
#[tokio::test]
async fn errors_when_offset_exceeds_entries() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("a.txt"), "").unwrap();
let err = list_dir_slice(tmp.path(), 100, 25, 2).await.unwrap_err();
assert_eq!(err, "offset exceeds directory entry count");
}
#[tokio::test]
async fn respects_depth_parameter() {
let tmp = TempDir::new().unwrap();
let sub = tmp.path().join("a");
let subsub = sub.join("b");
std::fs::create_dir_all(&subsub).unwrap();
std::fs::write(subsub.join("deep.txt"), "").unwrap();
// depth=1: only top-level entries (the dir "a" but not its children)
let depth1 = list_dir_slice(tmp.path(), 1, 100, 1).await.unwrap();
let joined1 = depth1.join("\n");
assert!(joined1.contains("a/"), "should see dir a/");
assert!(!joined1.contains("b/"), "should NOT see b/ at depth 1");
assert!(
!joined1.contains("deep.txt"),
"should NOT see deep.txt at depth 1"
);
// depth=2: top-level + children of a
let depth2 = list_dir_slice(tmp.path(), 1, 100, 2).await.unwrap();
let joined2 = depth2.join("\n");
assert!(joined2.contains("a/"), "should see dir a/");
assert!(joined2.contains("b/"), "should see b/ at depth 2");
assert!(
!joined2.contains("deep.txt"),
"should NOT see deep.txt at depth 2"
);
// depth=3: everything
let depth3 = list_dir_slice(tmp.path(), 1, 100, 3).await.unwrap();
let joined3 = depth3.join("\n");
assert!(joined3.contains("a/"), "should see dir a/");
assert!(joined3.contains("b/"), "should see b/ at depth 3");
assert!(
joined3.contains("deep.txt"),
"should see deep.txt at depth 3"
);
}
#[tokio::test]
async fn paginates_in_sorted_order() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("c.txt"), "").unwrap();
std::fs::write(tmp.path().join("a.txt"), "").unwrap();
std::fs::write(tmp.path().join("b.txt"), "").unwrap();
// First page: limit=2, should get a.txt, b.txt + overflow message
let page1 = list_dir_slice(tmp.path(), 1, 2, 1).await.unwrap();
assert!(page1[0].contains("a.txt"), "first entry should be a.txt");
assert!(page1[1].contains("b.txt"), "second entry should be b.txt");
assert!(
page1.last().unwrap().contains("More than 2 entries found"),
"should have overflow message"
);
// Second page: offset=3 → c.txt only, no overflow
let page2 = list_dir_slice(tmp.path(), 3, 2, 1).await.unwrap();
assert_eq!(page2.len(), 1, "second page should have 1 entry");
assert!(page2[0].contains("c.txt"), "should be c.txt");
}
#[tokio::test]
async fn handles_large_limit_without_overflow() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("only.txt"), "").unwrap();
// usize::MAX as limit should not panic
let result = list_dir_slice(tmp.path(), 1, usize::MAX, 1).await.unwrap();
assert_eq!(result.len(), 1);
assert!(result[0].contains("only.txt"));
}
#[tokio::test]
async fn indicates_truncated_results() {
let tmp = TempDir::new().unwrap();
// Create 40 files
for i in 0..40 {
std::fs::write(tmp.path().join(format!("file_{:03}.txt", i)), "").unwrap();
}
let result = list_dir_slice(tmp.path(), 1, 25, 1).await.unwrap();
assert!(
result
.last()
.unwrap()
.contains("More than 25 entries found"),
"should indicate truncation"
);
}
#[tokio::test]
async fn truncation_respects_sorted_order() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("z.txt"), "").unwrap();
std::fs::write(tmp.path().join("a.txt"), "").unwrap();
std::fs::write(tmp.path().join("m.txt"), "").unwrap();
// limit=2, offset=2 → should get m.txt (2nd sorted entry)
let result = list_dir_slice(tmp.path(), 2, 1, 1).await.unwrap();
assert!(result[0].contains("m.txt"), "offset=2 should land on m.txt");
}
// ── Tool integration tests ──────────────────────────────────
#[tokio::test]
async fn tool_lists_directory() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("hello.txt"), "world").unwrap();
std::fs::create_dir(tmp.path().join("sub")).unwrap();
let tool = CodexListDirTool;
let ctx = kigi_tool_runtime::ToolCallContext::default();
let input = CodexListDirInput {
dir_path: tmp.path().to_string_lossy().to_string(),
offset: 1,
limit: 25,
depth: 2,
};
let result = kigi_tool_runtime::Tool::run(&tool, ctx, input)
.await
.unwrap();
match result {
ListDirOutput::Content(content) => {
assert!(
content.content.contains("Absolute path:"),
"should have header"
);
assert!(content.content.contains("hello.txt"), "should list file");
assert!(content.content.contains("sub/"), "should list dir");
assert_eq!(content.absolute_root_path, tmp.path());
}
other => panic!("Expected Content, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_returns_error_for_invalid_offset() {
let tmp = TempDir::new().unwrap();
let tool = CodexListDirTool;
let ctx = kigi_tool_runtime::ToolCallContext::default();
let input = CodexListDirInput {
dir_path: tmp.path().to_string_lossy().to_string(),
offset: 0,
limit: 25,
depth: 2,
};
let result = kigi_tool_runtime::Tool::run(&tool, ctx, input)
.await
.unwrap();
match result {
ListDirOutput::Error(msg) => {
assert_eq!(msg, "offset must be a 1-indexed entry number");
}
other => panic!("Expected Error, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_returns_error_for_nonexistent_dir() {
let tmp = TempDir::new().unwrap();
let tool = CodexListDirTool;
let ctx = kigi_tool_runtime::ToolCallContext::default();
let input = CodexListDirInput {
dir_path: tmp.path().join("nonexistent").to_string_lossy().to_string(),
offset: 1,
limit: 25,
depth: 2,
};
let result = kigi_tool_runtime::Tool::run(&tool, ctx, input)
.await
.unwrap();
match result {
ListDirOutput::Error(msg) => {
assert!(
msg.contains("Failed to read directory"),
"Expected filesystem error, got: {msg}"
);
}
other => panic!("Expected Error, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_returns_error_for_relative_path() {
let tool = CodexListDirTool;
let ctx = kigi_tool_runtime::ToolCallContext::default();
let input = CodexListDirInput {
dir_path: "relative/path".to_string(),
offset: 1,
limit: 25,
depth: 2,
};
let result = kigi_tool_runtime::Tool::run(&tool, ctx, input)
.await
.unwrap();
match result {
ListDirOutput::Error(msg) => {
assert_eq!(msg, "dir_path must be an absolute path");
}
other => panic!("Expected Error, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_returns_error_for_zero_limit() {
let tmp = TempDir::new().unwrap();
let tool = CodexListDirTool;
let ctx = kigi_tool_runtime::ToolCallContext::default();
let input = CodexListDirInput {
dir_path: tmp.path().to_string_lossy().to_string(),
offset: 1,
limit: 0,
depth: 2,
};
let result = kigi_tool_runtime::Tool::run(&tool, ctx, input)
.await
.unwrap();
match result {
ListDirOutput::Error(msg) => {
assert_eq!(msg, "limit must be greater than zero");
}
other => panic!("Expected Error, got: {other:?}"),
}
}
#[tokio::test]
async fn tool_returns_error_for_zero_depth() {
let tmp = TempDir::new().unwrap();
let tool = CodexListDirTool;
let ctx = kigi_tool_runtime::ToolCallContext::default();
let input = CodexListDirInput {
dir_path: tmp.path().to_string_lossy().to_string(),
offset: 1,
limit: 25,
depth: 0,
};
let result = kigi_tool_runtime::Tool::run(&tool, ctx, input)
.await
.unwrap();
match result {
ListDirOutput::Error(msg) => {
assert_eq!(msg, "depth must be greater than zero");
}
other => panic!("Expected Error, got: {other:?}"),
}
}
#[tokio::test]
async fn empty_directory_returns_success() {
let tmp = TempDir::new().unwrap();
let result = list_dir_slice(tmp.path(), 1, 25, 2).await.unwrap();
assert!(result.is_empty(), "empty dir should return empty vec");
// Also verify the tool-level wrapper returns Content, not Error.
let tool = CodexListDirTool;
let ctx = kigi_tool_runtime::ToolCallContext::default();
let input = CodexListDirInput {
dir_path: tmp.path().to_string_lossy().to_string(),
offset: 1,
limit: 25,
depth: 2,
};
let output = kigi_tool_runtime::Tool::run(&tool, ctx, input)
.await
.unwrap();
match output {
ListDirOutput::Content(content) => {
assert!(
content.content.contains("Absolute path:"),
"should have header even for empty dir"
);
}
other => panic!("Empty directory should not error, got: {other:?}"),
}
}
#[cfg(unix)]
#[tokio::test]
async fn symlink_to_directory_classified_as_symlink() {
let tmp = TempDir::new().unwrap();
std::fs::create_dir(tmp.path().join("real_dir")).unwrap();
std::os::unix::fs::symlink(tmp.path().join("real_dir"), tmp.path().join("link_to_dir"))
.unwrap();
let result = list_dir_slice(tmp.path(), 1, 25, 1).await.unwrap();
let joined = result.join("\n");
assert!(
joined.contains("link_to_dir@"),
"symlink-to-dir should render with @ suffix, got: {joined}"
);
assert!(
joined.contains("real_dir/"),
"real dir should render with / suffix, got: {joined}"
);
}
#[tokio::test]
async fn capped_limit_in_overflow_message() {
let tmp = TempDir::new().unwrap();
for i in 0..30 {
std::fs::write(tmp.path().join(format!("f_{:03}.txt", i)), "").unwrap();
}
let result = list_dir_slice(tmp.path(), 1, 10, 1).await.unwrap();
let last = result.last().unwrap();
assert!(
last.contains("More than 10 entries found"),
"overflow message should use capped limit, got: {last}"
);
}
}
@@ -0,0 +1,22 @@
//! Codex-specific tool implementations.
//!
//! This module contains tools ported from the [openai/codex] project
//! (`codex-rs/core/src/tools/handlers/`), which is licensed under the
//! Apache License, Version 2.0. Copyright 2025 OpenAI. The ports have
//! been modified from the originals; see `THIRD_PARTY_NOTICES.md` at the
//! crate root for the full license text and change notice.
//!
//! Each tool lives in `ToolNamespace::Codex` and is a faithful port of
//! its codex counterpart.
//!
//! [openai/codex]: https://github.com/openai/codex
pub mod apply_patch;
pub mod grep_files;
pub mod list_dir;
pub mod read_file;
pub use apply_patch::ApplyPatchTool;
pub use grep_files::CodexGrepFilesTool;
pub use list_dir::CodexListDirTool;
pub use read_file::CodexReadFileTool;
@@ -0,0 +1,708 @@
//! Indentation-mode reader — exact port of codex `indentation::*`.
//!
//! Loads all file lines, computes effective indents (blank lines inherit
//! from previous non-blank), then expands bidirectionally from an anchor
//! line using the codex interleaved single-loop algorithm with inline
//! sibling filtering and inline header-comment handling.
use std::collections::VecDeque;
use super::text_utils::format_display;
/// Tab width used for indent measurement (spaces per tab).
const TAB_WIDTH: usize = 4;
/// Comment prefixes recognized for `include_header`.
const COMMENT_PREFIXES: &[&str] = &["#", "//", "--"];
/// Configuration for indentation-mode reading.
///
/// Mirrors codex `IndentationModeOptions`.
#[derive(Debug, Clone)]
pub(crate) struct IndentationOptions {
pub anchor_line: Option<usize>,
pub max_levels: usize,
pub include_siblings: bool,
pub include_header: bool,
pub max_lines: Option<usize>,
}
// ─── LineRecord ──────────────────────────────────────────────────────
/// Per-line record: number, raw (untruncated), display (truncated), indent.
///
/// Matches codex `LineRecord { number, raw, display, indent }`.
/// `raw` is used for `trimmed()` / `is_blank()` / `is_comment()`;
/// `display` is used for output formatting.
#[derive(Debug)]
struct LineRecord {
/// 1-indexed line number.
number: usize,
/// Raw untruncated line content (UTF-8 lossy).
raw: String,
/// Display string (UTF-8 lossy, truncated at MAX_LINE_LENGTH).
display: String,
/// Raw indent level (number of leading spaces, tabs counted as TAB_WIDTH).
indent: usize,
}
impl LineRecord {
/// Leading-whitespace-stripped raw content. Codex uses `raw.trim_start()`.
fn trimmed(&self) -> &str {
self.raw.trim_start()
}
/// Whether the line is blank (only whitespace).
fn is_blank(&self) -> bool {
self.trimmed().is_empty()
}
/// Whether the line is a comment (starts with a known prefix).
/// Codex uses `self.raw.trim().starts_with(prefix)`.
fn is_comment(&self) -> bool {
let t = self.raw.trim();
COMMENT_PREFIXES.iter().any(|p| t.starts_with(p))
}
}
// ─── Core functions ──────────────────────────────────────────────────
/// Collect line records from raw file bytes.
fn collect_lines(bytes: &[u8]) -> Vec<LineRecord> {
if bytes.is_empty() {
return vec![];
}
let mut records = Vec::new();
let mut line_num = 0usize;
let mut start = 0;
for i in 0..bytes.len() {
if bytes[i] == b'\n' {
line_num += 1;
let mut end = i;
if end > start && bytes[end - 1] == b'\r' {
end -= 1;
}
let raw_bytes = &bytes[start..end];
let raw = String::from_utf8_lossy(raw_bytes).into_owned();
let display = format_display(raw_bytes);
let indent = measure_indent(&raw);
records.push(LineRecord {
number: line_num,
raw,
display,
indent,
});
start = i + 1;
}
}
// Handle remaining content after last \n (or file without trailing \n).
if start < bytes.len() {
line_num += 1;
let mut end = bytes.len();
if end > start && bytes[end - 1] == b'\r' {
end -= 1;
}
let raw_bytes = &bytes[start..end];
let raw = String::from_utf8_lossy(raw_bytes).into_owned();
let display = format_display(raw_bytes);
let indent = measure_indent(&raw);
records.push(LineRecord {
number: line_num,
raw,
display,
indent,
});
}
records
}
/// Measure indent: count leading spaces (tabs = TAB_WIDTH spaces).
fn measure_indent(line: &str) -> usize {
let mut indent = 0;
for ch in line.chars() {
match ch {
' ' => indent += 1,
'\t' => indent += TAB_WIDTH,
_ => break,
}
}
indent
}
/// Compute effective indents: blank lines inherit the indent of the
/// previous non-blank line. Returns a vec parallel to `records`.
fn compute_effective_indents(records: &[LineRecord]) -> Vec<usize> {
let mut effective = Vec::with_capacity(records.len());
let mut last_non_blank_indent = 0usize;
for rec in records {
if rec.is_blank() {
effective.push(last_non_blank_indent);
} else {
last_non_blank_indent = rec.indent;
effective.push(rec.indent);
}
}
effective
}
/// Read a block of lines using indentation-based expansion from an anchor.
///
/// This is the main entry point for indentation mode.
///
/// Ported from codex `indentation::read_block` — uses the codex interleaved
/// single-loop algorithm with two cursors (i going up, j going down) that
/// alternate. Sibling filtering and header-comment inclusion are handled
/// **inline** during expansion, not as post-processing passes.
pub(crate) fn read_block(
bytes: &[u8],
offset: usize,
limit: usize,
options: IndentationOptions,
) -> Result<Vec<String>, String> {
let collected = collect_lines(bytes);
if collected.is_empty() {
return Ok(vec![]);
}
let anchor = options.anchor_line.unwrap_or(offset);
if anchor == 0 || anchor > collected.len() {
return Err("anchor_line exceeds file length".to_string());
}
let effective = compute_effective_indents(&collected);
// guard_limit = max_lines.unwrap_or(limit). Codex validates this > 0.
let guard_limit = options.max_lines.unwrap_or(limit);
if guard_limit == 0 {
return Err("max_lines must be greater than zero".to_string());
}
// final_limit = min(limit, guard_limit, collected.len())
let final_limit = limit.min(guard_limit).min(collected.len());
let anchor_idx = anchor - 1; // 0-indexed
let anchor_indent = effective[anchor_idx];
// Compute min_indent threshold.
let min_indent = if options.max_levels == 0 {
0
} else {
anchor_indent.saturating_sub(options.max_levels * TAB_WIDTH)
};
// Early return: final_limit == 1 → just the anchor line.
if final_limit == 1 {
let rec = &collected[anchor_idx];
return Ok(vec![format!("L{}: {}", rec.number, rec.display)]);
}
// ── Interleaved bidirectional expansion ──────────────────────
//
// Codex algorithm (lines 293–357): single `while out.len() < final_limit`
// loop. BOTH cursors are tried every iteration (up first, then down).
// A `progressed` counter tracks whether either direction added a line;
// if 0, both are exhausted and we break.
//
// `i` starts at anchor_idx - 1 going down to 0 (or -1 = exhausted).
// `j` starts at anchor_idx + 1 going up to collected.len() (= exhausted).
let mut out: VecDeque<usize> = VecDeque::new();
out.push_back(anchor_idx);
// Use isize for i so we can represent -1 as "exhausted"
let mut i: isize = anchor_idx as isize - 1;
let mut j: usize = anchor_idx + 1;
let n = collected.len();
// Counters: track boundary-level lines accepted in each direction.
let mut i_counter_min_indent: usize = 0;
let mut j_counter_min_indent: usize = 0;
while out.len() < final_limit {
let mut progressed = 0usize;
// ── ALWAYS try upward cursor (if available) ─────────────
if i >= 0 {
let added = expand_up(
&collected,
&effective,
&mut out,
&mut i,
min_indent,
options.include_siblings,
options.include_header,
&mut i_counter_min_indent,
);
if added {
progressed += 1;
}
// Short-cut: codex breaks after up if limit reached.
if out.len() >= final_limit {
break;
}
}
// ── ALWAYS try downward cursor (if available) ───────────
if j < n {
let added = expand_down(
&effective,
&mut out,
&mut j,
n,
min_indent,
options.include_siblings,
&mut j_counter_min_indent,
);
if added {
progressed += 1;
}
}
if progressed == 0 {
break;
}
}
// Trim leading/trailing blank lines.
trim_empty_lines(&collected, &mut out);
// Format output.
let lines: Vec<String> = out
.iter()
.map(|&idx| {
let rec = &collected[idx];
format!("L{}: {}", rec.number, rec.display)
})
.collect();
Ok(lines)
}
/// Expand the upward cursor by one step. Returns true if a line was
/// added to `out` (net gain — not reverted).
///
/// Codex logic (lines 296–320):
/// 1. If `eff >= min_indent`: push_front (line 300).
/// 2. If `eff == min_indent && !include_siblings`:
/// - `can_take_line = allow_header_comment || counter == 0`
/// - If can_take_line: increment counter (line is kept).
/// - If !can_take_line: pop_front (revert THIS just-pushed line), stop cursor.
/// 3. If `eff < min_indent`: stop cursor, return false.
#[allow(clippy::too_many_arguments)]
fn expand_up(
collected: &[LineRecord],
effective: &[usize],
out: &mut VecDeque<usize>,
i: &mut isize,
min_indent: usize,
include_siblings: bool,
include_header: bool,
counter: &mut usize,
) -> bool {
if *i < 0 {
return false;
}
let iu = *i as usize;
let eff = effective[iu];
if eff < min_indent {
// Below threshold — stop cursor.
*i = -1;
return false;
}
// eff >= min_indent — push first (codex line 300), then filter.
out.push_front(iu);
*i -= 1;
// Sibling filter: only applies when eff == min_indent && !include_siblings.
if eff == min_indent && !include_siblings {
let allow_header_comment = include_header && collected[iu].is_comment();
let can_take_line = allow_header_comment || *counter == 0;
if can_take_line {
*counter += 1; // line is kept, increment counter
} else {
// Revert THIS just-pushed line and stop cursor.
out.pop_front();
*i = -1;
return false; // net: no line added
}
}
true
}
/// Expand the downward cursor by one step. Returns true if a line was
/// added to `out` (net gain — not reverted).
///
/// Codex logic (lines 332–348):
/// 1. If `eff >= min_indent`: push_back (line 334).
/// 2. If `eff == min_indent && !include_siblings`:
/// - If `counter > 0`: pop_back (revert THIS just-pushed line), stop cursor.
/// - Always increment counter (line 346).
/// 3. If `eff < min_indent`: stop cursor, return false.
fn expand_down(
effective: &[usize],
out: &mut VecDeque<usize>,
j: &mut usize,
n: usize,
min_indent: usize,
include_siblings: bool,
counter: &mut usize,
) -> bool {
if *j >= n {
return false;
}
let ju = *j;
let eff = effective[ju];
if eff < min_indent {
// Below threshold — stop cursor.
*j = n;
return false;
}
// eff >= min_indent — push first (codex line 334), then filter.
out.push_back(ju);
*j += 1;
// Sibling filter: only applies when eff == min_indent && !include_siblings.
if eff == min_indent && !include_siblings {
if *counter > 0 {
// Second+ boundary-level line — revert THIS just-pushed line.
out.pop_back();
*j = n; // stop cursor
// Still increment counter (codex line 346: always increments).
*counter += 1;
return false; // net: no line added
}
*counter += 1; // always increment (codex line 346)
}
true
}
/// Trim leading and trailing blank lines from the result deque.
fn trim_empty_lines(records: &[LineRecord], deque: &mut VecDeque<usize>) {
while let Some(&idx) = deque.front() {
if records[idx].is_blank() {
deque.pop_front();
} else {
break;
}
}
while let Some(&idx) = deque.back() {
if records[idx].is_blank() {
deque.pop_back();
} else {
break;
}
}
}
// ─── Tests ───────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
fn make_opts(
anchor_line: Option<usize>,
max_levels: usize,
include_siblings: bool,
include_header: bool,
max_lines: Option<usize>,
) -> IndentationOptions {
IndentationOptions {
anchor_line,
max_levels,
include_siblings,
include_header,
max_lines,
}
}
// ── Exact-output tests ───────────────────────────────────────
#[test]
fn captures_function_block_with_limit() {
// anchor=2 (x=1, indent 4), max_levels=1, min_indent = 4-4 = 0.
// With min_indent=0, the entire file is reachable (no indent is below 0).
// Sibling filter: going up, def foo is first boundary (counter=1, accepted).
// Going down: y, return, blank (effective=4 > 0), def bar (effective=0 == min,
// counter=1, accepted), pass (effective=4 > 0, accepted). No second boundary hit,
// so downward includes everything.
let content =
b"def foo():\n x = 1\n y = 2\n return x + y\n\ndef bar():\n pass\n";
// Without limit: entire file (minus blank trim)
let opts_full = make_opts(Some(2), 1, false, true, None);
let result_full = read_block(content, 1, 2000, opts_full).unwrap();
assert!(result_full.iter().any(|l| l.contains("def foo():")));
assert!(result_full.iter().any(|l| l.contains("x = 1")));
assert!(result_full.iter().any(|l| l.contains("return x + y")));
// With max_lines=4: capped to 4 lines
let opts_limited = make_opts(Some(2), 1, false, true, Some(4));
let result = read_block(content, 1, 4, opts_limited).unwrap();
assert_eq!(
result,
vec![
"L1: def foo():",
"L2: x = 1",
"L3: y = 2",
"L4: return x + y",
]
);
}
#[test]
fn expands_to_parent_class() {
// L1: class MyClass: (indent 0)
// L2: def method(self): (indent 4)
// L3: x = 1 (indent 8) ← ANCHOR
// L4: y = 2 (indent 8)
// L5: return x + y (indent 8)
// L6: (blank, effective=8)
// L7: def other(self): (indent 4)
// L8: pass (indent 8)
// anchor=3, max_levels=2, min_indent = 8-8 = 0.
// Both directions try every iteration. Up first: class MyClass (indent 0,
// boundary counter=1, kept). Down: y (eff=8>0, kept). Up: exhausted (i=-1).
// Down: return, blank, def other (boundary counter=1, kept since counter was 0),
// pass. All accepted because min_indent=0.
let content = b"class MyClass:\n def method(self):\n x = 1\n y = 2\n return x + y\n\n def other(self):\n pass\n";
let opts = make_opts(Some(3), 2, false, true, None);
let result = read_block(content, 1, 2000, opts).unwrap();
assert_eq!(result[0], "L1: class MyClass:");
assert_eq!(result[1], "L2: def method(self):");
assert_eq!(result[2], "L3: x = 1");
assert_eq!(result[3], "L4: y = 2");
assert_eq!(result[4], "L5: return x + y");
}
#[test]
fn sibling_filter_at_nonzero_min_indent() {
// Layout:
// L1: class C: (indent 0)
// L2: def a(self): (indent 4, boundary)
// L3: pass (indent 8)
// L4: def b(self): (indent 4, boundary)
// L5: pass (indent 8)
// L6: def anchor(self): (indent 4, boundary)
// L7: x = 1 (indent 8) ← ANCHOR
// L8: def d(self): (indent 4, boundary)
// L9: pass (indent 8)
// L10: def e(self): (indent 4, boundary)
// L11: pass (indent 8)
//
// anchor=7, max_levels=1, min_indent = 8-4 = 4.
let content = b"\
class C:
def a(self):
pass
def b(self):
pass
def anchor(self):
x = 1
def d(self):
pass
def e(self):
pass
";
// Without siblings: up hits def anchor (boundary, counter 0→1, kept),
// then def b (boundary, counter==1, can_take_line=false → REVERT anchor, stop).
// Wait: up goes from anchor_idx=6 upward. i starts at 5 (def anchor line).
// L6 (idx 5) = " def anchor(self):" → eff=4 == min=4. counter==0, can_take=true.
// Push. counter=1. i=4.
// L5 (idx 4) = " pass" → eff=8 > 4. Push. i=3.
// L4 (idx 3) = " def b(self):" → eff=4 == min=4. counter==1, can_take=false.
// REVERT (pop front = L4 just pushed). Stop. i=-1.
// Wait that's not right. Let me retrace...
// Actually: push L4 first, THEN check. can_take_line = false (counter==1, not comment).
// Revert = pop front = L4 (the just-pushed one). i=-1.
//
// Down: j starts at 7 (def d).
// L8 (idx 7) = " def d(self):" → eff=4 == min=4. counter==0 → kept. counter=1.
// L9 (idx 8) = " pass" → eff=8>4 → kept.
// L10 (idx 9) = " def e(self):" → eff=4 == min=4. counter>0 → REVERT L10, stop.
//
// Result (before trim): [L5:pass, L6:def anchor, L7:x=1, L8:def d, L9:pass]
// After blank trim (no blanks): same.
let opts_no_sibs = make_opts(Some(7), 1, false, true, None);
let result_no_sibs = read_block(content, 1, 2000, opts_no_sibs).unwrap();
assert_eq!(
result_no_sibs,
vec![
"L5: pass",
"L6: def anchor(self):",
"L7: x = 1",
"L8: def d(self):",
"L9: pass",
]
);
// With siblings: all methods at indent 4 should be included
let opts_sibs = make_opts(Some(7), 1, true, true, None);
let result_sibs = read_block(content, 1, 2000, opts_sibs).unwrap();
assert!(result_sibs.iter().any(|l| l.contains("def a(")));
assert!(result_sibs.iter().any(|l| l.contains("def anchor")));
assert!(result_sibs.iter().any(|l| l.contains("def e(")));
assert!(result_sibs.len() > result_no_sibs.len());
}
#[test]
fn include_header_adds_comments() {
// L1: # Helper function (indent 0, comment)
// L2: # for computation (indent 0, comment)
// L3: def compute(x): (indent 0)
// L4: return x * 2 (indent 4) ← ANCHOR
//
// anchor=4, max_levels=1, min_indent = 4-4 = 0.
// With include_header=true: comments at indent 0 pass via allow_header_comment.
let content = b"# Helper function\n# for computation\ndef compute(x):\n return x * 2\n";
let opts_header = make_opts(Some(4), 1, false, true, None);
let result = read_block(content, 1, 2000, opts_header).unwrap();
assert_eq!(
result,
vec![
"L1: # Helper function",
"L2: # for computation",
"L3: def compute(x):",
"L4: return x * 2",
]
);
// Without header: comments at boundary are rejected by sibling filter
let opts_no_header = make_opts(Some(4), 1, false, false, None);
let result_no = read_block(content, 1, 2000, opts_no_header).unwrap();
assert_eq!(
result_no,
vec!["L3: def compute(x):", "L4: return x * 2",]
);
}
#[test]
fn limit_caps_output_size() {
// anchor=3 (b=2), max_levels=0, limit=3.
// Codex: both up+down each iteration. final_limit = min(3, 3, 6) = 3.
// Iter 1: up: push a=1 → [a,b,c...wait]
// out starts as [b]. Iter 1: up push foo → [foo, b]. down push c → [foo, b, c].
// out.len()=3 → done.
let content = b"def foo():\n a = 1\n b = 2\n c = 3\n d = 4\n e = 5\n";
let opts = make_opts(Some(3), 0, false, true, Some(3));
let result = read_block(content, 1, 3, opts).unwrap();
assert_eq!(result.len(), 3);
assert_eq!(
result,
vec!["L2: a = 1", "L3: b = 2", "L4: c = 3",]
);
}
#[test]
fn final_limit_1_returns_anchor_only() {
let content = b"line1\nline2\nline3\n";
let opts = make_opts(Some(2), 0, false, true, Some(1));
let result = read_block(content, 1, 2000, opts).unwrap();
assert_eq!(result, vec!["L2: line2"]);
}
// ── Edge cases ───────────────────────────────────────────────
#[test]
fn anchor_exceeds_file_length_error() {
let content = b"one\ntwo\n";
let opts = make_opts(Some(100), 0, false, true, None);
let result = read_block(content, 1, 2000, opts);
assert!(result.is_err());
assert_eq!(result.unwrap_err(), "anchor_line exceeds file length");
}
#[test]
fn empty_file_returns_empty() {
let content = b"";
let opts = make_opts(None, 0, false, true, None);
let result = read_block(content, 1, 2000, opts).unwrap();
assert!(result.is_empty());
}
#[test]
fn guard_limit_zero_returns_error() {
let content = b"line1\nline2\n";
let opts = make_opts(Some(1), 0, false, true, Some(0));
let result = read_block(content, 1, 2000, opts);
assert!(result.is_err());
assert_eq!(result.unwrap_err(), "max_lines must be greater than zero");
}
#[test]
fn trims_leading_trailing_blank_lines() {
// Blank lines at the edges of the expansion should be trimmed.
let content = b"\ndef foo():\n x = 1\n\n";
let opts = make_opts(Some(3), 1, false, true, None);
let result = read_block(content, 1, 2000, opts).unwrap();
// First and last lines of result should not be blank
assert!(!result.first().unwrap().ends_with(": "));
assert!(!result.last().unwrap().ends_with(": "));
}
#[test]
fn trimmed_uses_trim_start() {
// Verify that trimmed() strips only leading whitespace.
// A line like " hello " should have trimmed() = "hello "
let rec = LineRecord {
number: 1,
raw: " hello ".to_string(),
display: " hello ".to_string(),
indent: 2,
};
assert_eq!(rec.trimmed(), "hello ");
assert!(!rec.is_blank());
}
#[test]
fn is_comment_uses_raw_trim() {
// Verify is_comment uses raw.trim() (both sides), not trim_start().
let rec = LineRecord {
number: 1,
raw: " // comment ".to_string(),
display: " // comment ".to_string(),
indent: 2,
};
assert!(rec.is_comment());
}
#[test]
fn cpp_switch_shallow_expansion() {
let content = b"#include <iostream>\n\nint main() {\n switch (x) {\n case 1:\n std::cout << \"one\";\n break;\n case 2:\n std::cout << \"two\";\n break;\n }\n return 0;\n}\n";
// anchor=6 (std::cout << "one", indent 12), max_levels=1, min_indent=12-4=8.
let opts = make_opts(Some(6), 1, false, true, None);
let result = read_block(content, 1, 2000, opts).unwrap();
// Should include case 1: and its body
assert!(result.iter().any(|l| l.contains("case 1:")));
assert!(result.iter().any(|l| l.contains("\"one\"")));
}
#[test]
fn cpp_switch_deeper_expansion() {
let content = b"// Main entry point\n#include <iostream>\n\nint main() {\n switch (x) {\n case 1:\n std::cout << \"one\";\n break;\n case 2:\n std::cout << \"two\";\n break;\n }\n return 0;\n}\n";
// anchor=7 (std::cout << "one", indent 12), max_levels=2, min_indent=12-8=4.
let opts = make_opts(Some(7), 2, false, true, None);
let result = read_block(content, 1, 2000, opts).unwrap();
assert!(result.iter().any(|l| l.contains("switch (x)")));
}
}
@@ -0,0 +1,22 @@
//! Codex `read_file` — text file reader in codex `L{n}: {content}` format.
//!
//! This module ports the codex read_file tool as a separate tool under
//! `ToolNamespace::Codex`. It supports two modes:
//!
//! - **Slice mode** — reads a contiguous range of lines (default).
//! - **Indentation mode** — reads a block based on indentation structure.
//!
//! # Submodules
//!
//! - [`text_utils`] — shared text helpers (char-boundary truncation).
//! - [`slice`] — slice-mode reader (exact port of codex `slice::read()`).
//! - [`indentation`] — indentation-mode reader (exact port of codex `indentation::*`).
//! - [`tool`] — `CodexReadFileTool` implementation, input types, description.
pub mod indentation;
pub mod slice;
pub(crate) mod text_utils;
pub mod tool;
// Re-exports for convenience.
pub use tool::{CodexReadFileInput, CodexReadFileTool};
@@ -0,0 +1,190 @@
//! Slice-mode reader — exact port of codex `slice::read()`.
//!
//! Reads lines from `offset` (1-indexed) up to `limit`, formatting each
//! as `L{line_number}: {content}`. Lines are truncated at `MAX_LINE_LENGTH`
//! at a char boundary.
/// Maximum number of characters per line before truncation.
pub(crate) const MAX_LINE_LENGTH: usize = 500;
/// Read a contiguous range of lines from file bytes in slice mode.
///
/// Returns formatted lines as `L{n}: {content}`, or an error string if
/// `offset` exceeds the number of lines in the file.
pub(crate) fn read_slice(
file_bytes: &[u8],
offset: usize,
limit: usize,
) -> Result<Vec<String>, String> {
let mut collected = Vec::new();
let mut seen = 0usize;
for raw_line in split_lines(file_bytes) {
seen += 1;
if seen < offset {
continue;
}
if collected.len() == limit {
break;
}
let formatted = format_line(raw_line);
collected.push(format!("L{seen}: {formatted}"));
if collected.len() == limit {
break;
}
}
if seen < offset {
return Err("offset exceeds file length".to_string());
}
Ok(collected)
}
/// Format a raw byte line: decode as UTF-8 (lossy) and truncate at
/// `MAX_LINE_LENGTH` at a char boundary.
fn format_line(bytes: &[u8]) -> String {
super::text_utils::format_display(bytes)
}
/// Split raw bytes into lines, stripping `\n` and `\r\n` line endings.
///
/// Every byte sequence separated by `\n` becomes a line. Trailing `\r`
/// on each line is also stripped. A final `\n` produces an empty trailing
/// entry (matching codex `BufReader::read_until(b'\n')` behavior).
fn split_lines(bytes: &[u8]) -> Vec<&[u8]> {
if bytes.is_empty() {
return vec![];
}
let mut lines = Vec::new();
let mut start = 0;
for i in 0..bytes.len() {
if bytes[i] == b'\n' {
let mut end = i;
// Strip trailing \r for \r\n endings.
if end > start && bytes[end - 1] == b'\r' {
end -= 1;
}
lines.push(&bytes[start..end]);
start = i + 1;
}
}
// Remaining bytes after the last \n (or all bytes if no \n found).
if start < bytes.len() {
let mut end = bytes.len();
if end > start && bytes[end - 1] == b'\r' {
end -= 1;
}
lines.push(&bytes[start..end]);
} else if start == bytes.len() && !bytes.is_empty() && bytes[bytes.len() - 1] == b'\n' {
// File ends with \n — BufReader::read_until would NOT produce an
// empty trailing line for this case. The codex implementation reads
// until EOF and each read_until(b'\n') call consumes the delimiter.
// A trailing \n means the last read produces the line before it;
// no additional empty line is generated.
// So we do NOT push an empty trailing entry here.
}
lines
}
// ─── Tests ───────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reads_requested_range() {
let content = b"first\nsecond\nthird\nfourth\n";
let result = read_slice(content, 2, 2).unwrap();
assert_eq!(result, vec!["L2: second", "L3: third"]);
}
#[test]
fn errors_when_offset_exceeds_length() {
let content = b"one\ntwo\n";
let result = read_slice(content, 100, 10);
assert!(result.is_err());
assert_eq!(result.unwrap_err(), "offset exceeds file length");
}
#[test]
fn reads_non_utf8_lines() {
let content = b"\xff\xfe\n";
let result = read_slice(content, 1, 10).unwrap();
assert_eq!(result.len(), 1);
// Non-UTF8 bytes should be replaced with U+FFFD
assert!(result[0].contains('\u{FFFD}'));
}
#[test]
fn trims_crlf_endings() {
let content = b"hello\r\nworld\r\n";
let result = read_slice(content, 1, 10).unwrap();
assert_eq!(result, vec!["L1: hello", "L2: world"]);
}
#[test]
fn respects_limit_even_with_more_lines() {
let content = b"a\nb\nc\nd\ne\n";
let result = read_slice(content, 1, 3).unwrap();
assert_eq!(result.len(), 3);
assert_eq!(result, vec!["L1: a", "L2: b", "L3: c"]);
}
#[test]
fn truncates_lines_longer_than_max_length() {
let long_line = "x".repeat(600);
let content = format!("{}\n", long_line);
let result = read_slice(content.as_bytes(), 1, 10).unwrap();
assert_eq!(result.len(), 1);
// Line content should be truncated to MAX_LINE_LENGTH
let expected_content = &long_line[..MAX_LINE_LENGTH];
assert_eq!(result[0], format!("L1: {}", expected_content));
}
#[test]
fn reads_single_line_no_trailing_newline() {
let content = b"hello";
let result = read_slice(content, 1, 10).unwrap();
assert_eq!(result, vec!["L1: hello"]);
}
#[test]
fn reads_from_offset_1() {
let content = b"first\nsecond\nthird\n";
let result = read_slice(content, 1, 10).unwrap();
assert_eq!(result, vec!["L1: first", "L2: second", "L3: third"]);
}
#[test]
fn empty_file_returns_error() {
// Codex behavior: empty file has 0 lines, offset=1 exceeds file length.
let content = b"";
let result = read_slice(content, 1, 10);
assert!(result.is_err());
assert_eq!(result.unwrap_err(), "offset exceeds file length");
}
#[test]
fn truncation_at_multibyte_char_boundary() {
// Create a string that has multi-byte chars near the 500 boundary
let mut s = "a".repeat(498);
s.push('é'); // 2 bytes in UTF-8
s.push('x');
assert!(s.len() > MAX_LINE_LENGTH);
let content = format!("{}\n", s);
let result = read_slice(content.as_bytes(), 1, 10).unwrap();
// The truncated line should be valid UTF-8 and <= MAX_LINE_LENGTH bytes
let line_content = result[0].strip_prefix("L1: ").unwrap();
assert!(line_content.len() <= MAX_LINE_LENGTH);
assert!(line_content.is_char_boundary(line_content.len()));
}
}
@@ -0,0 +1,30 @@
//! Shared text utilities for the codex read_file tool.
use super::slice::MAX_LINE_LENGTH;
/// Truncate a string at a char boundary, returning at most `max_bytes`
/// bytes. Port of codex `take_bytes_at_char_boundary`.
pub(crate) fn take_at_char_boundary(s: &str, max_bytes: usize) -> &str {
if s.len() <= max_bytes {
return s;
}
let mut last_ok = 0;
for (i, ch) in s.char_indices() {
let nb = i + ch.len_utf8();
if nb > max_bytes {
break;
}
last_ok = nb;
}
&s[..last_ok]
}
/// UTF-8 lossy decode + truncate at MAX_LINE_LENGTH.
pub(crate) fn format_display(raw: &[u8]) -> String {
let decoded = String::from_utf8_lossy(raw);
if decoded.len() > MAX_LINE_LENGTH {
take_at_char_boundary(&decoded, MAX_LINE_LENGTH).to_string()
} else {
decoded.into_owned()
}
}
@@ -0,0 +1,643 @@
//! `CodexReadFileTool` — Tool trait implementation for the codex read_file format.
//!
//! Reads files via `AsyncFileSystem` and produces output in the codex
//! `L{n}: {content}` format. Supports both slice mode and indentation mode.
use std::path::PathBuf;
use crate::types::output::{FileContent, ReadFileOutput};
use crate::types::requirements::Expr;
#[allow(unused_imports)]
use crate::types::resources::{FileSystem, SharedResources};
use crate::types::tool::{ToolKind, ToolNamespace};
use super::{indentation, slice};
// ─── Description ─────────────────────────────────────────────────────
/// Tool description — word-for-word copy from codex `create_read_file_tool()` in
/// `codex-rs/core/src/tools/spec.rs` (line 1233).
const DESCRIPTION: &str = "Reads a local file with 1-indexed line numbers, supporting slice and indentation-aware block modes.";
// ─── Input ───────────────────────────────────────────────────────────
/// Input for the codex `read_file` tool.
///
/// Field descriptions match codex `create_read_file_tool()` parameter descriptions.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct CodexReadFileInput {
/// Absolute path to the file
pub file_path: String,
/// The line number to start reading from. Must be 1 or greater.
#[serde(default = "defaults::offset")]
pub offset: usize,
/// The maximum number of lines to return.
#[serde(default = "defaults::limit")]
pub limit: usize,
/// Optional mode selector: "slice" for simple ranges (default) or "indentation"
/// to expand around an anchor line.
#[serde(default)]
pub mode: ReadMode,
/// Indentation-mode configuration. Only used when mode is "indentation".
#[serde(default)]
pub indentation: Option<IndentationArgs>,
}
/// Read mode selector.
#[derive(Debug, Clone, Default, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum ReadMode {
#[default]
Slice,
Indentation,
}
/// Arguments for indentation-mode reading.
///
/// Field descriptions match codex `create_read_file_tool()` indentation_properties.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct IndentationArgs {
/// Anchor line to center the indentation lookup on (defaults to offset).
#[serde(default)]
pub anchor_line: Option<usize>,
/// How many parent indentation levels (smaller indents) to include.
#[serde(default = "defaults::max_levels")]
pub max_levels: usize,
/// When true, include additional blocks that share the anchor indentation.
#[serde(
default = "defaults::include_siblings",
deserialize_with = "crate::types::schema::deserialize_lenient_bool"
)]
pub include_siblings: bool,
/// Include doc comments or attributes directly above the selected block.
#[serde(
default = "defaults::include_header",
deserialize_with = "crate::types::schema::deserialize_lenient_bool"
)]
pub include_header: bool,
/// Hard cap on the number of lines returned when using indentation mode.
#[serde(default)]
pub max_lines: Option<usize>,
}
mod defaults {
pub fn offset() -> usize {
1
}
pub fn limit() -> usize {
2000
}
pub fn max_levels() -> usize {
0
}
pub fn include_siblings() -> bool {
false
}
pub fn include_header() -> bool {
true
}
}
impl Default for IndentationArgs {
fn default() -> Self {
Self {
anchor_line: None,
max_levels: defaults::max_levels(),
include_siblings: defaults::include_siblings(),
include_header: defaults::include_header(),
max_lines: None,
}
}
}
// ─── Tool ────────────────────────────────────────────────────────────
/// Codex read_file tool — reads files in the codex `L{n}: {content}` format.
#[derive(Debug, Default)]
pub struct CodexReadFileTool;
// ─── Tests ───────────────────────────────────────────────────────────
impl crate::types::tool_metadata::ToolMetadata for CodexReadFileTool {
fn kind(&self) -> ToolKind {
ToolKind::Read
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::Codex
}
fn description_template(&self) -> &str {
DESCRIPTION
}
fn requires_expr(&self) -> Expr<crate::types::requirements::ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for CodexReadFileTool {
type Args = CodexReadFileInput;
type Output = ReadFileOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("read_file").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"read_file",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(name = "tool.codex_read_file", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: CodexReadFileInput,
) -> Result<ReadFileOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
// 1. Validate. Codex raises here, but we surface these as a structured
// `FileReadError` (a model-facing error) instead of a hard `Err`, so
// otherwise-benign validation failures (empty/short files, relative
// paths) do not surface as tool-execution failures.
// `FileReadError` rides the structured-output path and maps cleanly to
// `ReadFileErrorTypes::FILE_READ_ERROR`.
if input.offset == 0 {
return Ok(ReadFileOutput::FileReadError(
"offset must be a 1-indexed line number".to_string(),
));
}
if input.limit == 0 {
return Ok(ReadFileOutput::FileReadError(
"limit must be greater than zero".to_string(),
));
}
let path = PathBuf::from(&input.file_path);
if !path.is_absolute() {
return Ok(ReadFileOutput::FileReadError(
"file_path must be an absolute path".to_string(),
));
}
// 2. Read file via AsyncFileSystem.
let fs;
{
fs = resources.lock().await.require::<FileSystem>()?.0.clone();
}
let file_bytes = match fs.read_file(&path).await {
Ok(bytes) => bytes,
Err(e) => {
return Ok(ReadFileOutput::FileReadError(format!(
"Failed to read file: {}, {e}",
path.display()
)));
}
};
// 3. Branch on mode. Out-of-range / empty-file reads return a structured
// `FileReadError` (see note above) instead of a hard `Err`.
let collected = match input.mode {
ReadMode::Slice => match slice::read_slice(&file_bytes, input.offset, input.limit) {
Ok(lines) => lines,
Err(e) => return Ok(ReadFileOutput::FileReadError(e)),
},
ReadMode::Indentation => {
let args = input.indentation.unwrap_or_default();
let options = indentation::IndentationOptions {
anchor_line: args.anchor_line,
max_levels: args.max_levels,
include_siblings: args.include_siblings,
include_header: args.include_header,
max_lines: args.max_lines,
};
match indentation::read_block(&file_bytes, input.offset, input.limit, options) {
Ok(lines) => lines,
Err(e) => return Ok(ReadFileOutput::FileReadError(e)),
}
}
};
// 4. Build formatted output (L{n}: {content} lines joined by \n).
let content = collected.join("\n");
// 5. Build raw_output — the unformatted file content for the read
// range. This matches the grok-build ReadFileTool semantics where
// raw_output is the actual file text without line-number prefixes.
let raw_output = String::from_utf8_lossy(&file_bytes).into_owned();
// 6. Compute total lines.
let total_lines = file_bytes.iter().filter(|&&b| b == b'\n').count()
+ if file_bytes.last() != Some(&b'\n') && !file_bytes.is_empty() {
1
} else {
0
};
// 7. Return.
Ok(ReadFileOutput::FileContent(FileContent {
content,
content_concise: None,
absolute_path: path,
offset: Some(input.offset),
limit: Some(input.limit),
raw_output,
total_lines,
extracted_images: Vec::new(),
}))
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::computer::local::LocalFs;
use crate::notification::types::ToolNotificationHandle;
use crate::types::resources::{Cwd, NotificationHandle, Resources};
use crate::types::tool_metadata::test_ctx;
use std::sync::Arc;
use tempfile::TempDir;
/// Set up Resources with real filesystem for tests.
fn test_resources(cwd: &std::path::Path) -> Resources {
let mut resources = Resources::new();
resources.insert(Cwd(cwd.to_path_buf()));
resources.insert(FileSystem(Arc::new(LocalFs)));
resources.insert(NotificationHandle(ToolNotificationHandle::noop()));
resources
}
/// Build a runtime `ToolCallContext` with the given shared resources.
// ── Slice mode tests ─────────────────────────────────────────
#[tokio::test]
async fn slice_reads_requested_range() {
let tmp = TempDir::new().unwrap();
let file_path = tmp.path().join("test.txt");
std::fs::write(&file_path, "first\nsecond\nthird\nfourth\n").unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: file_path.to_string_lossy().to_string(),
offset: 2,
limit: 2,
mode: ReadMode::Slice,
indentation: None,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileContent(fc) => {
assert_eq!(fc.content, "L2: second\nL3: third");
}
other => panic!("Expected FileContent, got {:?}", other),
}
}
#[tokio::test]
async fn slice_offset_exceeds_length_returns_read_error() {
let tmp = TempDir::new().unwrap();
let file_path = tmp.path().join("test.txt");
std::fs::write(&file_path, "one\ntwo\n").unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: file_path.to_string_lossy().to_string(),
offset: 100,
limit: 10,
mode: ReadMode::Slice,
indentation: None,
};
// Out-of-range reads are surfaced as a structured `FileReadError` (a
// model-facing error), not a hard `Err`.
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileReadError(msg) => {
assert!(msg.contains("offset exceeds file length"), "got: {msg}");
}
other => panic!("Expected FileReadError, got {:?}", other),
}
}
#[tokio::test]
async fn empty_file_returns_read_error() {
// An empty file has 0 lines, so the default offset=1 is out of range.
// Codex treats this as a read error; we surface it as a structured
// FileReadError instead of a hard Err.
let tmp = TempDir::new().unwrap();
let file_path = tmp.path().join("empty.txt");
std::fs::write(&file_path, "").unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: file_path.to_string_lossy().to_string(),
offset: 1,
limit: 10,
mode: ReadMode::Slice,
indentation: None,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileReadError(msg) => {
assert!(msg.contains("offset exceeds file length"), "got: {msg}");
}
other => panic!("Expected FileReadError, got {:?}", other),
}
}
#[tokio::test]
async fn slice_reads_non_utf8() {
let tmp = TempDir::new().unwrap();
let file_path = tmp.path().join("binary.txt");
std::fs::write(&file_path, b"\xff\xfe\n").unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: file_path.to_string_lossy().to_string(),
offset: 1,
limit: 10,
mode: ReadMode::Slice,
indentation: None,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileContent(fc) => {
assert_eq!(fc.content, format!("L1: {}{}", '\u{FFFD}', '\u{FFFD}'));
}
other => panic!("Expected FileContent, got {:?}", other),
}
}
#[tokio::test]
async fn slice_trims_crlf() {
let tmp = TempDir::new().unwrap();
let file_path = tmp.path().join("crlf.txt");
std::fs::write(&file_path, b"hello\r\nworld\r\n").unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: file_path.to_string_lossy().to_string(),
offset: 1,
limit: 10,
mode: ReadMode::Slice,
indentation: None,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileContent(fc) => {
assert_eq!(fc.content, "L1: hello\nL2: world");
}
other => panic!("Expected FileContent, got {:?}", other),
}
}
// ── Indentation mode tests ───────────────────────────────────
#[tokio::test]
async fn indentation_mode_captures_block() {
let tmp = TempDir::new().unwrap();
let file_path = tmp.path().join("code.py");
std::fs::write(
&file_path,
"def foo():\n x = 1\n y = 2\n return x + y\n\ndef bar():\n pass\n",
)
.unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: file_path.to_string_lossy().to_string(),
offset: 1,
limit: 2000,
mode: ReadMode::Indentation,
indentation: Some(IndentationArgs {
anchor_line: Some(2),
max_levels: 1,
include_siblings: false,
include_header: true,
max_lines: None,
}),
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileContent(fc) => {
assert!(fc.content.contains("def foo():"));
assert!(fc.content.contains("x = 1"));
assert!(fc.content.contains("y = 2"));
assert!(fc.content.contains("return x + y"));
}
other => panic!("Expected FileContent, got {:?}", other),
}
}
// ── Validation tests ─────────────────────────────────────────
#[tokio::test]
async fn indentation_anchor_past_eof_returns_read_error() {
// Indentation-mode range errors flow through the same read_block match
// arm as slice mode, so they must also surface as a structured
// FileReadError rather than a hard Err.
let tmp = TempDir::new().unwrap();
let file_path = tmp.path().join("code.py");
std::fs::write(&file_path, "def foo():\n x = 1\n").unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: file_path.to_string_lossy().to_string(),
offset: 1,
limit: 2000,
mode: ReadMode::Indentation,
indentation: Some(IndentationArgs {
anchor_line: Some(100),
max_levels: 0,
include_siblings: false,
include_header: true,
max_lines: None,
}),
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileReadError(msg) => {
assert!(
msg.contains("anchor_line exceeds file length"),
"got: {msg}"
);
}
other => panic!("Expected FileReadError, got {:?}", other),
}
}
#[tokio::test]
async fn validation_offset_zero() {
let tmp = TempDir::new().unwrap();
let file_path = tmp.path().join("test.txt");
std::fs::write(&file_path, "content\n").unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: file_path.to_string_lossy().to_string(),
offset: 0,
limit: 10,
mode: ReadMode::Slice,
indentation: None,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileReadError(msg) => {
assert!(
msg.contains("offset must be a 1-indexed line number"),
"got: {msg}"
);
}
other => panic!("Expected FileReadError, got {:?}", other),
}
}
#[tokio::test]
async fn validation_limit_zero() {
let tmp = TempDir::new().unwrap();
let file_path = tmp.path().join("test.txt");
std::fs::write(&file_path, "content\n").unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: file_path.to_string_lossy().to_string(),
offset: 1,
limit: 0,
mode: ReadMode::Slice,
indentation: None,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileReadError(msg) => {
assert!(
msg.contains("limit must be greater than zero"),
"got: {msg}"
);
}
other => panic!("Expected FileReadError, got {:?}", other),
}
}
#[tokio::test]
async fn validation_relative_path() {
let tmp = TempDir::new().unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: "relative/path.txt".to_string(),
offset: 1,
limit: 10,
mode: ReadMode::Slice,
indentation: None,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileReadError(msg) => {
assert!(
msg.contains("file_path must be an absolute path"),
"got: {msg}"
);
}
other => panic!("Expected FileReadError, got {:?}", other),
}
}
#[tokio::test]
async fn file_not_found() {
let tmp = TempDir::new().unwrap();
let tool = CodexReadFileTool;
let shared = test_resources(tmp.path()).into_shared();
let input = CodexReadFileInput {
file_path: tmp
.path()
.join("nonexistent.txt")
.to_string_lossy()
.to_string(),
offset: 1,
limit: 10,
mode: ReadMode::Slice,
indentation: None,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(shared.clone()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileReadError(msg) => {
assert!(msg.contains("Failed to read file"));
}
other => panic!("Expected FileReadError, got {:?}", other),
}
}
}
@@ -0,0 +1,767 @@
//! Cursor project-rule reminders attached after successful file reads.
use std::collections::HashSet;
use std::collections::hash_map::Entry;
use std::path::{Path, PathBuf};
use globset::GlobBuilder;
use serde::{Deserialize, Serialize};
use crate::types::resources::{SharedResources, State};
#[derive(Debug, Default, Serialize, Deserialize)]
pub struct CursorRulesOnReadTracker {
#[serde(default)]
scanned_scope_dirs: HashSet<PathBuf>,
#[serde(default)]
rules: Vec<ParsedCursorRule>,
#[serde(default)]
injected_rule_paths: HashSet<PathBuf>,
}
crate::register_resource!("cursor", "RulesOnRead", CursorRulesOnReadTracker);
#[derive(Debug, Clone, Serialize, Deserialize)]
struct ParsedCursorRule {
full_path: PathBuf,
scope_dir: PathBuf,
body: String,
kind: CursorRuleKind,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
enum CursorRuleKind {
Global,
FileGlobbed(Vec<String>),
AgentFetched,
Manual,
}
#[derive(Debug, Default, Deserialize)]
#[serde(rename_all = "camelCase")]
struct CursorRuleFrontmatter {
#[serde(default)]
always_apply: bool,
#[serde(default)]
globs: Option<GlobField>,
#[serde(default)]
description: Option<String>,
}
#[derive(Debug, Clone, Deserialize)]
#[serde(untagged)]
enum GlobField {
String(String),
List(Vec<String>),
}
impl GlobField {
fn into_patterns(self) -> Vec<String> {
match self {
Self::String(value) => value
.split(',')
.map(str::trim)
.filter(|s| !s.is_empty())
.map(str::to_owned)
.collect(),
Self::List(values) => values
.into_iter()
.flat_map(|value| {
value
.split(',')
.map(str::trim)
.filter(|s| !s.is_empty())
.map(str::to_owned)
.collect::<Vec<_>>()
})
.collect(),
}
}
}
pub async fn append_cursor_rules_for_read(
enabled: bool,
resources: SharedResources,
workspace_root: &Path,
read_path: &Path,
content: &mut String,
content_concise: &mut Option<String>,
) {
if !enabled {
return;
}
let Some(reminder) =
cursor_rule_reminder_for_read_inner(resources, workspace_root, read_path).await
else {
return;
};
append_reminder(content, &reminder);
if let Some(concise) = content_concise {
append_reminder(concise, &reminder);
}
}
fn append_reminder(content: &mut String, reminder: &str) {
if !content.is_empty() {
content.push_str("\n\n");
}
content.push_str(reminder);
}
async fn cursor_rule_reminder_for_read_inner(
resources: SharedResources,
workspace_root: &Path,
read_path: &Path,
) -> Option<String> {
let workspace_root = normalize_existing(workspace_root).await;
let read_path = normalize_existing(read_path).await;
let scope_dirs = ancestor_scope_dirs(&workspace_root, &read_path);
if scope_dirs.is_empty() {
return None;
}
let (dirs_to_scan, injected_rule_paths_to_normalize) = {
let mut res = resources.lock().await;
let tracker = &mut res.get_or_default::<State<CursorRulesOnReadTracker>>().0;
let dirs_to_scan = scope_dirs
.iter()
.filter(|dir| !tracker.scanned_scope_dirs.contains(*dir))
.cloned()
.collect::<Vec<_>>();
(
dirs_to_scan,
tracker
.injected_rule_paths
.iter()
.cloned()
.collect::<Vec<_>>(),
)
};
let mut discovered = Vec::new();
for scope_dir in &dirs_to_scan {
discovered.extend(scan_scope_dir(scope_dir).await);
}
let normalized_injected_rule_paths =
normalize_existing_paths(injected_rule_paths_to_normalize, &workspace_root).await;
let matching_rules = {
let mut res = resources.lock().await;
let tracker = &mut res.get_or_default::<State<CursorRulesOnReadTracker>>().0;
tracker
.injected_rule_paths
.extend(normalized_injected_rule_paths);
tracker.scanned_scope_dirs.extend(dirs_to_scan);
append_new_rules(&mut tracker.rules, discovered);
let mut matching_rules = Vec::new();
for rule in &tracker.rules {
if tracker.injected_rule_paths.contains(&rule.full_path) {
continue;
}
if rule_matches_read_path(rule, &workspace_root, &read_path) {
tracker.injected_rule_paths.insert(rule.full_path.clone());
matching_rules.push(rule.clone());
}
}
matching_rules
};
render_cursor_rule_reminder(&matching_rules)
}
fn append_new_rules(existing: &mut Vec<ParsedCursorRule>, discovered: Vec<ParsedCursorRule>) {
let mut by_path = existing
.iter()
.enumerate()
.map(|(idx, rule)| (rule.full_path.clone(), idx))
.collect::<std::collections::HashMap<_, _>>();
for rule in discovered {
match by_path.entry(rule.full_path.clone()) {
Entry::Occupied(_) => {}
Entry::Vacant(entry) => {
entry.insert(existing.len());
existing.push(rule);
}
}
}
}
fn ancestor_scope_dirs(workspace_root: &Path, read_path: &Path) -> Vec<PathBuf> {
let Some(parent) = read_path.parent() else {
return Vec::new();
};
if !parent.starts_with(workspace_root) {
return Vec::new();
}
let mut dirs = Vec::new();
let mut current = Some(parent);
while let Some(dir) = current {
if !dir.starts_with(workspace_root) {
break;
}
dirs.push(dir.to_path_buf());
if dir == workspace_root {
break;
}
current = dir.parent();
}
dirs.reverse();
dirs
}
async fn normalize_existing(path: &Path) -> PathBuf {
crate::util::fs::canonicalize_with_timeout(path.to_path_buf()).await
}
async fn normalize_existing_paths(
paths: impl IntoIterator<Item = PathBuf>,
workspace_root: &Path,
) -> HashSet<PathBuf> {
let mut normalized = HashSet::new();
for path in paths {
let absolute_path = if path.is_relative() {
workspace_root.join(&path)
} else {
path
};
normalized.insert(normalize_existing(&absolute_path).await);
}
normalized
}
async fn scan_scope_dir(scope_dir: &Path) -> Vec<ParsedCursorRule> {
let rules_dir = scope_dir.join(".cursor").join("rules");
let Ok(metadata) = tokio::fs::metadata(&rules_dir).await else {
return Vec::new();
};
if !metadata.is_dir() {
return Vec::new();
}
let mut rules = Vec::new();
let mut stack = vec![rules_dir];
while let Some(dir) = stack.pop() {
let Ok(mut read_dir) = tokio::fs::read_dir(&dir).await else {
continue;
};
let mut dirs = Vec::new();
let mut files = Vec::new();
while let Ok(Some(entry)) = read_dir.next_entry().await {
let path = entry.path();
match entry.file_type().await {
Ok(file_type) if file_type.is_dir() => dirs.push(path),
Ok(file_type) if file_type.is_file() && is_rule_file(&path) => files.push(path),
_ => {}
}
}
dirs.sort();
files.sort();
stack.extend(dirs.into_iter().rev());
for file in files {
let Ok(content) = tokio::fs::read_to_string(&file).await else {
continue;
};
let full_path = normalize_existing(&file).await;
rules.push(parse_rule(scope_dir.to_path_buf(), full_path, &content));
}
}
rules
}
fn is_rule_file(path: &Path) -> bool {
path.extension()
.and_then(|ext| ext.to_str())
.is_some_and(|ext| ext.eq_ignore_ascii_case("mdc") || ext.eq_ignore_ascii_case("md"))
}
fn parse_rule(scope_dir: PathBuf, full_path: PathBuf, content: &str) -> ParsedCursorRule {
let (frontmatter, body) = split_frontmatter(content)
.map(|(frontmatter, body)| (Some(frontmatter), body.to_owned()))
.unwrap_or((None, content.to_owned()));
let parsed = frontmatter.and_then(parse_frontmatter);
let kind = match parsed {
Some(fm) if fm.always_apply => CursorRuleKind::Global,
Some(fm) => {
let globs = fm.globs.map(GlobField::into_patterns).unwrap_or_default();
if !globs.is_empty() {
CursorRuleKind::FileGlobbed(globs)
} else if fm
.description
.is_some_and(|description| !description.trim().is_empty())
{
CursorRuleKind::AgentFetched
} else {
CursorRuleKind::Manual
}
}
None => CursorRuleKind::Manual,
};
ParsedCursorRule {
full_path,
scope_dir,
body: body.trim_start().to_owned(),
kind,
}
}
fn parse_frontmatter(yaml: &str) -> Option<CursorRuleFrontmatter> {
if let Ok(parsed) = serde_yaml::from_str::<CursorRuleFrontmatter>(yaml) {
return Some(parsed);
}
let mut parsed = CursorRuleFrontmatter::default();
let mut saw_field = false;
for line in yaml.lines() {
let Some((key, value)) = line.split_once(':') else {
continue;
};
let key = key.trim();
let value = value.trim().trim_matches('"').trim_matches('\'');
match key {
"alwaysApply" => {
parsed.always_apply = value.eq_ignore_ascii_case("true");
saw_field = true;
}
"globs" if !value.is_empty() => {
parsed.globs = Some(GlobField::String(value.to_owned()));
saw_field = true;
}
"description" if !value.is_empty() => {
parsed.description = Some(value.to_owned());
saw_field = true;
}
_ => {}
}
}
saw_field.then_some(parsed)
}
fn split_frontmatter(content: &str) -> Option<(&str, &str)> {
let rest = content
.strip_prefix("---\n")
.or_else(|| content.strip_prefix("---\r\n"))?;
for delimiter in ["\n---\n", "\n---\r\n", "\r\n---\r\n", "\r\n---\n"] {
if let Some(index) = rest.find(delimiter) {
let body_start = index + delimiter.len();
return Some((&rest[..index], &rest[body_start..]));
}
}
if rest.trim_end() == "---" {
return Some(("", ""));
}
None
}
fn rule_matches_read_path(
rule: &ParsedCursorRule,
workspace_root: &Path,
read_path: &Path,
) -> bool {
match &rule.kind {
CursorRuleKind::Global => {
rule.scope_dir != workspace_root && read_path.starts_with(&rule.scope_dir)
}
CursorRuleKind::FileGlobbed(globs) => file_globs_match(&rule.scope_dir, read_path, globs),
CursorRuleKind::AgentFetched | CursorRuleKind::Manual => false,
}
}
fn file_globs_match(scope_dir: &Path, read_path: &Path, globs: &[String]) -> bool {
let absolute_read_path = path_to_unix(read_path);
let relative_candidate = read_path
.strip_prefix(scope_dir)
.ok()
.map(path_to_unix)
.unwrap_or_default();
globs.iter().any(|glob| {
let normalized = glob.replace('\\', "/");
if Path::new(&normalized).is_absolute() {
return glob_matches(&normalized, &absolute_read_path);
}
glob_matches(&normalized, &relative_candidate)
|| glob_matches(&normalize_relative_glob(&normalized), &relative_candidate)
})
}
fn normalize_relative_glob(glob: &str) -> String {
if !glob.contains('/') || glob.ends_with('/') {
format!("**/{glob}")
} else {
glob.to_owned()
}
}
fn glob_matches(pattern: &str, candidate: &str) -> bool {
GlobBuilder::new(pattern)
.literal_separator(true)
.backslash_escape(false)
.build()
.is_ok_and(|glob| glob.compile_matcher().is_match(candidate))
}
fn path_to_unix(path: impl AsRef<Path>) -> String {
path.as_ref().to_string_lossy().replace('\\', "/")
}
fn render_cursor_rule_reminder(rules: &[ParsedCursorRule]) -> Option<String> {
if rules.is_empty() {
return None;
}
let mut lines =
vec!["The following rule files are relevant to the files you just read:".to_owned()];
for rule in rules {
let body = if rule.body.trim().is_empty() {
"(Rule file is empty.)"
} else {
rule.body.trim_end()
};
lines.push(format!("- {}\n{body}", rule.full_path.display()));
}
lines.push("Consider these rules if they affect your changes.".to_owned());
Some(lines.join("\n\n"))
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use tempfile::TempDir;
use super::*;
use crate::computer::local::LocalFs;
use crate::implementations::grok_build::read_file::{
ReadFileInput, ReadFileParams, ReadFileTool as GrokReadFileTool,
};
use crate::notification::types::ToolNotificationHandle;
use crate::types::output::ReadFileOutput;
use crate::types::resources::NotificationHandle;
use crate::types::resources::{Cwd, FileSystem, Params, Resources, State};
use crate::types::template_renderer::TemplateRenderer;
use crate::types::tool::ToolKind;
use crate::types::tool_metadata::test_ctx;
fn resources(root: &Path) -> SharedResources {
let mut resources = Resources::new();
resources.insert(Cwd(root.to_path_buf()));
resources.insert(FileSystem(Arc::new(LocalFs)));
resources.insert(NotificationHandle(ToolNotificationHandle::noop()));
resources.insert(TemplateRenderer::new(
[(ToolKind::Search, "grep".to_owned())]
.into_iter()
.collect(),
Default::default(),
));
resources.into_shared()
}
fn resources_with_grok_rules_on_read(root: &Path) -> SharedResources {
let mut resources = Resources::new();
resources.insert(Cwd(root.to_path_buf()));
resources.insert(FileSystem(Arc::new(LocalFs)));
resources.insert(NotificationHandle(ToolNotificationHandle::noop()));
resources.insert(TemplateRenderer::new(
[(ToolKind::Search, "grep".to_owned())]
.into_iter()
.collect(),
Default::default(),
));
resources.insert(Params(ReadFileParams {
cursor_rules_on_read: true,
}));
resources.into_shared()
}
#[tokio::test]
async fn file_globbed_rule_matches_read_path_once() {
let temp = TempDir::new().unwrap();
let root = temp.path();
let rules_dir = root.join(".cursor/rules");
std::fs::create_dir_all(&rules_dir).unwrap();
std::fs::write(
rules_dir.join("rust.mdc"),
"---\nglobs: *.rs\nalwaysApply: false\n---\nUse Rust rules.",
)
.unwrap();
std::fs::write(root.join("main.rs"), "fn main() {}\n").unwrap();
let shared = resources(root);
let first =
cursor_rule_reminder_for_read_inner(shared.clone(), root, &root.join("main.rs")).await;
assert!(
first
.as_deref()
.is_some_and(|text| text.contains("Use Rust rules."))
);
let second = cursor_rule_reminder_for_read_inner(shared, root, &root.join("main.rs")).await;
assert!(second.is_none(), "rule reminders are deduped per session");
}
#[tokio::test]
async fn seeded_injected_rule_paths_suppress_prefix_duplicates() {
let temp = TempDir::new().unwrap();
let root = temp.path();
let rule_path = root.join(".cursor/rules/rust.mdc");
std::fs::create_dir_all(rule_path.parent().unwrap()).unwrap();
std::fs::write(
&rule_path,
"---\nglobs: *.rs\nalwaysApply: false\n---\nUse Rust rules.",
)
.unwrap();
std::fs::write(root.join("main.rs"), "fn main() {}\n").unwrap();
let shared = resources(root);
{
let mut res = shared.lock().await;
let tracker = &mut res.get_or_default::<State<CursorRulesOnReadTracker>>().0;
tracker
.injected_rule_paths
.insert(dunce::canonicalize(&rule_path).unwrap());
}
let reminder =
cursor_rule_reminder_for_read_inner(shared, root, &root.join("main.rs")).await;
assert!(
reminder.is_none(),
"rules injected in the prefix should not be re-emitted"
);
}
#[tokio::test]
async fn non_canonical_seeded_injected_rule_paths_suppress_prefix_duplicates() {
let temp = TempDir::new().unwrap();
let root = temp.path();
let rule_path = root.join(".cursor/rules/rust.mdc");
std::fs::create_dir_all(rule_path.parent().unwrap()).unwrap();
std::fs::create_dir_all(root.join("subdir")).unwrap();
std::fs::write(
&rule_path,
"---\nglobs: *.rs\nalwaysApply: false\n---\nUse Rust rules.",
)
.unwrap();
std::fs::write(root.join("main.rs"), "fn main() {}\n").unwrap();
let shared = resources(root);
{
let mut res = shared.lock().await;
let tracker = &mut res.get_or_default::<State<CursorRulesOnReadTracker>>().0;
tracker
.injected_rule_paths
.insert(root.join("subdir/../.cursor/rules/rust.mdc"));
}
let reminder =
cursor_rule_reminder_for_read_inner(shared, root, &root.join("main.rs")).await;
assert!(
reminder.is_none(),
"non-canonical prefix paths should still dedupe against scanned rule paths"
);
}
#[tokio::test]
async fn relative_seeded_injected_rule_paths_resolve_against_workspace_root() {
let temp = TempDir::new().unwrap();
let root = temp.path();
let rule_path = root.join(".cursor/rules/rust.mdc");
std::fs::create_dir_all(rule_path.parent().unwrap()).unwrap();
std::fs::write(
&rule_path,
"---\nglobs: *.rs\nalwaysApply: false\n---\nUse Rust rules.",
)
.unwrap();
std::fs::write(root.join("main.rs"), "fn main() {}\n").unwrap();
let shared = resources(root);
{
let mut res = shared.lock().await;
let tracker = &mut res.get_or_default::<State<CursorRulesOnReadTracker>>().0;
tracker
.injected_rule_paths
.insert(PathBuf::from(".cursor/rules/rust.mdc"));
}
let reminder =
cursor_rule_reminder_for_read_inner(shared, root, &root.join("main.rs")).await;
assert!(
reminder.is_none(),
"relative prefix paths should resolve from the workspace root before dedupe"
);
}
#[tokio::test]
async fn root_always_apply_rule_is_not_read_scoped() {
let temp = TempDir::new().unwrap();
let root = temp.path();
let rules_dir = root.join(".cursor/rules");
std::fs::create_dir_all(&rules_dir).unwrap();
std::fs::write(
rules_dir.join("global.mdc"),
"---\nalwaysApply: true\n---\nGlobal rule.",
)
.unwrap();
std::fs::write(root.join("main.ts"), "const x = 1;\n").unwrap();
let reminder =
cursor_rule_reminder_for_read_inner(resources(root), root, &root.join("main.ts")).await;
assert!(reminder.is_none());
}
#[tokio::test]
async fn nested_always_apply_rule_is_read_scoped() {
let temp = TempDir::new().unwrap();
let root = temp.path();
let frontend = root.join("frontend");
let rules_dir = frontend.join(".cursor/rules");
std::fs::create_dir_all(&rules_dir).unwrap();
std::fs::write(
rules_dir.join("frontend.mdc"),
"---\nalwaysApply: true\n---\nFrontend rule.",
)
.unwrap();
std::fs::create_dir_all(frontend.join("src")).unwrap();
std::fs::write(
frontend.join("src/App.tsx"),
"export const App = () => null;\n",
)
.unwrap();
std::fs::write(root.join("server.ts"), "export const server = true;\n").unwrap();
let shared = resources(root);
let outside =
cursor_rule_reminder_for_read_inner(shared.clone(), root, &root.join("server.ts"))
.await;
assert!(outside.is_none());
let inside =
cursor_rule_reminder_for_read_inner(shared, root, &frontend.join("src/App.tsx")).await;
assert!(
inside
.as_deref()
.is_some_and(|text| text.contains("Frontend rule."))
);
}
#[tokio::test]
async fn duplicate_scans_do_not_duplicate_rule_cache() {
let temp = TempDir::new().unwrap();
let root = temp.path();
let rules_dir = root.join(".cursor/rules");
std::fs::create_dir_all(&rules_dir).unwrap();
std::fs::write(
rules_dir.join("rust.mdc"),
"---\nglobs: *.rs\nalwaysApply: false\n---\nUse Rust rules.",
)
.unwrap();
let first = scan_scope_dir(root).await;
let second = scan_scope_dir(root).await;
let mut rules = Vec::new();
append_new_rules(&mut rules, first);
append_new_rules(&mut rules, second);
assert_eq!(rules.len(), 1, "repeated scans should dedupe by rule path");
assert!(rules[0].body.contains("Use Rust rules."));
}
#[tokio::test]
async fn grok_read_file_output_includes_matching_rule_once() {
let temp = TempDir::new().unwrap();
let root = temp.path();
std::fs::create_dir_all(root.join(".cursor/rules")).unwrap();
std::fs::write(
root.join(".cursor/rules/rust.mdc"),
"---\nglobs: *.rs\nalwaysApply: false\n---\nUse Rust rules.",
)
.unwrap();
std::fs::write(root.join("main.rs"), "fn main() {}\n").unwrap();
let shared = resources_with_grok_rules_on_read(root);
let input = ReadFileInput {
path: "main.rs".to_owned(),
offset: None,
limit: None,
pages: None,
format: None,
};
let first = kigi_tool_runtime::Tool::run(&GrokReadFileTool, test_ctx(shared), input)
.await
.unwrap();
let ReadFileOutput::FileContent(first) = first else {
panic!("expected file content");
};
assert!(
first
.content
.contains("The following rule files are relevant to the files you just read:")
);
assert!(first.content.contains("Use Rust rules."));
assert!(!first.content.contains("cursor rule files"));
assert!(
first
.content_concise
.as_deref()
.is_some_and(|content| content.contains("Use Rust rules."))
);
}
#[tokio::test]
async fn parallel_reads_do_not_lose_matching_rule() {
let temp = TempDir::new().unwrap();
let root = temp.path();
std::fs::create_dir_all(root.join(".cursor/rules")).unwrap();
std::fs::write(
root.join(".cursor/rules/rust.mdc"),
"---\nglobs: *.rs\nalwaysApply: false\n---\nUse Rust rules.",
)
.unwrap();
std::fs::write(root.join("main.rs"), "fn main() {}\n").unwrap();
std::fs::write(root.join("README.md"), "# docs\n").unwrap();
let shared = resources(root);
let matching_path = root.join("main.rs");
let non_matching_path = root.join("README.md");
let matching = cursor_rule_reminder_for_read_inner(shared.clone(), root, &matching_path);
let non_matching = cursor_rule_reminder_for_read_inner(shared, root, &non_matching_path);
let (matching, non_matching) = tokio::join!(matching, non_matching);
assert!(
matching
.as_deref()
.is_some_and(|text| text.contains("Use Rust rules.")),
"matching parallel read should receive the rule"
);
assert!(
non_matching.is_none(),
"non-matching parallel read should not receive the rule"
);
}
#[test]
fn reminder_header_omits_cursor_product_name() {
let reminder = render_cursor_rule_reminder(&[ParsedCursorRule {
full_path: PathBuf::from("/repo/.cursor/rules/example.mdc"),
scope_dir: PathBuf::from("/repo"),
body: "Rule body.".to_owned(),
kind: CursorRuleKind::FileGlobbed(vec!["*.rs".to_owned()]),
}])
.unwrap();
assert!(
reminder
.starts_with("The following rule files are relevant to the files you just read:")
);
assert!(!reminder.contains("cursor rule files"));
}
}
@@ -0,0 +1,270 @@
//! File operation lock manager — serializes concurrent file operations.
//!
//! diagnostics for each file. Multiple reads for *different* paths can proceed
//! concurrently; reads for the *same* path are serialized.
//! - **Exclusive lock** (`wait_for_exclusive_lock`): used by `Write` and
//! `StrReplace` before mutating files. Blocks all per-path locks and
//! vice-versa.
//!
//! The queue is FIFO with priority inversion avoidance: per-path waiters
//! will not jump ahead of a queued exclusive waiter, preventing writer
//! starvation.
use std::collections::{HashSet, VecDeque};
use std::sync::Arc;
use tokio::sync::{Mutex, oneshot};
/// Shared file operation lock manager stored in tool shared resources.
#[derive(Clone)]
pub struct FileOperationLockManager {
inner: Arc<Mutex<LockInner>>,
}
struct LockInner {
locked_files: HashSet<String>,
exclusive_lock_active: bool,
wait_queue: VecDeque<QueuedWaiter>,
}
enum QueuedWaiter {
File {
path: String,
tx: oneshot::Sender<()>,
},
Exclusive {
tx: oneshot::Sender<()>,
},
}
impl FileOperationLockManager {
pub fn new() -> Self {
Self {
inner: Arc::new(Mutex::new(LockInner {
locked_files: HashSet::new(),
exclusive_lock_active: false,
wait_queue: VecDeque::new(),
})),
}
}
/// Acquire a per-path lock. Blocks if:
/// - An exclusive lock is active, OR
/// - The same path is already locked, OR
/// - An exclusive waiter is ahead in the queue.
///
/// Returns a guard that releases the lock on drop.
pub async fn wait_for_lock(&self, path: &str) -> FileOperationLockGuard {
let rx = {
let mut inner = self.inner.lock().await;
let needs_wait = inner.exclusive_lock_active
|| inner.locked_files.contains(path)
|| inner.has_exclusive_waiter_ahead();
if needs_wait {
let (tx, rx) = oneshot::channel();
inner.wait_queue.push_back(QueuedWaiter::File {
path: path.to_string(),
tx,
});
Some(rx)
} else {
inner.locked_files.insert(path.to_string());
None
}
};
if let Some(rx) = rx {
// Wait for our turn (ignore error — sender dropped means lock manager was dropped).
let _ = rx.await;
}
FileOperationLockGuard {
manager: self.clone(),
kind: LockKind::File(path.to_string()),
}
}
/// Acquire an exclusive lock. Blocks until all per-path locks are released
/// and no other exclusive lock is active.
///
/// Returns a guard that releases the lock on drop.
pub async fn wait_for_exclusive_lock(&self) -> FileOperationLockGuard {
let rx = {
let mut inner = self.inner.lock().await;
let needs_wait = inner.exclusive_lock_active || !inner.locked_files.is_empty();
if needs_wait {
let (tx, rx) = oneshot::channel();
inner.wait_queue.push_back(QueuedWaiter::Exclusive { tx });
Some(rx)
} else {
inner.exclusive_lock_active = true;
None
}
};
if let Some(rx) = rx {
let _ = rx.await;
}
FileOperationLockGuard {
manager: self.clone(),
kind: LockKind::Exclusive,
}
}
}
impl Default for FileOperationLockManager {
fn default() -> Self {
Self::new()
}
}
enum LockKind {
File(String),
Exclusive,
}
/// RAII guard that releases the lock when dropped.
pub struct FileOperationLockGuard {
manager: FileOperationLockManager,
kind: LockKind,
}
impl Drop for FileOperationLockGuard {
fn drop(&mut self) {
let manager = self.manager.clone();
let kind = std::mem::replace(&mut self.kind, LockKind::Exclusive);
// Use `spawn` to release asynchronously — `drop` can't be async.
tokio::spawn(async move {
let mut inner = manager.inner.lock().await;
match kind {
LockKind::File(path) => {
inner.locked_files.remove(&path);
}
LockKind::Exclusive => {
inner.exclusive_lock_active = false;
}
}
inner.process_queue();
});
}
}
impl LockInner {
fn has_exclusive_waiter_ahead(&self) -> bool {
self.wait_queue
.iter()
.any(|w| matches!(w, QueuedWaiter::Exclusive { .. }))
}
/// Process the wait queue, granting locks to eligible waiters.
///
/// If a waiter's receiver has been dropped (task cancelled), the send
/// will fail. In that case, we undo the lock grant and continue to the
/// next waiter. This prevents phantom locks from cancelled tool calls.
fn process_queue(&mut self) {
while let Some(front) = self.wait_queue.front() {
match front {
QueuedWaiter::Exclusive { .. } => {
if !self.locked_files.is_empty() || self.exclusive_lock_active {
break;
}
if let Some(QueuedWaiter::Exclusive { tx }) = self.wait_queue.pop_front() {
self.exclusive_lock_active = true;
if tx.send(()).is_err() {
// Receiver dropped (cancelled) — undo the grant.
self.exclusive_lock_active = false;
continue;
}
}
// Exclusive lock granted — stop processing.
break;
}
QueuedWaiter::File { path, .. } => {
if self.exclusive_lock_active || self.locked_files.contains(path) {
break;
}
let path = path.clone();
if let Some(QueuedWaiter::File { tx, .. }) = self.wait_queue.pop_front() {
self.locked_files.insert(path.clone());
if tx.send(()).is_err() {
// Receiver dropped (cancelled) — undo the grant.
self.locked_files.remove(&path);
continue;
}
}
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn serializes_same_path() {
let mgr = FileOperationLockManager::new();
let order = Arc::new(Mutex::new(Vec::new()));
let guard1 = mgr.wait_for_lock("a.ts").await;
let order2 = order.clone();
let mgr2 = mgr.clone();
let handle = tokio::spawn(async move {
order2.lock().await.push("2-waiting");
let _guard2 = mgr2.wait_for_lock("a.ts").await;
order2.lock().await.push("2-acquired");
});
// Give spawned task time to queue.
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
order.lock().await.push("1-releasing");
drop(guard1);
handle.await.unwrap();
let log = order.lock().await;
assert_eq!(log.as_slice(), &["2-waiting", "1-releasing", "2-acquired"]);
}
#[tokio::test]
async fn allows_different_paths_concurrently() {
let mgr = FileOperationLockManager::new();
let _guard_a = mgr.wait_for_lock("a.ts").await;
// Different path should acquire immediately.
let mgr2 = mgr.clone();
let handle = tokio::spawn(async move {
let _guard_b = mgr2.wait_for_lock("b.ts").await;
true
});
let result = tokio::time::timeout(std::time::Duration::from_millis(100), handle)
.await
.expect("should not timeout")
.expect("should not panic");
assert!(result);
}
#[tokio::test]
async fn exclusive_blocks_file_locks() {
let mgr = FileOperationLockManager::new();
let exclusive = mgr.wait_for_exclusive_lock().await;
let acquired = Arc::new(Mutex::new(false));
let mgr2 = mgr.clone();
let acquired2 = acquired.clone();
let handle = tokio::spawn(async move {
let _guard = mgr2.wait_for_lock("a.ts").await;
*acquired2.lock().await = true;
});
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
assert!(!*acquired.lock().await);
drop(exclusive);
handle.await.unwrap();
assert!(*acquired.lock().await);
}
}
@@ -0,0 +1,6 @@
//! Shared editor infrastructure used by the default grok_build tools.
pub mod file_operation_lock;
pub use file_operation_lock::{FileOperationLockGuard, FileOperationLockManager};
/// Default `block_until_ms` when the model omits the field on blocking shell
/// tools. Shared so ACP UI labels and tool implementations stay aligned.
pub const DEFAULT_BLOCK_UNTIL_MS: u64 = 30_000;
@@ -0,0 +1,711 @@
//! Formatting functions for AskUserQuestion tool results.
//!
//! Each function produces the **exact** model-visible string for one of the
//! four user-action paths.
//!
//! The tests below pin the exact output strings and serve as the
//! source-of-truth specification.
use std::collections::HashMap;
use indexmap::IndexMap;
use super::Question;
use super::types::QuestionAnnotation;
// ── Path D: Cancel ──────────────────────────────────────────────────────
/// Tool result text when the user cancels / dismisses the question UI.
///
/// Cancel is a normal user decision, not a tool failure, so this is a
/// purpose-built message rather than a generic permission-denial string.
pub const CANCEL_TEXT: &str = "User declined to answer the questions. Continue with the task using your best judgment, or ask different questions.";
// ── Path A: Accepted ────────────────────────────────────────────────────
/// Format the tool result for Path A (user accepted and submitted answers).
///
/// Produces the accepted-answers tool result:
///
/// ```text
/// User has answered your questions: "<q>"="<label>" ..., "<q>"="<label>" .... You can now continue with the user's answers in mind.
/// ```
///
/// Rules:
/// - Only answered questions appear (unanswered are omitted by the caller).
/// - Multi-select: each selected label is its own `Vec` element on the
/// wire; this function joins them with `, ` at format time.
/// - Freeform-only: a single-element vec containing `"Other"`, free text
/// in `annotations[q].notes`.
/// - Preview is appended only when present in annotations.
/// - Notes are appended only when present in annotations.
/// - Questions/labels are interpolated raw (no escaping).
pub fn format_accepted_tool_result(
answers: &IndexMap<String, Vec<String>>,
annotations: &Option<HashMap<String, QuestionAnnotation>>,
) -> String {
let entries: Vec<String> = answers
.iter()
.map(|(question_text, selected_labels)| {
let selected_label = selected_labels.join(", ");
let mut parts = vec![format!("\"{}\"=\"{}\"", question_text, selected_label)];
if let Some(anns) = annotations
&& let Some(ann) = anns.get(question_text)
{
if let Some(ref preview) = ann.preview {
parts.push(format!("selected preview:\n{}", preview));
}
if let Some(ref notes) = ann.notes {
parts.push(format!("user notes: {}", notes));
}
}
parts.join(" ")
})
.collect();
format!(
"User has answered your questions: {}. You can now continue with the user's answers in mind.",
entries.join(", ")
)
}
// ── Alternate id-keyed tool-result formatting ────
/// Format the tool result in the alternate id-keyed shape (Path A).
///
/// Answers are keyed by **id**, one question per line, with no trailing
/// sentence:
///
/// ```text
/// User questions responses:
/// Question <qid>: Selected option(s) <oid>(, <oid>)*
/// Question <qid>: Selected option(s) <oid>(, <oid>)*
/// ```
///
/// Examples:
///
/// - Single question, single-select:
/// `User questions responses:\nQuestion demo_pick: Selected option(s) a`
/// - Three questions, last with `allow_multiple: true` (one selection):
/// `User questions responses:\nQuestion q1: Selected option(s) tea\nQuestion q2: Selected option(s) code\nQuestion q3: Selected option(s) tests`
///
/// Multi-select labels arrive as separate `Vec` elements; this function
/// joins their resolved ids with `, ` (`Selected option(s) a, b, c`).
/// The multi-select join shape is exercised by the test below.
///
/// `input_questions` carries both `id` and the option `label`/`id` map
/// so we can resolve the answer values (which arrive label-keyed from
/// the client) back to the option ids.
///
/// `annotations` carries per-question freeform notes (the text the user
/// typed when picking the freeform "Other" path or dismissing). When no
/// option labels resolve to ids and `notes` is non-empty, the result is
/// `Question <qid>: <raw_text>` (no `Selected option(s)` prefix).
///
/// Question order follows `input_questions`. Unanswered questions are
/// omitted -- only answered questions appear in the result.
pub fn format_id_keyed_accepted_tool_result(
input_questions: &[super::Question],
answers: &IndexMap<String, Vec<String>>,
annotations: &Option<HashMap<String, QuestionAnnotation>>,
) -> String {
let lines: Vec<String> = input_questions
.iter()
.filter_map(|q| {
let qid = q.id.as_ref()?;
let labels = answers.get(&q.question)?;
// Each selected label is its own `Vec` element (the wire
// format no longer joins labels with `", "`), so we look each
// one up directly. No splitting, no ambiguity around labels
// that contain commas or share substrings with other labels.
let oids: Vec<&str> = labels
.iter()
.filter_map(|label| {
q.options
.iter()
.find(|o| &o.label == label)
.and_then(|o| o.id.as_deref())
})
.collect();
if oids.is_empty() {
// Freeform / dismissed: emit the raw text from the freeform
// input directly after `Question <qid>: `.
let notes = annotations
.as_ref()
.and_then(|m| m.get(&q.question))
.and_then(|a| a.notes.as_deref())
.map(|s| s.trim())
.filter(|s| !s.is_empty())?;
return Some(format!("Question {qid}: {notes}"));
}
Some(format!(
"Question {qid}: Selected option(s) {}",
oids.join(", ")
))
})
.collect();
if lines.is_empty() {
return "User questions responses:".to_string();
}
format!(
"User questions responses:
{}",
lines.join(
"
"
)
)
}
// ── Path B: Chat about this (plan mode) ─────────────────────────────────
/// Format the tool result for Path B ("Chat about this" / respond-to-agent).
///
/// Iterates ALL original questions. Answered questions show their label;
/// unanswered questions show "(No answer provided)".
///
/// Whitespace is intentional:
/// - Lines 2-4 and "Questions asked:" have 4-space indentation.
/// - Question bullets have no indentation.
/// - Answer lines have 2-space indentation.
pub fn format_chat_about_this(
questions: &[Question],
partial_answers: &HashMap<String, String>,
) -> String {
let question_lines: Vec<String> = questions
.iter()
.map(|q| {
if let Some(answer) = partial_answers.get(&q.question) {
format!("- \"{}\"\n Answer: {}", q.question, answer)
} else {
format!("- \"{}\"\n (No answer provided)", q.question)
}
})
.collect();
format!(
"The user wants to clarify these questions.\n\
\x20\x20\x20\x20This means they may have additional information, context or questions for you.\n\
\x20\x20\x20\x20Take their response into account and then reformulate the questions if appropriate.\n\
\x20\x20\x20\x20Start by asking them what they would like to clarify.\n\
\n\
\x20\x20\x20\x20Questions asked:\n\
{}",
question_lines.join("\n")
)
}
// ── Path C: Skip interview (plan mode) ──────────────────────────────────
/// Format the tool result for Path C ("Skip interview and plan immediately").
///
/// Same per-question format as Path B, but different header and NO indentation.
pub fn format_skip_interview(
questions: &[Question],
partial_answers: &HashMap<String, String>,
) -> String {
let question_lines: Vec<String> = questions
.iter()
.map(|q| {
if let Some(answer) = partial_answers.get(&q.question) {
format!("- \"{}\"\n Answer: {}", q.question, answer)
} else {
format!("- \"{}\"\n (No answer provided)", q.question)
}
})
.collect();
format!(
"The user has indicated they have provided enough answers for the plan interview.\n\
Stop asking clarifying questions and proceed to finish the plan with the information you have.\n\
\n\
Questions asked and answers provided:\n\
{}",
question_lines.join("\n")
)
}
// ── Tests ────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::super::QuestionOption;
use super::*;
// -- Helpers --
fn make_question(text: &str, labels: &[&str]) -> Question {
Question {
question: text.to_string(),
options: labels
.iter()
.map(|l| QuestionOption {
label: l.to_string(),
description: format!("Description for {l}"),
preview: None,
id: None,
})
.collect(),
multi_select: None,
id: None,
}
}
// ── Path A: format_accepted_tool_result ──────────────────────────────
#[test]
fn format_accepted_single_no_annotations() {
let mut answers = IndexMap::new();
answers.insert(
"Which database?".to_string(),
vec!["Redis (Recommended)".to_string()],
);
let result = format_accepted_tool_result(&answers, &None);
assert_eq!(
result,
"User has answered your questions: \"Which database?\"=\"Redis (Recommended)\". You can now continue with the user's answers in mind."
);
}
#[test]
fn format_accepted_multiple_with_annotations() {
let mut answers = IndexMap::new();
answers.insert("Which database?".to_string(), vec!["Redis".to_string()]);
answers.insert("Which framework?".to_string(), vec!["React".to_string()]);
let mut anns = HashMap::new();
anns.insert(
"Which database?".to_string(),
QuestionAnnotation {
preview: Some("<div>redis preview</div>".to_string()),
notes: None,
},
);
anns.insert(
"Which framework?".to_string(),
QuestionAnnotation {
preview: None,
notes: Some("I prefer React hooks".to_string()),
},
);
let result = format_accepted_tool_result(&answers, &Some(anns));
assert_eq!(
result,
"User has answered your questions: \"Which database?\"=\"Redis\" selected preview:\n<div>redis preview</div>, \"Which framework?\"=\"React\" user notes: I prefer React hooks. You can now continue with the user's answers in mind."
);
}
#[test]
fn format_accepted_multi_select() {
let mut answers = IndexMap::new();
answers.insert(
"Which features?".to_string(),
vec!["Auth".to_string(), "Logging".to_string()],
);
let result = format_accepted_tool_result(&answers, &None);
assert_eq!(
result,
"User has answered your questions: \"Which features?\"=\"Auth, Logging\". You can now continue with the user's answers in mind."
);
}
#[test]
fn format_accepted_freeform_only() {
// Freeform-only: label is "Other", typed text in annotations.notes
let mut answers = IndexMap::new();
answers.insert("Which database?".to_string(), vec!["Other".to_string()]);
let mut anns = HashMap::new();
anns.insert(
"Which database?".to_string(),
QuestionAnnotation {
preview: None,
notes: Some("I want to use DynamoDB".to_string()),
},
);
let result = format_accepted_tool_result(&answers, &Some(anns));
assert_eq!(
result,
"User has answered your questions: \"Which database?\"=\"Other\" user notes: I want to use DynamoDB. You can now continue with the user's answers in mind."
);
}
#[test]
fn format_accepted_preview_and_notes() {
let mut answers = IndexMap::new();
answers.insert("Which layout?".to_string(), vec!["Grid".to_string()]);
let mut anns = HashMap::new();
anns.insert(
"Which layout?".to_string(),
QuestionAnnotation {
preview: Some("<div class=\"grid\">...</div>".to_string()),
notes: Some("Use CSS Grid for the main layout".to_string()),
},
);
let result = format_accepted_tool_result(&answers, &Some(anns));
assert_eq!(
result,
"User has answered your questions: \"Which layout?\"=\"Grid\" selected preview:\n<div class=\"grid\">...</div> user notes: Use CSS Grid for the main layout. You can now continue with the user's answers in mind."
);
}
#[test]
fn format_accepted_empty() {
let answers = IndexMap::new();
let result = format_accepted_tool_result(&answers, &None);
assert_eq!(
result,
"User has answered your questions: . You can now continue with the user's answers in mind."
);
}
#[test]
fn format_accepted_partial() {
// Only answered questions appear. Unanswered questions are omitted by the caller
// (the answers IndexMap simply doesn't contain them).
let mut answers = IndexMap::new();
answers.insert("Which database?".to_string(), vec!["Redis".to_string()]);
// "Which framework?" is unanswered => not in the map
let result = format_accepted_tool_result(&answers, &None);
assert_eq!(
result,
"User has answered your questions: \"Which database?\"=\"Redis\". You can now continue with the user's answers in mind."
);
}
// ── Alternate id-keyed formatter tests ────────────────────
//
// Pin both result strings (single question and three questions) so any
// drift in the formatter trips a deterministic failure. Update the
// literal strings deliberately if the wire format ever changes.
fn id_keyed_q(qid: &str, prompt: &str, opts: &[(&str, &str)]) -> super::super::Question {
super::super::Question {
question: prompt.to_string(),
options: opts
.iter()
.map(|(oid, label)| super::super::QuestionOption {
label: (*label).to_string(),
description: (*label).to_string(),
preview: None,
id: Some((*oid).to_string()),
})
.collect(),
multi_select: None,
id: Some(qid.to_string()),
}
}
#[test]
fn format_id_keyed_single_question_single_select_matches_capture() {
let questions = vec![id_keyed_q(
"demo_pick",
"This is a demo of the question tool. Which outcome should we pick?",
&[
("a", "Option A — fast path (minimal scope)"),
("b", "Option B — thorough path (extra validation)"),
("c", "Option C — I'll decide later"),
],
)];
let mut answers = IndexMap::new();
answers.insert(
"This is a demo of the question tool. Which outcome should we pick?".to_string(),
vec!["Option A — fast path (minimal scope)".to_string()],
);
let result = format_id_keyed_accepted_tool_result(&questions, &answers, &None);
assert_eq!(
result, "User questions responses:\nQuestion demo_pick: Selected option(s) a",
"must match the id-keyed shape"
);
}
#[test]
fn format_id_keyed_three_questions_matches_capture() {
let questions = vec![
id_keyed_q(
"q1",
"Question 1 of 3: Morning drink?",
&[("coffee", "Coffee"), ("tea", "Tea"), ("water", "Water")],
),
id_keyed_q(
"q2",
"Question 2 of 3: How do you usually start a new task?",
&[
("read", "Read docs first"),
("code", "Jump into code"),
("plan", "Sketch a plan first"),
],
),
id_keyed_q(
"q3",
"Question 3 of 3: What do you lean on before a push? (pick any)",
&[
("tests", "Tests"),
("types", "Types"),
("lint", "Lint/format"),
],
),
];
let mut answers = IndexMap::new();
answers.insert(
"Question 1 of 3: Morning drink?".to_string(),
vec!["Tea".to_string()],
);
answers.insert(
"Question 2 of 3: How do you usually start a new task?".to_string(),
vec!["Jump into code".to_string()],
);
answers.insert(
"Question 3 of 3: What do you lean on before a push? (pick any)".to_string(),
vec!["Tests".to_string()],
);
let result = format_id_keyed_accepted_tool_result(&questions, &answers, &None);
assert_eq!(
result,
"User questions responses:\nQuestion q1: Selected option(s) tea\nQuestion q2: Selected option(s) code\nQuestion q3: Selected option(s) tests",
"must match the id-keyed shape"
);
}
#[test]
fn format_id_keyed_multi_select_inferred_csv() {
// Multi-select with multiple selections joins option ids with ", "
// at format time. Each selected label arrives as its own Vec
// element (the wire format no longer joins them).
let questions = vec![id_keyed_q(
"q3",
"What do you lean on before a push? (pick any)",
&[
("tests", "Tests"),
("types", "Types"),
("lint", "Lint/format"),
],
)];
let mut answers = IndexMap::new();
answers.insert(
"What do you lean on before a push? (pick any)".to_string(),
vec![
"Tests".to_string(),
"Types".to_string(),
"Lint/format".to_string(),
],
);
let result = format_id_keyed_accepted_tool_result(&questions, &answers, &None);
assert_eq!(
result,
"User questions responses:\nQuestion q3: Selected option(s) tests, types, lint"
);
}
#[test]
fn format_id_keyed_unanswered_questions_are_omitted() {
let questions = vec![
id_keyed_q("q1", "First?", &[("a", "Apple"), ("b", "Banana")]),
id_keyed_q("q2", "Second?", &[("x", "Xenon"), ("y", "Yttrium")]),
];
let mut answers = IndexMap::new();
answers.insert("First?".to_string(), vec!["Apple".to_string()]);
let result = format_id_keyed_accepted_tool_result(&questions, &answers, &None);
assert_eq!(
result,
"User questions responses:\nQuestion q1: Selected option(s) a"
);
}
#[test]
fn format_id_keyed_no_answers_emits_header_only() {
let questions = vec![id_keyed_q("q1", "First?", &[("a", "Apple")])];
let answers = IndexMap::new();
let result = format_id_keyed_accepted_tool_result(&questions, &answers, &None);
assert_eq!(result, "User questions responses:");
}
/// Freeform/dismiss:
/// when the user dismisses or types freeform text instead of picking
/// an option, the wire format emits the raw text directly after
/// `Question <qid>: ` with NO `Selected option(s)` prefix. The pager
/// sends `answers["..."] = ["Other"]` plus the typed text in
/// `annotations[q].notes`; the formatter falls through to the notes.
#[test]
fn format_id_keyed_freeform_dismissal_uses_notes_without_selected_prefix() {
let questions = vec![id_keyed_q(
"random_one",
"Pick one",
&[
("ocean", "Ocean"),
("mountains", "Mountains"),
("city", "City"),
],
)];
let mut answers = IndexMap::new();
answers.insert("Pick one".to_string(), vec!["Other".to_string()]);
let mut anns = HashMap::new();
anns.insert(
"Pick one".to_string(),
QuestionAnnotation {
preview: None,
notes: Some("nvm".to_string()),
},
);
let result = format_id_keyed_accepted_tool_result(&questions, &answers, &Some(anns));
assert_eq!(
result, "User questions responses:\nQuestion random_one: nvm",
"must match the id-keyed shape"
);
}
/// Freeform with no notes (just `["Other"]` and no annotation) is
/// indistinguishable from a no-answer to the formatter, so the
/// question is dropped (matching the `oids.is_empty()` branch).
#[test]
fn format_id_keyed_freeform_without_notes_is_dropped() {
let questions = vec![id_keyed_q("q1", "Pick", &[("a", "A")])];
let mut answers = IndexMap::new();
answers.insert("Pick".to_string(), vec!["Other".to_string()]);
let result = format_id_keyed_accepted_tool_result(&questions, &answers, &None);
assert_eq!(result, "User questions responses:");
}
#[test]
fn format_accepted_special_chars() {
// Quotes and newlines in labels appear verbatim (no escaping)
let mut answers = IndexMap::new();
answers.insert(
"Which \"option\"?".to_string(),
vec!["Option with\nnewline".to_string()],
);
let result = format_accepted_tool_result(&answers, &None);
assert_eq!(
result,
"User has answered your questions: \"Which \"option\"?\"=\"Option with\nnewline\". You can now continue with the user's answers in mind."
);
}
// ── Path B: format_chat_about_this ───────────────────────────────────
#[test]
fn format_chat_about_this_mixed() {
let questions = vec![
make_question("Which database?", &["Redis", "Postgres"]),
make_question("Which framework?", &["React", "Vue"]),
];
let mut partial = HashMap::new();
partial.insert("Which database?".to_string(), "Redis".to_string());
let result = format_chat_about_this(&questions, &partial);
let expected = "\
The user wants to clarify these questions.
This means they may have additional information, context or questions for you.
Take their response into account and then reformulate the questions if appropriate.
Start by asking them what they would like to clarify.
Questions asked:
- \"Which database?\"
Answer: Redis
- \"Which framework?\"
(No answer provided)";
assert_eq!(result, expected);
}
#[test]
fn format_chat_about_this_all_answered() {
let questions = vec![make_question("Which database?", &["Redis"])];
let mut partial = HashMap::new();
partial.insert("Which database?".to_string(), "Redis".to_string());
let result = format_chat_about_this(&questions, &partial);
assert!(result.contains("Answer: Redis"));
assert!(!result.contains("(No answer provided)"));
}
#[test]
fn format_chat_about_this_none_answered() {
let questions = vec![make_question("Q1?", &["A"]), make_question("Q2?", &["B"])];
let result = format_chat_about_this(&questions, &HashMap::new());
assert!(result.contains("- \"Q1?\"\n (No answer provided)"));
assert!(result.contains("- \"Q2?\"\n (No answer provided)"));
}
// ── Path C: format_skip_interview ────────────────────────────────────
#[test]
fn format_skip_interview_all_answered() {
let questions = vec![
make_question("Which database?", &["Redis", "Postgres"]),
make_question("Which framework?", &["React", "Vue"]),
];
let mut partial = HashMap::new();
partial.insert("Which database?".to_string(), "Redis".to_string());
partial.insert("Which framework?".to_string(), "React".to_string());
let result = format_skip_interview(&questions, &partial);
let expected = "\
The user has indicated they have provided enough answers for the plan interview.
Stop asking clarifying questions and proceed to finish the plan with the information you have.
Questions asked and answers provided:
- \"Which database?\"
Answer: Redis
- \"Which framework?\"
Answer: React";
assert_eq!(result, expected);
}
#[test]
fn format_skip_interview_mixed() {
let questions = vec![
make_question("Which database?", &["Redis"]),
make_question("Which framework?", &["React"]),
];
let mut partial = HashMap::new();
partial.insert("Which database?".to_string(), "Redis".to_string());
let result = format_skip_interview(&questions, &partial);
assert!(result.contains("Answer: Redis"));
assert!(result.contains("- \"Which framework?\"\n (No answer provided)"));
}
#[test]
fn format_skip_interview_no_indentation() {
// Path C has NO indentation on any header lines (unlike Path B)
let questions = vec![make_question("Q?", &["A"])];
let result = format_skip_interview(&questions, &HashMap::new());
// First line has no leading spaces
let first_line = result.lines().next().unwrap();
assert!(!first_line.starts_with(' '));
// Second line has no leading spaces
let second_line = result.lines().nth(1).unwrap();
assert!(!second_line.starts_with(' '));
// "Questions asked" line has no leading spaces
assert!(result.contains("\nQuestions asked and answers provided:\n"));
}
// ── Path D: CANCEL_TEXT ─────────────────────────────────────────────
#[test]
fn format_cancel() {
assert_eq!(
CANCEL_TEXT,
"User declined to answer the questions. Continue with the task using your best judgment, or ask different questions."
);
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,684 @@
//! Shared protocol and channel types for the AskUserQuestion blocking flow.
//!
//! These types define the request/response contract between three crates:
//!
//! - **`kigi-tools`** — tool blocks on a oneshot, formats the result.
//! - **`kigi-shell`** — coordinator receives requests over mpsc, calls the
//! client via ACP `ext_method`, sends results back over the oneshot.
//! - **`kigi-tui`** — handles the `ExtMethod`, renders UI, returns a
//! typed response.
//!
//! All three crates import these types from `kigi-tools`.
use std::collections::HashMap;
use educe::Educe;
use indexmap::IndexMap;
use tokio::sync::{mpsc, oneshot};
use super::Question;
use crate::register_resource;
// ── ACP wire-format types ────────────────────────────────────────────────
/// Annotation on a single question's answer.
///
/// Carried inside the `accepted` response alongside the selected label.
/// - `preview`: verbatim `Option.preview` of the selected option (single-select only).
/// - `notes`: free-text the user typed in the freeform input.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct QuestionAnnotation {
#[serde(skip_serializing_if = "Option::is_none")]
pub preview: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub notes: Option<String>,
}
/// Mode context for the question UI.
///
/// Sent as part of the ACP `ext_method` request so the pager knows whether
/// to show plan-mode-only actions (Chat about this / Skip interview).
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum AskUserQuestionMode {
/// Normal mode. Client shows only Accept and Cancel.
Default,
/// Plan mode. Client shows Accept, Cancel, Chat about this, Skip interview.
Plan,
}
/// ACP `ext_method` request payload (shell coordinator sends to client/pager).
///
/// Serialized as `camelCase` for the ACP JSON-RPC wire format.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct AskUserQuestionExtRequest {
pub session_id: String,
pub tool_call_id: String,
pub questions: Vec<Question>,
/// Controls whether the client shows plan-mode-only actions.
pub mode: AskUserQuestionMode,
}
/// Accepts both `"value"` (old wire format) and `["value"]` (new wire format)
/// for each answer entry, normalizing strings into single-element vectors.
fn deserialize_string_or_vec_answers<'de, D>(
deserializer: D,
) -> Result<IndexMap<String, Vec<String>>, D::Error>
where
D: serde::Deserializer<'de>,
{
#[derive(serde::Deserialize)]
#[serde(untagged)]
enum StringOrVec {
Vec(Vec<String>),
String(String),
}
let raw: IndexMap<String, StringOrVec> = serde::Deserialize::deserialize(deserializer)?;
Ok(raw
.into_iter()
.map(|(k, v)| match v {
StringOrVec::Vec(vec) => (k, vec),
StringOrVec::String(s) => (k, vec![s]),
})
.collect())
}
/// ACP `ext_method` response payload (client/pager returns to shell coordinator).
///
/// Internally tagged on `"outcome"` with `snake_case` variant names so the
/// JSON looks like `{ "outcome": "accepted", "answers": { ... } }`.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
#[serde(tag = "outcome", rename_all = "snake_case")]
pub enum AskUserQuestionExtResponse {
/// User accepted and submitted answers (Path A).
Accepted {
/// Answered questions in original order; unanswered omitted.
/// One element per selected option; freeform-only is `["Other"]`
/// with typed text in `annotations[q].notes`.
#[serde(deserialize_with = "deserialize_string_or_vec_answers")]
answers: IndexMap<String, Vec<String>>,
/// Per-question annotations (preview, notes). Absent when empty.
#[serde(default, skip_serializing_if = "Option::is_none")]
annotations: Option<HashMap<String, QuestionAnnotation>>,
},
/// User chose "Chat about this" (Path B, plan mode only).
ChatAboutThis {
/// Partial answers: answered questions only, label only (no notes).
/// Freeform-only => `"Other"` (notes dropped in plan-mode paths).
#[serde(default)]
partial_answers: HashMap<String, String>,
},
/// User chose "Skip interview and plan immediately" (Path C, plan mode only).
SkipInterview {
/// Same partial-answer rules as `ChatAboutThis`.
#[serde(default)]
partial_answers: HashMap<String, String>,
},
/// User cancelled / dismissed (Path D). NOT an error.
Cancelled,
}
// ── In-process types (coordinator <-> tool) ──────────────────────────────
/// In-process result: coordinator -> tool.
///
/// Uses `Result` so the tool can distinguish user actions from infrastructure
/// failures:
/// - `Ok(UserQuestionResponse)` for all 4 user paths (accepted, chat, skip, cancel).
/// - `Err(UserQuestionError)` for transport failures or malformed responses.
pub type UserQuestionResult = Result<UserQuestionResponse, UserQuestionError>;
/// Successful user response (all 4 user paths).
///
/// Every variant here produces `Ok(UserAnswered { message })` at the tool
/// level with `ToolCall` status `Completed`.
#[derive(Debug, Clone)]
pub enum UserQuestionResponse {
/// User accepted and submitted answers (Path A).
Accepted {
/// See `AskUserQuestionExtResponse::Accepted::answers`.
answers: IndexMap<String, Vec<String>>,
annotations: Option<HashMap<String, QuestionAnnotation>>,
},
/// User chose "Chat about this" (Path B, plan mode only).
/// Carries the original questions so the formatter can iterate all of them.
ChatAboutThis {
questions: Vec<Question>,
partial_answers: HashMap<String, String>,
},
/// User chose "Skip interview" (Path C, plan mode only).
/// Carries the original questions so the formatter can iterate all of them.
SkipInterview {
questions: Vec<Question>,
partial_answers: HashMap<String, String>,
},
/// User explicitly dismissed (Esc). NOT an error.
Cancelled,
}
/// Infrastructure failure (NOT a user action).
///
/// These produce `Err(ToolError::ExecutionError { .. })` at the tool level
/// with `ToolCall` status `Failed`.
#[derive(Debug, Clone)]
pub enum UserQuestionError {
/// ACP `ext_method` call failed (client disconnect, timeout, etc.).
TransportError(String),
/// Client returned JSON that could not be deserialized into
/// `AskUserQuestionExtResponse`.
MalformedResponse(String),
}
/// In-process request: tool -> coordinator (carries oneshot for reply).
///
/// Sent over the `mpsc` channel. The coordinator receives this, performs the
/// ACP `ext_method` round-trip, and sends the result back on `result_tx`.
#[derive(Educe)]
#[educe(Debug)]
pub struct UserQuestionRequest {
pub tool_call_id: String,
pub questions: Vec<Question>,
#[educe(Debug(ignore))]
pub result_tx: oneshot::Sender<UserQuestionResult>,
}
// ── Resource type ────────────────────────────────────────────────────────
/// Resource: `mpsc` sender injected into `SharedResources`.
///
/// Same injection pattern as `SubagentEventSender`. Cloned into each
/// session so that any `AskUserQuestionTool` invocation can emit a
/// `UserQuestionRequest` to the session's coordinator.
#[derive(Clone, Educe)]
#[educe(Debug)]
pub struct UserQuestionSender(
#[educe(Debug(ignore))] pub mpsc::UnboundedSender<UserQuestionRequest>,
);
register_resource!("grok_build", "UserQuestionSender", UserQuestionSender);
// ── Conversion helper ────────────────────────────────────────────────────
impl AskUserQuestionExtResponse {
/// Convert the wire-format ACP response into the in-process response type.
///
/// Called by the shell coordinator after deserializing the client's JSON.
/// The `questions` parameter carries the original question list so that
/// `ChatAboutThis` and `SkipInterview` responses can iterate all questions
/// (answered and unanswered) when formatting the tool result.
pub fn into_response(self, questions: Vec<Question>) -> UserQuestionResponse {
match self {
Self::Accepted {
answers,
annotations,
} => UserQuestionResponse::Accepted {
answers,
annotations,
},
Self::ChatAboutThis { partial_answers } => UserQuestionResponse::ChatAboutThis {
questions,
partial_answers,
},
Self::SkipInterview { partial_answers } => UserQuestionResponse::SkipInterview {
questions,
partial_answers,
},
Self::Cancelled => UserQuestionResponse::Cancelled,
}
}
}
// ── Tests ────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
// -- Helpers --
fn sample_questions() -> Vec<Question> {
vec![
Question {
question: "Which database?".to_string(),
options: vec![
super::super::QuestionOption {
label: "Redis".to_string(),
description: "In-memory store".to_string(),
preview: Some("<div>redis preview</div>".to_string()),
id: None,
},
super::super::QuestionOption {
label: "Postgres".to_string(),
description: "Relational DB".to_string(),
preview: None,
id: None,
},
],
multi_select: None,
id: None,
},
Question {
question: "Which framework?".to_string(),
options: vec![
super::super::QuestionOption {
label: "React".to_string(),
description: "UI library".to_string(),
preview: None,
id: None,
},
super::super::QuestionOption {
label: "Vue".to_string(),
description: "Progressive framework".to_string(),
preview: None,
id: None,
},
],
multi_select: None,
id: None,
},
]
}
// -- AskUserQuestionMode serde --
#[test]
fn mode_serializes_as_snake_case() {
assert_eq!(
serde_json::to_value(AskUserQuestionMode::Default).unwrap(),
serde_json::json!("default")
);
assert_eq!(
serde_json::to_value(AskUserQuestionMode::Plan).unwrap(),
serde_json::json!("plan")
);
}
#[test]
fn mode_round_trips() {
for mode in [AskUserQuestionMode::Default, AskUserQuestionMode::Plan] {
let json = serde_json::to_string(&mode).unwrap();
let back: AskUserQuestionMode = serde_json::from_str(&json).unwrap();
assert_eq!(mode, back);
}
}
// -- AskUserQuestionExtRequest serde --
#[test]
fn ext_request_serializes_camel_case() {
let req = AskUserQuestionExtRequest {
session_id: "sess-1".to_string(),
tool_call_id: "tc-1".to_string(),
questions: vec![],
mode: AskUserQuestionMode::Plan,
};
let json = serde_json::to_value(&req).unwrap();
// camelCase field names
assert!(json.get("sessionId").is_some());
assert!(json.get("toolCallId").is_some());
assert_eq!(json["mode"], "plan");
}
#[test]
fn ext_request_round_trips() {
let req = AskUserQuestionExtRequest {
session_id: "sess-1".to_string(),
tool_call_id: "tc-1".to_string(),
questions: sample_questions(),
mode: AskUserQuestionMode::Default,
};
let json = serde_json::to_string(&req).unwrap();
let back: AskUserQuestionExtRequest = serde_json::from_str(&json).unwrap();
assert_eq!(back.session_id, "sess-1");
assert_eq!(back.tool_call_id, "tc-1");
assert_eq!(back.questions.len(), 2);
assert_eq!(back.mode, AskUserQuestionMode::Default);
}
// -- AskUserQuestionExtResponse serde --
#[test]
fn ext_response_accepted_serializes_tagged() {
let mut answers = IndexMap::new();
answers.insert("Which database?".to_string(), vec!["Redis".to_string()]);
let mut annotations = HashMap::new();
annotations.insert(
"Which database?".to_string(),
QuestionAnnotation {
preview: Some("<div>redis preview</div>".to_string()),
notes: None,
},
);
let resp = AskUserQuestionExtResponse::Accepted {
answers,
annotations: Some(annotations),
};
let json = serde_json::to_value(&resp).unwrap();
assert_eq!(json["outcome"], "accepted");
assert!(json["answers"].is_object());
assert!(json["annotations"].is_object());
}
#[test]
fn ext_response_accepted_omits_empty_annotations() {
let mut answers = IndexMap::new();
answers.insert("Q?".to_string(), vec!["A".to_string()]);
let resp = AskUserQuestionExtResponse::Accepted {
answers,
annotations: None,
};
let json = serde_json::to_value(&resp).unwrap();
assert_eq!(json["outcome"], "accepted");
assert!(json.get("annotations").is_none());
}
#[test]
fn ext_response_chat_about_this_serializes() {
let mut partial = HashMap::new();
partial.insert("Which database?".to_string(), "Redis".to_string());
let resp = AskUserQuestionExtResponse::ChatAboutThis {
partial_answers: partial,
};
let json = serde_json::to_value(&resp).unwrap();
assert_eq!(json["outcome"], "chat_about_this");
assert!(json["partial_answers"].is_object());
}
#[test]
fn ext_response_skip_interview_serializes() {
let resp = AskUserQuestionExtResponse::SkipInterview {
partial_answers: HashMap::new(),
};
let json = serde_json::to_value(&resp).unwrap();
assert_eq!(json["outcome"], "skip_interview");
}
#[test]
fn ext_response_cancelled_serializes() {
let resp = AskUserQuestionExtResponse::Cancelled;
let json = serde_json::to_value(&resp).unwrap();
assert_eq!(json["outcome"], "cancelled");
}
#[test]
fn ext_response_round_trips_all_variants() {
// Accepted
let mut answers = IndexMap::new();
answers.insert("Q1?".to_string(), vec!["A1".to_string()]);
let accepted = AskUserQuestionExtResponse::Accepted {
answers,
annotations: None,
};
let json = serde_json::to_string(&accepted).unwrap();
let back: AskUserQuestionExtResponse = serde_json::from_str(&json).unwrap();
assert!(matches!(back, AskUserQuestionExtResponse::Accepted { .. }));
// ChatAboutThis
let chat = AskUserQuestionExtResponse::ChatAboutThis {
partial_answers: HashMap::new(),
};
let json = serde_json::to_string(&chat).unwrap();
let back: AskUserQuestionExtResponse = serde_json::from_str(&json).unwrap();
assert!(matches!(
back,
AskUserQuestionExtResponse::ChatAboutThis { .. }
));
// SkipInterview
let skip = AskUserQuestionExtResponse::SkipInterview {
partial_answers: HashMap::new(),
};
let json = serde_json::to_string(&skip).unwrap();
let back: AskUserQuestionExtResponse = serde_json::from_str(&json).unwrap();
assert!(matches!(
back,
AskUserQuestionExtResponse::SkipInterview { .. }
));
// Cancelled
let cancel = AskUserQuestionExtResponse::Cancelled;
let json = serde_json::to_string(&cancel).unwrap();
let back: AskUserQuestionExtResponse = serde_json::from_str(&json).unwrap();
assert!(matches!(back, AskUserQuestionExtResponse::Cancelled));
}
// -- into_response conversion --
#[test]
fn into_response_accepted() {
let mut answers = IndexMap::new();
answers.insert("Which database?".to_string(), vec!["Redis".to_string()]);
let mut annotations = HashMap::new();
annotations.insert(
"Which database?".to_string(),
QuestionAnnotation {
preview: Some("preview".to_string()),
notes: Some("my notes".to_string()),
},
);
let ext = AskUserQuestionExtResponse::Accepted {
answers: answers.clone(),
annotations: Some(annotations),
};
let resp = ext.into_response(sample_questions());
match resp {
UserQuestionResponse::Accepted {
answers: a,
annotations: ann,
} => {
assert_eq!(a, answers);
assert!(ann.is_some());
let ann = ann.unwrap();
assert!(ann.contains_key("Which database?"));
}
other => panic!("Expected Accepted, got {:?}", other),
}
}
#[test]
fn into_response_chat_about_this_carries_questions() {
let mut partial = HashMap::new();
partial.insert("Which database?".to_string(), "Redis".to_string());
let questions = sample_questions();
let ext = AskUserQuestionExtResponse::ChatAboutThis {
partial_answers: partial.clone(),
};
let resp = ext.into_response(questions.clone());
match resp {
UserQuestionResponse::ChatAboutThis {
questions: q,
partial_answers: p,
} => {
assert_eq!(q.len(), questions.len());
assert_eq!(p, partial);
}
other => panic!("Expected ChatAboutThis, got {:?}", other),
}
}
#[test]
fn into_response_skip_interview_carries_questions() {
let questions = sample_questions();
let ext = AskUserQuestionExtResponse::SkipInterview {
partial_answers: HashMap::new(),
};
let resp = ext.into_response(questions.clone());
match resp {
UserQuestionResponse::SkipInterview {
questions: q,
partial_answers: p,
} => {
assert_eq!(q.len(), questions.len());
assert!(p.is_empty());
}
other => panic!("Expected SkipInterview, got {:?}", other),
}
}
#[test]
fn into_response_cancelled() {
let ext = AskUserQuestionExtResponse::Cancelled;
let resp = ext.into_response(sample_questions());
assert!(matches!(resp, UserQuestionResponse::Cancelled));
}
// -- QuestionAnnotation serde --
#[test]
fn annotation_omits_none_fields() {
let ann = QuestionAnnotation {
preview: None,
notes: None,
};
let json = serde_json::to_value(&ann).unwrap();
assert!(json.get("preview").is_none());
assert!(json.get("notes").is_none());
}
#[test]
fn annotation_includes_present_fields() {
let ann = QuestionAnnotation {
preview: Some("prev".to_string()),
notes: Some("note".to_string()),
};
let json = serde_json::to_value(&ann).unwrap();
assert_eq!(json["preview"], "prev");
assert_eq!(json["notes"], "note");
}
// -- Backwards-compatible deserialization (string -> vec) --
#[test]
fn deserialize_accepted_old_string_format() {
let raw = r#"{
"outcome": "accepted",
"answers": {"Which cache?": "Only hot-path caches"}
}"#;
let resp: AskUserQuestionExtResponse = serde_json::from_str(raw).unwrap();
match resp {
AskUserQuestionExtResponse::Accepted { answers, .. } => {
assert_eq!(
answers["Which cache?"],
vec!["Only hot-path caches".to_string()]
);
}
other => panic!("Expected Accepted, got {:?}", other),
}
}
#[test]
fn deserialize_accepted_mixed_string_and_vec() {
let raw = r#"{
"outcome": "accepted",
"answers": {"Q1?": "old-style", "Q2?": ["new-style"]}
}"#;
let resp: AskUserQuestionExtResponse = serde_json::from_str(raw).unwrap();
match resp {
AskUserQuestionExtResponse::Accepted { answers, .. } => {
assert_eq!(answers["Q1?"], vec!["old-style".to_string()]);
assert_eq!(answers["Q2?"], vec!["new-style".to_string()]);
}
other => panic!("Expected Accepted, got {:?}", other),
}
}
// -- Deserialization from raw JSON (simulating pager responses) --
#[test]
fn deserialize_accepted_from_raw_json() {
let raw = r#"{
"outcome": "accepted",
"answers": {"Which database?": ["Redis"]},
"annotations": {
"Which database?": {
"preview": "<div>redis preview</div>"
}
}
}"#;
let resp: AskUserQuestionExtResponse = serde_json::from_str(raw).unwrap();
match resp {
AskUserQuestionExtResponse::Accepted {
answers,
annotations,
} => {
assert_eq!(answers["Which database?"], vec!["Redis".to_string()]);
let ann = annotations.unwrap();
assert_eq!(
ann["Which database?"].preview.as_deref(),
Some("<div>redis preview</div>")
);
assert!(ann["Which database?"].notes.is_none());
}
other => panic!("Expected Accepted, got {:?}", other),
}
}
#[test]
fn deserialize_accepted_without_annotations() {
let raw = r#"{"outcome": "accepted", "answers": {"Q?": ["A"]}}"#;
let resp: AskUserQuestionExtResponse = serde_json::from_str(raw).unwrap();
match resp {
AskUserQuestionExtResponse::Accepted { annotations, .. } => {
assert!(annotations.is_none());
}
other => panic!("Expected Accepted, got {:?}", other),
}
}
#[test]
fn deserialize_chat_about_this_empty_partials() {
let raw = r#"{"outcome": "chat_about_this"}"#;
let resp: AskUserQuestionExtResponse = serde_json::from_str(raw).unwrap();
match resp {
AskUserQuestionExtResponse::ChatAboutThis { partial_answers } => {
assert!(partial_answers.is_empty());
}
other => panic!("Expected ChatAboutThis, got {:?}", other),
}
}
#[test]
fn deserialize_cancelled() {
let raw = r#"{"outcome": "cancelled"}"#;
let resp: AskUserQuestionExtResponse = serde_json::from_str(raw).unwrap();
assert!(matches!(resp, AskUserQuestionExtResponse::Cancelled));
}
// -- IndexMap ordering preservation --
#[test]
fn accepted_answers_preserve_insertion_order() {
let mut answers = IndexMap::new();
answers.insert("Third?".to_string(), vec!["C".to_string()]);
answers.insert("First?".to_string(), vec!["A".to_string()]);
answers.insert("Second?".to_string(), vec!["B".to_string()]);
let resp = AskUserQuestionExtResponse::Accepted {
answers,
annotations: None,
};
// Round-trip through JSON
let json = serde_json::to_string(&resp).unwrap();
let back: AskUserQuestionExtResponse = serde_json::from_str(&json).unwrap();
if let AskUserQuestionExtResponse::Accepted { answers, .. } = back {
let keys: Vec<&String> = answers.keys().collect();
assert_eq!(keys, vec!["Third?", "First?", "Second?"]);
} else {
panic!("Expected Accepted");
}
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,16 @@
//! Stub surface when the deploy feature is off.
/// Placeholder config — deploy is unavailable in this build.
#[derive(Debug, Clone, Default)]
pub enum AppBuilderDeployerConfig {
#[default]
Disabled,
}
impl AppBuilderDeployerConfig {
pub fn is_enabled(&self) -> bool {
false
}
}
pub const DEPLOY_APP_TOOL_NAME: &str = "deploy_app";
@@ -0,0 +1,707 @@
//! `EnterPlanMode` tool — new architecture (`Tool` trait).
//!
//! Gateway tool that the agent calls when it decides a task is complex enough
//! to warrant a planning phase before writing code. This is the
//! **agent-initiated** entry path into plan mode.
//!
//! On success it notifies orchestration (`PlanModeEntered`) and seeds an empty
//! session plan file if missing (never truncating existing content), so the
//! model can read it before writing. Read-only enforcement and plan-file gating
//! stay in orchestration.
//!
//! ## User Consent
//!
//! This tool requires user approval before executing. The UI should present a
//! confirmation dialog. If the user declines, the tool result is rejected and
//! the model receives `"User declined to enter plan mode."`.
use crate::computer::types::AsyncFileSystem;
use crate::notification::types::PlanModeEntered;
use crate::types::output::{
EnterPlanModeOutput, EnterPlanModeToolHints, PlanFileSeedFailure, PlanFileSeedStatus,
};
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::resources::{FileSystem, NotificationHandle, resolve_plan_file_path};
use crate::types::template_renderer::TemplateRenderer;
use crate::types::tool::{ToolKind, ToolNamespace};
use std::path::Path;
use std::sync::Arc;
/// Input for the `EnterPlanMode` tool.
///
/// Empty object — no parameters. The decision to enter plan mode is a binary
/// gate. All configuration (workflow variant, explore agent count, etc.) comes
/// from feature flags and environment variables, not from the tool call.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct EnterPlanModeInput {}
/// `EnterPlanMode` tool: signals plan mode entry and seeds the session plan
/// file, returning a [`PlanFileSeedStatus`].
///
/// Params: `()` — no per-tool configuration.
#[derive(Debug, Default)]
pub struct EnterPlanModeTool;
impl crate::types::tool_metadata::ToolMetadata for EnterPlanModeTool {
fn kind(&self) -> ToolKind {
ToolKind::EnterPlan
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn emitted_notifications(&self) -> &'static [&'static str] {
&["PlanModeEntered"]
}
fn description_template(&self) -> &str {
r#"Use this tool when a task has ambiguity about the right approach or when the user asks you to write a plan. This tool enables a read-only plan mode where you explore the codebase and create an implementation plan for the user."#
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
// EnterPlanMode can only exist if ExitPlanMode is also registered —
// entering plan mode without the ability to exit would be a dead-end.
use crate::implementations::grok_build::exit_plan_mode::ExitPlanModeTool;
Expr::Value(ToolRequirement::Tool {
namespace: crate::types::tool_metadata::ToolMetadata::tool_namespace(&ExitPlanModeTool)
.to_string(),
id: kigi_tool_runtime::Tool::id(&ExitPlanModeTool).to_string(),
if_params: None,
})
}
}
impl kigi_tool_runtime::Tool for EnterPlanModeTool {
type Args = EnterPlanModeInput;
type Output = EnterPlanModeOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("enter_plan_mode").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"enter_plan_mode",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
// Read-only for permission UX; only FS write is seeding the session plan file.
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(name = "tool.enter_plan_mode", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
_input: EnterPlanModeInput,
) -> Result<EnterPlanModeOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let (seed_target, plan_file_path, tool_hints, fs) = {
let res = resources.lock().await;
// Send notification first.
if let Some(handle) = res.get::<NotificationHandle>() {
handle.0.send_plan_mode_entered(PlanModeEntered {
tool_call_id: ctx.call_id.as_str().to_owned(),
});
}
let (seed_target, plan_file_path) = resolve_plan_file_path(&res);
// Resolve client-facing tool names via TemplateRenderer.
let hints = if let Some(renderer) = res.get::<TemplateRenderer>() {
EnterPlanModeToolHints {
ask_user: renderer
.render("${{ tools.by_kind.ask_user }}")
.unwrap_or_else(|_| "ask_user_question".to_owned()),
exit_plan: renderer
.render("${{ tools.by_kind.exit_plan }}")
.unwrap_or_else(|_| "exit_plan_mode".to_owned()),
task: renderer
.render("${{ tools.by_kind.task }}")
.unwrap_or_default(),
}
} else {
EnterPlanModeToolHints::default()
};
let fs = res.get::<FileSystem>().map(|f| Arc::clone(&f.0));
(seed_target, plan_file_path, hints, fs)
};
// Seed only with both an FS and an absolute target; never write a relative path or truncate.
let plan_file_seed = match (fs.as_ref(), seed_target.as_deref()) {
(Some(fs), Some(target)) => probe_or_create_empty_plan_file(fs.as_ref(), target).await,
_ => {
tracing::warn!(
%plan_file_path,
"No FileSystem resource or no absolute plan path; not seeding plan file"
);
PlanFileSeedStatus::Missing(PlanFileSeedFailure::Unavailable)
}
};
tracing::info!(
%plan_file_path,
?plan_file_seed,
"Entered plan mode"
);
Ok(EnterPlanModeOutput::Entered {
message: "You have entered plan mode. You should now focus on exploring the codebase \
and creating an implementation plan."
.to_string(),
plan_file_path,
tool_hints,
plan_file_seed,
})
}
}
/// Probe the plan file; create an empty one only on not-found.
///
/// Never truncates existing content. Non-NotFound read errors fail closed as
/// [`PlanFileSeedStatus::Missing`] without calling `write_file`.
async fn probe_or_create_empty_plan_file(
fs: &dyn AsyncFileSystem,
path: &Path,
) -> PlanFileSeedStatus {
match fs.read_file(path).await {
Ok(bytes) if bytes.is_empty() => PlanFileSeedStatus::Empty,
Ok(_) => PlanFileSeedStatus::NonEmpty,
Err(e) if e.io_error_kind() == Some(std::io::ErrorKind::NotFound) => {
match fs.write_file(path, b"").await {
Ok(()) => PlanFileSeedStatus::Empty,
Err(e) => {
tracing::warn!(
error = %e,
path = %path.display(),
"Failed to create empty plan file"
);
PlanFileSeedStatus::Missing(PlanFileSeedFailure::NotCreated)
}
}
}
Err(e) => {
// Non-NotFound read error: a directory at the path reads as IsADirectory;
// anything else is treated as inaccessible. Never write (avoid truncate risk).
let reason = match e.io_error_kind() {
Some(std::io::ErrorKind::IsADirectory) => PlanFileSeedFailure::NotAFile,
_ => PlanFileSeedFailure::Inaccessible,
};
tracing::warn!(
error = %e,
?reason,
path = %path.display(),
"Failed to probe plan file; not creating (avoid truncate risk)"
);
PlanFileSeedStatus::Missing(reason)
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::computer::local::LocalFs;
use crate::computer::types::ComputerError;
use crate::types::output::ToolOutput;
use crate::types::resources::{Cwd, PlanFilePath, Resources};
use crate::types::tool_metadata::test_ctx_with_call_id;
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use tempfile::TempDir;
fn resources_with_plan_fs(tmp: &TempDir) -> (Resources, PathBuf) {
let plan = tmp.path().join("session").join("plan.md");
let mut resources = Resources::new();
resources.insert(FileSystem(Arc::new(LocalFs)));
resources.insert(PlanFilePath(plan.clone()));
(resources, plan)
}
/// Parametrized FS mock: injects the read/write outcomes and counts calls
/// so a test can assert the tool never touched the FS.
struct ProbeMockFs {
read: Result<Vec<u8>, ComputerError>,
write: Result<(), ComputerError>,
reads: AtomicUsize,
writes: AtomicUsize,
}
impl ProbeMockFs {
fn new(read: Result<Vec<u8>, ComputerError>, write: Result<(), ComputerError>) -> Self {
Self {
read,
write,
reads: AtomicUsize::new(0),
writes: AtomicUsize::new(0),
}
}
fn err(kind: std::io::ErrorKind) -> ComputerError {
ComputerError::IOError(format!("{kind:?}"), Some(kind))
}
}
#[async_trait::async_trait]
impl AsyncFileSystem for ProbeMockFs {
async fn read_file(&self, _path: &Path) -> Result<Vec<u8>, ComputerError> {
self.reads.fetch_add(1, Ordering::SeqCst);
self.read.clone()
}
async fn write_file(&self, _path: &Path, _data: &[u8]) -> Result<(), ComputerError> {
self.writes.fetch_add(1, Ordering::SeqCst);
self.write.clone()
}
async fn delete_file(&self, _path: &Path) -> Result<(), ComputerError> {
Ok(())
}
}
#[test]
fn tool_name_and_description() {
let tool = EnterPlanModeTool;
assert_eq!(
kigi_tool_runtime::Tool::id(&tool).as_str(),
"enter_plan_mode"
);
let desc = crate::types::tool_metadata::ToolMetadata::description_template(&tool);
assert!(desc.contains("plan mode"));
}
#[test]
fn tool_is_read_only() {
let tool = EnterPlanModeTool;
assert!(kigi_tool_runtime::Tool::capabilities(&tool).is_read_only);
}
#[test]
fn tool_kind_is_enter_plan() {
let tool = EnterPlanModeTool;
assert_eq!(
crate::types::tool_metadata::ToolMetadata::kind(&tool),
ToolKind::EnterPlan
);
}
#[tokio::test]
async fn enter_plan_mode_returns_confirmation() {
let tmp = TempDir::new().unwrap();
let (resources, _) = resources_with_plan_fs(&tmp);
let shared = resources.into_shared();
let tool = EnterPlanModeTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "test-call"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered {
ref message,
ref plan_file_path,
plan_file_seed,
..
} = result;
assert!(message.contains("entered plan mode"));
assert!(message.contains("exploring the codebase"));
assert!(message.contains("implementation plan"));
assert!(plan_file_path.contains("plan.md"));
assert_eq!(plan_file_seed, PlanFileSeedStatus::Empty);
}
#[tokio::test]
async fn sends_plan_mode_entered_notification() {
use crate::notification::types::{ToolNotification, ToolNotificationHandle};
let (handle, mut rx) = ToolNotificationHandle::channel();
let mut resources = Resources::new();
resources.insert(NotificationHandle(handle));
let shared = resources.into_shared();
let tool = EnterPlanModeTool;
kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "call-42"),
EnterPlanModeInput {},
)
.await
.unwrap();
let notification = rx.try_recv().expect("should have received a notification");
match notification {
ToolNotification::PlanModeEntered(entered) => {
assert_eq!(entered.tool_call_id, "call-42");
}
other => panic!("Expected PlanModeEntered, got {:?}", other),
}
}
#[tokio::test]
async fn works_without_notification_handle() {
let resources = Resources::new();
let shared = resources.into_shared();
let tool = EnterPlanModeTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "test-call"),
EnterPlanModeInput {},
)
.await;
assert!(result.is_ok());
}
#[tokio::test]
async fn without_filesystem_resource_plan_not_ready() {
let resources = Resources::new();
let shared = resources.into_shared();
let result = kigi_tool_runtime::Tool::run(
&EnterPlanModeTool,
test_ctx_with_call_id(shared, "test-call"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered { plan_file_seed, .. } = &result;
assert_eq!(
*plan_file_seed,
PlanFileSeedStatus::Missing(PlanFileSeedFailure::Unavailable)
);
let output: ToolOutput = result.into();
let prompt = output.to_prompt_format();
assert!(
prompt.contains("The plan file location is unavailable."),
"expected not-ready status: {prompt}"
);
assert!(
prompt.contains("5. Write your plan to the plan file above"),
"expected constant write-plan step: {prompt}"
);
}
#[tokio::test]
async fn no_absolute_path_does_not_write() {
// FileSystem present but no PlanFilePath and no Cwd: nothing to anchor an
// absolute path on, so the tool must not write a relative path.
let fs = Arc::new(ProbeMockFs::new(
Err(ProbeMockFs::err(std::io::ErrorKind::NotFound)),
Ok(()),
));
let mut resources = Resources::new();
resources.insert(FileSystem(fs.clone()));
let shared = resources.into_shared();
let result = kigi_tool_runtime::Tool::run(
&EnterPlanModeTool,
test_ctx_with_call_id(shared, "no-anchor"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered {
plan_file_seed,
plan_file_path,
..
} = &result;
assert_eq!(
*plan_file_seed,
PlanFileSeedStatus::Missing(PlanFileSeedFailure::Unavailable)
);
assert_eq!(plan_file_path, ".kigi/plan.md");
assert_eq!(fs.reads.load(Ordering::SeqCst), 0, "must not probe");
assert_eq!(fs.writes.load(Ordering::SeqCst), 0, "must not write");
}
#[tokio::test]
async fn prompt_format_returns_message() {
let tmp = TempDir::new().unwrap();
let (resources, _) = resources_with_plan_fs(&tmp);
let shared = resources.into_shared();
let tool = EnterPlanModeTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "test-call"),
EnterPlanModeInput {},
)
.await
.unwrap();
let output: ToolOutput = result.into();
let prompt = output.to_prompt_format();
assert!(prompt.contains("entered plan mode"));
assert!(prompt.contains("plan.md"));
assert!(
prompt.contains("The file exists and is empty."),
"expected empty plan status: {prompt}"
);
assert!(prompt.contains("exit_plan_mode"));
assert!(prompt.contains("ask_user_question"));
assert!(prompt.contains("5. Write your plan to the plan file above"));
assert!(
prompt.contains("6. When ready, use exit_plan_mode to present your plan to the user")
);
assert!(
!prompt.contains("create it at that path first if needed"),
"ready path should not include not-ready create hint: {prompt}"
);
}
#[tokio::test]
async fn does_not_truncate_existing_nonempty_plan() {
let tmp = TempDir::new().unwrap();
let (resources, plan_path) = resources_with_plan_fs(&tmp);
let shared = resources.into_shared();
let fs = LocalFs;
fs.write_file(&plan_path, b"# prior plan\n").await.unwrap();
let result = kigi_tool_runtime::Tool::run(
&EnterPlanModeTool,
test_ctx_with_call_id(shared, "reentry"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered { plan_file_seed, .. } = &result;
assert_eq!(*plan_file_seed, PlanFileSeedStatus::NonEmpty);
let bytes = fs.read_file(&plan_path).await.unwrap();
assert_eq!(bytes, b"# prior plan\n");
let output: ToolOutput = result.into();
let prompt = output.to_prompt_format();
assert!(
prompt.contains("The file exists but is not empty."),
"expected nonempty status: {prompt}"
);
}
#[tokio::test]
async fn existing_empty_plan_reports_empty_without_rewrite() {
let tmp = TempDir::new().unwrap();
let (resources, plan_path) = resources_with_plan_fs(&tmp);
let shared = resources.into_shared();
let fs = LocalFs;
fs.write_file(&plan_path, b"").await.unwrap();
let result = kigi_tool_runtime::Tool::run(
&EnterPlanModeTool,
test_ctx_with_call_id(shared, "empty-reentry"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered { plan_file_seed, .. } = &result;
assert_eq!(*plan_file_seed, PlanFileSeedStatus::Empty);
let bytes = fs.read_file(&plan_path).await.unwrap();
assert_eq!(bytes, b"");
}
#[tokio::test]
async fn probe_or_create_empty_plan_file_via_fs_creates_parents() {
let tmp = TempDir::new().unwrap();
let path = tmp.path().join("nested").join("dir").join("plan.md");
let fs = LocalFs;
let status = probe_or_create_empty_plan_file(&fs, &path).await;
assert_eq!(status, PlanFileSeedStatus::Empty);
assert!(path.is_file());
assert_eq!(fs.read_file(&path).await.unwrap(), b"");
}
#[tokio::test]
async fn non_not_found_read_error_does_not_write() {
let fs = ProbeMockFs::new(
Err(ProbeMockFs::err(std::io::ErrorKind::PermissionDenied)),
Ok(()),
);
let path = Path::new("/session/plan.md");
let status = probe_or_create_empty_plan_file(&fs, path).await;
assert_eq!(
status,
PlanFileSeedStatus::Missing(PlanFileSeedFailure::Inaccessible)
);
assert_eq!(fs.writes.load(Ordering::SeqCst), 0);
}
#[tokio::test]
async fn write_failure_after_not_found_returns_missing() {
let fs = ProbeMockFs::new(
Err(ProbeMockFs::err(std::io::ErrorKind::NotFound)),
Err(ProbeMockFs::err(std::io::ErrorKind::Other)),
);
let path = Path::new("/session/plan.md");
let status = probe_or_create_empty_plan_file(&fs, path).await;
assert_eq!(
status,
PlanFileSeedStatus::Missing(PlanFileSeedFailure::NotCreated)
);
assert_eq!(fs.writes.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn directory_at_path_reports_not_a_file() {
let fs = ProbeMockFs::new(
Err(ProbeMockFs::err(std::io::ErrorKind::IsADirectory)),
Ok(()),
);
let path = Path::new("/session/plan.md");
let status = probe_or_create_empty_plan_file(&fs, path).await;
assert_eq!(
status,
PlanFileSeedStatus::Missing(PlanFileSeedFailure::NotAFile)
);
assert_eq!(
fs.writes.load(Ordering::SeqCst),
0,
"must not write over a directory"
);
}
// -- PlanFilePath resource tests --
#[tokio::test]
async fn uses_plan_file_path_resource_when_set() {
let mut resources = Resources::new();
let session_plan = PathBuf::from("/home/user/.kigi/sessions/abc123/plan.md");
resources.insert(PlanFilePath(session_plan.clone()));
let shared = resources.into_shared();
let result = kigi_tool_runtime::Tool::run(
&EnterPlanModeTool,
test_ctx_with_call_id(shared, "t1"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered {
ref plan_file_path, ..
} = result;
assert_eq!(plan_file_path, &session_plan.display().to_string());
assert!(!plan_file_path.contains(".kigi/plan.md"));
}
#[tokio::test]
async fn falls_back_to_cwd_when_no_plan_file_path_resource() {
let mut resources = Resources::new();
resources.insert(Cwd(PathBuf::from("/workspace/my-project")));
let shared = resources.into_shared();
let result = kigi_tool_runtime::Tool::run(
&EnterPlanModeTool,
test_ctx_with_call_id(shared, "t2"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered {
ref plan_file_path, ..
} = result;
assert_eq!(plan_file_path, "/workspace/my-project/.kigi/plan.md");
}
#[tokio::test]
async fn tool_hints_resolved_from_template_renderer() {
use std::collections::HashMap;
let mut resources = Resources::new();
let tools: HashMap<ToolKind, String> = [
(ToolKind::AskUser, "AskUser".to_owned()),
(ToolKind::ExitPlan, "FinishPlan".to_owned()),
(ToolKind::Task, "delegate".to_owned()),
]
.into();
resources.insert(TemplateRenderer::new(tools, HashMap::new()));
let shared = resources.into_shared();
let result = kigi_tool_runtime::Tool::run(
&EnterPlanModeTool,
test_ctx_with_call_id(shared, "t5"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered { tool_hints, .. } = &result;
assert_eq!(tool_hints.ask_user, "AskUser");
assert_eq!(tool_hints.exit_plan, "FinishPlan");
assert_eq!(tool_hints.task, "delegate");
}
#[tokio::test]
async fn tool_hints_default_without_template_renderer() {
let resources = Resources::new();
let shared = resources.into_shared();
let result = kigi_tool_runtime::Tool::run(
&EnterPlanModeTool,
test_ctx_with_call_id(shared, "t6"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered { tool_hints, .. } = &result;
assert_eq!(tool_hints.ask_user, "ask_user_question");
assert_eq!(tool_hints.exit_plan, "exit_plan_mode");
assert!(
tool_hints.task.is_empty(),
"task should be empty when no TemplateRenderer and no Task tool registered"
);
}
#[tokio::test]
async fn plan_file_path_prefers_resource_over_cwd() {
let mut resources = Resources::new();
resources.insert(Cwd(PathBuf::from("/workspace/my-project")));
resources.insert(PlanFilePath(PathBuf::from(
"/home/user/.kigi/sessions/xyz/plan.md",
)));
let shared = resources.into_shared();
let result = kigi_tool_runtime::Tool::run(
&EnterPlanModeTool,
test_ctx_with_call_id(shared, "t4"),
EnterPlanModeInput {},
)
.await
.unwrap();
let EnterPlanModeOutput::Entered {
ref plan_file_path, ..
} = result;
assert_eq!(plan_file_path, "/home/user/.kigi/sessions/xyz/plan.md");
assert!(!plan_file_path.contains(".kigi/plan.md"));
}
}
@@ -0,0 +1,442 @@
//! `ExitPlanMode` tool — new architecture (`Tool` trait).
//!
//! Signals that the agent has finished planning and is ready for the user to
//! review and approve the plan. The tool reads the plan file from disk (it does
//! NOT accept plan content as input) and surfaces it via:
//!
//! 1. A `PlanModeExited` **notification** sent to the gateway/client, carrying
//! the plan content so the client can present it for user approval.
//! 2. A structured **`ExitPlanModeOutput`** returned to the model, containing
//! the plan content (or an empty-plan message).
//!
//! The actual approval flow (yes/no with feedback, context clear, mode
//! transition) happens on the client side — this tool just says "I'm done,
//! here's the plan."
//!
//! ## Plan File
//!
//! The plan file path defaults to `.kigi/plan.md` relative to the session
//! `Cwd`. The tool reads it via the `FileSystem` resource (the same async FS
//! abstraction used by `ReadFile` and `SearchReplace`).
pub mod types;
pub use types::{ExitPlanModeExtRequest, ExitPlanModeExtResponse};
use crate::notification::types::PlanModeExited;
use crate::types::output::ExitPlanModeOutput;
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::resources::{FileSystem, NotificationHandle, require_plan_file_path};
use crate::types::tool::{ToolKind, ToolNamespace};
/// Input for the `ExitPlanMode` tool.
///
/// Empty object — the plan is read from the plan file on disk, NOT passed as
/// a parameter. This ensures the user sees exactly what was written to disk,
/// preventing divergence between the model's in-context plan and the actual
/// file content.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct ExitPlanModeInput {}
/// `ExitPlanMode` tool.
///
/// Reads the plan file from disk and signals to the orchestration layer that
/// the agent is done planning. The client receives a `PlanModeExited`
/// notification with the plan content and is responsible for presenting the
/// approval UI.
///
/// Params: `()` — no per-tool configuration.
#[derive(Debug, Default)]
pub struct ExitPlanModeTool;
impl crate::types::tool_metadata::ToolMetadata for ExitPlanModeTool {
fn kind(&self) -> ToolKind {
ToolKind::ExitPlan
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn emitted_notifications(&self) -> &'static [&'static str] {
&["PlanModeExited"]
}
fn description_template(&self) -> &str {
r#"Exit plan mode and present your plan to the user.
Use this after you have finished writing your plan to the plan file in plan mode."#
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
// ExitPlanMode can only exist if EnterPlanMode is also registered —
// exiting plan mode without the ability to enter would be nonsensical.
use crate::implementations::grok_build::enter_plan_mode::EnterPlanModeTool;
Expr::Value(ToolRequirement::Tool {
namespace: crate::types::tool_metadata::ToolMetadata::tool_namespace(
&EnterPlanModeTool,
)
.to_string(),
id: kigi_tool_runtime::Tool::id(&EnterPlanModeTool).to_string(),
if_params: None,
})
}
}
impl kigi_tool_runtime::Tool for ExitPlanModeTool {
type Args = ExitPlanModeInput;
type Output = ExitPlanModeOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("exit_plan_mode").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"exit_plan_mode",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(name = "tool.exit_plan_mode", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
_input: ExitPlanModeInput,
) -> Result<ExitPlanModeOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let (plan_file_path, plan_content) = {
let res = resources.lock().await;
let (plan_path, plan_file_path_display) = require_plan_file_path(&res)?;
// Read the plan file from disk via the FileSystem abstraction.
let content = if let Some(fs) = res.get::<FileSystem>() {
match fs.0.read_file(&plan_path).await {
Ok(bytes) => {
let text = String::from_utf8_lossy(&bytes).into_owned();
if text.trim().is_empty() {
None
} else {
Some(text)
}
}
Err(_) => None,
}
} else {
// Fallback: try tokio::fs if no FileSystem resource is available.
match tokio::fs::read_to_string(&plan_path).await {
Ok(text) if !text.trim().is_empty() => Some(text),
_ => None,
}
};
(plan_file_path_display, content)
};
// Notify the gateway / client.
{
let res = resources.lock().await;
if let Some(handle) = res.get::<NotificationHandle>() {
handle.0.send_plan_mode_exited(PlanModeExited {
tool_call_id: ctx.call_id.as_str().to_owned(),
plan_content: plan_content.clone(),
plan_file_path: plan_file_path.clone(),
});
}
}
match plan_content {
Some(content) => {
tracing::info!(
plan_chars = content.len(),
"Exiting plan mode with plan content"
);
let message = "Your plan has been approved. You can now start coding.".to_owned();
Ok(ExitPlanModeOutput::PlanReady {
message,
plan_content: content,
plan_file_path,
})
}
None => {
tracing::info!("Exiting plan mode with empty/missing plan file");
Ok(ExitPlanModeOutput::EmptyPlan {
message:
"Plan mode exit approved. No plan content was found — you can proceed."
.to_string(),
plan_file_path,
})
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::computer::local::LocalFs;
use crate::types::output::ToolOutput;
use crate::types::resources::{Cwd, PlanFilePath, Resources};
use crate::types::tool_metadata::test_ctx_with_call_id;
use std::sync::Arc;
use tempfile::TempDir;
fn resources_with_cwd(cwd: &std::path::Path) -> Resources {
let mut resources = Resources::new();
resources.insert(Cwd(cwd.to_path_buf()));
resources.insert(FileSystem(Arc::new(LocalFs)));
resources
}
#[test]
fn tool_name_and_description() {
let tool = ExitPlanModeTool;
assert_eq!(
kigi_tool_runtime::Tool::id(&tool).as_str(),
"exit_plan_mode"
);
let desc = crate::types::tool_metadata::ToolMetadata::description_template(&tool);
assert!(desc.contains("Exit plan mode"));
assert!(desc.contains("plan file"));
}
#[test]
fn tool_is_read_only() {
assert!(kigi_tool_runtime::Tool::capabilities(&ExitPlanModeTool).is_read_only);
}
#[test]
fn tool_kind_is_exit_plan() {
assert_eq!(
crate::types::tool_metadata::ToolMetadata::kind(&ExitPlanModeTool),
ToolKind::ExitPlan
);
}
#[tokio::test]
async fn exit_with_plan_content() {
let tmp = TempDir::new().unwrap();
let plan_dir = tmp.path().join(".kigi");
std::fs::create_dir_all(&plan_dir).unwrap();
std::fs::write(
plan_dir.join("plan.md"),
"# My Plan\n\n1. Do thing A\n2. Do thing B\n",
)
.unwrap();
let resources = resources_with_cwd(tmp.path());
let shared = resources.into_shared();
let tool = ExitPlanModeTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "test-call"),
ExitPlanModeInput {},
)
.await
.unwrap();
match result {
ExitPlanModeOutput::PlanReady {
ref message,
ref plan_content,
ref plan_file_path,
} => {
assert!(message.contains("plan has been approved"));
assert!(message.contains("start coding"));
assert!(plan_content.contains("Do thing A"));
assert!(plan_content.contains("Do thing B"));
// Cwd fallback now displays the resolved absolute path (shared resolver).
assert!(plan_file_path.ends_with(".kigi/plan.md"));
}
other => panic!("Expected PlanReady, got {:?}", other),
}
}
#[tokio::test]
async fn exit_with_empty_plan_file() {
let tmp = TempDir::new().unwrap();
let plan_dir = tmp.path().join(".kigi");
std::fs::create_dir_all(&plan_dir).unwrap();
std::fs::write(plan_dir.join("plan.md"), " \n \n").unwrap();
let resources = resources_with_cwd(tmp.path());
let shared = resources.into_shared();
let tool = ExitPlanModeTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "test-call"),
ExitPlanModeInput {},
)
.await
.unwrap();
match result {
ExitPlanModeOutput::EmptyPlan { ref message, .. } => {
assert!(message.contains("Plan mode exit approved"));
assert!(message.contains("No plan content was found"));
assert!(message.contains("you can proceed"));
}
other => panic!("Expected EmptyPlan, got {:?}", other),
}
}
#[tokio::test]
async fn exit_with_missing_plan_file() {
let tmp = TempDir::new().unwrap();
let resources = resources_with_cwd(tmp.path());
let shared = resources.into_shared();
let tool = ExitPlanModeTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "test-call"),
ExitPlanModeInput {},
)
.await
.unwrap();
match result {
ExitPlanModeOutput::EmptyPlan { ref message, .. } => {
assert!(message.contains("Plan mode exit approved"));
assert!(message.contains("No plan content was found"));
assert!(message.contains("you can proceed"));
}
other => panic!("Expected EmptyPlan, got {:?}", other),
}
}
#[tokio::test]
async fn sends_plan_mode_exited_notification_with_content() {
use crate::notification::types::{ToolNotification, ToolNotificationHandle};
let tmp = TempDir::new().unwrap();
let plan_dir = tmp.path().join(".kigi");
std::fs::create_dir_all(&plan_dir).unwrap();
std::fs::write(plan_dir.join("plan.md"), "The plan").unwrap();
let (handle, mut rx) = ToolNotificationHandle::channel();
let mut resources = resources_with_cwd(tmp.path());
resources.insert(NotificationHandle(handle));
let shared = resources.into_shared();
let tool = ExitPlanModeTool;
kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "call-99"),
ExitPlanModeInput {},
)
.await
.unwrap();
let notification = rx.try_recv().expect("should have received a notification");
match notification {
ToolNotification::PlanModeExited(exited) => {
assert_eq!(exited.tool_call_id, "call-99");
assert_eq!(exited.plan_content, Some("The plan".to_string()));
assert!(exited.plan_file_path.ends_with(".kigi/plan.md"));
}
other => panic!("Expected PlanModeExited, got {:?}", other),
}
}
#[tokio::test]
async fn works_without_notification_handle() {
let tmp = TempDir::new().unwrap();
let resources = resources_with_cwd(tmp.path());
let shared = resources.into_shared();
let tool = ExitPlanModeTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "test-call"),
ExitPlanModeInput {},
)
.await;
assert!(result.is_ok());
}
#[tokio::test]
async fn prompt_format_includes_plan_content() {
let tmp = TempDir::new().unwrap();
let plan_dir = tmp.path().join(".kigi");
std::fs::create_dir_all(&plan_dir).unwrap();
std::fs::write(plan_dir.join("plan.md"), "Step 1\nStep 2").unwrap();
let resources = resources_with_cwd(tmp.path());
let shared = resources.into_shared();
let tool = ExitPlanModeTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(shared, "test-call"),
ExitPlanModeInput {},
)
.await
.unwrap();
let output: ToolOutput = result.into();
let prompt = output.to_prompt_format();
assert!(prompt.contains("plan has been approved"));
assert!(prompt.contains("saved at:"));
assert!(prompt.contains("Step 1"));
assert!(prompt.contains("Step 2"));
assert!(prompt.contains(".kigi/plan.md"));
assert!(prompt.contains("## Plan:"));
}
// -- PlanFilePath resource tests --
#[tokio::test]
async fn reads_from_plan_file_path_resource() {
let tmp = TempDir::new().unwrap();
let plan_file = tmp.path().join("session-plan.md");
std::fs::write(&plan_file, "# Session Plan\nDo X then Y").unwrap();
let mut resources = Resources::new();
resources.insert(Cwd(tmp.path().to_path_buf()));
resources.insert(FileSystem(Arc::new(LocalFs)));
resources.insert(PlanFilePath(plan_file.clone()));
let shared = resources.into_shared();
let result = kigi_tool_runtime::Tool::run(
&ExitPlanModeTool,
test_ctx_with_call_id(shared, "t1"),
ExitPlanModeInput {},
)
.await
.unwrap();
match result {
ExitPlanModeOutput::PlanReady {
ref plan_content,
ref plan_file_path,
..
} => {
assert!(plan_content.contains("Do X then Y"));
assert_eq!(plan_file_path, &plan_file.display().to_string());
}
other => panic!("Expected PlanReady, got {:?}", other),
}
}
}
@@ -0,0 +1,122 @@
//! Wire types for the `x.ai/exit_plan_mode` ACP ext_method.
//!
//! Shared between the shell (serializer) and the pager/desktop/VS Code
//! (deserializer) so both sides stay in sync.
/// ACP `ext_method` request payload (shell coordinator sends to client/pager).
///
/// Serialized as `camelCase` for the ACP JSON-RPC wire format.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct ExitPlanModeExtRequest {
pub session_id: String,
pub tool_call_id: String,
pub plan_content: Option<String>,
}
/// ACP `ext_method` response payload (client/pager returns to shell coordinator).
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct ExitPlanModeExtResponse {
/// `"approved"`, `"cancelled"`, or `"abandoned"`.
pub outcome: String,
/// Only present on `"cancelled"` when the user typed feedback.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub feedback: Option<String>,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ext_request_serializes_camel_case() {
let req = ExitPlanModeExtRequest {
session_id: "sess-1".into(),
tool_call_id: "tc-1".into(),
plan_content: Some("# Plan".into()),
};
let json = serde_json::to_value(&req).unwrap();
assert!(json.get("sessionId").is_some());
assert!(json.get("toolCallId").is_some());
assert!(json.get("planContent").is_some());
// Must NOT contain snake_case keys
assert!(json.get("session_id").is_none());
assert!(json.get("tool_call_id").is_none());
assert!(json.get("plan_content").is_none());
}
#[test]
fn ext_request_round_trips() {
let req = ExitPlanModeExtRequest {
session_id: "sess-1".into(),
tool_call_id: "tc-1".into(),
plan_content: Some("# Plan\n\n## Step 1\nDo something".into()),
};
let json = serde_json::to_string(&req).unwrap();
let back: ExitPlanModeExtRequest = serde_json::from_str(&json).unwrap();
assert_eq!(back.session_id, "sess-1");
assert_eq!(back.tool_call_id, "tc-1");
assert_eq!(
back.plan_content.as_deref(),
Some("# Plan\n\n## Step 1\nDo something")
);
}
#[test]
fn ext_request_round_trips_no_plan() {
let req = ExitPlanModeExtRequest {
session_id: "sess-2".into(),
tool_call_id: "tc-2".into(),
plan_content: None,
};
let json = serde_json::to_string(&req).unwrap();
let back: ExitPlanModeExtRequest = serde_json::from_str(&json).unwrap();
assert!(back.plan_content.is_none());
}
#[test]
fn ext_response_approved_round_trips() {
let resp = ExitPlanModeExtResponse {
outcome: "approved".into(),
feedback: None,
};
let json = serde_json::to_string(&resp).unwrap();
let back: ExitPlanModeExtResponse = serde_json::from_str(&json).unwrap();
assert_eq!(back.outcome, "approved");
assert!(back.feedback.is_none());
}
#[test]
fn ext_response_cancelled_with_feedback_round_trips() {
let resp = ExitPlanModeExtResponse {
outcome: "cancelled".into(),
feedback: Some("Please add error handling".into()),
};
let json = serde_json::to_string(&resp).unwrap();
let back: ExitPlanModeExtResponse = serde_json::from_str(&json).unwrap();
assert_eq!(back.outcome, "cancelled");
assert_eq!(back.feedback.as_deref(), Some("Please add error handling"));
}
#[test]
fn ext_response_omits_none_feedback() {
let resp = ExitPlanModeExtResponse {
outcome: "approved".into(),
feedback: None,
};
let json = serde_json::to_value(&resp).unwrap();
assert!(json.get("feedback").is_none());
}
#[test]
fn ext_response_abandoned_round_trips() {
let resp = ExitPlanModeExtResponse {
outcome: "abandoned".into(),
feedback: None,
};
let json = serde_json::to_string(&resp).unwrap();
let back: ExitPlanModeExtResponse = serde_json::from_str(&json).unwrap();
assert_eq!(back.outcome, "abandoned");
assert!(back.feedback.is_none());
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,81 @@
use std::path::PathBuf;
use std::sync::OnceLock;
#[cfg(bundle_rg)]
const RG_BYTES: &[u8] = include_bytes!(concat!(
env!("OUT_DIR"),
"/bundle-rg/rg-",
env!("KIGI_TOOLS_RG_VER"),
"-",
env!("KIGI_TOOLS_RG_TARGET"),
".bin"
));
#[cfg(bundle_rg)]
fn resolve_bundled_rg() -> std::io::Result<PathBuf> {
use std::fs;
#[cfg(unix)]
use std::os::unix::fs::PermissionsExt;
let p = crate::util::kigi_home().join("vendor").join(concat!(
"rg-",
env!("KIGI_TOOLS_RG_VER"),
"-",
env!("KIGI_TOOLS_RG_TARGET")
));
if !p.exists() {
fs::create_dir_all(p.parent().unwrap())?;
fs::write(&p, RG_BYTES)?;
#[cfg(unix)]
{
let mut perms = fs::metadata(&p)?.permissions();
perms.set_mode(0o755);
fs::set_permissions(&p, perms)?;
}
}
Ok(p)
}
/// Get the path to the ripgrep executable.
///
/// In release builds with bundling enabled, this extracts the bundled ripgrep
/// binary to ~/.kigi/vendor/ and returns that path.
/// Otherwise, assumes `rg` is in PATH.
pub fn rg_path() -> PathBuf {
static RG_EXEC: OnceLock<PathBuf> = OnceLock::new();
RG_EXEC
.get_or_init(|| {
#[cfg(bundle_rg)]
{
resolve_bundled_rg().unwrap_or_else(|_| PathBuf::from("rg"))
}
#[cfg(not(bundle_rg))]
{
// RG_BIN_PATH: explicit override (tests / packaging can set this).
if let Ok(p) = std::env::var("RG_BIN_PATH") {
return PathBuf::from(p);
}
// Some hermetic test runners set RUNFILES_DIR and ship rg as a
// data dependency rather than on PATH. Scan for a directory
// entry containing "ripgrep_hermetic" and prefer arch-scoped
// paths when present.
if let Ok(rf) = std::env::var("RUNFILES_DIR") {
let base = PathBuf::from(rf);
if let Ok(entries) = std::fs::read_dir(&base) {
for entry in entries.flatten() {
let name = entry.file_name();
if name.to_string_lossy().contains("ripgrep_hermetic") {
for sub in ["amd64/rg", "arm64/rg", "rg"] {
let candidate = entry.path().join(sub);
if candidate.exists() {
return candidate;
}
}
}
}
}
}
PathBuf::from("rg")
}
})
.clone()
}
@@ -0,0 +1,704 @@
//! `image_edit` tool — edits or transforms images via the xAI Imagine
//! `/images/edits` endpoint using one or more reference images.
//!
//! Use cases include likeness preservation, style transfer, subject lock,
//! remixing, and general image-to-image editing. The model chooses this
//! tool (instead of `image_gen`) when the user provides reference photos.
//!
//! Reference images are specified as filesystem paths or
//! `data:image/...;base64,...` URLs. The tool reads the bytes, compresses
//! them to fit API limits, and POSTs to the edit endpoint.
//!
//! Shares the same [`ImageGenClient`] and session credentials as
//! `image_gen` — no additional configuration is needed.
use std::io::Cursor;
use base64::Engine as _;
use image::ImageReader;
use reqwest::header::AUTHORIZATION;
use crate::attribution::ToolConsumer;
use crate::implementations::grok_build::image_gen::{ImageGenClient, ImageGenResponse};
use crate::types::output::{MediaGenOutput, ToolOutput};
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::resources::SessionFolder;
use crate::types::tool::{ToolKind, ToolNamespace};
use crate::util::image_compress::{FilterType, ReEncodeParams, re_encode_under_limit};
const XAI_IMAGINE_MODEL: &str = "grok-imagine-image-quality";
/// Size/dimension limits for reference images sent to the Imagine API.
/// Tighter than the vision path; the backend returns 400 when exceeded.
const MAX_REF_RAW_BYTES: usize = 400 * 1024;
const MAX_REF_DIMENSION: u32 = 768;
const MIN_REF_DIMENSION: u32 = 256;
const REF_QUALITY_STEPS: &[u8] = &[80, 65, 50, 35];
const MAX_REF_DECODE_PIXELS: u64 = 12_000_000;
pub const IMAGE_EDIT_TOOL_NAME: &str = "image_edit";
// ---------------------------------------------------------------------------
// Compression
// ---------------------------------------------------------------------------
/// Compress a reference image to fit within Imagine API limits.
///
/// Returns `(bytes, mime)`. Small JPEG/PNG inputs pass through unchanged.
fn compress_reference(
raw_bytes: Vec<u8>,
) -> Result<(Vec<u8>, &'static str), kigi_tool_runtime::ToolError> {
// Fast path: small JPEG/PNG passes through unchanged. Other formats
// (WebP, GIF, etc.) always re-encode to guarantee API-compatible output.
if raw_bytes.len() <= MAX_REF_RAW_BYTES
&& let Some(kind) = infer::get(&raw_bytes)
{
match kind.mime_type() {
"image/jpeg" => return Ok((raw_bytes, "image/jpeg")),
"image/png" => return Ok((raw_bytes, "image/png")),
_ => {}
}
}
// Refuse to decode absurdly large images.
let reader = ImageReader::new(Cursor::new(&raw_bytes))
.with_guessed_format()
.map_err(|_| {
kigi_tool_runtime::ToolError::invalid_arguments(
"could not detect image format for reference",
)
})?;
if let Ok((w, h)) = reader.into_dimensions()
&& (w as u64) * (h as u64) > MAX_REF_DECODE_PIXELS
{
return Err(kigi_tool_runtime::ToolError::invalid_arguments(format!(
"image reference is too large to process ({w}\u{00d7}{h} pixels)",
)));
}
// `into_dimensions` consumed the reader; re-open to decode.
let img = ImageReader::new(Cursor::new(&raw_bytes))
.with_guessed_format()
.ok()
.and_then(|r| r.decode().ok())
.ok_or_else(|| {
kigi_tool_runtime::ToolError::invalid_arguments("failed to decode image reference")
})?;
let params = ReEncodeParams {
max_bytes: MAX_REF_RAW_BYTES,
max_side_px: MAX_REF_DIMENSION,
// Imagine backend limits are side-based; no pixel-area cap applies.
max_pixels: u64::MAX,
min_side_px: MIN_REF_DIMENSION,
quality_steps: REF_QUALITY_STEPS,
filter: FilterType::Lanczos3,
};
let (buf, _w, _h, mime) = re_encode_under_limit(&img, &params).map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"could not compress image reference small enough for Imagine API: {e}"
))
})?;
Ok((buf, mime))
}
// ---------------------------------------------------------------------------
// Reference resolution
// ---------------------------------------------------------------------------
/// Resolve a reference (filesystem path or `data:image/...;base64,...` URL)
/// into a compressed data URL for the Imagine API.
async fn resolve_to_data_url(value: &str) -> Result<String, kigi_tool_runtime::ToolError> {
let value = value.trim();
// Accept `file://` URIs (e.g. an attachment's durable URI) by reading
// the underlying path. Data URLs and bare paths are untouched.
let value = value.strip_prefix("file://").unwrap_or(value);
let raw_bytes = if value.starts_with("data:image/") {
let comma = value.find(',').ok_or_else(|| {
kigi_tool_runtime::ToolError::invalid_arguments("malformed data URL in image reference")
})?;
if !value[..comma].contains(";base64") {
return Err(kigi_tool_runtime::ToolError::invalid_arguments(
"image references only support base64 data URLs",
));
}
base64::engine::general_purpose::STANDARD
.decode(&value[comma + 1..])
.map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"invalid base64 in image reference: {e}"
))
})?
} else {
tokio::fs::read(value).await.map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"image reference not readable: {value} ({e})"
))
})?
};
if raw_bytes.is_empty() {
return Err(kigi_tool_runtime::ToolError::invalid_arguments(
"image reference contained no data",
));
}
let (compressed, mime) = compress_reference(raw_bytes)?;
let b64 = base64::engine::general_purpose::STANDARD.encode(&compressed);
Ok(format!("data:{mime};base64,{b64}"))
}
// ---------------------------------------------------------------------------
// Attachment reference resolution
// ---------------------------------------------------------------------------
/// Parse an attached-image reference token into its 1-based display number.
///
/// Accepts the forms the model naturally produces for an image the user
/// attached to the conversation: `[Image #1]`, `Image #1`, `image #1`, or
/// a bare `#1`. Returns `None` for anything else — filesystem paths and
/// `data:` / `file://` URLs fall through to direct resolution.
fn parse_attachment_token(value: &str) -> Option<usize> {
let trimmed = value.trim();
// Strip optional surrounding brackets: `[…]`.
let inner = trimmed
.strip_prefix('[')
.and_then(|s| s.strip_suffix(']'))
.unwrap_or(trimmed)
.trim();
// Strip an optional leading `image` label (case-insensitive). The
// 5-byte prefix is ASCII, so slicing at byte 5 stays on a boundary.
let rest = match inner.get(..5).map(str::to_ascii_lowercase).as_deref() {
Some("image") => inner[5..].trim_start(),
_ => inner,
};
// Require the `#` sigil followed by a bare positive integer.
let digits = rest.strip_prefix('#')?.trim();
match digits.parse::<usize>() {
Ok(n) if n >= 1 => Some(n),
_ => None,
}
}
/// Resolve a single `image` argument to a reference `resolve_to_data_url`
/// can read.
///
/// Attachment tokens (`[Image #N]`) are mapped to the durable reference
/// the shell recorded for the current turn; everything else (filesystem
/// paths, `data:` / `file://` URLs) passes through unchanged.
fn resolve_attachment_reference(
reference: &str,
attached: Option<&crate::types::resources::AttachedImages>,
) -> Result<String, kigi_tool_runtime::ToolError> {
let Some(n) = parse_attachment_token(reference) else {
return Ok(reference.to_owned());
};
let registry = attached.filter(|a| !a.0.is_empty()).ok_or_else(|| {
// Tokens only resolve against the current message's attachments. An
// empty registry usually means the image was attached in an earlier
// message (cross-turn editing isn't supported yet), so steer the
// model to ask for a re-attach rather than retry the dead token.
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"image reference {reference:?} matches no image attached to this message. If it was \
attached earlier in the conversation, ask the user to re-attach it here; otherwise \
pass an absolute filesystem path or a data: URL."
))
})?;
registry.reference_for(n).map(str::to_owned).ok_or_else(|| {
let available: Vec<String> = registry
.0
.iter()
.map(|(num, _)| format!("[Image #{num}]"))
.collect();
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"image reference {reference:?} does not match any attached image. Available: {}.",
available.join(", ")
))
})
}
// ---------------------------------------------------------------------------
// Tool input / schema
// ---------------------------------------------------------------------------
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct ImageEditInput {
#[schemars(
description = "A text description of the desired edit or transformation. Describe what the output image should look like, referencing the input image(s)."
)]
pub prompt: String,
#[schemars(
description = "Reference image(s) to condition the edit on. Each is one reference, in priority order: (1) a user attachment — its placeholder token, e.g. \"[Image #1]\" (attachments have no path you can see, so never invent one); (2) an absolute filesystem path the user gave you; (3) a `data:image/...;base64,...` URL."
)]
pub image: Vec<String>,
#[serde(default = "default_aspect_ratio")]
#[schemars(
description = "The aspect ratio of the output image. For single-image edits this is ignored — the output matches the input image's aspect ratio. For multi-image edits, defaults to 'auto'. Supported values: 1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3, 2:1, 1:2, 19.5:9, 9:19.5, 20:9, 9:20, auto."
)]
pub aspect_ratio: String,
}
fn default_aspect_ratio() -> String {
"auto".to_owned()
}
// ---------------------------------------------------------------------------
// Tool implementation
// ---------------------------------------------------------------------------
#[derive(Debug, Default)]
pub struct ImageEditTool;
impl crate::types::tool_metadata::ToolMetadata for ImageEditTool {
fn kind(&self) -> ToolKind {
ToolKind::ImageGen
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
r##"Edit or transform existing image(s) via the xAI Imagine API; use instead of image_gen for image-to-image work (preserve likeness, transfer style, remix). Returns the saved image's absolute path. When telling the user where it was saved, refer to it by its short session-relative path (e.g. `images/1.jpg`) rather than the absolute path, so it renders as a clickable link that opens the image. Each required `image` is one reference — a user-attachment token (e.g. "[Image #1]"), an absolute filesystem path, or a `data:image/...;base64,...` URL (see the `image` parameter for the resolution order and details)."##
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for ImageEditTool {
type Args = ImageEditInput;
type Output = ToolOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("image_edit").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"image_edit",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.image_edit",
skip_all,
fields(prompt_len = input.prompt.len(), num_images = input.image.len(), aspect_ratio = %input.aspect_ratio)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: ImageEditInput,
) -> Result<ToolOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
if input.image.is_empty() {
return Err(kigi_tool_runtime::ToolError::invalid_arguments(
"image_edit requires at least one reference image. \
Use image_gen for text-only generation.",
));
}
let client = {
let res = resources.lock().await;
res.require::<ImageGenClient>()?.clone()
};
// Free / X Basic users are zero-limited on Imagine server-side; return
// the upsell prose instead of a doomed request (shares `image_gen`'s
// message and short-circuits before resolving any attachments).
if client.is_tier_restricted() {
return Ok(ToolOutput::Text(
super::image_gen::TIER_RESTRICTED_UPSELL.into(),
));
}
// Snapshot the per-turn attachment registry so `[Image #N]` tokens
// resolve to the real attachment (see `resolve_attachment_reference`).
let attached_images = {
let res = resources.lock().await;
res.get::<crate::types::resources::AttachedImages>()
.cloned()
};
// Resolve all references to compressed data URLs.
let mut data_urls = Vec::with_capacity(input.image.len());
for r in &input.image {
let resolved = resolve_attachment_reference(r, attached_images.as_ref())?;
data_urls.push(resolve_to_data_url(&resolved).await?);
}
tracing::info!(count = data_urls.len(), "resolved image references");
let base = client.base_url().trim_end_matches('/');
let url = format!("{base}/images/edits");
let mut payload = serde_json::json!({
"model": XAI_IMAGINE_MODEL,
"prompt": input.prompt,
"n": 1,
"resolution": "1k",
"response_format": "b64_json",
});
// API: single ref → "image" object; multiple → "images" array.
// For single-image edits the API auto-detects aspect ratio from the
// input image and ignores the `aspect_ratio` field. Only send it
// for multi-image edits where the API needs an explicit ratio.
let mut imgs: Vec<serde_json::Value> = data_urls
.iter()
.map(|u| serde_json::json!({ "url": u }))
.collect();
if imgs.len() == 1 {
payload["image"] = imgs.pop().unwrap();
} else {
payload["images"] = serde_json::Value::Array(imgs);
payload["aspect_ratio"] = serde_json::json!(input.aspect_ratio);
}
let sent_bearer = client.current_bearer().await;
let mut req = client.http().post(&url).json(&payload);
if let Some(ref key) = sent_bearer {
req = req.header(AUTHORIZATION, format!("Bearer {key}"));
}
let response = req.send().await.map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Image edit API request failed: {e}"
))
})?;
let status = response.status();
if status == reqwest::StatusCode::UNAUTHORIZED {
client.record_401_attribution(ToolConsumer::ImageGen, sent_bearer.as_deref());
}
if !status.is_success() {
let body = response.text().await.unwrap_or_default();
let truncated: String = body.chars().take(200).collect();
tracing::warn!(http_status = %status, "Imagine edit API error: {truncated}");
return Err(kigi_tool_runtime::ToolError::new(
kigi_tool_runtime::ToolErrorKind::Custom,
format!("Image edit failed with HTTP {status}: {truncated}"),
)
.with_details(serde_json::json!({"code": "http_failure", "status": status.as_u16()})));
}
let body = response.text().await.map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Failed to read image edit response body: {e}"
))
})?;
let resp_json: ImageGenResponse = serde_json::from_str(&body).map_err(|e| {
let preview: String = body.chars().take(500).collect();
tracing::warn!("Imagine edit API returned unparseable body: {preview}");
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Failed to parse image edit response: {e} — body preview: {preview}"
))
})?;
let b64_data = resp_json.b64_data().unwrap_or("");
if b64_data.is_empty() {
return Err(kigi_tool_runtime::ToolError::invalid_arguments(
"Image edit returned no image data.",
));
}
let image_bytes = base64::engine::general_purpose::STANDARD
.decode(b64_data)
.map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Failed to decode base64 image data: {e}"
))
})?;
let session_folder = {
let res = resources.lock().await;
res.require::<SessionFolder>()?.0.clone()
};
let absolute_path = client
.writer()
.save(&session_folder, &image_bytes, None)
.await
.map_err(|e| kigi_tool_runtime::ToolError::invalid_arguments(e.to_string()))?;
tracing::info!(
path = %absolute_path.display(),
bytes = image_bytes.len(),
"edited image saved to disk"
);
Ok(ToolOutput::ImageEdit(MediaGenOutput::new(absolute_path)))
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::tool_metadata::test_ctx_with_call_id;
#[test]
fn tool_name_and_description() {
let tool = ImageEditTool;
assert_eq!(kigi_tool_runtime::Tool::id(&tool).as_str(), "image_edit");
let desc = crate::types::tool_metadata::ToolMetadata::description_template(&tool);
assert!(desc.contains("Edit or transform"));
}
#[test]
fn input_deserialization() {
let input: ImageEditInput =
serde_json::from_str(r#"{"prompt": "anime style", "image": ["/Users/me/photo.jpg"]}"#)
.unwrap();
assert_eq!(input.prompt, "anime style");
assert_eq!(input.image, vec!["/Users/me/photo.jpg"]);
assert_eq!(input.aspect_ratio, "auto");
}
#[test]
fn input_requires_image() {
// image field is required by schema — empty array is a runtime check.
let input: ImageEditInput =
serde_json::from_str(r#"{"prompt": "test", "image": []}"#).unwrap();
assert!(input.image.is_empty());
}
#[tokio::test]
async fn rejects_empty_image_array() {
let tool = ImageEditTool;
let resources = crate::types::resources::Resources::new();
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "test-call"),
ImageEditInput {
prompt: "test".into(),
image: vec![],
aspect_ratio: "auto".into(),
},
)
.await;
let err = result.unwrap_err().to_string();
assert!(err.contains("at least one reference image"), "got: {err}");
}
#[tokio::test]
async fn errors_when_client_missing() {
let tool = ImageEditTool;
let resources = crate::types::resources::Resources::new();
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "test-call"),
ImageEditInput {
prompt: "test".into(),
image: vec!["/some/path.jpg".into()],
aspect_ratio: "auto".into(),
},
)
.await;
let err = result.unwrap_err().to_string();
assert!(err.contains("missing required resource"), "got: {err}");
}
// ── compress_reference ───────────────────────────────────────────
fn tiny_jpeg() -> Vec<u8> {
use image::{DynamicImage, RgbImage};
let img = DynamicImage::ImageRgb8(RgbImage::new(2, 2));
let mut buf = Vec::new();
img.write_to(
&mut std::io::Cursor::new(&mut buf),
image::ImageFormat::Jpeg,
)
.unwrap();
buf
}
fn tiny_png() -> Vec<u8> {
use image::{DynamicImage, RgbaImage};
let img = DynamicImage::ImageRgba8(RgbaImage::new(2, 2));
let mut buf = Vec::new();
img.write_to(&mut std::io::Cursor::new(&mut buf), image::ImageFormat::Png)
.unwrap();
buf
}
#[test]
fn compress_small_jpeg_passthrough() {
let jpeg = tiny_jpeg();
let (out, mime) = compress_reference(jpeg.clone()).unwrap();
assert_eq!(out, jpeg);
assert_eq!(mime, "image/jpeg");
}
#[test]
fn compress_small_png_passthrough() {
let png = tiny_png();
let (out, mime) = compress_reference(png.clone()).unwrap();
assert_eq!(out, png);
assert_eq!(mime, "image/png");
}
#[test]
fn compress_oversized_shrinks() {
use image::{DynamicImage, RgbImage};
let mut img = RgbImage::new(1600, 1600);
for (i, px) in img.pixels_mut().enumerate() {
let v = (i * 37 + 13) as u8;
*px = image::Rgb([v, v.wrapping_add(80), v.wrapping_add(160)]);
}
let mut buf = Vec::new();
let enc = image::codecs::jpeg::JpegEncoder::new_with_quality(&mut buf, 100);
DynamicImage::ImageRgb8(img)
.write_with_encoder(enc)
.unwrap();
assert!(buf.len() > MAX_REF_RAW_BYTES);
let (out, mime) = compress_reference(buf).unwrap();
assert!(out.len() <= MAX_REF_RAW_BYTES);
assert!(mime == "image/jpeg" || mime == "image/png");
}
// ── resolve_to_data_url ──────────────────────────────────────────
#[tokio::test]
async fn resolve_filesystem_path() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("test.jpg");
std::fs::write(&path, tiny_jpeg()).unwrap();
let url = resolve_to_data_url(path.to_str().unwrap()).await.unwrap();
assert!(url.starts_with("data:image/jpeg;base64,"));
}
#[tokio::test]
async fn resolve_data_url_roundtrip() {
let jpeg = tiny_jpeg();
let b64 = base64::engine::general_purpose::STANDARD.encode(&jpeg);
let input = format!("data:image/jpeg;base64,{b64}");
let url = resolve_to_data_url(&input).await.unwrap();
assert!(url.starts_with("data:image/jpeg;base64,"));
}
#[tokio::test]
async fn resolve_missing_file_errors() {
assert!(resolve_to_data_url("/nonexistent/image.jpg").await.is_err());
}
#[tokio::test]
async fn resolve_malformed_data_url_errors() {
assert!(resolve_to_data_url("data:image/jpeg").await.is_err());
}
#[tokio::test]
async fn resolve_file_uri_reads_path() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("test.jpg");
std::fs::write(&path, tiny_jpeg()).unwrap();
let uri = format!("file://{}", path.display());
let url = resolve_to_data_url(&uri).await.unwrap();
assert!(url.starts_with("data:image/jpeg;base64,"));
}
// ── parse_attachment_token ───────────────────────────────────────
#[test]
fn parse_attachment_token_accepts_known_forms() {
assert_eq!(parse_attachment_token("[Image #1]"), Some(1));
assert_eq!(parse_attachment_token("Image #2"), Some(2));
assert_eq!(parse_attachment_token("image #3"), Some(3));
assert_eq!(parse_attachment_token("[image #4]"), Some(4));
assert_eq!(parse_attachment_token("Image#5"), Some(5));
assert_eq!(parse_attachment_token("#6"), Some(6));
assert_eq!(parse_attachment_token(" [Image #7] "), Some(7));
}
#[test]
fn parse_attachment_token_rejects_non_tokens() {
assert_eq!(parse_attachment_token("/Users/me/photo.jpg"), None);
assert_eq!(parse_attachment_token("data:image/png;base64,AAAA"), None);
assert_eq!(parse_attachment_token("file:///tmp/x.png"), None);
assert_eq!(parse_attachment_token("[Image #0]"), None);
assert_eq!(parse_attachment_token("[Image #]"), None);
assert_eq!(parse_attachment_token("Image one"), None);
assert_eq!(parse_attachment_token(""), None);
}
// ── resolve_attachment_reference ─────────────────────────────────
#[test]
fn resolve_reference_passes_through_non_tokens() {
let resolved = resolve_attachment_reference("/Users/me/photo.jpg", None).unwrap();
assert_eq!(resolved, "/Users/me/photo.jpg");
}
#[test]
fn resolve_reference_maps_token_to_registry() {
let attached = crate::types::resources::AttachedImages(vec![
(1, "/tmp/a.png".to_owned()),
(2, "/tmp/b.png".to_owned()),
]);
assert_eq!(
resolve_attachment_reference("[Image #1]", Some(&attached)).unwrap(),
"/tmp/a.png"
);
assert_eq!(
resolve_attachment_reference("Image #2", Some(&attached)).unwrap(),
"/tmp/b.png"
);
}
#[test]
fn resolve_reference_maps_by_number_not_position() {
// After a mid-compose chip removal the surviving numbers are
// non-contiguous (`#1`, `#3`). Resolution must key on the number,
// not the list position, or `[Image #3]` would resolve to the wrong
// file (or wrongly error).
let attached = crate::types::resources::AttachedImages(vec![
(1, "/tmp/first.png".to_owned()),
(3, "/tmp/third.png".to_owned()),
]);
assert_eq!(
resolve_attachment_reference("[Image #3]", Some(&attached)).unwrap(),
"/tmp/third.png"
);
// `[Image #2]` was removed → no match.
assert!(resolve_attachment_reference("[Image #2]", Some(&attached)).is_err());
}
#[test]
fn resolve_reference_token_without_registry_errors() {
let err = resolve_attachment_reference("[Image #1]", None)
.unwrap_err()
.to_string();
assert!(err.contains("re-attach"), "got: {err}");
}
#[test]
fn resolve_reference_unmatched_number_errors() {
let attached = crate::types::resources::AttachedImages(vec![(1, "/tmp/a.png".to_owned())]);
let err = resolve_attachment_reference("[Image #2]", Some(&attached))
.unwrap_err()
.to_string();
assert!(err.contains("does not match"), "got: {err}");
assert!(err.contains("[Image #1]"), "should list available: {err}");
}
}
@@ -0,0 +1,585 @@
//! `image_gen` tool — generates images via the xAI Imagine API and saves
//! them to the local filesystem so the model can reference them in code
//! (e.g. `<img src="images/hero.jpg">`).
//!
//! Architecture follows the same pattern as `web_search`:
//!
//! - [`ImageGenConfig`] is built from session credentials by the host and
//! injected into the tool registry.
//! - When `Enabled`, an [`ImageGenClient`] is constructed once and injected
//! into `Resources`. The tool reads it at runtime via `resources.require()`.
//! - When `Disabled`, the tool is not registered so the model never sees it.
//!
//! The generated image is written to `<session_folder>/images/<n>.jpg`
//! where `<n>` is a session-scoped counter (1, 2, 3, ... — 1 token each).
//! The tool returns the absolute path so the model can copy or move the
//! image into the project working directory when it needs a persistent asset.
use base64::Engine as _;
use reqwest::header::{AUTHORIZATION, CONTENT_TYPE, HeaderValue};
use crate::attribution::{SharedAttributionCallback, ToolConsumer};
use crate::types::SharedApiKeyProvider;
use crate::types::output::{MediaGenOutput, ToolOutput};
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::resources::SessionFolder;
use crate::types::tool::{ToolKind, ToolNamespace};
/// Default Imagine model for `image_gen`. Used unless an explicit
/// `model_override` is supplied via `ImageGenConfig::Enabled`.
const XAI_IMAGINE_MODEL: &str = "grok-imagine-image-quality";
// Some Imagine models (e.g. `grok-imagine-image`, selectable via `model_override`)
// expand the prompt then generate, and the proxy buffers
// the whole image before sending any bytes — so the client may receive nothing
// for well over a minute. Keep these generous so a slow-but-progressing
// generation isn't cut off.
const IMAGE_GEN_TIMEOUT_SECS: u64 = 300;
const IMAGE_GEN_READ_TIMEOUT_SECS: u64 = 240;
const DEFAULT_IMAGE_DIR: &str = "images";
pub use kigi_tools_api::slash_commands::{
IMAGE_GEN_TOOL_NAME, IMAGINE_COMMAND_NAME, imagine_instruction, imagine_usage_message,
};
/// Prose returned to the model (as a normal, successful tool result) when a
/// free / X Basic user calls `image_gen` or `image_edit`. The model relays it
/// to the user. The deliberate `/imagine` slash command shows the richer
/// SuperGrok upsell modal instead; this covers the natural-language path.
pub(crate) const TIER_RESTRICTED_UPSELL: &str = "Image generation is a SuperGrok feature and isn't available on the free or X Basic tier. Let the user know they can unlock image and video generation by upgrading to SuperGrok: https://grok.com/supergrok?referrer=grok-build. Do not retry this tool.";
/// HTTP client for xAI Imagine API. Cloned per-request; shares `Arc` state.
#[derive(Clone)]
pub struct ImageGenClient {
http: reqwest::Client,
base_url: String,
/// Imagine model slug used by `generate()`. Selected at construction
/// from `ImageGenConfig::model_override` (falling back to
/// [`XAI_IMAGINE_MODEL`]). `image_edit` uses its own model and is
/// unaffected.
model: String,
writer: super::storage::SessionFileWriter,
api_key_provider: Option<SharedApiKeyProvider>,
/// Optional 401-attribution hook. Hosts wire this so a 401 from the
/// Imagine API emits an `auth_401_attribution` event with
/// `consumer == "ImageGen"` for unified auth-failure telemetry.
attribution_callback: Option<SharedAttributionCallback>,
/// When `true`, the user is on a tier the Imagine server zero-limits
/// (free / X Basic). `image_gen` / `image_edit` short-circuit before any
/// HTTP call and return the SuperGrok upsell prose instead. See
/// [`ImageGenClient::is_tier_restricted`].
tier_restricted: bool,
}
impl ImageGenClient {
pub fn new(
config: &ImageGenConfig,
api_key_provider: Option<SharedApiKeyProvider>,
) -> Result<Self, kigi_tool_runtime::ToolError> {
let ImageGenConfig::Enabled {
api_key,
base_url,
extra_headers,
model_override,
tier_restricted,
..
} = config
else {
return Err(kigi_tool_runtime::ToolError::invalid_arguments(
"Cannot create ImageGenClient from disabled config",
));
};
let model = model_override
.clone()
.filter(|m| !m.trim().is_empty())
.unwrap_or_else(|| XAI_IMAGINE_MODEL.to_owned());
let mut headers = reqwest::header::HeaderMap::new();
headers.insert(CONTENT_TYPE, HeaderValue::from_static("application/json"));
// Always bake the static api_key as the default Authorization header.
// The dynamic provider overrides per-request; this is the fallback.
headers.insert(
AUTHORIZATION,
HeaderValue::from_str(&format!("Bearer {api_key}")).map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Invalid API key for header: {e}"
))
})?,
);
extra_headers.into_iter().try_for_each(|(key, value)| {
let header_name =
reqwest::header::HeaderName::from_bytes(key.as_bytes()).map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Invalid header name '{key}': {e}"
))
})?;
let header_value = HeaderValue::from_str(value).map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Invalid header value for '{key}': {e}"
))
})?;
headers.insert(header_name, header_value);
Ok::<(), kigi_tool_runtime::ToolError>(())
})?;
let http = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(IMAGE_GEN_TIMEOUT_SECS))
.read_timeout(std::time::Duration::from_secs(IMAGE_GEN_READ_TIMEOUT_SECS))
.default_headers(headers)
.build()
.map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Failed to build HTTP client: {e}"
))
})?;
Ok(Self {
http,
base_url: base_url.clone(),
model,
writer: super::storage::SessionFileWriter::new(DEFAULT_IMAGE_DIR, "jpg"),
api_key_provider,
attribution_callback: None,
tier_restricted: *tier_restricted,
})
}
/// Whether the current user's tier (free / X Basic) is zero-limited on
/// Imagine server-side. `image_gen` / `image_edit` use this to short-circuit
/// with the SuperGrok upsell instead of issuing a doomed request.
pub(crate) fn is_tier_restricted(&self) -> bool {
self.tier_restricted
}
/// Wire a 401-attribution callback into this client. Idempotent;
/// safe to call before or after the first request. Builder-style
/// so `new()` callers that don't care can ignore it.
pub fn with_attribution_callback(
mut self,
callback: Option<SharedAttributionCallback>,
) -> Self {
self.attribution_callback = callback;
self
}
pub(crate) async fn current_bearer(&self) -> Option<String> {
crate::types::api_key_provider::resolve_bearer(self.api_key_provider.as_ref()).await
}
pub(crate) fn record_401_attribution(&self, consumer: ToolConsumer, sent_bearer: Option<&str>) {
crate::attribution::emit_401(self.attribution_callback.as_ref(), consumer, sent_bearer);
}
pub(crate) fn base_url(&self) -> &str {
&self.base_url
}
pub(crate) fn http(&self) -> &reqwest::Client {
&self.http
}
pub(crate) fn writer(&self) -> &super::storage::SessionFileWriter {
&self.writer
}
pub async fn generate(
&self,
prompt: &str,
aspect_ratio: &str,
) -> Result<Vec<u8>, kigi_tool_runtime::ToolError> {
let url = format!("{}/images/generations", self.base_url.trim_end_matches('/'));
let payload = serde_json::json!({
"model": self.model,
"prompt": prompt,
"n": 1,
"aspect_ratio": aspect_ratio,
"resolution": "1k",
"response_format": "b64_json",
});
// Capture the bearer once so the request and the 401-attribution
// emit see the same value (even if the provider rotates between
// the send and the response handling).
let sent_bearer = self.current_bearer().await;
let mut req = self.http.post(&url).json(&payload);
if let Some(ref key) = sent_bearer {
req = req.header(AUTHORIZATION, format!("Bearer {key}"));
}
let response = req.send().await.map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Image generation API request failed: {e}"
))
})?;
let status = response.status();
if status == reqwest::StatusCode::UNAUTHORIZED {
self.record_401_attribution(ToolConsumer::ImageGen, sent_bearer.as_deref());
}
if !status.is_success() {
let body = response.text().await.unwrap_or_default();
let truncated: String = body.chars().take(200).collect();
tracing::warn!(http_status = %status, "Imagine API error: {truncated}");
return Err(kigi_tool_runtime::ToolError::new(
kigi_tool_runtime::ToolErrorKind::Custom,
format!("Image generation failed with HTTP {status}: {truncated}"),
)
.with_details(serde_json::json!({"code": "http_failure", "status": status.as_u16()})));
}
let body = response.text().await.map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Failed to read image generation response body: {e}"
))
})?;
let resp_json: ImageGenResponse = serde_json::from_str(&body).map_err(|e| {
let preview: String = body.chars().take(500).collect();
tracing::warn!("Imagine API returned unparseable body: {preview}");
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Failed to parse image generation response: {e} — body preview: {preview}"
))
})?;
let b64_data = resp_json.b64_data().unwrap_or("");
if b64_data.is_empty() {
return Err(kigi_tool_runtime::ToolError::invalid_arguments(
"Image generation returned no image data.",
));
}
base64::engine::general_purpose::STANDARD
.decode(b64_data)
.map_err(|e| {
kigi_tool_runtime::ToolError::invalid_arguments(format!(
"Failed to decode base64 image data: {e}"
))
})
}
}
/// `Enabled` means credentials are present; each tool has its own gate.
#[derive(Debug, Clone, Default)]
pub enum ImageGenConfig {
#[default]
Disabled,
Enabled {
api_key: String,
base_url: String,
extra_headers: indexmap::IndexMap<String, String>,
image_gen_enabled: bool,
image_edit_enabled: bool,
/// Optional Imagine model override for `image_gen`. When `Some(non-empty)`,
/// `image_gen` calls that model instead of the default quality model
/// ([`XAI_IMAGINE_MODEL`]). Driven by the remote
/// `image_gen_model_override` config flag. `image_edit` is unaffected.
model_override: Option<String>,
/// `true` when the user is on a tier the Imagine server zero-limits
/// (free / X Basic). The tools stay advertised to the model, but
/// `image_gen` / `image_edit` short-circuit at call time with the
/// SuperGrok upsell prose instead of a doomed request. Set by the
/// host from the subscription tier; always `false` for team /
/// API-key / workspace callers.
tier_restricted: bool,
},
}
impl ImageGenConfig {
/// Credentials present — required to construct any of the clients.
pub fn has_credentials(&self) -> bool {
matches!(self, Self::Enabled { .. })
}
pub fn image_gen_enabled(&self) -> bool {
matches!(
self,
Self::Enabled {
image_gen_enabled: true,
..
}
)
}
pub fn image_edit_enabled(&self) -> bool {
matches!(
self,
Self::Enabled {
image_edit_enabled: true,
..
}
)
}
/// The configured `image_gen` model override, if any. `None` means the
/// default quality model ([`XAI_IMAGINE_MODEL`]) is used.
pub fn model_override(&self) -> Option<&str> {
match self {
Self::Enabled { model_override, .. } => {
model_override.as_deref().filter(|m| !m.trim().is_empty())
}
Self::Disabled => None,
}
}
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct ImageGenInput {
#[schemars(description = "Text description of the image to generate.")]
pub prompt: String,
#[serde(default = "default_aspect_ratio")]
#[schemars(
description = "Aspect ratio of the generated image, decide it based on the user's request. Defaults to 'auto'. 1:1 for square (icons, profiles), 16:9 for wide (landscapes, cinematic), 9:16 for tall (phone wallpapers, stories), 3:2 for horizontal photos, 2:3 for vertical (portraits, posters)."
)]
pub aspect_ratio: String,
}
fn default_aspect_ratio() -> String {
"auto".to_owned()
}
#[derive(Debug, serde::Deserialize)]
pub struct ImageGenResponse {
#[serde(default)]
data: Vec<ImageGenData>,
}
impl ImageGenResponse {
pub fn b64_data(&self) -> Option<&str> {
self.data.first().and_then(|d| d.b64_json.as_deref())
}
}
#[derive(Debug, serde::Deserialize)]
struct ImageGenData {
b64_json: Option<String>,
}
#[derive(Debug, Default)]
pub struct ImageGenTool;
impl crate::types::tool_metadata::ToolMetadata for ImageGenTool {
fn kind(&self) -> ToolKind {
ToolKind::ImageGen
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
"Generate a new image from a text description using Imagine; returns the saved image's absolute path. When telling the user where it was saved, refer to it by its short session-relative path (e.g. `images/1.jpg`) rather than the absolute path, so it renders as a clickable link that opens the image. To produce multiple images, emit multiple tool calls with distinct prompts."
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for ImageGenTool {
type Args = ImageGenInput;
type Output = ToolOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("image_gen").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"image_gen",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.image_gen",
skip_all,
fields(prompt_len = input.prompt.len(), aspect_ratio = %input.aspect_ratio)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: ImageGenInput,
) -> Result<ToolOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let client = {
let res = resources.lock().await;
res.require::<ImageGenClient>()?.clone()
};
// Free / X Basic users are zero-limited on Imagine server-side; return
// the upsell prose instead of a doomed request (the tool stays
// advertised so the model can surface the nudge in-conversation).
if client.is_tier_restricted() {
return Ok(ToolOutput::Text(TIER_RESTRICTED_UPSELL.into()));
}
let image_bytes = client.generate(&input.prompt, &input.aspect_ratio).await?;
let session_folder = {
let res = resources.lock().await;
res.require::<SessionFolder>()?.0.clone()
};
let absolute_path = client
.writer
.save(&session_folder, &image_bytes, None)
.await
.map_err(|e| kigi_tool_runtime::ToolError::invalid_arguments(e.to_string()))?;
tracing::info!(
path = %absolute_path.display(),
bytes = image_bytes.len(),
"image saved to disk"
);
Ok(ToolOutput::ImageGen(MediaGenOutput::new(absolute_path)))
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::tool_metadata::test_ctx_with_call_id;
#[test]
fn tool_name_and_description() {
let tool = ImageGenTool;
assert_eq!(kigi_tool_runtime::Tool::id(&tool).as_str(), "image_gen");
assert!(
crate::types::tool_metadata::ToolMetadata::description_template(&tool)
.contains("Generate a new image from a text description")
);
}
#[test]
fn default_aspect_ratio_is_auto() {
let input: ImageGenInput = serde_json::from_str(r#"{"prompt": "test"}"#).unwrap();
assert_eq!(input.aspect_ratio, "auto");
}
#[test]
fn per_tool_gates_are_independent() {
let cfg = ImageGenConfig::Enabled {
api_key: "k".into(),
base_url: "https://api.x.ai/v1".into(),
extra_headers: indexmap::IndexMap::new(),
image_gen_enabled: false,
image_edit_enabled: true,
model_override: Some("grok-imagine-image".into()),
tier_restricted: false,
};
assert!(cfg.has_credentials());
assert!(!cfg.image_gen_enabled());
assert!(cfg.image_edit_enabled());
assert_eq!(cfg.model_override(), Some("grok-imagine-image"));
assert!(!ImageGenConfig::Disabled.has_credentials());
}
#[test]
fn client_selects_model_from_override() {
let mk = |model_override: Option<&str>| ImageGenConfig::Enabled {
api_key: "k".into(),
base_url: "https://api.x.ai/v1".into(),
extra_headers: indexmap::IndexMap::new(),
image_gen_enabled: true,
image_edit_enabled: true,
model_override: model_override.map(String::from),
tier_restricted: false,
};
// No override → default quality model.
assert_eq!(
ImageGenClient::new(&mk(None), None).unwrap().model,
XAI_IMAGINE_MODEL
);
// Empty override → treated as no override.
assert_eq!(
ImageGenClient::new(&mk(Some("")), None).unwrap().model,
XAI_IMAGINE_MODEL
);
// Override → that exact model slug.
assert_eq!(
ImageGenClient::new(&mk(Some("grok-imagine-image")), None)
.unwrap()
.model,
"grok-imagine-image"
);
}
#[tokio::test]
async fn errors_when_client_missing() {
let tool = ImageGenTool;
let resources = crate::types::resources::Resources::new();
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "test-call"),
ImageGenInput {
prompt: "a test image".into(),
aspect_ratio: "auto".into(),
},
)
.await;
assert!(result.is_err());
let err_msg = result.unwrap_err().to_string();
assert!(
err_msg.contains("missing required resource"),
"Expected MissingResource error, got: {err_msg}"
);
}
#[tokio::test]
async fn tier_restricted_short_circuits_with_upsell() {
// A free / X Basic user's image_gen call returns the SuperGrok upsell
// prose as a normal result (no HTTP, no error card) so the model can
// relay it. Only the client is inserted — the short-circuit returns
// before any other resource (e.g. SessionFolder) is required.
let cfg = ImageGenConfig::Enabled {
api_key: "k".into(),
base_url: "https://api.x.ai/v1".into(),
extra_headers: indexmap::IndexMap::new(),
image_gen_enabled: true,
image_edit_enabled: true,
model_override: None,
tier_restricted: true,
};
let mut resources = crate::types::resources::Resources::new();
resources.insert(ImageGenClient::new(&cfg, None).unwrap());
let result = kigi_tool_runtime::Tool::run(
&ImageGenTool,
test_ctx_with_call_id(resources.into_shared(), "test-call"),
ImageGenInput {
prompt: "a cat".into(),
aspect_ratio: "auto".into(),
},
)
.await
.expect("tier-restricted call must succeed with upsell prose");
match result {
ToolOutput::Text(t) => {
assert!(t.text.contains("SuperGrok"), "got: {}", t.text);
assert!(t.text.contains("supergrok?referrer=grok-build"));
}
other => panic!("expected Text upsell, got {other:?}"),
}
}
}
@@ -0,0 +1,758 @@
//! `kill_task` tool — new architecture (`Tool` trait).
//!
//! Terminates a running background task. Reads the `Terminal` resource
//! from Resources to access the terminal backend.
pub mod terminal_command;
pub use terminal_command::KillTerminalCommandTool;
use crate::computer::types::KillOutcome;
use crate::implementations::grok_build::task::TaskTool;
use crate::implementations::grok_build::task::backend::SubagentBackendResource;
use crate::implementations::grok_build::task::types::SubagentCancelOutcome;
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::resources::Terminal;
use crate::types::template_renderer::TemplateRenderer;
use crate::types::tool::ToolKind;
use crate::types::tool::ToolNamespace;
use kigi_tool_types::{KillTaskOutput, KillTaskResult, KillTaskToolInput};
// ───────────────────────────────────────────────────────────────────────────
// Tool implementation
// ───────────────────────────────────────────────────────────────────────────
#[derive(Debug, Default)]
pub struct KillTaskTool;
// ── Legacy message helpers ───────────────────────────────────────────────
//
// Historical fixture captured from an earlier (0.4.10) revision of this tool.
//
// In 0.4.10, kill_task returned:
// Err(ToolError::ProcessManagerError(format!("Task {} not found", input.task_id)))
//
// The meaningful customer-facing message content is the inner string.
// Subagent wording is out of scope — subagents didn't exist in 0.4.10.
/// Exact historical not-found message for `kill_task` in legacy-0.4.10.
fn render_legacy_kill_task_not_found(task_id: &str) -> String {
format!("Task {} not found", task_id)
}
/// Format a "not found" response for kill_task.
async fn not_found_response(
task_id: &str,
terminal: &std::sync::Arc<dyn crate::computer::types::TerminalBackend>,
is_legacy: bool,
) -> KillTaskOutput {
if is_legacy {
// Legacy: simple error without task ID enumeration.
// Subagent wording is out of scope — subagents didn't exist in 0.4.10.
return KillTaskOutput::TaskNotFound(render_legacy_kill_task_not_found(task_id));
}
// Current: include known task IDs for discoverability.
let known = terminal.list_tasks().await;
let msg = if known.is_empty() {
format!(
"Task or subagent {task_id} not found. No background tasks or subagents exist in this session.",
)
} else {
let ids: Vec<&str> = known.iter().map(|t| t.task_id.as_str()).collect();
format!(
"Task or subagent {task_id} not found. Known bash task IDs: [{}]",
ids.join(", ")
)
};
KillTaskOutput::TaskNotFound(msg)
}
impl crate::types::tool_metadata::ToolMetadata for KillTaskTool {
fn kind(&self) -> ToolKind {
ToolKind::KillTaskAction
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
// Canonical wording lives in the shared builder; `versioned_definition`
// renders it context-aware from the finalized toolset. This static
// fallback mirrors the default grok-build toolset on the current OS.
static DESC: std::sync::LazyLock<String> = std::sync::LazyLock::new(|| {
kigi_tool_types::build_kill_task_description(&kigi_tool_types::KillTaskToolNaming {
monitor_tool: Some("monitor"),
subagent_present: true,
bash_present: true,
is_windows: cfg!(not(unix)),
})
});
&DESC
}
fn versioned_definition(
&self,
_contract_version: Option<&str>,
client_name: &str,
description_override: Option<&str>,
renderer: &TemplateRenderer,
param_map: &std::collections::HashMap<String, String>,
input_schema: &serde_json::Value,
_effective_params: &serde_json::Value,
) -> crate::types::definition::ToolDefinition {
let description = kill_task_description(renderer, description_override);
let remapped_schema = if param_map.is_empty() {
input_schema.clone()
} else {
crate::util::remap::remap_schema_properties(input_schema, param_map)
};
crate::types::definition::ToolDefinition::function(
client_name,
Some(&description),
remapped_schema,
)
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
use crate::types::tool_metadata::ToolMetadata as TM;
let task_tool = Expr::Value(ToolRequirement::Tool {
namespace: TM::tool_namespace(&TaskTool).to_string(),
id: kigi_tool_runtime::Tool::id(&TaskTool).to_string(),
if_params: None,
});
let mut arms =
crate::implementations::grok_build::task_output::background_bash_requires_exprs();
arms.push(task_tool);
Expr::Or(arms)
}
}
/// Resolve the model-facing `kill_task` description from the finalized toolset,
/// honoring an explicit config override. Wording lives in the shared
/// [`kigi_tool_types::build_kill_task_description`] builder so the CLI and
/// prod-chat can't drift; the monitor / subagent / bash clauses follow the
/// tools actually registered this turn, and the kill verb follows the host OS.
fn kill_task_description(
renderer: &TemplateRenderer,
description_override: Option<&str>,
) -> String {
if let Some(ovr) = description_override {
return renderer.render(ovr).unwrap_or_else(|e| {
tracing::warn!("kill_task description override render failed, using raw: {e}");
ovr.to_string()
});
}
kigi_tool_types::build_kill_task_description(&kigi_tool_types::KillTaskToolNaming {
monitor_tool: renderer.tool_for_kind(ToolKind::Monitor),
subagent_present: renderer.tool_for_kind(ToolKind::Task).is_some(),
bash_present: renderer.tool_for_kind(ToolKind::Execute).is_some(),
is_windows: cfg!(not(unix)),
})
}
impl kigi_tool_runtime::Tool for KillTaskTool {
type Args = KillTaskToolInput;
type Output = KillTaskOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("kill_task").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"kill_task",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.kill_task",
skip_all,
fields(task_id = %input.task_id)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: KillTaskToolInput,
) -> Result<KillTaskOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let is_legacy = crate::versions::is_legacy_contract(
crate::types::tool_metadata::behavior_version(&ctx).as_deref(),
);
let terminal;
{
let res = resources.lock().await;
terminal = res.require::<Terminal>()?.0.clone();
}
match terminal.kill_task(&input.task_id).await {
KillOutcome::Killed => Ok(KillTaskOutput::Result(KillTaskResult {
task_id: input.task_id.clone(),
outcome: "killed".to_string(),
message: "Task was terminated successfully".to_string(),
})),
KillOutcome::AlreadyExited => Ok(KillTaskOutput::Result(KillTaskResult {
task_id: input.task_id.clone(),
outcome: "already_exited".to_string(),
message: "Task had already completed".to_string(),
})),
KillOutcome::NotFound => {
// Try subagent cancel via backend
let backend = {
resources
.lock()
.await
.get::<SubagentBackendResource>()
.cloned()
};
if let Some(backend) = backend {
let outcome = backend.backend().cancel(&input.task_id).await;
return Ok(match outcome {
SubagentCancelOutcome::Cancelled => {
KillTaskOutput::Result(KillTaskResult {
task_id: input.task_id.clone(),
outcome: "killed".to_string(),
message: "Subagent cancellation initiated".to_string(),
})
}
SubagentCancelOutcome::AlreadyFinished { status } => {
KillTaskOutput::Result(KillTaskResult {
task_id: input.task_id.clone(),
outcome: "already_exited".to_string(),
message: format!("Subagent already {status}"),
})
}
SubagentCancelOutcome::NotFound => {
not_found_response(&input.task_id, &terminal, is_legacy).await
}
});
}
Ok(not_found_response(&input.task_id, &terminal, is_legacy).await)
}
}
}
}
// ───────────────────────────────────────────────────────────────────────────
// Tests
// ───────────────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
use crate::computer::types::{
BackgroundHandle, KillOutcome as KO, TaskSnapshot, TerminalBackend, TerminalRunRequest,
TerminalRunResult,
};
use crate::implementations::grok_build::task::backend::{
ChannelBackend, SubagentBackendResource,
};
use crate::implementations::grok_build::task::types::{
SubagentCancelRequest, SubagentCancelTarget, SubagentEvent,
};
use crate::types::resources::{Resources, SharedResources};
use crate::types::tool_metadata::test_ctx_with_call_id;
use std::sync::Arc;
use std::time::Duration;
fn make_ctx_with_version(
call_id: &str,
resources: SharedResources,
version: &str,
) -> kigi_tool_runtime::ToolCallContext {
let mut ctx = test_ctx_with_call_id(resources, call_id);
ctx.extensions
.insert(kigi_tool_runtime::BehaviorVersion(version.to_owned()));
ctx
}
/// Minimal mock backend for testing kill_task.
struct MockTerminal {
/// Pre-configured outcome for `kill_task` calls.
outcome: KO,
}
#[async_trait::async_trait]
impl TerminalBackend for MockTerminal {
async fn run(
&self,
_request: TerminalRunRequest,
) -> Result<TerminalRunResult, crate::computer::types::ComputerError> {
unimplemented!()
}
async fn run_background(
&self,
_request: TerminalRunRequest,
) -> Result<BackgroundHandle, crate::computer::types::ComputerError> {
unimplemented!()
}
async fn kill_task(&self, _task_id: &str) -> KO {
self.outcome
}
async fn get_task(&self, _task_id: &str) -> Option<TaskSnapshot> {
None
}
async fn wait_for_completion(
&self,
_task_id: &str,
_timeout: Option<Duration>,
) -> Option<TaskSnapshot> {
None
}
async fn list_tasks(&self) -> Vec<TaskSnapshot> {
vec![]
}
}
fn resources_with_terminal(outcome: KO) -> Resources {
let mut resources = Resources::new();
let backend: Arc<dyn TerminalBackend> = Arc::new(MockTerminal { outcome });
resources.insert(Terminal(backend));
resources
}
#[test]
fn tool_name_and_description() {
let tool = KillTaskTool;
assert_eq!(kigi_tool_runtime::Tool::id(&tool).as_str(), "kill_task");
// The static fallback is the shared builder's default grok-build
// rendering (monitor + task + bash present) for the current OS.
let desc = crate::types::tool_metadata::ToolMetadata::description_template(&tool);
assert!(desc.contains("Terminate"));
assert!(desc.contains("subagent"));
// Must name "monitor" so the model connects stopping a monitor to this tool.
assert!(desc.contains("monitor"));
// The kill verb is OS-specific (SIGTERM on POSIX, Job Object on Windows).
if cfg!(not(unix)) {
assert!(desc.contains("Job Object"), "windows verb: {desc}");
} else {
assert!(desc.contains("SIGTERM"), "posix verb: {desc}");
}
assert!(
!desc.contains("${"),
"fallback must not leak template markers: {desc}"
);
}
/// No dead references: `monitor` / `subagent` appear only when their tool is
/// present. The context-aware `versioned_definition` path drives this via
/// `kill_task_description`; render every subset (including none) and check.
#[test]
fn description_context_aware_for_tool_subsets() {
use crate::types::template_renderer::TemplateRenderer;
use crate::types::tool::ToolKind;
use std::collections::HashMap;
let cases: &[(&str, &[(ToolKind, &str)])] = &[
("no tools", &[]),
(
"execute only",
&[(ToolKind::Execute, "run_terminal_command")],
),
("monitor only", &[(ToolKind::Monitor, "monitor")]),
("task only", &[(ToolKind::Task, "spawn_subagent")]),
(
"execute + monitor",
&[
(ToolKind::Execute, "run_terminal_command"),
(ToolKind::Monitor, "monitor"),
],
),
(
"execute + task",
&[
(ToolKind::Execute, "run_terminal_command"),
(ToolKind::Task, "spawn_subagent"),
],
),
(
"monitor + task",
&[
(ToolKind::Monitor, "monitor"),
(ToolKind::Task, "spawn_subagent"),
],
),
(
"execute + monitor + task",
&[
(ToolKind::Execute, "run_terminal_command"),
(ToolKind::Monitor, "monitor"),
(ToolKind::Task, "spawn_subagent"),
],
),
];
for (label, kinds) in cases {
let tools: HashMap<ToolKind, String> =
kinds.iter().map(|(k, n)| (*k, n.to_string())).collect();
let renderer = TemplateRenderer::new(tools, HashMap::new());
let rendered = kill_task_description(&renderer, None);
println!("\n===== kill_task [{label}] =====\n{rendered}");
let has_monitor = kinds.iter().any(|(k, _)| *k == ToolKind::Monitor);
let has_task = kinds.iter().any(|(k, _)| *k == ToolKind::Task);
assert!(
!rendered.contains("${"),
"[{label}] left an unrendered template marker:\n{rendered}"
);
assert!(
!rendered.contains(" ,") && !rendered.contains(", ,") && !rendered.contains("()"),
"[{label}] dangling punctuation:\n{rendered}"
);
assert!(
rendered.contains("background task"),
"[{label}] must always mention background task:\n{rendered}"
);
assert_eq!(
rendered.contains("monitor"),
has_monitor,
"[{label}] monitor mention must match monitor-tool presence:\n{rendered}"
);
assert_eq!(
rendered.contains("subagent"),
has_task,
"[{label}] subagent mention must match task-tool presence:\n{rendered}"
);
}
}
/// The kill mechanism is OS-level: Windows describes Job Object termination,
/// Unix/Git Bash describe SIGTERM/SIGKILL.
#[test]
fn kill_verb_matches_platform() {
use crate::types::template_renderer::TemplateRenderer;
use crate::types::tool::ToolKind;
use std::collections::HashMap;
let tools: HashMap<ToolKind, String> =
HashMap::from([(ToolKind::Execute, "run_terminal_command".to_string())]);
let renderer = TemplateRenderer::new(tools, HashMap::new());
let desc = kill_task_description(&renderer, None);
if cfg!(not(unix)) {
assert!(
desc.contains("- Terminates the Job Object of a bash task"),
"windows verb, got:\n{desc}"
);
assert!(
!desc.contains("SIGTERM"),
"Unix kill jargon leaked:\n{desc}"
);
} else {
assert!(
desc.contains("- Sends SIGTERM/SIGKILL to a bash task"),
"posix verb, got:\n{desc}"
);
assert!(
!desc.contains("Job Object"),
"Windows jargon leaked:\n{desc}"
);
}
}
#[tokio::test]
async fn kill_task_killed() {
let resources = resources_with_terminal(KO::Killed);
let tool = KillTaskTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "tool_call"),
KillTaskToolInput {
task_id: "task-1".into(),
},
)
.await
.unwrap();
match result {
KillTaskOutput::Result(r) => {
assert_eq!(r.outcome, "killed");
assert_eq!(r.task_id, "task-1");
}
other => panic!("Expected Success, got {:?}", other),
}
}
#[tokio::test]
async fn kill_task_already_exited() {
let resources = resources_with_terminal(KO::AlreadyExited);
let tool = KillTaskTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "tool_call"),
KillTaskToolInput {
task_id: "task-2".into(),
},
)
.await
.unwrap();
match result {
KillTaskOutput::Result(r) => assert_eq!(r.outcome, "already_exited"),
other => panic!("Expected Success, got {:?}", other),
}
}
#[tokio::test]
async fn kill_task_not_found_returns_typed_output() {
let resources = resources_with_terminal(KO::NotFound);
let tool = KillTaskTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "tool_call"),
KillTaskToolInput {
task_id: "task-3".into(),
},
)
.await
.unwrap(); // Should be Ok, not Err
match result {
KillTaskOutput::TaskNotFound(msg) => {
assert!(msg.contains("not found"), "message: {msg}");
assert!(
msg.contains("task-3"),
"message should include task ID: {msg}"
);
}
other => panic!("Expected TaskNotFound, got {:?}", other),
}
}
#[tokio::test]
async fn errors_when_terminal_not_in_resources() {
let resources = Resources::new();
let tool = KillTaskTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "tool_call"),
KillTaskToolInput {
task_id: "task-x".into(),
},
)
.await;
assert!(result.is_err());
assert!(
result
.unwrap_err()
.to_string()
.contains("missing required resource")
);
}
// ── MP-3: Legacy message parity fixture tests ────────────────────────
#[tokio::test]
async fn legacy_kill_task_not_found_exact_historical_message() {
let resources = resources_with_terminal(KO::NotFound);
let tool = KillTaskTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
make_ctx_with_version("test-call", resources.into_shared(), "legacy-0.4.10"),
KillTaskToolInput {
task_id: "task-abc".into(),
},
)
.await
.unwrap();
match result {
KillTaskOutput::TaskNotFound(msg) => {
// Exact historical fixture — no trailing period.
assert_eq!(msg, "Task task-abc not found");
}
other => panic!("Expected TaskNotFound, got {:?}", other),
}
}
#[tokio::test]
async fn current_kill_task_not_found_includes_discoverability() {
// Current (non-legacy) path must still include known task IDs
// or "No background tasks or subagents exist" text.
let resources = resources_with_terminal(KO::NotFound);
let tool = KillTaskTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "test-call"),
KillTaskToolInput {
task_id: "task-abc".into(),
},
)
.await
.unwrap();
match result {
KillTaskOutput::TaskNotFound(msg) => {
assert!(
msg.contains("No background tasks or subagents exist"),
"Current path must include discoverability text, got: {msg}"
);
}
other => panic!("Expected TaskNotFound, got {:?}", other),
}
}
// ── Subagent cancel via backend tests ─────────────────────────────
/// Build resources with a terminal that returns `NotFound` and a
/// `SubagentBackendResource` backed by channels, returning the cancel
/// receiver so the test can simulate the coordinator.
fn resources_with_backend_cancel() -> (
Resources,
tokio::sync::mpsc::UnboundedReceiver<SubagentEvent>,
) {
let mut resources = Resources::new();
let terminal: Arc<dyn TerminalBackend> = Arc::new(MockTerminal {
outcome: KO::NotFound,
});
resources.insert(Terminal(terminal));
let (tx, rx) = tokio::sync::mpsc::unbounded_channel::<SubagentEvent>();
resources.insert(SubagentBackendResource(Arc::new(ChannelBackend::new(tx))));
(resources, rx)
}
fn unwrap_cancel(event: SubagentEvent) -> SubagentCancelRequest {
match event {
SubagentEvent::Cancel(r) => r,
_ => panic!("Expected SubagentEvent::Cancel"),
}
}
#[tokio::test]
async fn kill_task_subagent_cancelled() {
let (resources, mut cancel_rx) = resources_with_backend_cancel();
let shared = resources.into_shared();
let handle = tokio::spawn(async move {
let req = unwrap_cancel(cancel_rx.recv().await.unwrap());
match &req.target {
SubagentCancelTarget::SubagentId(id) => assert_eq!(id, "sub-1"),
other => panic!("Expected SubagentId, got {:?}", other),
}
req.respond_to
.send(SubagentCancelOutcome::Cancelled)
.unwrap();
});
let result = kigi_tool_runtime::Tool::run(
&KillTaskTool,
test_ctx_with_call_id(shared, "test-call"),
KillTaskToolInput {
task_id: "sub-1".into(),
},
)
.await
.unwrap();
handle.await.unwrap();
match result {
KillTaskOutput::Result(r) => {
assert_eq!(r.outcome, "killed");
assert!(
r.message.contains("Subagent cancellation"),
"msg: {}",
r.message
);
}
other => panic!("Expected Result(killed), got {:?}", other),
}
}
#[tokio::test]
async fn kill_task_subagent_already_finished() {
let (resources, mut cancel_rx) = resources_with_backend_cancel();
let shared = resources.into_shared();
let handle = tokio::spawn(async move {
let req = unwrap_cancel(cancel_rx.recv().await.unwrap());
req.respond_to
.send(SubagentCancelOutcome::AlreadyFinished {
status: "completed".to_string(),
})
.unwrap();
});
let result = kigi_tool_runtime::Tool::run(
&KillTaskTool,
test_ctx_with_call_id(shared, "test-call"),
KillTaskToolInput {
task_id: "sub-done".into(),
},
)
.await
.unwrap();
handle.await.unwrap();
match result {
KillTaskOutput::Result(r) => {
assert_eq!(r.outcome, "already_exited");
assert!(
r.message.contains("Subagent already completed"),
"msg: {}",
r.message
);
}
other => panic!("Expected Result(already_exited), got {:?}", other),
}
}
#[tokio::test]
async fn kill_task_subagent_not_found_falls_through() {
let (resources, mut cancel_rx) = resources_with_backend_cancel();
let shared = resources.into_shared();
let handle = tokio::spawn(async move {
let req = unwrap_cancel(cancel_rx.recv().await.unwrap());
req.respond_to
.send(SubagentCancelOutcome::NotFound)
.unwrap();
});
let result = kigi_tool_runtime::Tool::run(
&KillTaskTool,
test_ctx_with_call_id(shared, "test-call"),
KillTaskToolInput {
task_id: "sub-nope".into(),
},
)
.await
.unwrap();
handle.await.unwrap();
match result {
KillTaskOutput::TaskNotFound(msg) => {
assert!(msg.contains("not found"), "msg: {msg}");
}
other => panic!("Expected TaskNotFound, got {:?}", other),
}
}
}
@@ -0,0 +1,216 @@
//! Workspace-only, subagent-free variant of `kill_task` (delegates to [`KillTaskTool`]).
use super::KillTaskTool;
use crate::implementations::grok_build::task_output::background_bash_requires_exprs;
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
use kigi_tool_types::{KillTaskOutput, KillTaskToolInput};
fn kill_terminal_command_requires_expr() -> Expr<ToolRequirement> {
Expr::Or(background_bash_requires_exprs())
}
#[derive(Debug, Default)]
pub struct KillTerminalCommandTool;
impl crate::types::tool_metadata::ToolMetadata for KillTerminalCommandTool {
fn kind(&self) -> ToolKind {
ToolKind::KillTaskAction
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
r#"Terminate a running background terminal command${%- if tools.by_kind.monitor %} or monitor${%- endif %}.
Usage notes:
- Pass its task_id${%- if tools.by_kind.monitor %} (a monitor's task_id is returned by ${{ tools.by_kind.monitor }})${%- endif %}.
- ${%- if is_windows %} Terminates the Job Object of${%- else %} Sends SIGTERM/SIGKILL to${%- endif %} a background command${%- if tools.by_kind.monitor %} or monitor${%- endif %}.
- Returns success if the command was killed or had already exited."#
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
kill_terminal_command_requires_expr()
}
}
impl kigi_tool_runtime::Tool for KillTerminalCommandTool {
type Args = KillTaskToolInput;
type Output = KillTaskOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("kill_terminal_command").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"kill_terminal_command",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.kill_terminal_command",
skip_all,
fields(task_id = %input.task_id)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: KillTaskToolInput,
) -> Result<KillTaskOutput, kigi_tool_runtime::ToolError> {
kigi_tool_runtime::Tool::run(&KillTaskTool, ctx, input).await
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::computer::types::{
BackgroundHandle, KillOutcome, TaskSnapshot, TerminalBackend, TerminalRunRequest,
TerminalRunResult,
};
use crate::types::resources::{Resources, Terminal};
use crate::types::tool_metadata::ToolMetadata;
use crate::types::tool_metadata::test_ctx_with_call_id;
use std::sync::Arc;
use std::time::Duration;
struct MockTerminal {
outcome: KillOutcome,
}
#[async_trait::async_trait]
impl TerminalBackend for MockTerminal {
async fn run(
&self,
_request: TerminalRunRequest,
) -> Result<TerminalRunResult, crate::computer::types::ComputerError> {
unimplemented!()
}
async fn run_background(
&self,
_request: TerminalRunRequest,
) -> Result<BackgroundHandle, crate::computer::types::ComputerError> {
unimplemented!()
}
async fn kill_task(&self, _task_id: &str) -> KillOutcome {
self.outcome
}
async fn get_task(&self, _task_id: &str) -> Option<TaskSnapshot> {
None
}
async fn wait_for_completion(
&self,
_task_id: &str,
_timeout: Option<Duration>,
) -> Option<TaskSnapshot> {
None
}
async fn list_tasks(&self) -> Vec<TaskSnapshot> {
vec![]
}
}
fn resources_with_terminal(outcome: KillOutcome) -> Resources {
let mut resources = Resources::new();
let backend: Arc<dyn TerminalBackend> = Arc::new(MockTerminal { outcome });
resources.insert(Terminal(backend));
resources
}
#[test]
fn tool_name_and_description_are_subagent_free() {
let tool = KillTerminalCommandTool;
assert_eq!(
kigi_tool_runtime::Tool::id(&tool).as_str(),
"kill_terminal_command"
);
let tmpl = ToolMetadata::description_template(&tool);
assert!(tmpl.contains("background terminal command"));
assert!(tmpl.contains("Terminate"));
assert!(
!tmpl.to_lowercase().contains("subagent"),
"workspace tool must not mention subagents: {tmpl}"
);
}
#[tokio::test]
async fn delegates_kill_killed() {
let resources = resources_with_terminal(KillOutcome::Killed);
let result = kigi_tool_runtime::Tool::run(
&KillTerminalCommandTool,
test_ctx_with_call_id(resources.into_shared(), "tool_call"),
KillTaskToolInput {
task_id: "tc-1".into(),
},
)
.await
.unwrap();
match result {
KillTaskOutput::Result(r) => {
assert_eq!(r.outcome, "killed");
assert_eq!(r.task_id, "tc-1");
}
other => panic!("Expected Result(killed), got {other:?}"),
}
}
#[tokio::test]
async fn delegates_kill_already_exited() {
let resources = resources_with_terminal(KillOutcome::AlreadyExited);
let result = kigi_tool_runtime::Tool::run(
&KillTerminalCommandTool,
test_ctx_with_call_id(resources.into_shared(), "tool_call"),
KillTaskToolInput {
task_id: "tc-2".into(),
},
)
.await
.unwrap();
match result {
KillTaskOutput::Result(r) => assert_eq!(r.outcome, "already_exited"),
other => panic!("Expected Result(already_exited), got {other:?}"),
}
}
#[tokio::test]
async fn delegates_kill_not_found() {
let resources = resources_with_terminal(KillOutcome::NotFound);
let result = kigi_tool_runtime::Tool::run(
&KillTerminalCommandTool,
test_ctx_with_call_id(resources.into_shared(), "tool_call"),
KillTaskToolInput {
task_id: "tc-3".into(),
},
)
.await
.unwrap();
match result {
KillTaskOutput::TaskNotFound(msg) => {
assert!(msg.contains("not found"), "message: {msg}");
assert!(
msg.contains("tc-3"),
"message should include task ID: {msg}"
);
}
other => panic!("Expected TaskNotFound, got {other:?}"),
}
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,452 @@
//! Legacy (0.4.10) depth-threshold directory rendering.
//!
//! Extracted from an earlier revision of the codebase. The current renderer uses
//! BFS character-budget expansion; this module preserves the old depth-based
//! summarization algorithm for `contract_version = "legacy-0.4.10"`.
use std::collections::HashMap;
use std::path::{Path, PathBuf};
// ───────────────────────────────────────────────────────────────────────────
// Configuration
// ───────────────────────────────────────────────────────────────────────────
const ROOT_SUMMARIZATION_THRESHOLD: usize = 1500;
const SUBDIR_SUMMARIZATION_THRESHOLD: usize = 15;
const TOP_K_EXTENSIONS_TO_RENDER: usize = 3;
const DEFAULT_MAX_OUTPUT_BYTES: usize = crate::DEFAULT_TOOL_OUTPUT_BYTES;
#[derive(Debug, Clone)]
struct RenderConfig {
summarization_threshold_by_depth: Vec<usize>,
top_k_extensions_to_render: usize,
max_output_bytes: usize,
}
impl Default for RenderConfig {
fn default() -> Self {
Self {
summarization_threshold_by_depth: vec![
ROOT_SUMMARIZATION_THRESHOLD,
SUBDIR_SUMMARIZATION_THRESHOLD,
],
top_k_extensions_to_render: TOP_K_EXTENSIONS_TO_RENDER,
max_output_bytes: DEFAULT_MAX_OUTPUT_BYTES,
}
}
}
impl RenderConfig {
fn with_sub_dirs_always_summarized() -> Self {
RenderConfig {
summarization_threshold_by_depth: vec![ROOT_SUMMARIZATION_THRESHOLD],
top_k_extensions_to_render: TOP_K_EXTENSIONS_TO_RENDER,
max_output_bytes: DEFAULT_MAX_OUTPUT_BYTES,
}
}
}
// ───────────────────────────────────────────────────────────────────────────
// Accumulator + helpers
// ───────────────────────────────────────────────────────────────────────────
#[derive(Debug, Default)]
struct DirAccum {
total_files: usize,
by_ext: HashMap<String, usize>,
}
impl DirAccum {
fn add_ext(&mut self, ext: &str) {
self.total_files += 1;
*self.by_ext.entry(ext.to_owned()).or_default() += 1;
}
fn to_summary(&self, top_n: usize) -> String {
if self.by_ext.is_empty() {
return String::new();
}
let mut items: Vec<(String, usize)> =
self.by_ext.iter().map(|(k, v)| (k.clone(), *v)).collect();
items.sort_by(|a, b| match b.1.cmp(&a.1) {
std::cmp::Ordering::Equal => a.0.cmp(&b.0),
other => other,
});
let mut parts: Vec<String> = Vec::new();
let mut top_sum: usize = 0;
for (ext, count) in items.iter().take(top_n) {
top_sum += *count;
if ext == "no-ext" {
parts.push(format!("{} *no-ext", count));
} else {
parts.push(format!("{} *.{}", count, ext));
}
}
let ellipsis = if top_sum < self.total_files {
", ..."
} else {
""
};
let file_word = if self.total_files == 1 {
"file"
} else {
"files"
};
format!(
"[{} {} in subtree: {}{}]",
self.total_files,
file_word,
parts.join(", "),
ellipsis
)
}
}
fn ext_key_from_path(path: &Path) -> String {
path.extension().map_or("no-ext".to_string(), |s| {
s.to_string_lossy().to_ascii_lowercase()
})
}
fn filename(path: &Path) -> String {
path.file_name()
.map_or("<unknown_filename>".to_string(), |s| {
s.to_string_lossy().into_owned()
})
}
// ───────────────────────────────────────────────────────────────────────────
// Tree structures
// ───────────────────────────────────────────────────────────────────────────
#[derive(Debug)]
struct ChildEntry {
is_dir: bool,
name: String,
path: PathBuf,
}
#[derive(Debug, Default)]
struct DirectoryView {
depth: usize,
child_count: usize,
children: Option<Vec<ChildEntry>>,
subtree: DirAccum,
}
#[derive(Debug)]
struct Collected {
dirs: HashMap<PathBuf, DirectoryView>,
}
fn should_summarize(depth: usize, child_count: usize, cfg: &RenderConfig) -> bool {
cfg.summarization_threshold_by_depth
.get(depth)
.is_none_or(|threshold| child_count >= *threshold)
}
fn get_or_init_dir<'a>(
dirs: &'a mut HashMap<PathBuf, DirectoryView>,
path: &Path,
depth: usize,
cfg: &RenderConfig,
) -> &'a mut DirectoryView {
dirs.entry(path.to_path_buf())
.or_insert_with(|| DirectoryView {
depth,
child_count: 0,
children: if depth < cfg.summarization_threshold_by_depth.len() {
Some(Vec::new())
} else {
None
},
subtree: DirAccum::default(),
})
}
// ───────────────────────────────────────────────────────────────────────────
// Collection + rendering
// ───────────────────────────────────────────────────────────────────────────
fn collect(root_path: &Path, walker: ignore::Walk, cfg: &RenderConfig) -> Collected {
let mut dirs: HashMap<PathBuf, DirectoryView> = HashMap::new();
for directory_entry in walker {
let Ok(entry) = directory_entry else { continue };
let Some(file_type) = entry.file_type() else {
continue;
};
let path = entry.path();
let depth = entry.depth();
if file_type.is_dir() && depth <= cfg.summarization_threshold_by_depth.len() {
get_or_init_dir(&mut dirs, path, depth, cfg);
}
if depth == 0 {
continue;
}
let Some(parent_path) = path.parent() else {
continue;
};
let parent_depth = depth.saturating_sub(1);
if parent_depth <= cfg.summarization_threshold_by_depth.len() {
let parent_view = get_or_init_dir(&mut dirs, parent_path, parent_depth, cfg);
parent_view.child_count += 1;
if let Some(children) = parent_view.children.as_mut() {
let name = filename(path);
children.push(ChildEntry {
is_dir: file_type.is_dir(),
name,
path: path.to_path_buf(),
});
if should_summarize(parent_view.depth, parent_view.child_count, cfg) {
parent_view.children = None;
}
}
}
if file_type.is_file() {
let ext_key = ext_key_from_path(path);
let mut ancestors: Vec<&Path> = path
.ancestors()
.take_while(|p| p.starts_with(root_path))
.collect();
ancestors.pop();
ancestors.reverse();
for (depth, p) in ancestors
.into_iter()
.take(cfg.summarization_threshold_by_depth.len())
.enumerate()
{
let view = get_or_init_dir(&mut dirs, p, depth, cfg);
view.subtree.add_ext(&ext_key);
}
}
}
for view in dirs.values_mut() {
if let Some(children) = view.children.as_mut() {
children.sort_by(|a, b| {
a.name
.to_ascii_lowercase()
.cmp(&b.name.to_ascii_lowercase())
});
}
}
Collected { dirs }
}
fn render(root: &Path, collected: &Collected, cfg: &RenderConfig) -> Vec<String> {
fn render_dir(
lines: &mut Vec<String>,
dir_path: &Path,
collected: &Collected,
cfg: &RenderConfig,
) {
let view = match collected.dirs.get(dir_path) {
Some(v) => v,
None => return,
};
let indent = " ".repeat(view.depth);
lines.push(format!("{}- {}/", indent, filename(dir_path)));
if should_summarize(view.depth, view.child_count, cfg) {
let summary = view.subtree.to_summary(cfg.top_k_extensions_to_render);
if summary.is_empty() {
return;
}
let indent_child = " ".repeat(view.depth + 1);
lines.push(format!("{}{}", indent_child, summary));
} else if let Some(children) = &view.children {
for child in children {
if child.is_dir {
render_dir(lines, &child.path, collected, cfg);
} else {
let indent_child = " ".repeat(view.depth + 1);
lines.push(format!("{}- {}", indent_child, child.name));
}
}
}
}
let mut lines: Vec<String> = Vec::new();
render_dir(&mut lines, root, collected, cfg);
lines
}
fn output_byte_count(lines: &[String]) -> usize {
if lines.is_empty() {
return 0;
}
lines.iter().map(|l| l.len()).sum::<usize>() + lines.len() - 1
}
fn render_with_fallback(root: &Path, collected: &Collected, cfg: &RenderConfig) -> Vec<String> {
let lines = render(root, collected, cfg);
let byte_count = output_byte_count(&lines);
if byte_count > cfg.max_output_bytes {
let mut lines = render(
root,
collected,
&RenderConfig::with_sub_dirs_always_summarized(),
);
tracing::debug!(
path = %root.display(),
original_bytes = byte_count,
limit = cfg.max_output_bytes,
"legacy list_dir output exceeded size limit, falling back to shallow rendering"
);
lines.push(
"[Subdirectories not expanded: listing exceeds size limit. \
Use list_dir on a specific subdirectory to see its contents.]"
.to_string(),
);
lines
} else {
lines
}
}
// ───────────────────────────────────────────────────────────────────────────
// Public entry point
// ───────────────────────────────────────────────────────────────────────────
/// Render a directory listing using the legacy (0.4.10) depth-threshold algorithm.
///
/// Returns the body text (without the root path header line).
pub(crate) fn render_legacy(root: &Path, max_output_bytes: usize) -> String {
let cfg = RenderConfig {
max_output_bytes,
..Default::default()
};
let walker = ignore::WalkBuilder::new(root)
.standard_filters(true)
.build();
let collected = collect(root, walker, &cfg);
let output_lines = render_with_fallback(root, &collected, &cfg);
// Skip the first line (root dir name) — the caller prepends its own.
if output_lines.len() > 1 {
output_lines[1..].join("\n")
} else {
String::new()
}
}
// ───────────────────────────────────────────────────────────────────────────
// Tests — fixture-based historical verification
// ───────────────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
use tempfile::TempDir;
/// Create a reference directory tree for fixture comparison.
fn create_fixture_tree(root: &std::path::Path) {
// src/
// main.rs
// lib.rs
// util/
// helpers.rs
// tests/
// test_main.rs
// README.md
// Cargo.toml
let src = root.join("src");
std::fs::create_dir_all(src.join("util")).unwrap();
std::fs::write(src.join("main.rs"), "fn main() {}").unwrap();
std::fs::write(src.join("lib.rs"), "pub mod util;").unwrap();
std::fs::write(src.join("util").join("helpers.rs"), "pub fn help() {}").unwrap();
let tests = root.join("tests");
std::fs::create_dir_all(&tests).unwrap();
std::fs::write(tests.join("test_main.rs"), "#[test] fn it_works() {}").unwrap();
std::fs::write(root.join("README.md"), "# Project").unwrap();
std::fs::write(root.join("Cargo.toml"), "[package]\nname = \"test\"").unwrap();
}
/// The legacy depth-based renderer expands small directories fully,
/// showing individual files with indentation per depth level.
/// Archived-output fixture test: asserts exact string equality against
/// the known output of the earlier depth-threshold algorithm.
///
/// Fixture captured from `render_legacy()` on the reference tree defined
/// by `create_fixture_tree()`. If this test fails after a change to the
/// legacy renderer, the change has drifted from historical behavior.
#[test]
fn legacy_renders_small_tree_exact_fixture() {
let tmp = TempDir::new().unwrap();
create_fixture_tree(tmp.path());
let body = render_legacy(tmp.path(), 40_000);
// Exact archived output from the depth-threshold algorithm.
// Root files listed alphabetically, src/ expanded (< 15 children),
// util/ summarized (depth >= 2), tests/ expanded.
let expected = " - Cargo.toml\n - README.md\n - src/\n - lib.rs\n - main.rs\n - util/\n [1 file in subtree: 1 *.rs]\n - tests/\n - test_main.rs";
assert_eq!(
body, expected,
"legacy renderer output must match archived fixture exactly.\n\
Got:\n{body}\n\nExpected:\n{expected}"
);
}
/// Empty directories should produce empty output (no "no children found"
/// — that is added by the caller in mod.rs, not by the renderer).
#[test]
fn legacy_empty_directory_returns_empty_string() {
let tmp = TempDir::new().unwrap();
let body = render_legacy(tmp.path(), 40_000);
assert!(
body.is_empty(),
"empty dir should produce empty body, got: {body}"
);
}
/// The depth-based algorithm summarizes directories when child count
/// exceeds the threshold (15 for subdirs by default). Verify that a
/// large directory gets a summary line instead of full expansion.
#[test]
fn legacy_summarizes_large_subdirectory() {
let tmp = TempDir::new().unwrap();
let large_dir = tmp.path().join("many_files");
std::fs::create_dir_all(&large_dir).unwrap();
// Create 20 files to exceed the SUBDIR_SUMMARIZATION_THRESHOLD (15).
for i in 0..20 {
std::fs::write(large_dir.join(format!("file_{i}.rs")), "").unwrap();
}
let body = render_legacy(tmp.path(), 40_000);
// Should show a summary line with file count and extension breakdown.
assert!(
body.contains("files in subtree") || body.contains("file in subtree"),
"large dir should be summarized: {body}"
);
assert!(
body.contains("*.rs"),
"summary should mention .rs extension: {body}"
);
}
/// Verify structural equivalence: the depth-based renderer produces
/// lines with consistent 2-space indentation matching the historical
/// algorithm's output pattern.
#[test]
fn legacy_indentation_matches_historical_pattern() {
let tmp = TempDir::new().unwrap();
create_fixture_tree(tmp.path());
let body = render_legacy(tmp.path(), 40_000);
// Every line should start with some number of " " pairs followed by "- "
// or be a summary line (starts with spaces + "[").
for line in body.lines() {
let trimmed = line.trim_start();
let indent_chars = line.len() - trimmed.len();
assert!(
indent_chars % 2 == 0,
"indentation must be even (2-space per level), got {} in: {line}",
indent_chars
);
assert!(
trimmed.starts_with("- ") || trimmed.starts_with('['),
"line must start with '- ' or '[': {line}"
);
}
}
}
@@ -0,0 +1,7 @@
//! Version-specific behavior modules for `list_dir`.
//!
//! - `legacy_0_4_10`: depth-threshold rendering + generic error messages
//! - Current behavior remains in `list_dir/mod.rs` (BFS budget rendering
//! + structured error variants).
pub(crate) mod legacy_0_4_10;
@@ -0,0 +1,115 @@
//! `lsp` tool - code intelligence via language servers.
//!
//! Implementation is in `implementations::lsp`. This module provides the
//! `LspTool` (Tool trait impl) under the `GrokBuild` namespace.
use std::sync::Arc;
use crate::implementations::lsp::{LspBackend, LspToolInput};
use crate::types::output::ToolOutput;
use crate::types::tool::{ToolKind, ToolNamespace};
#[derive(serde::Serialize, serde::Deserialize)]
pub struct LspToolOutput(pub String);
impl kigi_tool_runtime::ToolOutput for LspToolOutput {}
impl From<LspToolOutput> for ToolOutput {
fn from(o: LspToolOutput) -> Self {
ToolOutput::Text(o.0.into())
}
}
#[derive(Debug, Default)]
pub struct LspTool;
impl crate::types::tool_metadata::ToolMetadata for LspTool {
fn kind(&self) -> ToolKind {
ToolKind::Lsp
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
r#"Code intelligence via language servers.${%- if tools.by_kind.search and tools.by_kind.read %} Prefer over ${{ tools.by_kind.search }}/${{ tools.by_kind.read }} for understanding code.${%- endif %}
Operations: goToDefinition (jump to where a symbol is defined), findReferences (all usages of a symbol), hover (type info/docs at a position), goToImplementation (trait/interface implementations), documentSymbol (list all symbols in a file), workspaceSymbol (search symbols by name across the workspace — requires query parameter, not file_path).
Requires file_path + line + character for position-based operations."#
}
fn emitted_notifications(&self) -> &'static [&'static str] {
&[
"LspServerCrashed",
"LspServerFailed",
"LspServerReady",
"LspServerRetrying",
"LspServerStarting",
]
}
}
impl kigi_tool_runtime::Tool for LspTool {
type Args = LspToolInput;
type Output = LspToolOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("lsp").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"lsp",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.lsp",
skip_all,
fields(operation = %input.operation)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: LspToolInput,
) -> Result<LspToolOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let handle;
{
let res = resources.lock().await;
handle = res
.get::<Arc<dyn LspBackend>>()
.ok_or_else(|| {
kigi_tool_runtime::ToolError::custom(
"process_manager",
"LSP tool is unavailable. Configure ~/.kigi/lsp.json or <cwd>/.kigi/lsp.json and ensure the language server can start.",
)
})?
.clone();
}
let result = handle.dispatch(&input).await;
if result.is_error {
Err(kigi_tool_runtime::ToolError::custom(
"process_manager",
result.text,
))
} else {
Ok(LspToolOutput(result.text))
}
}
}
@@ -0,0 +1,67 @@
//! New-architecture tool implementations (NewTool trait).
//!
//! Each sub-module here contains a tool that implements `NewTool` instead
//! of the old `Tool` trait. During migration, old implementations live in
//! `implementations/<tool>/` and new implementations live in
//! `implementations/grok_build/<tool>/`.
//!
//! The [`register_all()`] function is the single entry-point for wiring up
//! the standard toolset. It inserts shared resources (`Terminal`,
//! `AvailableSkills`, `BashParams`) and registers every built-in tool.
pub mod ask_user_question;
pub mod bash;
#[path = "deploy_app_stub.rs"]
pub mod deploy_app;
pub mod enter_plan_mode;
pub mod exit_plan_mode;
pub mod grep;
pub mod image_edit;
pub mod image_gen;
pub mod kill_task;
pub mod list_dir;
pub mod lsp;
pub mod monitor;
pub mod read_file;
pub mod scheduler;
pub mod search_replace;
pub(crate) mod storage;
pub mod task;
pub mod task_output;
pub mod todo;
pub mod update_goal;
pub mod video_gen;
pub mod web_fetch;
pub mod web_search;
pub use ask_user_question::AskUserQuestionTool;
pub use bash::BashTool;
pub use deploy_app::{AppBuilderDeployerConfig, DEPLOY_APP_TOOL_NAME};
pub use enter_plan_mode::EnterPlanModeTool;
pub use exit_plan_mode::ExitPlanModeTool;
pub use grep::GrepTool;
pub use image_edit::{IMAGE_EDIT_TOOL_NAME, ImageEditTool};
pub use image_gen::{
IMAGE_GEN_TOOL_NAME, IMAGINE_COMMAND_NAME, ImageGenTool, imagine_instruction,
imagine_usage_message,
};
pub use kill_task::{KillTaskTool, KillTerminalCommandTool};
pub use list_dir::ListDirTool;
pub use lsp::LspTool;
pub use monitor::tool::MonitorTool;
pub use read_file::ReadFileTool;
pub use scheduler::create::{
SCHEDULER_CREATE_TOOL_NAME, SchedulerCreateTool, loop_schedule_instruction, loop_usage_message,
};
pub use scheduler::delete::{SCHEDULER_DELETE_TOOL_NAME, SchedulerDeleteTool};
pub use scheduler::list::SchedulerListTool;
pub use search_replace::SearchReplaceTool;
pub use task::TaskTool;
pub use task_output::{GetTerminalCommandOutputTool, TaskOutputTool, WaitTasksTool};
pub use todo::TodoWriteTool;
pub use update_goal::{UPDATE_GOAL_TOOL_NAME, UpdateGoalTool};
pub use video_gen::{
IMAGE_TO_VIDEO_TOOL_NAME, IMAGINE_VIDEO_COMMAND_NAME, ImageToVideoTool,
REFERENCE_TO_VIDEO_TOOL_NAME, ReferenceToVideoTool, imagine_video_instruction,
imagine_video_usage_message,
};
pub use web_fetch::{WebFetchClient, WebFetchConfig, WebFetchParams, WebFetchTool};
pub use web_search::WebSearchTool;
@@ -0,0 +1,224 @@
use super::types::{BATCH_TRUNCATION_LIMIT, BUFFER_CAP_BYTES, LINE_TRUNCATION_LIMIT};
use crate::util::floor_char_boundary;
/// Processes raw stdout chunks into complete lines.
///
/// Buffers partial lines, splits on `\n`, truncates individual lines at
/// `LINE_TRUNCATION_LIMIT` chars, and caps the internal buffer at `BUFFER_CAP_BYTES`.
#[derive(Default)]
pub struct LineProcessor {
buffer: Vec<u8>,
}
impl LineProcessor {
pub fn new() -> Self {
Self::default()
}
/// Push a raw stdout chunk. Returns any complete lines extracted.
pub fn push(&mut self, chunk: &[u8]) -> Vec<String> {
self.buffer.extend_from_slice(chunk);
// Cap buffer at BUFFER_CAP_BYTES (keep the tail).
if self.buffer.len() > BUFFER_CAP_BYTES {
let start = self.buffer.len() - BUFFER_CAP_BYTES;
self.buffer = self.buffer[start..].to_vec();
}
let mut lines = Vec::new();
while let Some(nl_pos) = self.buffer.iter().position(|&b| b == b'\n') {
let raw = self.buffer.drain(..=nl_pos).collect::<Vec<_>>();
let text = String::from_utf8_lossy(&raw).trim().to_string();
if text.is_empty() {
continue;
}
lines.push(truncate_line(&text));
}
lines
}
/// Flush any remaining partial line from the buffer.
pub fn flush(&mut self) -> Option<String> {
if self.buffer.is_empty() {
return None;
}
let raw = std::mem::take(&mut self.buffer);
let text = String::from_utf8_lossy(&raw).trim().to_string();
if text.is_empty() {
return None;
}
Some(truncate_line(&text))
}
}
fn truncate_line(line: &str) -> String {
if line.len() > LINE_TRUNCATION_LIMIT {
let boundary = floor_char_boundary(line, LINE_TRUNCATION_LIMIT);
format!("{}...(truncated)", &line[..boundary])
} else {
line.to_string()
}
}
/// Batch multiple lines into a single event string, truncating at `BATCH_TRUNCATION_LIMIT`.
pub fn batch_lines(lines: &[String]) -> String {
let joined = lines.join("\n");
if joined.len() > BATCH_TRUNCATION_LIMIT {
let boundary = floor_char_boundary(&joined, BATCH_TRUNCATION_LIMIT);
format!("{}\n...(truncated)", &joined[..boundary])
} else {
joined
}
}
/// Sanitize a model-supplied monitor description for embedding in the
/// `<monitor-event …>` attribute and in line labels: `"` would break the
/// attribute / the parser's `" task_id="` anchor, and newlines would break
/// the single-line opening-tag shape (`>\n` anchor) and label lines.
pub fn sanitize_monitor_description(description: &str) -> String {
description.replace('"', "'").replace(['\n', '\r'], " ")
}
/// Wrap event text in XML tags for the LLM conversation.
pub fn wrap_monitor_event(description: &str, event_text: &str, task_id: &str) -> String {
let description = sanitize_monitor_description(description);
format!(
"<monitor-event description=\"{description}\" task_id=\"{task_id}\">\n\
{event_text}\n\
</monitor-event>"
)
}
#[cfg(test)]
mod tests {
use super::*;
/// Quotes and newlines in the model-supplied description are neutralized
/// before embedding — they would otherwise break the attribute quoting
/// (`" task_id="` anchor) or the single-line opening tag (`>\n` anchor).
#[test]
fn wrap_sanitizes_description() {
let wrapped = wrap_monitor_event("watch \"prod\"\nlogs", "line", "t-1");
assert!(
wrapped
.starts_with("<monitor-event description=\"watch 'prod' logs\" task_id=\"t-1\">"),
"{wrapped}"
);
}
#[test]
fn push_single_line() {
let mut proc = LineProcessor::new();
let lines = proc.push(b"hello world\n");
assert_eq!(lines, vec!["hello world"]);
}
#[test]
fn push_multiple_lines() {
let mut proc = LineProcessor::new();
let lines = proc.push(b"line1\nline2\nline3\n");
assert_eq!(lines, vec!["line1", "line2", "line3"]);
}
#[test]
fn partial_line_buffered() {
let mut proc = LineProcessor::new();
let lines = proc.push(b"partial");
assert!(lines.is_empty());
let lines = proc.push(b" line\n");
assert_eq!(lines, vec!["partial line"]);
}
#[test]
fn empty_lines_skipped() {
let mut proc = LineProcessor::new();
let lines = proc.push(b"hello\n\n\nworld\n");
assert_eq!(lines, vec!["hello", "world"]);
}
#[test]
fn long_line_truncated() {
let mut proc = LineProcessor::new();
let long = "x".repeat(600);
let input = format!("{long}\n");
let lines = proc.push(input.as_bytes());
assert_eq!(lines.len(), 1);
assert!(lines[0].ends_with("...(truncated)"));
assert!(lines[0].len() < 600);
}
#[test]
fn buffer_cap_enforced() {
let mut proc = LineProcessor::new();
// Push more than BUFFER_CAP_BYTES without newlines
let big = vec![b'a'; BUFFER_CAP_BYTES + 1000];
proc.push(&big);
assert!(proc.buffer.len() <= BUFFER_CAP_BYTES);
}
#[test]
fn flush_returns_partial() {
let mut proc = LineProcessor::new();
proc.push(b"no newline");
let flushed = proc.flush();
assert_eq!(flushed, Some("no newline".to_string()));
}
#[test]
fn flush_empty_returns_none() {
let mut proc = LineProcessor::new();
assert!(proc.flush().is_none());
}
#[test]
fn batch_lines_joins() {
let lines = vec!["line1".into(), "line2".into(), "line3".into()];
assert_eq!(batch_lines(&lines), "line1\nline2\nline3");
}
#[test]
fn batch_lines_truncates_at_limit() {
let long_line = "x".repeat(2000);
let lines = vec![long_line.clone(), long_line];
let batched = batch_lines(&lines);
assert!(batched.ends_with("...(truncated)"));
assert!(batched.len() < 5000);
}
#[test]
fn truncate_line_multibyte_no_panic() {
// 3-byte UTF-8 chars — truncation boundary may land mid-char
let line = "\u{4e16}\u{754c}".repeat(200); // CJK chars, 3 bytes each
let truncated = truncate_line(&line);
assert!(truncated.ends_with("...(truncated)"));
// Verify the result is valid UTF-8 (would panic if not)
let _ = truncated.as_bytes();
}
#[test]
fn truncate_line_emoji_no_panic() {
// 4-byte emoji chars
let line = "\u{1F600}".repeat(200);
let truncated = truncate_line(&line);
assert!(truncated.ends_with("...(truncated)"));
}
#[test]
fn batch_lines_multibyte_no_panic() {
let long_line = "\u{4e16}\u{754c}".repeat(1500);
let lines = vec![long_line];
let batched = batch_lines(&lines);
assert!(batched.ends_with("...(truncated)"));
}
#[test]
fn xml_wrapping() {
let result = wrap_monitor_event("errors in log", "ERROR: disk full", "task-123");
assert_eq!(
result,
"<monitor-event description=\"errors in log\" task_id=\"task-123\">\n\
ERROR: disk full\n\
</monitor-event>"
);
}
}
@@ -0,0 +1,4 @@
pub mod event;
pub mod rate_limiter;
pub mod tool;
pub mod types;
@@ -0,0 +1,275 @@
use std::time::{Duration, Instant};
use super::types::{AUTO_KILL_THRESHOLD_MS, RATE_LIMIT_REFILL_MS};
/// Token bucket rate limiter.
///
/// Starts full at `capacity` tokens. Each `try_consume()` takes one token.
/// Tokens refill at 1 per `refill_interval_ms`.
pub struct TokenBucket {
capacity: u32,
tokens: u32,
refill_interval: Duration,
last_refill: Instant,
}
impl TokenBucket {
pub fn new(capacity: u32, refill_interval_ms: u64) -> Self {
Self {
capacity,
tokens: capacity,
refill_interval: Duration::from_millis(refill_interval_ms),
last_refill: Instant::now(),
}
}
/// Try to consume one token. Returns true if a token was available.
pub fn try_consume(&mut self) -> bool {
let now = Instant::now();
let elapsed = now.duration_since(self.last_refill);
let refills = (elapsed.as_millis() / self.refill_interval.as_millis()) as u32;
if refills > 0 {
self.tokens = self.tokens.saturating_add(refills).min(self.capacity);
self.last_refill += self.refill_interval * refills;
}
if self.tokens > 0 {
self.tokens -= 1;
true
} else {
false
}
}
}
/// Tracks rate-limit suppression state and auto-kill logic.
///
/// Used alongside `TokenBucket` to detect sustained overload and generate
/// catch-up notices when the rate subsides.
#[derive(Default)]
pub struct SuppressionTracker {
pub suppressed_count: u64,
pub last_suppression: Option<Instant>,
pub suppression_start: Option<Instant>,
pub killed: bool,
/// Resolved model-facing name for the kill tool, used in suppression notices.
kill_tool_name: String,
}
/// Result of processing an event through the rate limiter + suppression tracker.
pub enum RateLimitOutcome {
/// Event is allowed through. If `catch_up_notice` is Some, a suppression
/// notice should be sent before the event.
Allowed { catch_up_notice: Option<String> },
/// Event is suppressed (token bucket empty).
Suppressed,
/// Monitor should be auto-killed (sustained overload for 30s+).
AutoKill { message: String },
}
impl SuppressionTracker {
pub fn new() -> Self {
Self::default()
}
pub fn with_kill_tool_name(mut self, name: String) -> Self {
self.kill_tool_name = name;
self
}
/// Process a rate-limit decision. Call after `TokenBucket::try_consume()`.
pub fn process(&mut self, token_available: bool, _description: &str) -> RateLimitOutcome {
if self.killed {
return RateLimitOutcome::Suppressed;
}
if token_available {
let catch_up = if self.suppressed_count > 0 {
let kill_name = if self.kill_tool_name.is_empty() {
"kill_command_or_subagent"
} else {
&self.kill_tool_name
};
let notice = format!(
"[{} events suppressed -- output rate too high. \
Consider using {} to restart this monitor \
with a more selective filter.]",
self.suppressed_count, kill_name
);
self.suppressed_count = 0;
// Reset suppression start if the burst has subsided
// (> 3x refill interval since last suppression).
if let Some(last) = self.last_suppression
&& last.elapsed() > Duration::from_millis(RATE_LIMIT_REFILL_MS * 3)
{
self.suppression_start = None;
}
Some(notice)
} else {
None
};
RateLimitOutcome::Allowed {
catch_up_notice: catch_up,
}
} else {
self.suppressed_count += 1;
self.last_suppression = Some(Instant::now());
if self.suppression_start.is_none() {
self.suppression_start = Some(Instant::now());
}
// Check auto-kill threshold.
if let Some(start) = self.suppression_start {
let elapsed = start.elapsed();
if elapsed > Duration::from_millis(AUTO_KILL_THRESHOLD_MS) {
self.killed = true;
let secs = elapsed.as_secs();
return RateLimitOutcome::AutoKill {
message: format!(
"[Monitor stopped -- your script produced too much output \
({} events suppressed over {secs}s). \
Write a new monitor command that filters more aggressively -- \
pipe through grep --line-buffered, awk, or a wrapper script \
that only emits the specific events you need.]",
self.suppressed_count
),
};
}
}
RateLimitOutcome::Suppressed
}
}
}
/// Combined rate limiter: token bucket + suppression tracker.
pub struct MonitorRateLimiter {
pub bucket: TokenBucket,
pub suppression: SuppressionTracker,
}
impl MonitorRateLimiter {
pub fn new(capacity: u32, refill_interval_ms: u64) -> Self {
Self {
bucket: TokenBucket::new(capacity, refill_interval_ms),
suppression: SuppressionTracker::new(),
}
}
pub fn with_kill_tool_name(mut self, name: String) -> Self {
self.suppression = self.suppression.with_kill_tool_name(name);
self
}
/// Process an event. Returns the rate limit decision.
pub fn process_event(&mut self, description: &str) -> RateLimitOutcome {
let available = self.bucket.try_consume();
self.suppression.process(available, description)
}
pub fn is_killed(&self) -> bool {
self.suppression.killed
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn bucket_starts_full() {
let mut bucket = TokenBucket::new(10, 2000);
for _ in 0..10 {
assert!(bucket.try_consume());
}
assert!(!bucket.try_consume());
}
#[test]
fn bucket_refills_after_interval() {
let mut bucket = TokenBucket::new(10, 50); // 50ms for test speed
for _ in 0..10 {
bucket.try_consume();
}
assert!(!bucket.try_consume());
std::thread::sleep(Duration::from_millis(60));
assert!(bucket.try_consume()); // one token refilled
}
#[test]
fn bucket_does_not_exceed_capacity() {
let mut bucket = TokenBucket::new(3, 50);
std::thread::sleep(Duration::from_millis(200)); // enough for many refills
// Should be capped at 3
assert!(bucket.try_consume());
assert!(bucket.try_consume());
assert!(bucket.try_consume());
assert!(!bucket.try_consume());
}
#[test]
fn suppression_tracker_counts() {
let mut tracker = SuppressionTracker::new();
let outcome = tracker.process(false, "test");
assert!(matches!(outcome, RateLimitOutcome::Suppressed));
assert_eq!(tracker.suppressed_count, 1);
}
#[test]
fn catch_up_notice_on_recovery() {
let mut tracker = SuppressionTracker::new();
// Suppress some events
tracker.process(false, "test");
tracker.process(false, "test");
tracker.process(false, "test");
assert_eq!(tracker.suppressed_count, 3);
// Now a token is available
let outcome = tracker.process(true, "test");
match outcome {
RateLimitOutcome::Allowed { catch_up_notice } => {
let notice = catch_up_notice.expect("should have catch-up notice");
assert!(notice.contains("3 events suppressed"));
}
_ => panic!("expected Allowed with catch-up notice"),
}
assert_eq!(tracker.suppressed_count, 0);
}
#[test]
fn no_catch_up_when_no_suppression() {
let mut tracker = SuppressionTracker::new();
let outcome = tracker.process(true, "test");
match outcome {
RateLimitOutcome::Allowed { catch_up_notice } => {
assert!(catch_up_notice.is_none());
}
_ => panic!("expected Allowed without catch-up"),
}
}
#[test]
fn killed_discards_events() {
let mut tracker = SuppressionTracker::new();
tracker.killed = true;
let outcome = tracker.process(true, "test");
assert!(matches!(outcome, RateLimitOutcome::Suppressed));
}
#[test]
fn combined_rate_limiter() {
let mut rl = MonitorRateLimiter::new(3, 2000);
// First 3 events pass
for _ in 0..3 {
assert!(matches!(
rl.process_event("test"),
RateLimitOutcome::Allowed { .. }
));
}
// 4th is suppressed
assert!(matches!(
rl.process_event("test"),
RateLimitOutcome::Suppressed
));
}
}
@@ -0,0 +1,641 @@
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::Mutex;
use crate::notification::MonitorEvent;
use crate::notification::types::ToolNotificationHandle;
use crate::types::requirements::Expr;
use crate::types::resources::Terminal;
use crate::types::tool::{ToolKind, ToolNamespace};
use super::event::{self, LineProcessor};
use super::rate_limiter::{MonitorRateLimiter, RateLimitOutcome};
use super::types::{MonitorInput, MonitorOutput, RATE_LIMIT_CAPACITY, RATE_LIMIT_REFILL_MS};
#[derive(Debug, Default)]
pub struct MonitorTool;
impl crate::types::tool_metadata::ToolMetadata for MonitorTool {
fn kind(&self) -> ToolKind {
ToolKind::Monitor
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
r#"Start a background monitor that streams events from a long-running script. Each stdout line is an event - you can keep working and notifications arrive in the chat. Exit ends the watch.
**Output volume**: Every stdout line becomes a message in the conversation, so write selective filters. In pipes use `grep --line-buffered` (plain `grep` buffers and delays events by minutes).
Set `persistent: true` for session-length watches (PR monitoring, log tails) -- the monitor runs${%- if tools.by_kind.kill_task_action %} until you call ${{ tools.by_kind.kill_task_action }} or${%- endif %} until the session ends. Otherwise it stops at `timeout_ms` (default 10h)."#
}
fn emitted_notifications(&self) -> &'static [&'static str] {
&["BashExecutionBackgrounded", "MonitorEvent", "TaskCompleted"]
}
fn requires_expr(&self) -> Expr<crate::types::requirements::ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for MonitorTool {
type Args = MonitorInput;
type Output = MonitorOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("monitor").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"monitor",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(name = "tool.monitor", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: MonitorInput,
) -> Result<MonitorOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
input
.validate()
.map_err(|e| kigi_tool_runtime::ToolError::invalid_arguments(e.to_string()))?;
let resolved_timeout = input.resolved_timeout_ms();
let description = input.description.clone();
let (terminal, notification_handle, cwd, session_folder, owner_session_id) = {
let res = resources.lock().await;
let terminal = res.require::<Terminal>()?.0.clone();
let notif = res
.get::<crate::types::resources::NotificationHandle>()
.map(|h| h.0.clone())
.unwrap_or_default();
let cwd = res
.get::<crate::types::resources::Cwd>()
.map(|c| c.0.clone())
.unwrap_or_else(|| std::path::PathBuf::from("."));
let session_folder = res
.require::<crate::types::resources::SessionFolder>()?
.0
.clone();
let owner = res
.get::<crate::types::resources::OwnerSessionId>()
.map(|o| o.0.clone());
(terminal, notif, cwd, session_folder, owner)
};
// Output file lives in the session folder alongside bash terminal logs.
let output_file = session_folder
.join("terminal")
.join(format!("monitor-{}.log", ctx.call_id.as_str()));
let bg_handle = terminal
.run_background(crate::computer::types::TerminalRunRequest {
command: input.command.clone(),
working_directory: cwd.clone(),
env: std::collections::HashMap::from([(
"PYTHONUNBUFFERED".to_string(),
"1".to_string(),
)]),
timeout: if resolved_timeout == 0 {
Duration::from_secs(86400 * 365) // long-running (until kill or session end)
} else {
Duration::from_millis(resolved_timeout)
},
output_byte_limit: 10 * 1024 * 1024,
output_file,
notification_handle: notification_handle.clone(),
tool_call_id: ctx.call_id.as_str().to_owned(),
display_command: Some(format!("[monitor] {}", input.description)),
auto_background_on_timeout: false,
foreground_block_budget: None,
kind: crate::computer::types::TaskKind::Monitor,
owner_session_id,
})
.await
.map_err(|e| kigi_tool_runtime::ToolError::custom("process_manager", e.to_string()))?;
let task_id = bg_handle.task_id.clone();
// Notify the pager so the monitor appears in the tasks pane
// (same notification that bash background tasks send).
notification_handle.send_backgrounded(crate::notification::BashExecutionBackgrounded {
base: crate::notification::BashNotificationBase {
tool_call_id: ctx.call_id.as_str().to_owned(),
// Send the real monitor command (so the block viewer shows the
// actual script). The human-readable description travels in
// `monitor_description` so the pager can render a "Monitor" tag
// instead of bash-highlighting a "[monitor] …" pseudo-command.
command: input.command.clone(),
output: Vec::new(),
total_bytes: 0,
truncated: false,
cwd,
},
output_file: bg_handle.output_file.clone(),
task_id: task_id.clone(),
monitor_description: Some(input.description.clone()),
description: None,
});
// Spawn the stdout processing pipeline.
// Reads the output file, processes lines through the rate limiter,
// and emits MonitorEvent notifications.
let pipeline_task_id = task_id.clone();
let pipeline_description = description.clone();
// Weak handle: the pipeline must not keep the session's terminal backend
// (and the monitored process) alive past session end. See
// `run_monitor_pipeline`.
let pipeline_terminal = std::sync::Arc::downgrade(&terminal);
let pipeline_notif = notification_handle.clone();
let pipeline_output_file = bg_handle.output_file;
// Resolve the kill tool name before spawning the pipeline (resources
// can't cross the spawn boundary).
let kill_tool_name = crate::types::template_renderer::TemplateRenderer::resolve_tool_name(
&resources,
crate::types::tool::ToolKind::KillTaskAction,
)
.await;
let pipeline_kill_name = kill_tool_name.clone();
tokio::spawn(async move {
run_monitor_pipeline(
&pipeline_task_id,
&pipeline_description,
pipeline_terminal,
&pipeline_notif,
&pipeline_output_file,
pipeline_kill_name,
0, // fresh pipeline — read from start
)
.await;
});
let kill_tool_display =
kill_tool_name.unwrap_or_else(|| "kill_command_or_subagent".to_string());
let result_message = if resolved_timeout == 0 {
format!(
"Monitor started (task {task_id}, persistent -- runs until {kill_tool_display} or session end).\n\
You will be notified on each event. Keep working -- do not poll or sleep.\n\
Events may arrive while you are waiting for the user -- an event is not their reply."
)
} else {
format!(
"Monitor started (task {task_id}, timeout {resolved_timeout}ms).\n\
You will be notified on each event. Keep working -- do not poll or sleep.\n\
Events may arrive while you are waiting for the user -- an event is not their reply."
)
};
tracing::info!(
task_id = %task_id,
resolved_timeout,
result_message = %result_message,
"Monitor started"
);
Ok(MonitorOutput {
task_id,
timeout_ms: resolved_timeout,
persistent: resolved_timeout == 0,
})
}
}
/// Background pipeline: polls the task output and feeds lines through
/// the processing pipeline (line processor -> rate limiter -> XML wrap -> notification).
///
/// `pub(crate)` so `reparent_notifications` can re-spawn the pipeline on the
/// parent's runtime when a subagent exits and its monitors are reparented.
///
/// Holds the backend as a [`Weak`](std::sync::Weak), never a strong `Arc`: a
/// persistent monitor loops for the session's whole lifetime, so a strong ref
/// would pin the terminal actor (and its process) and leak it across sessions
/// on shared-runtime hosts (hosts that build one backend per session on a
/// shared runtime). With a `Weak` the pipeline stops once the session drops
/// its backend, letting `shutdown_all()` reap the process.
pub(crate) async fn run_monitor_pipeline(
task_id: &str,
description: &str,
terminal: std::sync::Weak<dyn crate::computer::types::TerminalBackend>,
notification_handle: &ToolNotificationHandle,
output_file: &std::path::Path,
kill_tool_name: Option<String>,
// Starting file offset. Pass 0 for fresh pipelines or the current file
// size when re-spawning after reparent to avoid duplicate events.
start_offset: u64,
) {
let rate_limiter = Arc::new(Mutex::new(
MonitorRateLimiter::new(RATE_LIMIT_CAPACITY, RATE_LIMIT_REFILL_MS).with_kill_tool_name(
kill_tool_name.unwrap_or_else(|| "kill_command_or_subagent".to_string()),
),
));
let mut line_processor = LineProcessor::new();
let mut last_read_offset: u64 = start_offset;
let mut last_owner: Option<String> = None;
loop {
// Strong handle for this tick only; `None` means the session dropped
// the backend (torn down with the session), so stop streaming.
let Some(terminal) = terminal.upgrade() else {
break;
};
// Check if the task is still running.
let snapshot = terminal.get_task(task_id).await;
let completed = snapshot.is_none() || snapshot.as_ref().is_some_and(|s| s.completed);
// Re-read each tick (a reparented monitor follows its new owner); keep
// the last seen owner so an evicted-snapshot terminal event still routes.
let snapshot_owner = snapshot.as_ref().and_then(|s| s.owner_session_id.clone());
if snapshot_owner.is_some() {
last_owner.clone_from(&snapshot_owner);
}
let owner_session_id = snapshot_owner.or_else(|| last_owner.clone());
// Read new output from the file.
let new_bytes = read_new_bytes(output_file, &mut last_read_offset).await;
if !new_bytes.is_empty() {
let lines = line_processor.push(&new_bytes);
if !lines.is_empty() {
let batched = event::batch_lines(&lines);
process_event(
task_id,
description,
&batched,
&rate_limiter,
notification_handle,
owner_session_id.as_deref(),
)
.await;
}
}
if completed {
// Flush any remaining partial line.
if let Some(remaining) = line_processor.flush() {
process_event(
task_id,
description,
&remaining,
&rate_limiter,
notification_handle,
owner_session_id.as_deref(),
)
.await;
}
// Do NOT emit a terminal `[monitor ended: …]` MonitorEvent here.
// Natural exit auto-wakes via `TaskCompleted` → immediate Prompt
// (`format_monitor_completion` in the notification bridge). Emitting
// a terminal event as well produced a second NotificationDrain turn
// with the same ended signal. Stdout lines above still stream as
// events while the process is alive; the UI learns completion from
// `x.ai/task_completed`.
break;
}
if rate_limiter.lock().await.is_killed() {
break;
}
// Drop the strong handle so only the `Weak` is held between ticks.
drop(terminal);
tokio::time::sleep(Duration::from_millis(super::types::DEBOUNCE_MS)).await;
}
}
async fn read_new_bytes(path: &std::path::Path, offset: &mut u64) -> Vec<u8> {
use tokio::io::AsyncReadExt;
let Ok(mut file) = tokio::fs::File::open(path).await else {
return Vec::new();
};
let Ok(metadata) = file.metadata().await else {
return Vec::new();
};
let file_len = metadata.len();
if file_len <= *offset {
return Vec::new();
}
use tokio::io::AsyncSeekExt;
if file.seek(std::io::SeekFrom::Start(*offset)).await.is_err() {
return Vec::new();
}
let to_read = (file_len - *offset) as usize;
let mut buf = vec![0u8; to_read.min(1024 * 1024)]; // cap single read at 1MB
let Ok(n) = file.read(&mut buf).await else {
return Vec::new();
};
buf.truncate(n);
*offset += n as u64;
buf
}
async fn process_event(
task_id: &str,
description: &str,
event_text: &str,
rate_limiter: &Arc<Mutex<MonitorRateLimiter>>,
notification_handle: &ToolNotificationHandle,
owner_session_id: Option<&str>,
) {
let owner = || owner_session_id.map(str::to_string);
let mut rl = rate_limiter.lock().await;
match rl.process_event(description) {
RateLimitOutcome::Allowed { catch_up_notice } => {
if let Some(notice) = catch_up_notice {
let wrapped = event::wrap_monitor_event(description, &notice, task_id);
notification_handle.send_monitor_event(MonitorEvent {
task_id: task_id.to_string(),
description: description.to_string(),
event_text: wrapped,
raw_text: notice,
owner_session_id: owner(),
});
}
let wrapped = event::wrap_monitor_event(description, event_text, task_id);
notification_handle.send_monitor_event(MonitorEvent {
task_id: task_id.to_string(),
description: description.to_string(),
event_text: wrapped,
raw_text: event_text.to_string(),
owner_session_id: owner(),
});
}
RateLimitOutcome::Suppressed => {
// Silently dropped.
}
RateLimitOutcome::AutoKill { message } => {
let wrapped = event::wrap_monitor_event(description, &message, task_id);
notification_handle.send_monitor_event(MonitorEvent {
task_id: task_id.to_string(),
description: description.to_string(),
event_text: wrapped,
raw_text: message,
owner_session_id: owner(),
});
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::computer::local::LocalTerminalBackend;
use crate::computer::types::{TaskKind, TerminalBackend, TerminalRunRequest};
use std::time::Duration;
/// A persistent monitor must not keep the session's terminal backend (and
/// thus the monitored process) alive after the session releases its handle.
///
/// Regression for the cross-session monitor leak: the monitor pipeline held
/// a strong `Arc<dyn TerminalBackend>`, so on a shared runtime (hosts that
/// build one `LocalTerminalBackend` per session on one long-lived
/// multi-threaded runtime) a persistent monitor's pipeline kept the actor's
/// command channel open forever. The monitored process, the terminal actor,
/// and the pipeline all leaked once the session ended. The interactive CLI
/// only masked this because each session owns a dedicated thread+runtime
/// that is torn down on session exit.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn persistent_monitor_released_when_session_drops_backend() {
let tmp = tempfile::tempdir().unwrap();
let output_file = tmp.path().join("monitor.log");
// Session-owned terminal backend (mirrors multi-session hosts that
// build one LocalTerminalBackend per session on a shared runtime).
let backend: Arc<dyn TerminalBackend> = Arc::new(LocalTerminalBackend::new());
let weak = Arc::downgrade(&backend);
// Start a persistent monitor: a process that effectively never exits.
let handle = backend
.run_background(TerminalRunRequest {
command: "while true; do echo tick; sleep 0.1; done".to_string(),
working_directory: tmp.path().to_path_buf(),
env: std::collections::HashMap::new(),
timeout: Duration::from_secs(86_400 * 365),
output_byte_limit: 10 * 1024 * 1024,
output_file: output_file.clone(),
notification_handle: ToolNotificationHandle::noop(),
tool_call_id: "tc-monitor".to_string(),
display_command: Some("[monitor] tick".to_string()),
auto_background_on_timeout: false,
foreground_block_budget: None,
kind: TaskKind::Monitor,
owner_session_id: Some("session-A".to_string()),
})
.await
.expect("spawn monitor");
let task_id = handle.task_id.clone();
// Spawn the streaming pipeline exactly like `MonitorTool::run`.
let pipeline_terminal = Arc::downgrade(&backend);
let pipeline_output = output_file.clone();
let pipeline_task_id = task_id.clone();
tokio::spawn(async move {
run_monitor_pipeline(
&pipeline_task_id,
"tick",
pipeline_terminal,
&ToolNotificationHandle::noop(),
&pipeline_output,
None,
0,
)
.await;
});
// The monitor is running and visible.
tokio::time::sleep(Duration::from_millis(300)).await;
assert!(
backend.get_task(&task_id).await.is_some(),
"monitor task should be running before the session ends"
);
// Session ends: drop every session-owned reference to the backend.
// Only the pipeline's (weak) reference remains.
drop(handle);
drop(backend);
// The backend — and the monitored process it owns — must be reclaimed
// promptly. If the pipeline holds a strong `Arc`, `weak.upgrade()` never
// returns `None` and the actor's `shutdown_all` never reaps the process.
let mut reclaimed = false;
for _ in 0..50 {
tokio::time::sleep(Duration::from_millis(100)).await;
if weak.upgrade().is_none() {
reclaimed = true;
break;
}
}
assert!(
reclaimed,
"monitor pipeline leaked the terminal backend past session end"
);
}
/// On exit the pipeline must NOT emit a terminal `[monitor ended]` event
/// (wake is owned by `TaskCompleted` auto-wake). Stdout lines still stream
/// as MonitorEvents with the owner stamp so mid-run ticks reach the right
/// session.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn monitor_exit_does_not_emit_terminal_ended_event() {
let tmp = tempfile::tempdir().unwrap();
let output_file = tmp.path().join("monitor.log");
let backend: Arc<dyn TerminalBackend> = Arc::new(LocalTerminalBackend::new());
let handle = backend
.run_background(TerminalRunRequest {
command: "echo done".to_string(),
working_directory: tmp.path().to_path_buf(),
env: std::collections::HashMap::new(),
timeout: Duration::from_secs(60),
output_byte_limit: 10 * 1024 * 1024,
output_file: output_file.clone(),
notification_handle: ToolNotificationHandle::noop(),
tool_call_id: "tc-exit".to_string(),
display_command: Some("[monitor] done".to_string()),
auto_background_on_timeout: false,
foreground_block_budget: None,
kind: TaskKind::Monitor,
owner_session_id: Some("session-A".to_string()),
})
.await
.expect("spawn monitor");
let task_id = handle.task_id.clone();
tokio::time::sleep(Duration::from_millis(200)).await;
let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel();
let capture = ToolNotificationHandle::from_sender(tx);
run_monitor_pipeline(
&task_id,
"done",
Arc::downgrade(&backend),
&capture,
&output_file,
None,
0,
)
.await;
let mut saw_stdout = false;
while let Ok(n) = rx.try_recv() {
if let crate::notification::types::ToolNotification::MonitorEvent(e) = n {
assert!(
!e.raw_text.contains("monitor ended"),
"terminal ended event must not be emitted (TaskCompleted owns wake): {}",
e.raw_text
);
if e.raw_text.contains("done") {
saw_stdout = true;
assert_eq!(
e.owner_session_id.as_deref(),
Some("session-A"),
"stdout events must still carry the owner"
);
}
}
}
assert!(
saw_stdout,
"stdout line from the monitor command must still stream as a MonitorEvent"
);
}
/// When a subagent dies, its monitor is reparented to the parent: the owner
/// flips to the parent and the pipeline is re-spawned on the parent's
/// handle. The re-spawned pipeline must stamp the PARENT owner so the
/// parent's bridge delivers the events instead of dropping them.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn reparented_monitor_emits_events_with_parent_owner() {
let tmp = tempfile::tempdir().unwrap();
let output_file = tmp.path().join("monitor.log");
let backend: Arc<dyn TerminalBackend> = Arc::new(LocalTerminalBackend::new());
// Monitor owned by the (soon-dead) child session, emitting over time.
let handle = backend
.run_background(TerminalRunRequest {
command: "for i in $(seq 1 50); do echo tick $i; sleep 0.1; done".to_string(),
working_directory: tmp.path().to_path_buf(),
env: std::collections::HashMap::new(),
timeout: Duration::from_secs(60),
output_byte_limit: 10 * 1024 * 1024,
output_file: output_file.clone(),
notification_handle: ToolNotificationHandle::noop(),
tool_call_id: "tc-reparent".to_string(),
display_command: Some("[monitor] tick".to_string()),
auto_background_on_timeout: false,
foreground_block_budget: None,
kind: TaskKind::Monitor,
owner_session_id: Some("child-session".to_string()),
})
.await
.expect("spawn monitor");
let task_id = handle.task_id.clone();
// Subagent dies -> reparent its monitor to the parent. This flips the
// owner and re-spawns the pipeline on the parent's capture handle.
let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel();
let parent_handle = ToolNotificationHandle::from_sender(tx);
backend
.reparent_notifications(
"child-session",
"parent-session",
parent_handle,
Arc::downgrade(&backend),
)
.await;
// The re-spawned pipeline streams post-reparent ticks, all of which must
// carry the parent owner.
let mut saw_parent_owned_event = false;
for _ in 0..50 {
tokio::time::sleep(Duration::from_millis(100)).await;
while let Ok(n) = rx.try_recv() {
if let crate::notification::types::ToolNotification::MonitorEvent(e) = n {
assert_eq!(
e.owner_session_id.as_deref(),
Some("parent-session"),
"reparented monitor events must carry the parent owner"
);
saw_parent_owned_event = true;
}
}
if saw_parent_owned_event {
break;
}
}
assert!(
saw_parent_owned_event,
"re-spawned pipeline must emit monitor events to the parent after reparent"
);
backend.kill_task(&task_id).await;
}
}
@@ -0,0 +1,170 @@
/// Max characters per individual stdout line before truncation.
pub const LINE_TRUNCATION_LIMIT: usize = 500;
/// Max characters per batched event (multiple lines joined).
pub const BATCH_TRUNCATION_LIMIT: usize = 3_000;
/// Raw stdout buffer cap in bytes.
pub const BUFFER_CAP_BYTES: usize = 1_048_576; // 1 MB
/// Debounce window for batching concurrent stdout lines (ms).
pub const DEBOUNCE_MS: u64 = 200;
/// Token bucket capacity.
pub const RATE_LIMIT_CAPACITY: u32 = 10;
/// Token bucket refill interval in milliseconds.
pub const RATE_LIMIT_REFILL_MS: u64 = 2_000;
/// Auto-kill after this many ms of continuous rate-limit violations.
pub const AUTO_KILL_THRESHOLD_MS: u64 = 30_000;
/// Default monitor timeout (non-persistent). 10 hours to avoid short
/// unexpected cutoffs for monitors the model starts without an explicit deadline.
pub const DEFAULT_TIMEOUT_MS: u64 = 36_000_000; // 10 hours
/// Maximum monitor timeout.
pub const MAX_TIMEOUT_MS: u64 = 36_000_000; // 10 hours
/// Max result size for the tool_result response.
pub const MAX_RESULT_SIZE_CHARS: usize = 10_000;
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct MonitorInput {
/// Shell command or script. Each stdout line is an event; exit ends the watch.
#[schemars(
description = "Shell command or script. Each stdout line is an event; exit ends the watch."
)]
pub command: String,
/// Short human-readable description of what you are monitoring (shown in every notification).
#[schemars(
description = "Short human-readable description of what you are monitoring (shown in every notification)."
)]
pub description: String,
/// Kill the monitor after this deadline (ms). Ignored when persistent is true.
/// Default: 36000000 (10 hr). Max: 36000000 (10 hr).
#[serde(default)]
#[schemars(
description = "Kill the monitor after this deadline (ms). Default: 36000000 (10 hr)."
)]
pub timeout_ms: Option<u64>,
/// Run for the lifetime of the session (no timeout). Use for session-length watches.
/// Stop with kill_command_or_subagent.
#[serde(
default,
deserialize_with = "crate::types::schema::deserialize_lenient_option_bool"
)]
#[schemars(
description = "Run for the lifetime of the session (no timeout).${%- if tools.by_kind.kill_task_action %} Stop with ${{ tools.by_kind.kill_task_action }}.${%- endif %}"
)]
pub persistent: Option<bool>,
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
#[serde(rename_all = "camelCase")]
pub struct MonitorOutput {
/// ID of the background monitor task (used with kill_command_or_subagent to cancel).
pub task_id: String,
/// Timeout deadline in milliseconds. 0 when persistent.
pub timeout_ms: u64,
/// Whether the monitor runs until kill_command_or_subagent or session end.
pub persistent: bool,
}
impl kigi_tool_runtime::ToolOutput for MonitorOutput {}
#[derive(thiserror::Error, Debug)]
pub enum MonitorError {
#[error("persistent must be true when timeout_ms exceeds {MAX_TIMEOUT_MS}ms")]
TimeoutExceedsMax,
}
impl MonitorInput {
/// Validate input constraints.
pub fn validate(&self) -> Result<(), MonitorError> {
let persistent = self.persistent.unwrap_or(false);
if let Some(timeout) = self.timeout_ms
&& !persistent
&& timeout > MAX_TIMEOUT_MS
{
return Err(MonitorError::TimeoutExceedsMax);
}
Ok(())
}
/// Resolved timeout in milliseconds (0 for persistent / no-deadline monitors).
pub fn resolved_timeout_ms(&self) -> u64 {
if self.persistent.unwrap_or(false) {
0
} else {
self.timeout_ms.unwrap_or(DEFAULT_TIMEOUT_MS)
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn default_timeout_is_10h() {
let input = MonitorInput {
command: "tail -f log".into(),
description: "watch log".into(),
timeout_ms: None,
persistent: None,
};
assert_eq!(input.resolved_timeout_ms(), DEFAULT_TIMEOUT_MS);
assert!(input.validate().is_ok());
assert_eq!(DEFAULT_TIMEOUT_MS, 36_000_000);
}
#[test]
fn persistent_has_zero_timeout() {
let input = MonitorInput {
command: "tail -f log".into(),
description: "watch log".into(),
timeout_ms: None,
persistent: Some(true),
};
assert_eq!(input.resolved_timeout_ms(), 0);
assert!(input.validate().is_ok());
}
#[test]
fn explicit_timeout_within_max() {
let input = MonitorInput {
command: "cmd".into(),
description: "desc".into(),
timeout_ms: Some(600_000),
persistent: None,
};
assert_eq!(input.resolved_timeout_ms(), 600_000);
assert!(input.validate().is_ok());
}
#[test]
fn timeout_exceeds_max_without_persistent_fails() {
let input = MonitorInput {
command: "cmd".into(),
description: "desc".into(),
timeout_ms: Some(MAX_TIMEOUT_MS + 1),
persistent: Some(false),
};
assert!(input.validate().is_err());
}
#[test]
fn timeout_exceeds_max_with_persistent_ok() {
let input = MonitorInput {
command: "cmd".into(),
description: "desc".into(),
timeout_ms: Some(MAX_TIMEOUT_MS + 1),
persistent: Some(true),
};
assert!(input.validate().is_ok());
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,26 @@
//! Legacy (0.4.10) behavior for `read_file`.
//!
//! Centralizes all version-specific policy decisions for legacy-0.4.10:
//! - Generic error message for all filesystem failures (no structured variants)
//! - No gitignore enforcement (gitignored files are readable)
//!
//! The main `run_read_file()` flow calls these helpers to make version-specific
//! decisions. The execution path stays in `mod.rs`; the policy lives here.
use std::path::Path;
/// Exact historical read failure message for `read_file` in legacy-0.4.10.
///
/// Captured from the historical 0.4.10 implementation.
///
/// Historical 0.4.10 collapsed filesystem read failures (missing file,
/// directory path, permission denied, etc.) into the same generic message
/// without appending OS error detail.
pub(crate) fn render_read_error(path: &Path) -> String {
format!("Failed to read file: {}", path.display())
}
/// Legacy 0.4.10 does not enforce gitignore — gitignored files are readable.
pub(crate) fn allows_gitignored_reads() -> bool {
true
}
@@ -0,0 +1,8 @@
//! Version-specific behavior modules for `read_file`.
//!
//! - `legacy_0_4_10`: generic error messages, no gitignore enforcement,
//! legacy marker for reminder suppression.
//! - Current behavior remains in `read_file/mod.rs` (structured error
//! variants, gitignore enforcement, confusable reminders).
pub(crate) mod legacy_0_4_10;
@@ -0,0 +1,683 @@
use std::time::Duration;
use chrono::Utc;
use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken;
use crate::notification::types::ToolNotificationHandle;
use crate::notification::{ScheduledTaskCreated, ScheduledTaskFired, ScheduledTaskRemoved};
use crate::types::resources::{SharedResources, State};
use super::interval::interval_to_human;
use super::types::{ScheduledTask, SchedulerCommand, SchedulerError, SchedulerState};
const MAX_SCHEDULED_TASKS: usize = 50;
/// Build a `ScheduledTaskCreated` payload from a task. Shared between the
/// live `SchedulerCommand::Create` path and the post-restore re-announce so
/// the wire format stays in lockstep.
fn task_created_payload(task: &ScheduledTask) -> ScheduledTaskCreated {
ScheduledTaskCreated {
task_id: task.id.clone(),
prompt: task.prompt.clone(),
human_schedule: interval_to_human(task.interval_secs),
next_fire_at: Some(task.next_fire_at().to_rfc3339()),
}
}
pub struct SchedulerActor {
pub(crate) resources: SharedResources,
pub(crate) notification_handle: ToolNotificationHandle,
pub(crate) cmd_rx: mpsc::UnboundedReceiver<SchedulerCommand>,
pub(crate) cancel_token: CancellationToken,
}
impl SchedulerActor {
pub async fn run(mut self) {
self.handle_missed_tasks().await;
self.announce_existing_tasks().await;
loop {
let next_fire = self.compute_next_fire_delay().await;
tokio::select! {
biased;
_ = self.cancel_token.cancelled() => {
tracing::debug!("SchedulerActor shutting down (cancelled)");
break;
}
Some(cmd) = self.cmd_rx.recv() => {
self.handle_command(cmd).await;
}
_ = tokio::time::sleep(next_fire) => {
self.fire_next_task().await;
}
}
}
// Notify the pager to remove UI chips for any remaining tasks on shutdown.
let remaining: Vec<String> = {
let mut res = self.resources.lock().await;
let state = res.get_or_default::<State<SchedulerState>>();
state.tasks.drain(..).map(|t| t.id).collect()
};
for task_id in remaining {
self.notification_handle
.send_scheduled_task_removed(ScheduledTaskRemoved { task_id });
}
}
async fn compute_next_fire_delay(&self) -> Duration {
let res = self.resources.lock().await;
let scheduler_state = res.get::<State<SchedulerState>>();
scheduler_state
.map(|s| {
s.tasks
.iter()
.map(|t| t.next_fire_at())
.min()
.map(|next| {
let now = Utc::now();
if next <= now {
Duration::ZERO
} else {
(next - now).to_std().unwrap_or(Duration::ZERO)
}
})
.unwrap_or(Duration::MAX)
})
.unwrap_or(Duration::MAX)
}
async fn fire_next_task(&mut self) {
let now = Utc::now();
let mut res = self.resources.lock().await;
let state = res.get_or_default::<State<SchedulerState>>();
let idx = state.tasks.iter().position(|t| t.next_fire_at() <= now);
let Some(idx) = idx else {
return;
};
let task = &mut state.tasks[idx];
let task_id = task.id.clone();
let prompt = task.prompt.clone();
let human_schedule = interval_to_human(task.interval_secs);
// Advancing `last_fired_at` pushes `next_fire_at` forward by one
// interval, so the task is not re-selected until it is due again.
// Overlapping prompts are deduped downstream by stable queue-item-id.
task.last_fired_at = Some(now);
let next_fire_at = Some(task.next_fire_at().to_rfc3339());
let should_remove = if !task.recurring {
true
} else {
task.is_expired(now)
};
if should_remove {
state.tasks.remove(idx);
}
// Drop the lock before sending the notification to avoid holding it
// across potentially blocking operations.
drop(res);
tracing::info!(
task_id = %task_id,
schedule = %human_schedule,
"Firing scheduled task"
);
let removed_task_id = should_remove.then(|| task_id.clone());
self.notification_handle
.send_scheduled_task_fired(ScheduledTaskFired {
task_id,
prompt,
human_schedule,
next_fire_at,
});
if let Some(task_id) = removed_task_id {
self.notification_handle
.send_scheduled_task_removed(ScheduledTaskRemoved { task_id });
}
}
async fn handle_missed_tasks(&mut self) {
let now = Utc::now();
let mut res = self.resources.lock().await;
let state = res.get_or_default::<State<SchedulerState>>();
let missed: Vec<(String, String, u64)> = state
.tasks
.iter()
.filter(|t| t.is_missed(now))
.map(|t| (t.id.clone(), t.prompt.clone(), t.interval_secs))
.collect();
if missed.is_empty() {
return;
}
drop(res);
let mut fired_ids = Vec::new();
for (task_id, prompt, interval_secs) in missed {
tracing::info!(task_id = %task_id, "Firing missed one-shot task");
self.notification_handle
.send_scheduled_task_fired(ScheduledTaskFired {
task_id: task_id.clone(),
prompt,
human_schedule: interval_to_human(interval_secs),
next_fire_at: None,
});
fired_ids.push(task_id);
}
let mut res = self.resources.lock().await;
let state = res.get_or_default::<State<SchedulerState>>();
state.tasks.retain(|t| !fired_ids.contains(&t.id));
drop(res);
for task_id in fired_ids {
self.notification_handle
.send_scheduled_task_removed(ScheduledTaskRemoved { task_id });
}
}
/// Re-emit `ScheduledTaskCreated` for every task currently in state.
///
/// Runs after `handle_missed_tasks()` so missed one-shots that were just
/// pruned are not announced as ghost entries. The notification bridge
/// forwards each one to the client exactly like a fresh `Create`, which
/// lets the pager rebuild its `scheduled_tasks` view after a session
/// restore (only `session/update` payloads land in `updates.jsonl`, so the
/// replay path cannot recover these on its own).
async fn announce_existing_tasks(&self) {
let snapshot: Vec<ScheduledTaskCreated> = {
let res = self.resources.lock().await;
let Some(state) = res.get::<State<SchedulerState>>() else {
return;
};
if state.tasks.is_empty() {
return;
}
// Iteration order matters: the pager sorts the tasks pane by
// created_at, but every re-announced task gets a synthetic
// `Instant::now()` on the pager side, so the relative order is
// determined by the order we send notifications here. Keep
// state.tasks as a Vec so insertion order survives.
state.tasks.iter().map(task_created_payload).collect()
};
tracing::info!(
count = snapshot.len(),
"Re-announcing scheduled tasks after restore"
);
for created in snapshot {
self.notification_handle
.send_scheduled_task_created(created);
}
}
async fn handle_command(&mut self, cmd: SchedulerCommand) {
match cmd {
SchedulerCommand::Create { task, reply } => {
let mut res = self.resources.lock().await;
let state = res.get_or_default::<State<SchedulerState>>();
if state.tasks.len() >= MAX_SCHEDULED_TASKS {
let _ = reply.send(Err(SchedulerError::TaskLimitReached(MAX_SCHEDULED_TASKS)));
return;
}
self.notification_handle
.send_scheduled_task_created(task_created_payload(&task));
state.tasks.push(task.clone());
let _ = reply.send(Ok(task));
}
SchedulerCommand::Delete { id, reply } => {
let mut res = self.resources.lock().await;
let state = res.get_or_default::<State<SchedulerState>>();
let before = state.tasks.len();
state.tasks.retain(|t| t.id != id);
let removed = before != state.tasks.len();
drop(res);
let _ = reply.send(removed);
if removed {
self.notification_handle
.send_scheduled_task_removed(ScheduledTaskRemoved { task_id: id });
}
}
SchedulerCommand::List { reply } => {
let res = self.resources.lock().await;
let tasks = res
.get::<State<SchedulerState>>()
.map(|s| s.tasks.clone())
.unwrap_or_default();
let _ = reply.send(tasks);
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::implementations::grok_build::scheduler::types::{ScheduledTask, SchedulerHandle};
use crate::notification::ToolNotification;
use crate::types::resources::Resources;
use std::sync::Arc;
use tokio::sync::Mutex;
fn make_test_actor() -> (
SchedulerHandle,
CancellationToken,
mpsc::UnboundedReceiver<ToolNotification>,
) {
let mut resources = Resources::new();
resources.register_state::<SchedulerState>();
let shared = Arc::new(Mutex::new(resources));
let (notif_handle, notif_rx) = ToolNotificationHandle::channel();
let (cmd_tx, cmd_rx) = mpsc::unbounded_channel();
let cancel_token = CancellationToken::new();
let actor = SchedulerActor {
resources: shared,
notification_handle: notif_handle,
cmd_rx,
cancel_token: cancel_token.clone(),
};
tokio::spawn(actor.run());
(SchedulerHandle(cmd_tx), cancel_token, notif_rx)
}
#[tokio::test]
async fn create_and_list_task() {
let (handle, cancel, _notif_rx) = make_test_actor();
let task = ScheduledTask::new(300, "check deploy".into(), true, false);
let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
handle
.0
.send(SchedulerCommand::Create {
task: task.clone(),
reply: reply_tx,
})
.unwrap();
let result = reply_rx.await.unwrap();
assert!(result.is_ok());
let (list_tx, list_rx) = tokio::sync::oneshot::channel();
handle
.0
.send(SchedulerCommand::List { reply: list_tx })
.unwrap();
let tasks = list_rx.await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].prompt, "check deploy");
cancel.cancel();
}
#[tokio::test]
async fn delete_task() {
let (handle, cancel, _notif_rx) = make_test_actor();
let task = ScheduledTask::new(300, "test".into(), true, false);
let task_id = task.id.clone();
let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
handle
.0
.send(SchedulerCommand::Create {
task,
reply: reply_tx,
})
.unwrap();
reply_rx.await.unwrap().unwrap();
let (del_tx, del_rx) = tokio::sync::oneshot::channel();
handle
.0
.send(SchedulerCommand::Delete {
id: task_id,
reply: del_tx,
})
.unwrap();
let removed = del_rx.await.unwrap();
assert!(removed);
let (list_tx, list_rx) = tokio::sync::oneshot::channel();
handle
.0
.send(SchedulerCommand::List { reply: list_tx })
.unwrap();
let tasks = list_rx.await.unwrap();
assert!(tasks.is_empty());
cancel.cancel();
}
#[tokio::test]
async fn delete_nonexistent_returns_false() {
let (handle, cancel, _notif_rx) = make_test_actor();
let (del_tx, del_rx) = tokio::sync::oneshot::channel();
handle
.0
.send(SchedulerCommand::Delete {
id: "nonexistent".into(),
reply: del_tx,
})
.unwrap();
let removed = del_rx.await.unwrap();
assert!(!removed);
cancel.cancel();
}
#[tokio::test]
async fn max_task_limit_enforced() {
let (handle, cancel, _notif_rx) = make_test_actor();
for i in 0..MAX_SCHEDULED_TASKS {
let task = ScheduledTask::new(300, format!("task {i}"), true, false);
let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
handle
.0
.send(SchedulerCommand::Create {
task,
reply: reply_tx,
})
.unwrap();
reply_rx.await.unwrap().unwrap();
}
let task = ScheduledTask::new(300, "one too many".into(), true, false);
let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
handle
.0
.send(SchedulerCommand::Create {
task,
reply: reply_tx,
})
.unwrap();
let result = reply_rx.await.unwrap();
assert!(result.is_err());
cancel.cancel();
}
#[tokio::test]
async fn cancel_token_shuts_down_actor() {
let (handle, cancel, _notif_rx) = make_test_actor();
cancel.cancel();
// After cancellation, sends should eventually fail (receiver dropped).
tokio::time::sleep(Duration::from_millis(50)).await;
let task = ScheduledTask::new(300, "test".into(), true, false);
let (reply_tx, _reply_rx) = tokio::sync::oneshot::channel();
// The send may succeed (buffered) but the reply will never come
// because the actor has stopped.
let _ = handle.0.send(SchedulerCommand::Create {
task,
reply: reply_tx,
});
}
#[tokio::test]
async fn recurring_task_fires_repeatedly_without_external_clear() {
let (handle, cancel, mut notif_rx) = make_test_actor();
let mut task = ScheduledTask::new(1, "recurring".into(), true, false);
task.created_at = chrono::Utc::now() - chrono::Duration::seconds(10);
let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
handle
.0
.send(SchedulerCommand::Create {
task,
reply: reply_tx,
})
.unwrap();
reply_rx.await.unwrap().unwrap();
// Drain ScheduledTaskCreated.
let notif = tokio::time::timeout(Duration::from_secs(2), notif_rx.recv())
.await
.expect("created")
.expect("channel open");
assert!(matches!(notif, ToolNotification::ScheduledTaskCreated(_)));
// First fire.
let notif = tokio::time::timeout(Duration::from_secs(2), notif_rx.recv())
.await
.expect("first fire")
.expect("channel open");
assert!(matches!(notif, ToolNotification::ScheduledTaskFired(_)));
// The task re-fires purely from `next_fire_at` rescheduling, with no
// in-flight guard to clear.
let notif = tokio::time::timeout(Duration::from_secs(3), notif_rx.recv())
.await
.expect("second fire")
.expect("channel open");
assert!(matches!(notif, ToolNotification::ScheduledTaskFired(_)));
cancel.cancel();
}
#[tokio::test]
async fn announces_existing_tasks_on_startup() {
let mut resources = Resources::new();
resources.register_state::<SchedulerState>();
// Simulate session restore: scheduler state already contains two
// recurring tasks before the actor spawns. Both intervals are far
// enough in the future that neither will fire during the test.
let state = resources.get_or_default::<State<SchedulerState>>();
let mut task_a = ScheduledTask::new(300, "task A".into(), true, false);
task_a.id = "restored-A".to_string();
state.tasks.push(task_a);
let mut task_b = ScheduledTask::new(600, "task B".into(), true, false);
task_b.id = "restored-B".to_string();
state.tasks.push(task_b);
let shared = Arc::new(Mutex::new(resources));
let (notif_handle, mut notif_rx) = ToolNotificationHandle::channel();
let (_cmd_tx, cmd_rx) = mpsc::unbounded_channel();
let cancel_token = CancellationToken::new();
let actor = SchedulerActor {
resources: shared,
notification_handle: notif_handle,
cmd_rx,
cancel_token: cancel_token.clone(),
};
tokio::spawn(actor.run());
// The first two events on the channel must be ScheduledTaskCreated
// for the two restored tasks, in insertion order.
let n1 = tokio::time::timeout(Duration::from_secs(2), notif_rx.recv())
.await
.expect("first notification")
.expect("channel open");
let n2 = tokio::time::timeout(Duration::from_secs(2), notif_rx.recv())
.await
.expect("second notification")
.expect("channel open");
let ToolNotification::ScheduledTaskCreated(c1) = n1 else {
panic!("expected ScheduledTaskCreated, got {n1:?}");
};
let ToolNotification::ScheduledTaskCreated(c2) = n2 else {
panic!("expected ScheduledTaskCreated, got {n2:?}");
};
assert_eq!(c1.task_id, "restored-A");
assert_eq!(c1.prompt, "task A");
assert_eq!(c1.human_schedule, "every 5 minutes");
assert!(c1.next_fire_at.is_some());
assert_eq!(c2.task_id, "restored-B");
assert_eq!(c2.prompt, "task B");
assert_eq!(c2.human_schedule, "every 10 minutes");
assert!(c2.next_fire_at.is_some());
// No further events should arrive within a short window: tasks are
// not yet due and no commands are in flight.
let extra = tokio::time::timeout(Duration::from_millis(150), notif_rx.recv()).await;
assert!(
extra.is_err(),
"no further notifications expected, got {:?}",
extra.ok()
);
cancel_token.cancel();
}
#[tokio::test]
async fn announces_no_tasks_when_state_empty() {
let (_handle, cancel, mut notif_rx) = make_test_actor();
// Brand-new session: state.tasks is empty, the re-announce step must
// be a silent no-op.
let result = tokio::time::timeout(Duration::from_millis(150), notif_rx.recv()).await;
assert!(
result.is_err(),
"expected no notifications, got {:?}",
result.ok()
);
cancel.cancel();
}
#[tokio::test]
async fn missed_one_shots_all_fire_and_are_removed() {
let mut resources = Resources::new();
resources.register_state::<SchedulerState>();
let state = resources.get_or_default::<State<SchedulerState>>();
let past = chrono::Utc::now() - chrono::Duration::seconds(60);
for i in 0..3 {
let mut task = ScheduledTask::new(1, format!("missed-{i}"), false, false);
task.id = format!("missed-{i}");
task.created_at = past;
state.tasks.push(task);
}
let shared = Arc::new(Mutex::new(resources));
let (notif_handle, mut notif_rx) = ToolNotificationHandle::channel();
let (_cmd_tx, cmd_rx) = mpsc::unbounded_channel();
let cancel_token = CancellationToken::new();
let mut actor = SchedulerActor {
resources: shared.clone(),
notification_handle: notif_handle,
cmd_rx,
cancel_token: cancel_token.clone(),
};
actor.handle_missed_tasks().await;
// All three missed one-shots fire (fires first, then removes).
for _ in 0..3 {
let notif = notif_rx.try_recv().expect("missed task should fire");
assert!(matches!(notif, ToolNotification::ScheduledTaskFired(_)));
}
for _ in 0..3 {
let notif = notif_rx.try_recv().expect("missed task should be removed");
assert!(matches!(notif, ToolNotification::ScheduledTaskRemoved(_)));
}
assert!(notif_rx.try_recv().is_err());
// All fired missed one-shots are pruned from state.
let res = shared.lock().await;
let remaining = res
.get::<State<SchedulerState>>()
.map(|s| s.tasks.len())
.unwrap_or(0);
assert_eq!(remaining, 0, "all fired missed one-shots should be removed");
}
#[tokio::test]
async fn cancel_sends_removed_for_remaining_tasks() {
let mut resources = Resources::new();
resources.register_state::<SchedulerState>();
let state = resources.get_or_default::<State<SchedulerState>>();
let mut task_a = ScheduledTask::new(300, "task A".into(), true, false);
task_a.id = "cancel-A".to_string();
state.tasks.push(task_a);
let mut task_b = ScheduledTask::new(600, "task B".into(), true, false);
task_b.id = "cancel-B".to_string();
state.tasks.push(task_b);
let shared = Arc::new(Mutex::new(resources));
let (notif_handle, mut notif_rx) = ToolNotificationHandle::channel();
let (_cmd_tx, cmd_rx) = mpsc::unbounded_channel();
let cancel_token = CancellationToken::new();
let actor = SchedulerActor {
resources: shared,
notification_handle: notif_handle,
cmd_rx,
cancel_token: cancel_token.clone(),
};
let handle = tokio::spawn(actor.run());
// Drain the ScheduledTaskCreated announcements.
let _ = tokio::time::timeout(Duration::from_secs(2), notif_rx.recv()).await;
let _ = tokio::time::timeout(Duration::from_secs(1), notif_rx.recv()).await;
cancel_token.cancel();
handle.await.expect("actor should complete");
// Collect all ScheduledTaskRemoved notifications.
let mut removed_ids = Vec::new();
while let Ok(notif) = notif_rx.try_recv() {
if let ToolNotification::ScheduledTaskRemoved(r) = notif {
removed_ids.push(r.task_id);
}
}
removed_ids.sort();
assert_eq!(removed_ids, vec!["cancel-A", "cancel-B"]);
}
#[tokio::test]
async fn cancel_with_no_tasks_sends_no_removed() {
let mut resources = Resources::new();
resources.register_state::<SchedulerState>();
let shared = Arc::new(Mutex::new(resources));
let (notif_handle, mut notif_rx) = ToolNotificationHandle::channel();
let (_cmd_tx, cmd_rx) = mpsc::unbounded_channel();
let cancel_token = CancellationToken::new();
let actor = SchedulerActor {
resources: shared,
notification_handle: notif_handle,
cmd_rx,
cancel_token: cancel_token.clone(),
};
let handle = tokio::spawn(actor.run());
cancel_token.cancel();
handle.await.expect("actor should complete");
assert!(
notif_rx.try_recv().is_err(),
"no notifications expected when state is empty"
);
}
}
@@ -0,0 +1,225 @@
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
use super::interval::{interval_to_human, parse_interval};
use super::types::{ScheduledTask, SchedulerCommand, SchedulerHandle};
// Canonical /loop wording lives in the light API crate so other consumers can
// link it without the tools implementation crate; re-exported to keep paths stable.
pub use kigi_tools_api::slash_commands::{
SCHEDULER_CREATE_TOOL_NAME, loop_schedule_instruction, loop_usage_message,
};
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct SchedulerCreateInput {
/// Interval string: "5m", "2h", "1d", etc.
#[schemars(description = "Interval between executions, e.g. \"5m\", \"2h\", \"1d\"")]
pub interval: String,
/// The prompt to run on each fire.
#[schemars(description = "The prompt text to execute on each scheduled fire")]
pub prompt: String,
/// Whether the task recurs. Default true.
#[serde(
default = "default_true",
deserialize_with = "crate::types::schema::deserialize_lenient_bool"
)]
#[schemars(
description = "Whether the task repeats (true) or fires once (false). Default: true"
)]
pub recurring: bool,
/// Whether the task persists across sessions. Default false (session-only).
#[serde(
default,
deserialize_with = "crate::types::schema::deserialize_lenient_option_bool"
)]
#[schemars(description = "Whether the task persists across sessions. Default: false")]
pub durable: Option<bool>,
/// Whether to fire immediately on creation. Default false (wait for the
/// first interval — a "scheduled" task should not run on creation unless
/// explicitly asked to).
#[serde(
default,
deserialize_with = "crate::types::schema::deserialize_lenient_bool"
)]
#[schemars(
description = "Whether to fire immediately on creation (true) or wait for the first interval (false). Default: false"
)]
pub fire_immediately: bool,
}
fn default_true() -> bool {
true
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
#[serde(rename_all = "camelCase")]
pub struct SchedulerCreateOutput {
pub id: String,
pub human_schedule: String,
pub recurring: bool,
}
impl kigi_tool_runtime::ToolOutput for SchedulerCreateOutput {}
#[derive(Debug, Default)]
pub struct SchedulerCreateTool;
impl crate::types::tool_metadata::ToolMetadata for SchedulerCreateTool {
fn kind(&self) -> ToolKind {
ToolKind::Other
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
r#"Create a scheduled task that runs a prompt on a recurring interval.
Set fire_immediately: true to also fire once on creation; by default the first run waits for the interval.
Usage notes:
- Interval format: "5m" (minutes), "2h" (hours), "1d" (days), "60s" (seconds, min 60)
- Maximum 50 scheduled tasks at once
- Recurring tasks auto-expire after 7 days"#
// TODO: scheduler tools share ToolKind::Other so they can't be template-ized
// via ${{ tools.by_kind.* }}. If tool name randomization is needed, add
// dedicated ToolKind variants (SchedulerCreate, SchedulerDelete, SchedulerList).
}
fn emitted_notifications(&self) -> &'static [&'static str] {
// A create call only registers the task (the actor emits
// ScheduledTaskCreated). Fired/Removed come later from the actor timer,
// delete, or shutdown — not from this tool's execution.
&["ScheduledTaskCreated"]
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for SchedulerCreateTool {
type Args = SchedulerCreateInput;
type Output = SchedulerCreateOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new(SCHEDULER_CREATE_TOOL_NAME).expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"scheduler_create",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.scheduler_create",
skip_all,
fields(interval = %input.interval)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: SchedulerCreateInput,
) -> Result<SchedulerCreateOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let interval_secs = parse_interval(&input.interval)
.map_err(|e| kigi_tool_runtime::ToolError::invalid_arguments(e.to_string()))?;
let sender = {
let res = resources.lock().await;
res.get::<SchedulerHandle>()
.ok_or_else(|| {
kigi_tool_runtime::ToolError::custom("missing_resource", "SchedulerHandle")
})?
.0
.clone()
};
let durable = input.durable.unwrap_or(false);
let task = ScheduledTask::with_fire_immediately(
interval_secs,
input.prompt,
input.recurring,
durable,
input.fire_immediately,
);
let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
sender
.send(SchedulerCommand::Create {
task: task.clone(),
reply: reply_tx,
})
.map_err(|_| {
kigi_tool_runtime::ToolError::custom("process_manager", "Scheduler actor stopped")
})?;
let created = reply_rx
.await
.map_err(|_| {
kigi_tool_runtime::ToolError::custom(
"process_manager",
"Scheduler actor dropped reply",
)
})?
.map_err(|e| kigi_tool_runtime::ToolError::invalid_arguments(e.to_string()))?;
Ok(SchedulerCreateOutput {
id: created.id,
human_schedule: interval_to_human(interval_secs),
recurring: input.recurring,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn loop_usage_message_has_no_host_default() {
let usage = loop_usage_message();
assert!(usage.contains("Usage: /loop"));
assert!(
!usage.contains("10m"),
"usage must not claim a default: {usage}"
);
}
#[test]
fn loop_schedule_instruction_holds_invariants() {
let args = "every 30 minutes do x";
let instr = loop_schedule_instruction(args);
assert!(
!instr.contains("10m"),
"instruction must not default: {instr}"
);
assert!(instr.contains("Deriving the interval"));
assert!(instr.contains("<number><unit>"));
assert!(instr.contains("ask the user how often"));
assert!(instr.contains("Do NOT execute the prompt inline"));
// Raw request forwarded verbatim for the model to parse.
assert!(instr.contains(args));
}
}
@@ -0,0 +1,137 @@
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
use super::types::{SchedulerCommand, SchedulerHandle};
/// Canonical tool name advertised by `SchedulerDeleteTool::id()`.
/// See note on `SCHEDULER_CREATE_TOOL_NAME`.
pub const SCHEDULER_DELETE_TOOL_NAME: &str = "scheduler_delete";
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct SchedulerDeleteInput {
/// The scheduled task ID to cancel.
#[schemars(description = "The task ID to cancel (from scheduler_create output)")]
pub id: String,
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct SchedulerDeleteOutput {
pub success: bool,
pub message: String,
}
impl kigi_tool_runtime::ToolOutput for SchedulerDeleteOutput {}
#[derive(Debug, Default)]
pub struct SchedulerDeleteTool;
impl crate::types::tool_metadata::ToolMetadata for SchedulerDeleteTool {
fn kind(&self) -> ToolKind {
ToolKind::Other
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
r#"Cancel a scheduled task by ID.
Returns success: true if the task was found and removed, false if no task with that ID exists."#
}
fn emitted_notifications(&self) -> &'static [&'static str] {
&["ScheduledTaskRemoved"]
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
use super::create::SchedulerCreateTool;
use crate::types::tool_metadata::ToolMetadata as TM;
Expr::Value(ToolRequirement::Tool {
namespace: TM::tool_namespace(&SchedulerCreateTool).to_string(),
id: kigi_tool_runtime::Tool::id(&SchedulerCreateTool).to_string(),
if_params: None,
})
}
}
impl kigi_tool_runtime::Tool for SchedulerDeleteTool {
type Args = SchedulerDeleteInput;
type Output = SchedulerDeleteOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("scheduler_delete").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"scheduler_delete",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.scheduler_delete",
skip_all,
fields(id = %input.id)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: SchedulerDeleteInput,
) -> Result<SchedulerDeleteOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let sender = {
let res = resources.lock().await;
res.get::<SchedulerHandle>()
.ok_or_else(|| {
kigi_tool_runtime::ToolError::custom("missing_resource", "SchedulerHandle")
})?
.0
.clone()
};
let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
sender
.send(SchedulerCommand::Delete {
id: input.id.clone(),
reply: reply_tx,
})
.map_err(|_| {
kigi_tool_runtime::ToolError::custom("process_manager", "Scheduler actor stopped")
})?;
let removed = reply_rx.await.map_err(|_| {
kigi_tool_runtime::ToolError::custom("process_manager", "Scheduler actor dropped reply")
})?;
if removed {
Ok(SchedulerDeleteOutput {
success: true,
message: format!("Scheduled task {} cancelled.", input.id),
})
} else {
Ok(SchedulerDeleteOutput {
success: false,
message: format!(
"No scheduled task with ID {} found. Use scheduler_list to see active tasks.",
input.id
),
})
}
}
}
@@ -0,0 +1,165 @@
use super::types::SchedulerError;
const MINIMUM_INTERVAL_SECS: u64 = 60;
/// Parse an interval string like "5m", "2h", "30s", "1d" into seconds.
/// Minimum interval is 60 seconds; values below are clamped.
pub fn parse_interval(s: &str) -> Result<u64, SchedulerError> {
let s = s.trim();
if s.is_empty() {
return Err(SchedulerError::InvalidInterval(
"interval cannot be empty".into(),
));
}
let (digits, suffix) = s.split_at(s.len() - 1);
let value: u64 = digits.parse().map_err(|_| {
SchedulerError::InvalidInterval(format!(
"invalid interval format: {s:?} (expected e.g. 5m, 2h, 1d)"
))
})?;
if value == 0 {
return Err(SchedulerError::InvalidInterval(
"interval value must be greater than 0".into(),
));
}
let unit_secs: u64 = match suffix {
"s" => 1,
"m" => 60,
"h" => 3600,
"d" => 86400,
_ => {
return Err(SchedulerError::InvalidInterval(format!(
"invalid interval suffix: {suffix:?} (expected s, m, h, or d)"
)));
}
};
let secs = value
.checked_mul(unit_secs)
.ok_or_else(|| SchedulerError::InvalidInterval(format!("interval too large: {s:?}")))?;
Ok(secs.max(MINIMUM_INTERVAL_SECS))
}
/// Convert seconds to a human-readable interval string.
/// e.g. 300 -> "every 5 minutes", 3600 -> "every 1 hour"
pub fn interval_to_human(secs: u64) -> String {
if secs.is_multiple_of(86400) {
let n = secs / 86400;
if n == 1 {
"every 1 day".into()
} else {
format!("every {n} days")
}
} else if secs.is_multiple_of(3600) {
let n = secs / 3600;
if n == 1 {
"every 1 hour".into()
} else {
format!("every {n} hours")
}
} else if secs.is_multiple_of(60) {
let n = secs / 60;
if n == 1 {
"every 1 minute".into()
} else {
format!("every {n} minutes")
}
} else if secs == 1 {
"every 1 second".into()
} else {
format!("every {secs} seconds")
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_minutes() {
assert_eq!(parse_interval("5m").unwrap(), 300);
assert_eq!(parse_interval("10m").unwrap(), 600);
assert_eq!(parse_interval("1m").unwrap(), 60);
}
#[test]
fn parse_hours() {
assert_eq!(parse_interval("2h").unwrap(), 7200);
assert_eq!(parse_interval("1h").unwrap(), 3600);
}
#[test]
fn parse_days() {
assert_eq!(parse_interval("1d").unwrap(), 86400);
assert_eq!(parse_interval("7d").unwrap(), 604800);
}
#[test]
fn parse_seconds_clamped_to_minimum() {
assert_eq!(parse_interval("30s").unwrap(), 60);
assert_eq!(parse_interval("1s").unwrap(), 60);
assert_eq!(parse_interval("60s").unwrap(), 60);
assert_eq!(parse_interval("120s").unwrap(), 120);
}
#[test]
fn parse_empty_returns_error() {
assert!(parse_interval("").is_err());
}
#[test]
fn parse_invalid_format_returns_error() {
assert!(parse_interval("abc").is_err());
assert!(parse_interval("5x").is_err());
assert!(parse_interval("m").is_err());
}
#[test]
fn parse_zero_returns_error() {
assert!(parse_interval("0m").is_err());
assert!(parse_interval("0s").is_err());
}
#[test]
fn parse_overflow_returns_error() {
// Digits parse as u64 but the unit multiplication overflows — must
// surface an error rather than panicking (debug) or wrapping (release).
assert!(parse_interval("1000000000000000000d").is_err());
assert!(parse_interval(&format!("{}s", u64::MAX)).is_ok());
assert!(parse_interval(&format!("{}d", u64::MAX)).is_err());
}
#[test]
fn human_readable_minutes() {
assert_eq!(interval_to_human(300), "every 5 minutes");
assert_eq!(interval_to_human(60), "every 1 minute");
assert_eq!(interval_to_human(600), "every 10 minutes");
}
#[test]
fn human_readable_hours() {
assert_eq!(interval_to_human(3600), "every 1 hour");
assert_eq!(interval_to_human(7200), "every 2 hours");
}
#[test]
fn human_readable_days() {
assert_eq!(interval_to_human(86400), "every 1 day");
assert_eq!(interval_to_human(172800), "every 2 days");
}
#[test]
fn human_readable_seconds() {
assert_eq!(interval_to_human(45), "every 45 seconds");
assert_eq!(interval_to_human(1), "every 1 second");
}
#[test]
fn parse_with_whitespace() {
assert_eq!(parse_interval(" 5m ").unwrap(), 300);
}
}
@@ -0,0 +1,145 @@
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
use super::interval::interval_to_human;
use super::types::{SchedulerCommand, SchedulerHandle};
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct SchedulerListInput {}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
#[serde(rename_all = "camelCase")]
pub struct ScheduledTaskSummary {
pub id: String,
pub prompt: String,
pub interval_human: String,
pub next_fire_at: String,
pub created_at: String,
pub recurring: bool,
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct SchedulerListOutput {
pub tasks: Vec<ScheduledTaskSummary>,
}
impl kigi_tool_runtime::ToolOutput for SchedulerListOutput {}
#[derive(Debug, Default)]
pub struct SchedulerListTool;
impl crate::types::tool_metadata::ToolMetadata for SchedulerListTool {
fn kind(&self) -> ToolKind {
ToolKind::Other
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
"List all active scheduled tasks with their IDs, prompts, intervals, and next fire times."
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
use super::create::SchedulerCreateTool;
use crate::types::tool_metadata::ToolMetadata as TM;
Expr::Value(ToolRequirement::Tool {
namespace: TM::tool_namespace(&SchedulerCreateTool).to_string(),
id: kigi_tool_runtime::Tool::id(&SchedulerCreateTool).to_string(),
if_params: None,
})
}
}
impl kigi_tool_runtime::Tool for SchedulerListTool {
type Args = SchedulerListInput;
type Output = SchedulerListOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("scheduler_list").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"scheduler_list",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(name = "tool.scheduler_list", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
_input: SchedulerListInput,
) -> Result<SchedulerListOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let sender = {
let res = resources.lock().await;
res.get::<SchedulerHandle>()
.ok_or_else(|| {
kigi_tool_runtime::ToolError::custom(
"missing_dependency",
"missing dependency: SchedulerHandle",
)
})?
.0
.clone()
};
let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
sender
.send(SchedulerCommand::List { reply: reply_tx })
.map_err(|_| {
kigi_tool_runtime::ToolError::execution(
kigi_tool_protocol::ToolId::new("scheduler_list").expect("valid"),
"Scheduler actor stopped",
)
})?;
let tasks = reply_rx.await.map_err(|_| {
kigi_tool_runtime::ToolError::execution(
kigi_tool_protocol::ToolId::new("scheduler_list").expect("valid"),
"Scheduler actor dropped reply",
)
})?;
let summaries = tasks
.into_iter()
.map(|t| {
let next_fire = t.next_fire_at().to_rfc3339();
let created = t.created_at.to_rfc3339();
let prompt = if t.prompt.len() > 80 {
let cut = crate::util::floor_char_boundary(&t.prompt, 80);
format!("{}...", &t.prompt[..cut])
} else {
t.prompt
};
ScheduledTaskSummary {
id: t.id,
prompt,
interval_human: interval_to_human(t.interval_secs),
next_fire_at: next_fire,
created_at: created,
recurring: t.recurring,
}
})
.collect();
Ok(SchedulerListOutput { tasks: summaries })
}
}
@@ -0,0 +1,6 @@
pub mod actor;
pub mod create;
pub mod delete;
pub mod interval;
pub mod list;
pub mod types;
@@ -0,0 +1,196 @@
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use tokio::sync::{mpsc, oneshot};
#[derive(thiserror::Error, Debug)]
pub enum SchedulerError {
#[error("invalid interval: {0}")]
InvalidInterval(String),
#[error("maximum of {0} scheduled tasks reached")]
TaskLimitReached(usize),
}
/// A single scheduled recurring or one-shot task.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct ScheduledTask {
pub id: String,
pub interval_secs: u64,
pub prompt: String,
pub recurring: bool,
pub durable: bool,
pub created_at: DateTime<Utc>,
pub last_fired_at: Option<DateTime<Utc>>,
pub expires_at: Option<DateTime<Utc>>,
}
impl ScheduledTask {
pub fn new(interval_secs: u64, prompt: String, recurring: bool, durable: bool) -> Self {
Self::with_fire_immediately(interval_secs, prompt, recurring, durable, false)
}
pub fn with_fire_immediately(
interval_secs: u64,
prompt: String,
recurring: bool,
durable: bool,
fire_immediately: bool,
) -> Self {
let now = Utc::now();
// When fire_immediately is true, anchor created_at in the past so that
// next_fire_at() = created_at + interval = now, firing on the first tick.
let created_at = if fire_immediately {
now - chrono::Duration::seconds(interval_secs as i64)
} else {
now
};
Self {
id: uuid::Uuid::now_v7().to_string().replace('-', "")[..12].to_string(),
interval_secs,
prompt,
recurring,
durable,
created_at,
last_fired_at: None,
expires_at: if recurring {
Some(now + chrono::Duration::days(7))
} else {
None
},
}
}
/// Next fire time, computed from `last_fired_at` (or `created_at` if never fired).
pub fn next_fire_at(&self) -> DateTime<Utc> {
let anchor = self.last_fired_at.unwrap_or(self.created_at);
anchor + chrono::Duration::seconds(self.interval_secs as i64)
}
/// Whether this task has expired (recurring tasks only).
pub fn is_expired(&self, now: DateTime<Utc>) -> bool {
self.expires_at.is_some_and(|exp| now >= exp)
}
/// Whether this task was missed (one-shot: fire time already passed, never fired).
pub fn is_missed(&self, now: DateTime<Utc>) -> bool {
!self.recurring && self.last_fired_at.is_none() && self.next_fire_at() < now
}
}
/// Persisted state for the scheduler, stored via Resources + ResourcesPersistence.
/// Only durable tasks are serialized; non-durable tasks are filtered out before save.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct SchedulerState {
pub tasks: Vec<ScheduledTask>,
}
crate::register_resource!("grok_build", "Scheduler", SchedulerState);
/// Handle for tools to communicate with the SchedulerActor.
/// Ephemeral -- not serialized, not persisted. Inserted via `resources.insert()`.
#[derive(Clone)]
pub struct SchedulerHandle(pub mpsc::UnboundedSender<SchedulerCommand>);
pub enum SchedulerCommand {
Create {
task: ScheduledTask,
reply: oneshot::Sender<Result<ScheduledTask, SchedulerError>>,
},
Delete {
id: String,
reply: oneshot::Sender<bool>,
},
List {
reply: oneshot::Sender<Vec<ScheduledTask>>,
},
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn new_recurring_task_has_7_day_expiry() {
let task = ScheduledTask::new(300, "check deploy".into(), true, false);
assert!(task.expires_at.is_some());
let expiry = task.expires_at.unwrap();
let diff = expiry - task.created_at;
assert_eq!(diff.num_days(), 7);
}
#[test]
fn new_one_shot_task_has_no_expiry() {
let task = ScheduledTask::new(300, "check deploy".into(), false, false);
assert!(task.expires_at.is_none());
}
#[test]
fn next_fire_at_uses_created_at_when_never_fired() {
let task = ScheduledTask::new(300, "test".into(), true, false);
let expected = task.created_at + chrono::Duration::seconds(300);
assert_eq!(task.next_fire_at(), expected);
}
#[test]
fn next_fire_at_uses_last_fired_at_when_present() {
let mut task = ScheduledTask::new(300, "test".into(), true, false);
let fired = Utc::now();
task.last_fired_at = Some(fired);
let expected = fired + chrono::Duration::seconds(300);
assert_eq!(task.next_fire_at(), expected);
}
#[test]
fn is_expired_returns_true_when_past_expiry() {
let mut task = ScheduledTask::new(300, "test".into(), true, false);
task.expires_at = Some(Utc::now() - chrono::Duration::hours(1));
assert!(task.is_expired(Utc::now()));
}
#[test]
fn is_expired_returns_false_when_before_expiry() {
let task = ScheduledTask::new(300, "test".into(), true, false);
assert!(!task.is_expired(Utc::now()));
}
#[test]
fn is_expired_returns_false_for_one_shot() {
let task = ScheduledTask::new(300, "test".into(), false, false);
assert!(!task.is_expired(Utc::now()));
}
#[test]
fn is_missed_returns_true_for_unfired_one_shot_past_due() {
let mut task = ScheduledTask::new(1, "test".into(), false, false);
task.created_at = Utc::now() - chrono::Duration::seconds(10);
assert!(task.is_missed(Utc::now()));
}
#[test]
fn is_missed_returns_false_for_recurring() {
let mut task = ScheduledTask::new(1, "test".into(), true, false);
task.created_at = Utc::now() - chrono::Duration::seconds(10);
assert!(!task.is_missed(Utc::now()));
}
#[test]
fn is_missed_returns_false_if_already_fired() {
let mut task = ScheduledTask::new(1, "test".into(), false, false);
task.created_at = Utc::now() - chrono::Duration::seconds(10);
task.last_fired_at = Some(Utc::now());
assert!(!task.is_missed(Utc::now()));
}
#[test]
fn task_id_is_12_chars() {
let task = ScheduledTask::new(300, "test".into(), true, false);
assert_eq!(task.id.len(), 12);
}
#[test]
fn scheduler_state_default_is_empty() {
let state = SchedulerState::default();
assert!(state.tasks.is_empty());
}
}
@@ -0,0 +1,532 @@
//! SearchReplace tool implementation.
//!
//! This tool performs exact string replacements in files with support for:
//! - Exact string replacement (find/replace)
//! - New file creation (when `old_string` is empty)
//! - Replace all mode (`replace_all: true`)
//! - Read-before-edit validation (non-concise mode)
//! - External modification detection
use crate::types::output::SearchReplaceEditDetail;
// ============================================================================
// Shared string-edit helpers
// ============================================================================
/// Render a snippet of the file with line numbers around the edit.
pub(crate) fn render_snippet(
new_text: &str,
new_string: &str,
start_pos: usize,
context_size: usize,
) -> (String, String, String) {
let LineRange {
start_line,
end_line,
} = compute_line_range(new_text, start_pos, new_string);
let total_lines_count = new_text.split_inclusive('\n').count();
let lines = new_text.split_inclusive('\n').collect::<Vec<_>>();
let snippet_start = start_line.saturating_sub(context_size);
let snippet_end = (end_line + context_size).min(total_lines_count.saturating_sub(1));
let before_context = if snippet_start < start_line {
lines[snippet_start..start_line].join("")
} else {
String::new()
};
let after_context = if end_line < snippet_end {
lines[(end_line + 1)..=snippet_end].join("")
} else {
String::new()
};
let snippet = lines
.iter()
.enumerate()
.map(|(line_num, line)| format!("{}→{}", line_num + 1, line))
.skip(snippet_start)
.take(snippet_end - snippet_start + 1)
.collect::<String>();
(snippet, before_context, after_context)
}
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
/// 0-based inclusive start and end line indices
pub(crate) struct LineRange {
pub start_line: usize,
pub end_line: usize,
}
/// Compute the line range of the inserted text in the text
pub(crate) fn compute_line_range(text: &str, start_pos: usize, inserted_text: &str) -> LineRange {
let start_line = text[..start_pos].matches('\n').count();
let lines_in_inserted = inserted_text.split_inclusive('\n').count().max(1);
let end_line = start_line + lines_in_inserted - 1;
LineRange {
start_line,
end_line,
}
}
/// Replace text at specific positions and return new text with new positions.
pub(crate) fn replace_using_positions(
text: &str,
match_positions: &[usize],
old_string: &str,
new_string: &str,
) -> (String, Vec<usize>) {
let mut new_text = String::new();
let mut new_positions: Vec<usize> = Vec::with_capacity(match_positions.len());
let mut last_end: usize = 0;
for &pos in match_positions {
new_text.push_str(&text[last_end..pos]);
new_positions.push(new_text.len());
new_text.push_str(new_string);
last_end = pos + old_string.len();
}
new_text.push_str(&text[last_end..]);
(new_text, new_positions)
}
/// Build edit details for each replacement.
pub(crate) fn build_edit_details(
new_text: &str,
old_string: &str,
new_string: &str,
new_positions: &[usize],
context_lines: usize,
) -> Vec<SearchReplaceEditDetail> {
let mut details: Vec<SearchReplaceEditDetail> = Vec::with_capacity(new_positions.len());
for &start_pos in new_positions {
let (_snippet, context_before, context_after) =
render_snippet(new_text, new_string, start_pos, context_lines);
let line_range_new = compute_line_range(new_text, start_pos, new_string);
// Extract the leading text on the line before the match starts.
// This is the text between the last '\n' before start_pos and start_pos itself.
let line_start = new_text[..start_pos]
.rfind('\n')
.map(|i| i + 1)
.unwrap_or(0);
let line_prefix = new_text[line_start..start_pos].to_owned();
details.push(SearchReplaceEditDetail {
old_string: old_string.to_owned(),
old_line: line_range_new.start_line + 1,
new_string: new_string.to_owned(),
new_line: line_range_new.start_line + 1,
context_before,
context_after,
line_prefix,
});
}
details
}
// ============================================================================
// Normalized (confusable-aware) matching helpers
// ============================================================================
/// A single match found via confusable-normalized comparison, expressed in
/// the original text's byte coordinates.
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct NormalizedMatch {
/// Byte offset in the original text where the match starts.
pub original_start: usize,
/// Length in bytes of the matched region in the original text.
/// May differ from the search pattern's byte length because confusable
/// characters have different UTF-8 widths than their ASCII equivalents.
pub original_len: usize,
}
/// Result of normalized match search — distinguishes "no match exists" from
/// "matches exist but are unsafe to apply."
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) enum NormalizedMatchResult {
/// No normalized match exists in the text.
NoMatch,
/// One or more valid, non-overlapping matches were found.
Matches(Vec<NormalizedMatch>),
/// Normalized matching found candidates but they are ambiguous or unsafe
/// (overlapping remapped spans, or partial-expansion matches that don't
/// roundtrip correctly). The caller should treat this as an explicit
/// ambiguity error, NOT as "string not found."
Ambiguous,
}
/// Find match positions using confusable-normalized comparison and remap
/// them back to the original text's byte coordinates.
///
/// Algorithm:
/// 1. Build `(normalized_text, offset_map)` via [`build_offset_map`].
/// 2. Normalize the search pattern the same way.
/// 3. Find all non-overlapping matches in `normalized_text`.
/// 4. Remap each normalized `[start..end]` span back to original bytes
/// via `offset_map`.
/// 5. **Roundtrip validation:** For each candidate, verify that
/// `normalize_confusables(&text[orig_start..orig_end]) == norm_pattern`.
/// This rejects partial-expansion matches (e.g., pattern `-` matching
/// inside em-dash `—` which normalizes to `--`).
/// 6. Reject overlapping validated spans (fail closed → `Ambiguous`).
///
/// Returns [`NormalizedMatchResult`] to distinguish no-match from ambiguity.
pub(crate) fn find_normalized_match_positions(text: &str, pattern: &str) -> NormalizedMatchResult {
use crate::util::unicode_confusables::{build_offset_map, normalize_confusables};
let (norm_text, offset_map) = build_offset_map(text);
let norm_pattern = normalize_confusables(pattern);
if norm_pattern.is_empty() {
return NormalizedMatchResult::NoMatch;
}
// Collect all non-overlapping matches in normalized space and validate
// each candidate via roundtrip check.
let mut validated = Vec::new();
let mut had_rejected_candidates = false;
for (norm_start, _) in norm_text.match_indices(&norm_pattern) {
let norm_end = norm_start + norm_pattern.len();
let orig_start = offset_map[norm_start];
let orig_end = offset_map[norm_end];
// Reject zero-length or inverted spans.
if orig_end <= orig_start {
had_rejected_candidates = true;
continue;
}
let orig_slice = &text[orig_start..orig_end];
// Roundtrip validation: the normalized original slice must exactly
// equal the normalized pattern. This catches partial-expansion
// matches (e.g., pattern "-" matching inside "—" → "--").
if normalize_confusables(orig_slice) != norm_pattern {
had_rejected_candidates = true;
continue;
}
validated.push(NormalizedMatch {
original_start: orig_start,
original_len: orig_end - orig_start,
});
}
if validated.is_empty() {
// If we had candidates but all were rejected, that's ambiguity.
// If we had zero candidates at all, that's a genuine no-match.
return if had_rejected_candidates {
NormalizedMatchResult::Ambiguous
} else {
NormalizedMatchResult::NoMatch
};
}
// Reject overlapping remapped spans (fail closed on ambiguity).
for window in validated.windows(2) {
let end_of_prev = window[0].original_start + window[0].original_len;
if end_of_prev > window[1].original_start {
return NormalizedMatchResult::Ambiguous;
}
}
NormalizedMatchResult::Matches(validated)
}
/// Replace text at normalized-match positions and return the new text with
/// new byte offsets of each replacement.
///
/// Each `NormalizedMatch` specifies a region in the original text (which may
/// contain Unicode confusables) to be replaced with `new_string`.
pub(crate) fn replace_normalized_matches(
text: &str,
matches: &[NormalizedMatch],
new_string: &str,
) -> (String, Vec<usize>) {
let mut result = String::new();
let mut new_positions: Vec<usize> = Vec::with_capacity(matches.len());
let mut last_end: usize = 0;
for m in matches {
result.push_str(&text[last_end..m.original_start]);
new_positions.push(result.len());
result.push_str(new_string);
last_end = m.original_start + m.original_len;
}
result.push_str(&text[last_end..]);
(result, new_positions)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn render_snippet_middle_of_file_contexts() {
let new_text = "one\ntwo NEW here\nthree\nfour\nfive\nsix\nseven\neight\nnine\nten\n";
let inserted = "NEW here";
let start_pos = new_text.find(inserted).unwrap();
let (snippet, before_context, after_context) =
render_snippet(new_text, inserted, start_pos, 3);
let expected_snippet = "1→one\n2→two NEW here\n3→three\n4→four\n5→five\n";
assert_eq!(snippet, expected_snippet);
assert_eq!(before_context, "one\n");
assert_eq!(after_context, "three\nfour\nfive\n");
let line_range = compute_line_range(new_text, start_pos, inserted);
assert_eq!(
line_range,
LineRange {
start_line: 1,
end_line: 1
}
);
}
#[test]
fn render_snippet_start_of_file_contexts() {
let new_text = "AA1\nAA2\nb\nc\nd\ne\nf\n";
let inserted = "AA1\nAA2\n";
let start_pos = 0;
let (snippet, before_context, after_context) =
render_snippet(new_text, inserted, start_pos, 3);
let expected_snippet = "1→AA1\n2→AA2\n3→b\n4→c\n5→d\n";
assert_eq!(snippet, expected_snippet);
assert_eq!(before_context, "");
assert_eq!(after_context, "b\nc\nd\n");
let line_range = compute_line_range(new_text, start_pos, inserted);
assert_eq!(
line_range,
LineRange {
start_line: 0,
end_line: 1
}
);
}
#[test]
fn render_snippet_end_of_file_contexts() {
let new_text = "a\nb\nc\nd\ne\nNEW\n";
let inserted = "NEW\n";
let start_pos = new_text.find(inserted).unwrap();
let (snippet, before_context, after_context) =
render_snippet(new_text, inserted, start_pos, 3);
let expected_snippet = "3→c\n4→d\n5→e\n6→NEW\n";
assert_eq!(snippet, expected_snippet);
assert_eq!(before_context, "c\nd\ne\n");
assert_eq!(after_context, "");
let line_range = compute_line_range(new_text, start_pos, inserted);
assert_eq!(
line_range,
LineRange {
start_line: 5,
end_line: 5
}
);
}
#[test]
fn render_snippet_multiline_insertion_middle() {
let new_text = "1\n2\nAAA\nBBB\nCCC\n3\n4\n5\n6\n7\n";
let inserted = "AAA\nBBB\nCCC\n";
let start_pos = new_text.find(inserted).unwrap();
let (snippet, before_context, after_context) =
render_snippet(new_text, inserted, start_pos, 3);
let expected_snippet = "1→1\n2→2\n3→AAA\n4→BBB\n5→CCC\n6→3\n7→4\n8→5\n";
assert_eq!(snippet, expected_snippet);
assert_eq!(before_context, "1\n2\n");
assert_eq!(after_context, "3\n4\n5\n");
let line_range = compute_line_range(new_text, start_pos, inserted);
assert_eq!(
line_range,
LineRange {
start_line: 2,
end_line: 4
}
);
}
#[test]
fn test_replace_using_positions() {
let text = "hello world, hello again";
let positions = vec![0, 13];
let (new_text, new_positions) = replace_using_positions(text, &positions, "hello", "hi");
assert_eq!(new_text, "hi world, hi again");
assert_eq!(new_positions, vec![0, 10]);
}
// ── Normalized matching helpers ─────────────────────────────────────
fn unwrap_matches(result: NormalizedMatchResult) -> Vec<NormalizedMatch> {
match result {
NormalizedMatchResult::Matches(m) => m,
other => panic!("Expected Matches, got {:?}", other),
}
}
#[test]
fn normalized_match_smart_quotes() {
let text = "say \u{201C}hello\u{201D} world";
let matches = unwrap_matches(find_normalized_match_positions(text, "\"hello\""));
assert_eq!(matches.len(), 1);
let m = &matches[0];
assert_eq!(
&text[m.original_start..m.original_start + m.original_len],
"\u{201C}hello\u{201D}"
);
}
#[test]
fn normalized_match_em_dash() {
let text = "foo\u{2014}bar";
let matches = unwrap_matches(find_normalized_match_positions(text, "foo--bar"));
assert_eq!(matches.len(), 1);
}
#[test]
fn normalized_match_nbsp() {
let text = "hello\u{00A0}world";
let matches = unwrap_matches(find_normalized_match_positions(text, "hello world"));
assert_eq!(matches.len(), 1);
}
#[test]
fn normalized_match_ellipsis() {
let text = "wait\u{2026}";
let matches = unwrap_matches(find_normalized_match_positions(text, "wait..."));
assert_eq!(matches.len(), 1);
}
#[test]
fn normalized_match_returns_no_match_for_unrelated() {
assert_eq!(
find_normalized_match_positions("hello world", "xyz"),
NormalizedMatchResult::NoMatch
);
}
#[test]
fn normalized_match_pure_ascii_still_works() {
let matches = unwrap_matches(find_normalized_match_positions("hello world", "hello"));
assert_eq!(matches.len(), 1);
assert_eq!(matches[0].original_start, 0);
assert_eq!(matches[0].original_len, 5);
}
#[test]
fn normalized_match_multiple_occurrences() {
let text = "\u{201C}a\u{201D} and \u{201C}b\u{201D}";
let matches = unwrap_matches(find_normalized_match_positions(text, "\"a\""));
assert_eq!(matches.len(), 1);
}
#[test]
fn normalized_match_empty_pattern_returns_no_match() {
assert_eq!(
find_normalized_match_positions("hello", ""),
NormalizedMatchResult::NoMatch
);
}
// ── Partial-expansion rejection ─────────────────────────────────────
#[test]
fn partial_expansion_dash_inside_em_dash_rejected() {
let text = "\u{2014}";
assert_eq!(
find_normalized_match_positions(text, "-"),
NormalizedMatchResult::Ambiguous,
"partial match inside em-dash must be Ambiguous"
);
}
#[test]
fn partial_expansion_dot_inside_ellipsis_rejected() {
let text = "\u{2026}";
assert_eq!(
find_normalized_match_positions(text, "."),
NormalizedMatchResult::Ambiguous,
"partial match inside ellipsis must be Ambiguous"
);
}
#[test]
fn partial_expansion_double_dot_inside_ellipsis_rejected() {
let text = "\u{2026}";
assert_eq!(
find_normalized_match_positions(text, ".."),
NormalizedMatchResult::Ambiguous,
);
}
#[test]
fn full_expansion_em_dash_accepted() {
let text = "a\u{2014}b";
let matches = unwrap_matches(find_normalized_match_positions(text, "--"));
assert_eq!(matches.len(), 1);
assert_eq!(
&text[matches[0].original_start..matches[0].original_start + matches[0].original_len],
"\u{2014}"
);
}
#[test]
fn full_expansion_ellipsis_accepted() {
let text = "a\u{2026}b";
let matches = unwrap_matches(find_normalized_match_positions(text, "..."));
assert_eq!(matches.len(), 1);
}
// ── Replace with new return type ────────────────────────────────────
#[test]
fn replace_normalized_matches_basic() {
let text = "say \u{201C}hello\u{201D} world";
let matches = unwrap_matches(find_normalized_match_positions(text, "\"hello\""));
let (new_text, new_positions) = replace_normalized_matches(text, &matches, "\"goodbye\"");
assert_eq!(new_text, "say \"goodbye\" world");
assert_eq!(new_positions.len(), 1);
}
#[test]
fn replace_normalized_matches_preserves_surrounding() {
let text = "before \u{201C}target\u{201D} after";
let matches = unwrap_matches(find_normalized_match_positions(text, "\"target\""));
let (new_text, _) = replace_normalized_matches(text, &matches, "\"replaced\"");
assert!(new_text.starts_with("before "));
assert!(new_text.ends_with(" after"));
assert!(new_text.contains("\"replaced\""));
}
#[test]
fn replace_normalized_matches_at_end_of_string() {
let text = "prefix\u{2026}";
let matches = unwrap_matches(find_normalized_match_positions(text, "prefix..."));
let (new_text, _) = replace_normalized_matches(text, &matches, "done");
assert_eq!(new_text, "done");
}
#[test]
fn replace_normalized_result_is_valid_utf8() {
let text = "a\u{201C}b\u{2014}c\u{00A0}d";
let matches = unwrap_matches(find_normalized_match_positions(text, "\"b--c d"));
let (new_text, _) = replace_normalized_matches(text, &matches, "replaced");
assert_eq!(new_text, "areplaced");
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,92 @@
//! Legacy (0.4.10) error downgrade for `search_replace`.
//!
//! Restores exact historical 0.4.10 wording by collapsing structured error
//! variants (`FileNotFound`, `MultipleMatchesFound`, etc.) to generic
//! `InvalidInput`.
use crate::types::output::SearchReplaceOutput;
use crate::types::resources::SharedResources;
use crate::types::template_renderer::TemplateRenderer;
#[derive(Debug, Clone, Copy)]
pub(crate) enum ErrorKind {
FileAlreadyExists,
FileNotFound,
MultipleMatchesFound,
NoMatchesFound,
}
#[derive(Debug, Clone)]
struct RenderContext {
file_path: String,
read_tool_name: String,
replace_all_param_name: String,
old_string_param_name: String,
}
fn render_error(kind: ErrorKind, ctx: &RenderContext) -> String {
match kind {
ErrorKind::FileAlreadyExists => format!(
"{} is empty, which is only allowed when creating a new file or when the file is empty.",
ctx.old_string_param_name
),
ErrorKind::FileNotFound => {
format!(
"File not found: {}. Please check the path and try again.",
ctx.file_path
)
}
ErrorKind::MultipleMatchesFound => format!(
"The string to replace was found multiple times in the file. Use {} to replace all occurrences, or include more context to only edit one occurrence.",
ctx.replace_all_param_name
),
ErrorKind::NoMatchesFound => format!(
"The string to replace was not found in the file, use the {} tool to see the correct string.",
ctx.read_tool_name
),
}
}
async fn build_render_context(
resources: &SharedResources,
file_path: &str,
) -> Result<RenderContext, kigi_tool_runtime::ToolError> {
let res = resources.lock().await;
let renderer = res.require::<TemplateRenderer>()?;
let read_tool_name = renderer
.render("${{ tools.by_kind.read }}")
.map_err(|e| kigi_tool_runtime::ToolError::invalid_arguments(e.to_string()))?;
let replace_all_param_name = renderer
.render("${{ params.edit.replace_all }}")
.map_err(|e| kigi_tool_runtime::ToolError::invalid_arguments(e.to_string()))?;
let old_string_param_name = renderer
.render("${{ params.edit.old_string }}")
.map_err(|e| kigi_tool_runtime::ToolError::invalid_arguments(e.to_string()))?;
Ok(RenderContext {
file_path: file_path.to_string(),
read_tool_name,
replace_all_param_name,
old_string_param_name,
})
}
/// Downgrade structured error variants to generic `InvalidInput` for legacy,
/// restoring exact historical 0.4.10 wording.
pub(crate) async fn downgrade_structured_errors(
output: SearchReplaceOutput,
resources: &SharedResources,
file_path: &str,
) -> Result<SearchReplaceOutput, kigi_tool_runtime::ToolError> {
use SearchReplaceOutput::*;
let ctx = build_render_context(resources, file_path).await?;
Ok(match output {
FileAlreadyExists(_) => InvalidInput(render_error(ErrorKind::FileAlreadyExists, &ctx)),
FileNotFound(_) => InvalidInput(render_error(ErrorKind::FileNotFound, &ctx)),
FilenameTooLong(msg) => InvalidInput(msg),
MultipleMatchesFound(_) => {
InvalidInput(render_error(ErrorKind::MultipleMatchesFound, &ctx))
}
NoMatchesFound(_) => InvalidInput(render_error(ErrorKind::NoMatchesFound, &ctx)),
other => other,
})
}
@@ -0,0 +1,8 @@
//! Version-specific behavior modules for `search_replace`.
//!
//! - `legacy_0_4_10`: error downgrade logic that restores exact historical
//! 0.4.10 wording by collapsing structured error variants to `InvalidInput`.
//! - Current behavior remains in `search_replace/mod.rs` (structured errors,
//! gitignore enforcement, confusable diagnostics).
pub(crate) mod legacy_0_4_10;
@@ -0,0 +1,250 @@
//! Session-scoped file storage with crash-safe atomic writes and budgets.
//!
//! Each [`SessionFileWriter`] manages a single subdirectory and file
//! extension, producing files named `1.jpg`, `2.mp4`, `3.pdf`, etc.
//! The counter is lazily initialised from existing files on disk so
//! resumed sessions don't overwrite previous output.
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicU32, AtomicU64, Ordering};
use anyhow::Context;
const IMAGE_MAX_BYTES: u64 = 1024 * 1024 * 1024; // 1 GB
const VIDEO_MAX_BYTES: u64 = 2 * 1024 * 1024 * 1024; // 2 GB
const DEFAULT_MAX_BYTES: u64 = 1024 * 1024 * 1024; // 1 GB
fn budget_for(dir_name: &str) -> u64 {
match dir_name {
"images" => IMAGE_MAX_BYTES,
"videos" => VIDEO_MAX_BYTES,
_ => DEFAULT_MAX_BYTES,
}
}
/// Persists numbered files to `<session_folder>/<dir_name>/<N>.<ext>`.
///
/// Writes are crash-safe: data is written to a temp file, fsynced, then
/// atomically renamed into place via `tempfile::NamedTempFile::persist`.
#[derive(Clone, Debug)]
pub(crate) struct SessionFileWriter {
dir_name: &'static str,
ext: &'static str,
counter: Arc<AtomicU32>,
bytes_written: Arc<AtomicU64>,
max_total_bytes: u64,
}
impl SessionFileWriter {
pub(crate) fn new(dir_name: &'static str, ext: &'static str) -> Self {
let max_total_bytes = budget_for(dir_name);
Self {
dir_name,
ext,
counter: Arc::new(AtomicU32::new(0)),
bytes_written: Arc::new(AtomicU64::new(0)),
max_total_bytes,
}
}
/// Save `bytes` to the next numbered file, returning the absolute path.
///
/// `ext_override` writes a different file type without needing a
/// separate writer (e.g. saving a PNG from a JPG-default writer).
#[tracing::instrument(skip_all, fields(dir = self.dir_name))]
pub(crate) async fn save(
&self,
session_folder: &Path,
bytes: &[u8],
ext_override: Option<&str>,
) -> anyhow::Result<PathBuf> {
let dir = session_folder.join(self.dir_name);
if !dir.exists() {
tokio::fs::create_dir_all(&dir).await?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
tokio::fs::set_permissions(&dir, std::fs::Permissions::from_mode(0o700)).await?;
}
}
// Lazy-init: scan existing files for counter resume + byte accounting.
// compare_exchange ensures only the winner initialises; losers no-op.
if self.counter.load(Ordering::Relaxed) == 0
&& let Ok((max_n, total_bytes)) = scan_dir_stats(&dir).await
&& self
.counter
.compare_exchange(0, max_n, Ordering::Relaxed, Ordering::Relaxed)
.is_ok()
{
self.bytes_written.fetch_add(total_bytes, Ordering::Relaxed);
}
let n = self.counter.fetch_add(1, Ordering::Relaxed) + 1;
let size = bytes.len() as u64;
let new_total = self.bytes_written.fetch_add(size, Ordering::Relaxed) + size;
if new_total > self.max_total_bytes {
self.bytes_written.fetch_sub(size, Ordering::Relaxed);
self.counter.fetch_sub(1, Ordering::Relaxed);
anyhow::bail!(
"byte budget exceeded: {}/{} bytes in {}",
new_total,
self.max_total_bytes,
self.dir_name
);
}
let ext = ext_override.unwrap_or(self.ext);
let path = dir.join(format!("{n}.{ext}"));
// Atomic write: tempfile -> sync_all -> persist, all in one
// spawn_blocking call to avoid blocking the async executor.
let dest = path.clone();
let target_dir = dir;
let data = bytes.to_vec();
let result: anyhow::Result<()> = tokio::task::spawn_blocking(move || {
let tmp = tempfile::NamedTempFile::new_in(&target_dir).context("create temp file")?;
std::fs::write(tmp.path(), &data).context("write temp file")?;
tmp.as_file().sync_all().context("fsync temp file")?;
tmp.persist(&dest)
.with_context(|| format!("persist to {}", dest.display()))?;
Ok(())
})
.await
.context("spawn_blocking panicked")?;
if let Err(e) = result {
self.counter.fetch_sub(1, Ordering::Relaxed);
self.bytes_written.fetch_sub(size, Ordering::Relaxed);
return Err(e);
}
tracing::debug!(bytes = size, "file saved");
Ok(path)
}
}
/// Scan `dir` for stats: (max counter N from any `N.*` file, total bytes of all files).
/// Also removes orphan `.tmp*` files left behind by interrupted writes.
async fn scan_dir_stats(dir: &Path) -> Result<(u32, u64), std::io::Error> {
let mut max = 0u32;
let mut total_bytes = 0u64;
let mut entries = tokio::fs::read_dir(dir).await?;
while let Ok(Some(entry)) = entries.next_entry().await {
let name = entry.file_name();
let name_str = name.to_string_lossy();
// Remove orphan temp files from interrupted writes
if name_str.starts_with(".tmp") {
let _ = tokio::fs::remove_file(entry.path()).await;
continue;
}
// Track the highest numbered file (any extension)
if let Some(stem) = name_str.split_once('.').map(|(s, _)| s)
&& let Ok(n) = stem.parse::<u32>()
{
max = max.max(n);
}
// Sum bytes for budget init
if let Ok(meta) = entry.metadata().await {
total_bytes += meta.len();
}
}
Ok((max, total_bytes))
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn save_creates_numbered_files() {
let tmp = tempfile::tempdir().unwrap();
let writer = SessionFileWriter::new("images", "jpg");
let p1 = writer.save(tmp.path(), b"img1", None).await.unwrap();
let p2 = writer.save(tmp.path(), b"img2", None).await.unwrap();
assert_eq!(p1.file_name().unwrap(), "1.jpg");
assert_eq!(p2.file_name().unwrap(), "2.jpg");
assert_eq!(tokio::fs::read(&p1).await.unwrap(), b"img1");
assert_eq!(tokio::fs::read(&p2).await.unwrap(), b"img2");
}
#[tokio::test]
async fn save_resumes_counter_from_existing_files() {
let tmp = tempfile::tempdir().unwrap();
let dir = tmp.path().join("downloads");
tokio::fs::create_dir_all(&dir).await.unwrap();
tokio::fs::write(dir.join("1.pdf"), b"").await.unwrap();
tokio::fs::write(dir.join("5.pdf"), b"").await.unwrap();
tokio::fs::write(dir.join("readme.txt"), b"").await.unwrap();
let writer = SessionFileWriter::new("downloads", "pdf");
let path = writer.save(tmp.path(), b"new", None).await.unwrap();
assert_eq!(path.file_name().unwrap(), "6.pdf");
}
#[tokio::test]
async fn save_uses_ext_override() {
let tmp = tempfile::tempdir().unwrap();
let writer = SessionFileWriter::new("downloads", "pdf");
let p1 = writer.save(tmp.path(), b"doc", Some("docx")).await.unwrap();
assert_eq!(p1.file_name().unwrap(), "1.docx");
}
#[tokio::test]
async fn scan_dir_stats_empty_dir() {
let tmp = tempfile::tempdir().unwrap();
let (max, bytes) = scan_dir_stats(tmp.path()).await.unwrap();
assert_eq!(max, 0);
assert_eq!(bytes, 0);
}
#[tokio::test]
async fn scan_dir_stats_finds_max_and_bytes() {
let tmp = tempfile::tempdir().unwrap();
tokio::fs::write(tmp.path().join("1.mp4"), b"aaaa")
.await
.unwrap();
tokio::fs::write(tmp.path().join("3.mp4"), b"bbbbbb")
.await
.unwrap();
tokio::fs::write(tmp.path().join("2.mp4"), b"cc")
.await
.unwrap();
tokio::fs::write(tmp.path().join("hero.mp4"), b"")
.await
.unwrap();
let (max, bytes) = scan_dir_stats(tmp.path()).await.unwrap();
assert_eq!(max, 3);
assert_eq!(bytes, 4 + 6 + 2); // aaaa + bbbbbb + cc
}
#[tokio::test]
async fn scan_dir_stats_cleans_orphan_tmp() {
let tmp = tempfile::tempdir().unwrap();
tokio::fs::write(tmp.path().join(".tmp123"), b"orphan")
.await
.unwrap();
tokio::fs::write(tmp.path().join("1.jpg"), b"keep")
.await
.unwrap();
let (max, _) = scan_dir_stats(tmp.path()).await.unwrap();
assert_eq!(max, 1);
assert!(!tmp.path().join(".tmp123").exists());
}
#[tokio::test]
async fn scan_dir_stats_missing_dir() {
let dir = Path::new("/nonexistent/path/that/does/not/exist");
assert!(scan_dir_stats(dir).await.is_err());
}
}
@@ -0,0 +1,832 @@
//! Backend trait abstracting how subagent operations are dispatched.
//!
//! `SubagentBackend` decouples the tool implementations (`TaskTool`,
//! `TaskOutputTool`, `KillTaskTool`) from the transport mechanism used to
//! communicate with the subagent coordinator.
//!
//! Two implementations are planned:
//!
//! - [`ChannelBackend`] — wraps in-process `tokio::mpsc` channels used by
//! the local host shell. This is the only implementation today.
//! - `RemoteBackend` (future) — dispatches over a remote transport to an
//! out-of-process spawner.
use std::sync::Arc;
use tokio::sync::{mpsc, oneshot};
use super::types::{
SubagentCancelOutcome, SubagentCancelRequest, SubagentCancelTarget, SubagentDescribeOutcome,
SubagentDescribeRequest, SubagentEvent, SubagentQueryRequest, SubagentRequest, SubagentResult,
SubagentSnapshot, SubagentValidateTypeOutcome, SubagentValidateTypeRequest,
};
use crate::register_resource;
use kigi_tool_runtime::ToolError;
/// Abstraction over the mechanism used to spawn, query, and cancel subagents.
///
/// Injected into `Resources` as [`SubagentBackendResource`] so that
/// `TaskTool`, `TaskOutputTool`, and `KillTaskTool` can operate
/// identically regardless of the underlying transport.
#[async_trait::async_trait]
pub trait SubagentBackend: Send + Sync + 'static {
/// Spawn a subagent and await its result.
///
/// For blocking mode the caller awaits the returned future directly.
/// For background mode the caller spawns a tokio task around this call
/// and drops the receiver immediately.
async fn spawn(&self, request: SubagentRequest) -> Result<SubagentResult, ToolError>;
/// Query the current state of a subagent by ID.
///
/// When `block` is true the backend waits (up to `timeout_ms`) for the
/// subagent to reach a terminal state before responding.
async fn query(
&self,
id: &str,
block: bool,
timeout_ms: Option<u64>,
) -> Option<SubagentSnapshot>;
/// Request cancellation of a subagent by ID.
async fn cancel(&self, id: &str) -> SubagentCancelOutcome;
/// Validate a subagent type synchronously before spawning.
/// Returns `ValidationUnavailable` on channel close / responder drop / timeout.
async fn validate_type(
&self,
subagent_type: &str,
parent_session_id: &str,
) -> SubagentValidateTypeOutcome;
/// Describe a subagent type's resolved toolset (tool names + capability
/// flags) before spawning. Read-only: builds the agent definition and
/// applies the same parent-dependent toolset re-selection a spawn would,
/// then reports the result without starting a child session.
///
/// Returns [`SubagentDescribeOutcome::Unavailable`] on channel close /
/// responder drop / timeout (modeled exactly on [`Self::validate_type`]).
///
/// `harness_agent_type` is the `/goal`-only harness override (see
/// [`super::types::SubagentRuntimeOverrides::harness_agent_type`]); the
/// coordinator resolves the toolset for `(subagent_type,
/// harness_agent_type)`. `None` (every non-goal caller) defers the flavor
/// to the parent agent.
async fn describe_subagent_type(
&self,
subagent_type: &str,
harness_agent_type: Option<&str>,
parent_session_id: &str,
) -> SubagentDescribeOutcome;
}
/// Resource wrapper injected into every session's `Resources`.
///
/// Wraps an `Arc<dyn SubagentBackend>` so the backend can be shared across
/// concurrent tool invocations within the same session.
#[derive(Clone)]
pub struct SubagentBackendResource(pub Arc<dyn SubagentBackend>);
impl SubagentBackendResource {
pub fn backend(&self) -> &dyn SubagentBackend {
self.0.as_ref()
}
}
impl std::fmt::Debug for SubagentBackendResource {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("SubagentBackendResource").finish()
}
}
register_resource!(
"grok_build",
"SubagentBackendResource",
SubagentBackendResource
);
/// In-process channel-based backend for the local host shell.
///
/// Wraps a single `mpsc::UnboundedSender<SubagentEvent>` that carries
/// spawn, query, and cancel messages to the coordinator. The oneshot for
/// `spawn` is created inside the backend so callers never manage it.
pub struct ChannelBackend {
tx: mpsc::UnboundedSender<SubagentEvent>,
}
impl ChannelBackend {
pub fn new(tx: mpsc::UnboundedSender<SubagentEvent>) -> Self {
Self { tx }
}
}
#[async_trait::async_trait]
impl SubagentBackend for ChannelBackend {
async fn spawn(&self, request: SubagentRequest) -> Result<SubagentResult, ToolError> {
let (result_tx, result_rx) = oneshot::channel();
// Replace the dummy oneshot with our fresh one. Using struct update
// syntax (`..request`) ensures new fields added to `SubagentRequest`
// are forwarded automatically — a field-by-field copy would silently
// drop them.
self.tx
.send(SubagentEvent::Spawn(Box::new(SubagentRequest {
result_tx,
..request
})))
.map_err(|_| {
ToolError::custom(
"channel_closed",
"Subagent coordinator channel closed — cannot spawn subagent",
)
})?;
result_rx.await.map_err(|_| {
ToolError::custom(
"channel_closed",
"Subagent result channel dropped — child session may have crashed",
)
})
}
async fn query(
&self,
id: &str,
block: bool,
timeout_ms: Option<u64>,
) -> Option<SubagentSnapshot> {
let (respond_to, response_rx) = oneshot::channel();
let sent = self.tx.send(SubagentEvent::Query(SubagentQueryRequest {
subagent_id: id.to_string(),
block,
timeout_ms,
respond_to,
}));
if sent.is_err() {
return None;
}
response_rx.await.ok().flatten()
}
async fn cancel(&self, id: &str) -> SubagentCancelOutcome {
let (respond_to, response_rx) = oneshot::channel();
let sent = self.tx.send(SubagentEvent::Cancel(SubagentCancelRequest {
target: SubagentCancelTarget::SubagentId(id.to_string()),
respond_to,
}));
if sent.is_err() {
return SubagentCancelOutcome::NotFound;
}
response_rx.await.unwrap_or(SubagentCancelOutcome::NotFound)
}
async fn validate_type(
&self,
subagent_type: &str,
parent_session_id: &str,
) -> SubagentValidateTypeOutcome {
let (respond_to, response_rx) = oneshot::channel();
if self
.tx
.send(SubagentEvent::ValidateType(SubagentValidateTypeRequest {
subagent_type: subagent_type.to_string(),
parent_session_id: parent_session_id.to_string(),
respond_to,
}))
.is_err()
{
tracing::warn!(
subagent_type,
"coordinator validation channel closed, treating as ValidationUnavailable",
);
return SubagentValidateTypeOutcome::ValidationUnavailable;
}
let timeout = validate_type_timeout();
match tokio::time::timeout(timeout, response_rx).await {
Ok(Ok(outcome)) => outcome,
Ok(Err(_)) => {
tracing::warn!(
subagent_type,
"coordinator validation responder dropped, treating as ValidationUnavailable",
);
SubagentValidateTypeOutcome::ValidationUnavailable
}
Err(_) => {
tracing::warn!(
subagent_type,
timeout_ms = timeout.as_millis() as u64,
"coordinator validation timed out, treating as ValidationUnavailable",
);
SubagentValidateTypeOutcome::ValidationUnavailable
}
}
}
async fn describe_subagent_type(
&self,
subagent_type: &str,
harness_agent_type: Option<&str>,
parent_session_id: &str,
) -> SubagentDescribeOutcome {
let (respond_to, response_rx) = oneshot::channel();
if self
.tx
.send(SubagentEvent::DescribeType(SubagentDescribeRequest {
subagent_type: subagent_type.to_string(),
harness_agent_type: harness_agent_type.map(str::to_string),
parent_session_id: parent_session_id.to_string(),
respond_to,
}))
.is_err()
{
tracing::warn!(
subagent_type,
"coordinator describe channel closed, treating as Unavailable",
);
return SubagentDescribeOutcome::Unavailable;
}
let timeout = validate_type_timeout();
match tokio::time::timeout(timeout, response_rx).await {
Ok(Ok(outcome)) => outcome,
Ok(Err(_)) => {
tracing::warn!(
subagent_type,
"coordinator describe responder dropped, treating as Unavailable",
);
SubagentDescribeOutcome::Unavailable
}
Err(_) => {
tracing::warn!(
subagent_type,
timeout_ms = timeout.as_millis() as u64,
"coordinator describe timed out, treating as Unavailable",
);
SubagentDescribeOutcome::Unavailable
}
}
}
}
/// Default `validate_type` timeout. Override via [`VALIDATE_TYPE_TIMEOUT_ENV_VAR`].
pub const VALIDATE_TYPE_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(2);
/// Env-var override for [`VALIDATE_TYPE_TIMEOUT`] (positive milliseconds).
pub const VALIDATE_TYPE_TIMEOUT_ENV_VAR: &str = "XAI_VALIDATE_TYPE_TIMEOUT_MS";
/// Validation timeout, honoring the env-var override.
pub(crate) fn validate_type_timeout() -> std::time::Duration {
let raw = std::env::var(VALIDATE_TYPE_TIMEOUT_ENV_VAR).ok();
parse_timeout_ms(raw.as_deref())
.map(std::time::Duration::from_millis)
.unwrap_or(VALIDATE_TYPE_TIMEOUT)
}
/// Parse a positive `u64` millisecond value; `None` for unset, invalid, or zero.
pub(crate) fn parse_timeout_ms(value: Option<&str>) -> Option<u64> {
value?.parse::<u64>().ok().filter(|&ms| ms > 0)
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use tokio::sync::mpsc;
/// Helper: receive the next event, match the expected variant, or panic.
macro_rules! recv_event {
($rx:expr, Spawn) => {{
let event = $rx.recv().await.unwrap();
match event {
SubagentEvent::Spawn(inner) => *inner,
_ => panic!("Expected SubagentEvent::Spawn, got different variant"),
}
}};
($rx:expr, $variant:ident) => {{
let event = $rx.recv().await.unwrap();
match event {
SubagentEvent::$variant(inner) => inner,
_ => panic!(
"Expected SubagentEvent::{}, got different variant",
stringify!($variant)
),
}
}};
}
#[tokio::test]
async fn channel_backend_spawn_success() {
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
let req = recv_event!(rx, Spawn);
assert_eq!(req.id, "test-id");
assert_eq!(req.prompt, "do something");
req.result_tx
.send(SubagentResult {
success: true,
output: Arc::from("done"),
subagent_id: "test-id".to_string(),
child_session_id: "test-id".to_string(),
tool_calls: 3,
turns: 1,
duration_ms: 500,
..Default::default()
})
.unwrap();
});
let (dummy_tx, _dummy_rx) = oneshot::channel();
let request = SubagentRequest {
id: "test-id".to_string(),
prompt: "do something".to_string(),
description: "test".to_string(),
subagent_type: "general-purpose".to_string(),
parent_session_id: "parent".to_string(),
parent_prompt_id: None,
resume_from: None,
cwd: None,
runtime_overrides: Default::default(),
run_in_background: false,
surface_completion: true,
fork_context: false,
result_tx: dummy_tx,
};
let result = backend.spawn(request).await.unwrap();
assert!(result.success);
assert_eq!(result.subagent_id, "test-id");
assert_eq!(result.tool_calls, 3);
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_spawn_closed_channel() {
let (tx, rx) = mpsc::unbounded_channel::<SubagentEvent>();
drop(rx);
let backend = ChannelBackend::new(tx);
let (dummy_tx, _dummy_rx) = oneshot::channel();
let request = SubagentRequest {
id: "test-id".to_string(),
prompt: "do something".to_string(),
description: "test".to_string(),
subagent_type: "general-purpose".to_string(),
parent_session_id: "parent".to_string(),
parent_prompt_id: None,
resume_from: None,
cwd: None,
runtime_overrides: Default::default(),
run_in_background: false,
surface_completion: true,
fork_context: false,
result_tx: dummy_tx,
};
let err = backend.spawn(request).await.unwrap_err();
assert!(err.to_string().contains("channel closed"));
}
#[tokio::test]
async fn channel_backend_query_found() {
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
let req = recv_event!(rx, Query);
assert_eq!(req.subagent_id, "sub-1");
assert!(req.block);
assert_eq!(req.timeout_ms, Some(5000));
req.respond_to
.send(Some(SubagentSnapshot {
subagent_id: "sub-1".to_string(),
description: "find bugs".to_string(),
subagent_type: "explore".to_string(),
status: super::super::types::SubagentSnapshotStatus::Completed {
output: "result".to_string(),
tool_calls: 2,
turns: 1,
worktree_path: None,
},
started_at_epoch_ms: 1000,
duration_ms: 200,
persona: Some("reviewer".to_string()),
}))
.unwrap();
});
let snap = backend.query("sub-1", true, Some(5000)).await;
let snap = snap.expect("snapshot should be present");
assert_eq!(snap.subagent_id, "sub-1");
assert_eq!(snap.description, "find bugs");
assert_eq!(snap.subagent_type, "explore");
assert_eq!(snap.started_at_epoch_ms, 1000);
assert_eq!(snap.duration_ms, 200);
assert_eq!(snap.persona.as_deref(), Some("reviewer"));
match &snap.status {
super::super::types::SubagentSnapshotStatus::Completed {
output,
tool_calls,
turns,
worktree_path,
} => {
assert_eq!(output, "result");
assert_eq!(*tool_calls, 2);
assert_eq!(*turns, 1);
assert!(worktree_path.is_none());
}
other => panic!("Expected Completed, got {:?}", other),
}
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_query_non_blocking_passes_through() {
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
let req = recv_event!(rx, Query);
assert_eq!(req.subagent_id, "sub-nb");
assert!(!req.block, "block should be false");
assert_eq!(req.timeout_ms, None, "timeout_ms should be None");
req.respond_to.send(None).unwrap();
});
let snap = backend.query("sub-nb", false, None).await;
assert!(snap.is_none());
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_query_not_found() {
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
let req = recv_event!(rx, Query);
req.respond_to.send(None).unwrap();
});
let snap = backend.query("nonexistent", false, None).await;
assert!(snap.is_none());
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_cancel_success() {
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
let req = recv_event!(rx, Cancel);
match &req.target {
SubagentCancelTarget::SubagentId(id) => assert_eq!(id, "sub-cancel"),
other => panic!("Expected SubagentId, got {:?}", other),
}
req.respond_to
.send(SubagentCancelOutcome::Cancelled)
.unwrap();
});
let outcome = backend.cancel("sub-cancel").await;
assert!(matches!(outcome, SubagentCancelOutcome::Cancelled));
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_cancel_closed_channel() {
let (tx, rx) = mpsc::unbounded_channel::<SubagentEvent>();
drop(rx);
let backend = ChannelBackend::new(tx);
let outcome = backend.cancel("sub-cancel").await;
assert!(matches!(outcome, SubagentCancelOutcome::NotFound));
}
#[tokio::test]
async fn channel_backend_spawn_result_dropped() {
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
let req = recv_event!(rx, Spawn);
drop(req.result_tx);
});
let (dummy_tx, _dummy_rx) = oneshot::channel();
let request = SubagentRequest {
id: "drop-test".to_string(),
prompt: "test".to_string(),
description: "test".to_string(),
subagent_type: "general-purpose".to_string(),
parent_session_id: "parent".to_string(),
parent_prompt_id: None,
resume_from: None,
cwd: None,
runtime_overrides: Default::default(),
run_in_background: false,
surface_completion: true,
fork_context: false,
result_tx: dummy_tx,
};
let err = backend.spawn(request).await.unwrap_err();
assert!(
err.to_string().contains("result channel dropped"),
"error: {err}"
);
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_query_closed_channel() {
let (tx, rx) = mpsc::unbounded_channel::<SubagentEvent>();
drop(rx);
let backend = ChannelBackend::new(tx);
let snap = backend.query("sub-1", false, None).await;
assert!(snap.is_none());
}
// ── validate_type ────────────────────────────────────────────────
#[tokio::test]
async fn channel_backend_validate_type_round_trips_outcome() {
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
let event = rx.recv().await.unwrap();
match event {
SubagentEvent::ValidateType(req) => {
assert_eq!(req.subagent_type, "explore");
assert_eq!(req.parent_session_id, "parent-1");
req.respond_to
.send(SubagentValidateTypeOutcome::Ok)
.unwrap();
}
_ => panic!("Expected ValidateType event"),
}
});
let outcome = backend.validate_type("explore", "parent-1").await;
assert!(matches!(outcome, SubagentValidateTypeOutcome::Ok));
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_validate_type_propagates_unknown_outcome() {
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
if let Some(SubagentEvent::ValidateType(req)) = rx.recv().await {
req.respond_to
.send(SubagentValidateTypeOutcome::Unknown {
available: vec!["explore".into(), "plan".into()],
})
.unwrap();
}
});
let outcome = backend.validate_type("invented", "p").await;
match outcome {
SubagentValidateTypeOutcome::Unknown { available } => {
assert_eq!(available, vec!["explore".to_string(), "plan".to_string()]);
}
other => panic!("expected Unknown, got {other:?}"),
}
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_validate_type_returns_validation_unavailable_when_channel_closed() {
let (tx, rx) = mpsc::unbounded_channel::<SubagentEvent>();
drop(rx);
let backend = ChannelBackend::new(tx);
let outcome = backend.validate_type("explore", "p").await;
assert!(matches!(
outcome,
SubagentValidateTypeOutcome::ValidationUnavailable
));
}
#[tokio::test]
async fn channel_backend_validate_type_returns_validation_unavailable_when_responder_dropped() {
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
if let Some(SubagentEvent::ValidateType(req)) = rx.recv().await {
drop(req.respond_to);
}
});
let outcome = backend.validate_type("explore", "p").await;
assert!(matches!(
outcome,
SubagentValidateTypeOutcome::ValidationUnavailable,
));
handle.await.unwrap();
}
use super::super::types::test_capture;
#[tokio::test(start_paused = true)]
async fn channel_backend_validate_type_logs_warn_on_timeout() {
let captured = test_capture::capture();
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
// Coordinator receives but never replies; keeps the responder
// alive so the timeout arm fires (not responder-dropped).
let holder = tokio::spawn(async move {
if let Some(SubagentEvent::ValidateType(req)) = rx.recv().await {
std::mem::forget(req.respond_to);
std::future::pending::<()>().await;
}
});
let validate = tokio::spawn(async move { backend.validate_type("explore", "p").await });
tokio::time::advance(VALIDATE_TYPE_TIMEOUT + std::time::Duration::from_millis(1)).await;
let outcome = validate.await.unwrap();
assert!(matches!(
outcome,
SubagentValidateTypeOutcome::ValidationUnavailable
));
let mut events_rx = captured.events_rx;
let mut saw_timeout_warn = false;
while let Ok(event) = events_rx.try_recv() {
if event.level == tracing::Level::WARN
&& event.fields.contains("coordinator validation timed out")
&& event.fields.contains("subagent_type=explore")
&& event.fields.contains("timeout_ms=")
{
saw_timeout_warn = true;
break;
}
}
assert!(saw_timeout_warn, "must emit WARN with timeout_ms field");
holder.abort();
}
// ── describe_subagent_type ───────────────────────────────────────
#[tokio::test]
async fn channel_backend_describe_round_trips_summary() {
use super::super::types::{SubagentDescribeOutcome, SubagentTypeSummary};
use crate::types::tool::ToolKind;
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
match rx.recv().await.unwrap() {
SubagentEvent::DescribeType(req) => {
assert_eq!(req.subagent_type, "explore");
assert_eq!(req.harness_agent_type.as_deref(), Some("cursor"));
assert_eq!(req.parent_session_id, "parent-1");
let mut summary = SubagentTypeSummary {
can_read: true,
can_search: true,
..Default::default()
};
summary
.tool_names
.insert(ToolKind::Read, "read_file".to_string());
req.respond_to
.send(SubagentDescribeOutcome::Ok(summary))
.unwrap();
}
_ => panic!("Expected DescribeType event"),
}
});
let outcome = backend
.describe_subagent_type("explore", Some("cursor"), "parent-1")
.await;
match outcome {
SubagentDescribeOutcome::Ok(summary) => {
assert!(summary.can_read && summary.can_search && !summary.can_execute);
assert_eq!(
summary.tool_names.get(&ToolKind::Read).unwrap(),
"read_file"
);
}
other => panic!("expected Ok, got {other:?}"),
}
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_describe_propagates_not_allowed_outcome() {
use super::super::types::SubagentDescribeOutcome;
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
if let Some(SubagentEvent::DescribeType(req)) = rx.recv().await {
req.respond_to
.send(SubagentDescribeOutcome::NotAllowed {
allowed: vec!["explore".into()],
})
.unwrap();
}
});
match backend.describe_subagent_type("plan", None, "p").await {
SubagentDescribeOutcome::NotAllowed { allowed } => {
assert_eq!(allowed, vec!["explore".to_string()]);
}
other => panic!("expected NotAllowed, got {other:?}"),
}
handle.await.unwrap();
}
#[tokio::test]
async fn channel_backend_describe_returns_unavailable_when_channel_closed() {
use super::super::types::SubagentDescribeOutcome;
let (tx, rx) = mpsc::unbounded_channel::<SubagentEvent>();
drop(rx);
let backend = ChannelBackend::new(tx);
assert!(matches!(
backend.describe_subagent_type("explore", None, "p").await,
SubagentDescribeOutcome::Unavailable
));
}
#[tokio::test]
async fn channel_backend_describe_returns_unavailable_when_responder_dropped() {
use super::super::types::SubagentDescribeOutcome;
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let handle = tokio::spawn(async move {
if let Some(SubagentEvent::DescribeType(req)) = rx.recv().await {
drop(req.respond_to);
}
});
assert!(matches!(
backend.describe_subagent_type("explore", None, "p").await,
SubagentDescribeOutcome::Unavailable
));
handle.await.unwrap();
}
#[tokio::test(start_paused = true)]
async fn channel_backend_describe_returns_unavailable_on_timeout() {
use super::super::types::SubagentDescribeOutcome;
let (tx, mut rx) = mpsc::unbounded_channel::<SubagentEvent>();
let backend = ChannelBackend::new(tx);
let holder = tokio::spawn(async move {
if let Some(SubagentEvent::DescribeType(req)) = rx.recv().await {
std::mem::forget(req.respond_to);
std::future::pending::<()>().await;
}
});
let describe =
tokio::spawn(async move { backend.describe_subagent_type("explore", None, "p").await });
tokio::time::advance(VALIDATE_TYPE_TIMEOUT + std::time::Duration::from_millis(1)).await;
assert!(matches!(
describe.await.unwrap(),
SubagentDescribeOutcome::Unavailable
));
holder.abort();
}
#[test]
fn parse_timeout_ms_returns_none_for_unset() {
assert_eq!(parse_timeout_ms(None), None);
}
#[test]
fn parse_timeout_ms_returns_none_for_unparseable() {
assert_eq!(parse_timeout_ms(Some("not-a-number")), None);
assert_eq!(parse_timeout_ms(Some("")), None);
assert_eq!(parse_timeout_ms(Some("3.14")), None);
assert_eq!(parse_timeout_ms(Some("-100")), None);
}
#[test]
fn parse_timeout_ms_returns_none_for_zero() {
assert_eq!(parse_timeout_ms(Some("0")), None);
}
#[test]
fn parse_timeout_ms_returns_value_for_positive_integer() {
assert_eq!(parse_timeout_ms(Some("5000")), Some(5000));
assert_eq!(parse_timeout_ms(Some("1")), Some(1));
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,160 @@
//! Workspace-only, subagent-free variant of `get_task_output` (delegates to [`TaskOutputTool`]).
use super::{TaskOutputTool, background_bash_requires_exprs};
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
use kigi_tool_types::{TaskOutputOutput, TaskOutputToolInput};
fn terminal_command_output_requires_expr() -> Expr<ToolRequirement> {
Expr::Or(background_bash_requires_exprs())
}
#[derive(Debug, Default)]
pub struct GetTerminalCommandOutputTool;
impl crate::types::tool_metadata::ToolMetadata for GetTerminalCommandOutputTool {
fn kind(&self) -> ToolKind {
ToolKind::BackgroundTaskAction
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
r#"Get output and status from a background terminal command${%- if tools.by_kind.monitor %} or monitor${%- endif %}.
Usage notes:
- Pass task_ids with one or more ids from ${{ params.execute.is_background }}=true commands${%- if tools.by_kind.monitor %} (a monitor's task_id is returned by ${{ tools.by_kind.monitor }})${%- endif %}; for a single task use a one-element array. Multiple ids with a positive timeout_ms wait until all complete
- Omit timeout_ms or pass 0 for a non-blocking status snapshot; set a positive timeout_ms to wait up to that many milliseconds, capped at ~10 min
- Returns current output, status, and exit code if completed${%- if tools.by_kind.read %}
- If output is large, use ${{ tools.by_kind.read }} on the output_file path${%- endif %}"#
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
terminal_command_output_requires_expr()
}
fn is_read_only(&self) -> bool {
true
}
}
impl kigi_tool_runtime::Tool for GetTerminalCommandOutputTool {
type Args = TaskOutputToolInput;
type Output = TaskOutputOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("get_terminal_command_output").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"get_terminal_command_output",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.get_terminal_command_output",
skip_all,
fields(waits = %input.waits())
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: TaskOutputToolInput,
) -> Result<TaskOutputOutput, kigi_tool_runtime::ToolError> {
kigi_tool_runtime::Tool::run(&TaskOutputTool, ctx, input).await
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::implementations::grok_build::task_output::test_helpers::{
make_snapshot, resources_with_terminal,
};
use crate::types::tool_metadata::ToolMetadata;
use crate::types::tool_metadata::test_ctx;
#[test]
fn tool_name_and_description_are_subagent_free() {
let tool = GetTerminalCommandOutputTool;
assert_eq!(
kigi_tool_runtime::Tool::id(&tool).as_str(),
"get_terminal_command_output"
);
let tmpl = ToolMetadata::description_template(&tool);
assert!(tmpl.contains("background terminal command"));
assert!(
!tmpl.to_lowercase().contains("subagent"),
"workspace tool must not mention subagents: {tmpl}"
);
}
#[test]
fn is_read_only() {
let tool = GetTerminalCommandOutputTool;
assert!(ToolMetadata::is_read_only(&tool));
}
#[tokio::test]
async fn delegates_to_task_output_for_running_task() {
let snapshot = make_snapshot("tc-1", false, None);
let resources = resources_with_terminal(Some(snapshot));
let result = kigi_tool_runtime::Tool::run(
&GetTerminalCommandOutputTool,
test_ctx(resources.into_shared()),
TaskOutputToolInput {
task_ids: vec!["tc-1".into()],
timeout_ms: None,
},
)
.await
.unwrap();
match result {
TaskOutputOutput::Result(r) => {
assert_eq!(r.task_id, "tc-1");
assert_eq!(r.status, "running");
}
other => panic!("Expected Result, got {other:?}"),
}
}
#[tokio::test]
async fn delegates_to_task_output_for_completed_task() {
let snapshot = make_snapshot("tc-2", true, Some(0));
let resources = resources_with_terminal(Some(snapshot));
let result = kigi_tool_runtime::Tool::run(
&GetTerminalCommandOutputTool,
test_ctx(resources.into_shared()),
TaskOutputToolInput {
task_ids: vec!["tc-2".into()],
timeout_ms: None,
},
)
.await
.unwrap();
match result {
TaskOutputOutput::Result(r) => {
assert_eq!(r.status, "completed");
assert_eq!(r.exit_code, Some(0));
}
other => panic!("Expected Result, got {other:?}"),
}
}
}
@@ -0,0 +1,308 @@
//! `wait_tasks` tool — blocks until multiple background tasks complete.
//!
//! Prefer `get_task_output` / `get_command_or_subagent_output` with `task_ids`
//! and a positive `timeout_ms` (wait-all). This tool remains as a thin alias
//! for older prompts that still emit `wait_tasks` / `wait_commands_or_subagents`.
//!
//! `mode: wait_any` is still honored here for compatibility; the unified get
//! tool only supports wait-all for multi-id waits.
use crate::DEFAULT_TOOL_OUTPUT_BYTES;
use crate::implementations::grok_build::task::backend::SubagentBackendResource;
use crate::implementations::grok_build::task_output::{
MAX_MULTI_WAIT_IDS, TaskOutputTool, resolve_tasks, wait_any_event_driven,
};
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::resources::{Terminal, TruncationCfg};
use crate::types::template_renderer::TemplateRenderer;
use crate::types::tool::{ToolKind, ToolNamespace};
use kigi_tool_types::{MultiTaskOutputResult, TaskOutputOutput, WaitMode, WaitTasksToolInput};
#[derive(Debug, Default)]
pub struct WaitTasksTool;
impl crate::types::tool_metadata::ToolMetadata for WaitTasksTool {
fn kind(&self) -> ToolKind {
ToolKind::WaitTasksAction
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
// Canonical wording lives in the shared builder; `versioned_definition`
// renders it context-aware from the finalized toolset. This static
// fallback mirrors the default grok-build toolset.
static DESC: std::sync::LazyLock<String> = std::sync::LazyLock::new(|| {
kigi_tool_types::build_wait_tasks_description(&kigi_tool_types::WaitTasksToolNaming {
background_retrieval_tool: "get_command_or_subagent_output",
bash_background_param: Some("is_background"),
subagent_background_param: Some("run_in_background"),
})
});
&DESC
}
fn versioned_definition(
&self,
_contract_version: Option<&str>,
client_name: &str,
description_override: Option<&str>,
renderer: &TemplateRenderer,
param_map: &std::collections::HashMap<String, String>,
input_schema: &serde_json::Value,
_effective_params: &serde_json::Value,
) -> crate::types::definition::ToolDefinition {
let description = wait_tasks_description(renderer, description_override);
let remapped_schema = if param_map.is_empty() {
input_schema.clone()
} else {
crate::util::remap::remap_schema_properties(input_schema, param_map)
};
crate::types::definition::ToolDefinition::function(
client_name,
Some(&description),
remapped_schema,
)
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
super::task_output_requires_expr()
}
fn is_read_only(&self) -> bool {
true
}
}
/// Resolve the model-facing `wait_tasks` description from the finalized toolset,
/// honoring an explicit config override. Wording lives in the shared
/// [`kigi_tool_types::build_wait_tasks_description`] builder so the CLI and
/// prod-chat can't drift. When no dedicated background-retrieval tool is
/// registered, fall back to naming this tool's own get-output sibling.
fn wait_tasks_description(
renderer: &TemplateRenderer,
description_override: Option<&str>,
) -> String {
if let Some(ovr) = description_override {
return renderer.render(ovr).unwrap_or_else(|e| {
tracing::warn!("wait_tasks description override render failed, using raw: {e}");
ovr.to_string()
});
}
kigi_tool_types::build_wait_tasks_description(&kigi_tool_types::WaitTasksToolNaming {
background_retrieval_tool: renderer
.tool_for_kind(ToolKind::BackgroundTaskAction)
.unwrap_or("get_task_output"),
bash_background_param: renderer.param_for_kind(ToolKind::Execute, "is_background"),
subagent_background_param: renderer.param_for_kind(ToolKind::Task, "run_in_background"),
})
}
impl kigi_tool_runtime::Tool for WaitTasksTool {
type Args = WaitTasksToolInput;
type Output = TaskOutputOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("wait_tasks").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"wait_tasks",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.wait_tasks",
skip_all,
fields(task_count = %input.task_ids.len(), mode = ?input.mode)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: WaitTasksToolInput,
) -> Result<TaskOutputOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
if input.task_ids.is_empty() {
return Err(kigi_tool_runtime::ToolError::invalid_arguments(
"task_ids must not be empty.".to_string(),
));
}
if input.task_ids.len() > MAX_MULTI_WAIT_IDS {
return Err(kigi_tool_runtime::ToolError::invalid_arguments(format!(
"task_ids exceeds maximum of {MAX_MULTI_WAIT_IDS} entries."
)));
}
// wait_all (the common case) shares the unified multi path on get_task_output.
if matches!(input.mode, WaitMode::WaitAll) {
use super::DEFAULT_WAIT_TIMEOUT;
// Legacy wait always blocks: omit or 0 => default budget (not a snapshot).
let ms = input
.timeout_ms
.filter(|ms| *ms > 0)
.unwrap_or(DEFAULT_WAIT_TIMEOUT.as_millis() as u64);
return TaskOutputTool::run_multi_tasks(
&input.task_ids,
Some(ms),
resources,
"wait_commands_or_subagents",
)
.await;
}
// wait_any: keep legacy event-driven path (not exposed on get_task_output).
let timeout =
crate::implementations::grok_build::task_output::capped_wait_timeout(input.timeout_ms);
let (terminal, backend, read_file_name, max_output_bytes) = {
let res = resources.lock().await;
let terminal = res.require::<Terminal>()?.0.clone();
let backend = res.get::<SubagentBackendResource>().cloned();
let renderer = res.require::<TemplateRenderer>()?;
let rfn = renderer
.render("${{ tools.by_kind.read }}")
.map_err(|e| kigi_tool_runtime::ToolError::invalid_arguments(e.to_string()))?;
let mob = res
.get::<TruncationCfg>()
.map(|cfg| {
cfg.0.max_output_bytes_for(
"wait_commands_or_subagents",
DEFAULT_TOOL_OUTPUT_BYTES,
)
})
.unwrap_or(DEFAULT_TOOL_OUTPUT_BYTES);
(terminal, backend, rfn, mob)
};
let initial = resolve_tasks(
&input.task_ids,
&terminal,
&backend,
&read_file_name,
max_output_bytes,
)
.await;
let has_pending =
!initial.pending_bash_ids.is_empty() || !initial.pending_subagent_ids.is_empty();
let results = if has_pending {
let deadline = tokio::time::Instant::now() + timeout;
wait_any_event_driven(
&terminal,
&backend,
&initial.pending_bash_ids,
&initial.pending_subagent_ids,
deadline,
)
.await;
resolve_tasks(
&input.task_ids,
&terminal,
&backend,
&read_file_name,
max_output_bytes,
)
.await
.results
} else {
initial.results
};
let completed_count = results
.iter()
.filter(|r| r.status == "completed" || r.status == "failed" || r.status == "cancelled")
.count();
let total = results.len();
let summary = format!("{completed_count}/{total} tasks completed (wait_any)");
Ok(TaskOutputOutput::MultiResult(MultiTaskOutputResult {
mode: "wait_any".to_string(),
results,
summary,
}))
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::tool_metadata::ToolMetadata;
use crate::types::tool_metadata::test_ctx;
use kigi_tool_runtime::Tool;
#[test]
fn tool_name_and_kind() {
let tool = WaitTasksTool;
assert_eq!(Tool::id(&tool).as_str(), "wait_tasks");
assert_eq!(ToolMetadata::kind(&tool), ToolKind::WaitTasksAction);
}
#[test]
fn description_mentions_key_concepts() {
let tool = WaitTasksTool;
let d = ToolMetadata::description_template(&tool);
assert!(d.contains("task_ids"));
}
#[tokio::test]
async fn rejects_empty_task_ids() {
use crate::types::resources::Resources;
use std::sync::Arc;
let tool = WaitTasksTool;
let resources = Arc::new(tokio::sync::Mutex::new(Resources::new()));
let err = Tool::run(
&tool,
test_ctx(resources),
WaitTasksToolInput {
task_ids: vec![],
mode: WaitMode::WaitAll,
timeout_ms: None,
},
)
.await
.unwrap_err();
let s = format!("{err:?}");
assert!(s.contains("empty") || s.contains("task_ids"));
}
#[tokio::test]
async fn rejects_too_many_ids() {
use crate::types::resources::Resources;
use std::sync::Arc;
let tool = WaitTasksTool;
let resources = Arc::new(tokio::sync::Mutex::new(Resources::new()));
let ids: Vec<String> = (0..=MAX_MULTI_WAIT_IDS).map(|i| format!("t{i}")).collect();
let err = Tool::run(
&tool,
test_ctx(resources),
WaitTasksToolInput {
task_ids: ids,
mode: WaitMode::WaitAll,
timeout_ms: Some(1000),
},
)
.await
.unwrap_err();
let s = format!("{err:?}");
assert!(s.contains("maximum") || s.contains("exceeds"));
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,512 @@
//! `update_goal` — model-driven goal progress reporting.
//!
//! Each invocation is paired with an `oneshot::Sender<UpdateGoalAck>`
//! over the channel to `SessionActor`; the tool blocks on that ack so
//! the model's tool reply reflects the real outcome (classifier
//! verdict / transition / rejection), not a misleading instant success.
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
pub use kigi_tools_api::slash_commands::UPDATE_GOAL_TOOL_NAME;
// ---------------------------------------------------------------------------
// Input schema
// ---------------------------------------------------------------------------
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct UpdateGoalInput {
#[serde(
default,
deserialize_with = "crate::types::schema::deserialize_lenient_option_bool"
)]
#[schemars(
description = "Set to true ONLY when the goal is fully achieved. This ends goal mode. Use together with `message` to include a completion summary."
)]
pub completed: Option<bool>,
#[serde(default)]
#[schemars(
description = "Optional short message logged as progress (visible in tool response, not surfaced to the pager dashboard). Use with `completed: true` for a completion summary."
)]
pub message: Option<String>,
#[serde(default)]
#[schemars(
description = "Set only when truly stuck after 3+ consecutive failed attempts at the same problem. If set, the goal is paused as blocked. This is a FAILURE signal — never put success text here. For success, use `completed: true` with `message`."
)]
pub blocked_reason: Option<String>,
}
// ---------------------------------------------------------------------------
// Channel types — inserted into Resources, read by SessionActor
// ---------------------------------------------------------------------------
/// Outcome of an `update_goal` call as delivered by the session actor.
#[derive(Debug)]
pub enum UpdateGoalAck {
/// `message`-only or `blocked_reason` update accepted.
Accepted { summary: String },
/// Classifier judged the goal achieved.
ClassifierAchieved { details_path: String },
/// Classifier could not produce a verdict (infra failure); the
/// harness fails open and treats the goal as achieved.
ClassifierFailOpenAchieved { reason: &'static str },
/// Classifier rejected the completion; `attempt < max_runs` so
/// another attempt is still available.
ClassifierNotAchieved {
details_path: String,
attempt: u32,
max_runs: u32,
},
/// Classifier rejected the completion AND the per-goal cap was
/// reached; the goal has been auto-paused with `BackOff`.
ClassifierCapReached { details_path: String, attempt: u32 },
/// Classifier rejected the completion with the same flagged gaps as
/// the prior attempt (no progress); the goal auto-paused early
/// before the cap.
ClassifierStalled { details_path: String, attempt: u32 },
/// Verification found no model-fixable path (every refuter flagged a
/// contradiction or environment-unverifiable blocker); the goal
/// paused for a user decision.
ClassifierBlocked { details_path: String },
/// Classifier disabled by policy; goal marked complete directly.
CompletedWithoutClassifier,
/// Second `update_goal(completed: true)` arrived while a
/// classifier was already verifying the previous attempt; routed
/// through the synthetic-NotAchieved accounting.
ClassifierConcurrentInFlight {
details_path: String,
attempt: u32,
max_runs: u32,
},
/// Mid-turn `completed: true` was queued for classifier
/// verification at turn-end. The verdict arrives as a system
/// reminder in the next user turn; the model must NOT call
/// `update_goal(completed: true)` again until then.
///
/// Invariant: the ack is resolved IMMEDIATELY at defer time, NOT
/// parked — parking deadlocks the single-task actor.
DeferredToTurnEnd { pending_depth: u32 },
/// Update was rejected; `reason` discriminates the cause and
/// drives the tool-error code, `detail` is the model-facing
/// message.
Rejected {
reason: RejectReason,
detail: String,
},
}
/// Structured cause for `UpdateGoalAck::Rejected`; each variant maps
/// to a stable `error_code()` the model sees as the `ToolError` kind.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum RejectReason {
/// A prior cmd in this drain transitioned the goal to Blocked.
BlockSeenInDrain,
/// `blocked_reason` set but the goal was not Active.
BlockedAgainstNonActive,
/// `completed: true` arrived after the classifier cap auto-paused
/// the goal — model must wait for user resume.
PostCap,
/// `completed: true` against a non-Active goal for reasons OTHER
/// than the classifier cap.
NonActive,
/// The goal harness is not enabled for this session (no `/goal`
/// run in progress), so there is no orchestration to update. The
/// `update_goal` tool and its `GoalUpdateHandle` are always
/// exposed, so a model can call the tool outside goal mode; the
/// drain rejects cleanly with this reason instead of dropping the
/// ack oneshot (which would surface as the misleading
/// `harness_no_ack` "dropped the response channel" error).
HarnessDisabled,
/// Reserved for strict-mode eviction surfacing; not currently
/// constructed (the new design acks evicted entries as
/// `DeferredToTurnEnd` at their own defer time).
PendingQueueEvicted,
/// The goal auto-paused mid-drain (cap, stall/no_progress, or
/// blocked); this strictly-later entry was dropped without
/// re-verification.
DroppedAfterPauseInDrain,
/// `GoalOrchestration` snapshot vanished between guard and reserve.
OrchestrationVanished,
/// Goal transitioned out of Active while the classifier awaited
/// a verdict (user paused mid-fire).
StatusChangedDuringClassifier,
/// In-flight short-circuit but the orchestration snapshot
/// vanished mid-flight.
InFlightOrchestrationVanished,
}
impl RejectReason {
/// Stable error code surfaced as the tool's `ToolError.kind`.
pub fn error_code(self) -> &'static str {
match self {
Self::BlockSeenInDrain => "goal_update_block_seen",
Self::BlockedAgainstNonActive => "goal_update_blocked_against_non_active",
Self::PostCap => "goal_update_post_cap",
Self::NonActive => "goal_update_non_active",
Self::HarnessDisabled => "goal_update_harness_disabled",
Self::PendingQueueEvicted => "goal_update_evicted",
Self::DroppedAfterPauseInDrain => "goal_update_dropped_after_pause",
Self::OrchestrationVanished => "goal_update_no_orchestration",
Self::StatusChangedDuringClassifier => "goal_update_status_changed",
Self::InFlightOrchestrationVanished => "goal_update_in_flight_orchestration_vanished",
}
}
}
/// Item posted across the goal-update channel: the model's input
/// paired with the oneshot the tool will await for its reply.
pub type UpdateGoalEnvelope = (UpdateGoalInput, tokio::sync::oneshot::Sender<UpdateGoalAck>);
/// Wrap an `UpdateGoalInput` in an envelope whose ack receiver is
/// discarded. Test-only helper; `pub` is needed for cross-crate test
/// access from `kigi-shell`.
#[doc(hidden)]
pub fn envelope_for_test(input: UpdateGoalInput) -> UpdateGoalEnvelope {
let (ack_tx, _ack_rx) = tokio::sync::oneshot::channel();
(input, ack_tx)
}
/// Handle for the `update_goal` tool to send commands to the session.
/// Inserted into Resources as an ephemeral (non-serialized) resource.
pub struct GoalUpdateHandle(pub tokio::sync::mpsc::UnboundedSender<UpdateGoalEnvelope>);
impl std::fmt::Debug for GoalUpdateHandle {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("GoalUpdateHandle").finish()
}
}
// ---------------------------------------------------------------------------
// Output
// ---------------------------------------------------------------------------
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct UpdateGoalOutput {
pub success: bool,
pub summary: String,
}
impl kigi_tool_runtime::ToolOutput for UpdateGoalOutput {}
// ---------------------------------------------------------------------------
// Tool implementation
// ---------------------------------------------------------------------------
#[derive(Debug, Default)]
pub struct UpdateGoalTool;
impl crate::types::tool_metadata::ToolMetadata for UpdateGoalTool {
fn kind(&self) -> ToolKind {
ToolKind::GoalUpdate
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
"Report progress on the active goal. Use the parameters to log a status message, mark the goal completed, or flag that you're blocked."
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for UpdateGoalTool {
type Args = UpdateGoalInput;
type Output = UpdateGoalOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new(UPDATE_GOAL_TOOL_NAME).expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
UPDATE_GOAL_TOOL_NAME,
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(name = "new_tool.update_goal", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: UpdateGoalInput,
) -> Result<UpdateGoalOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
// Fallback summary used only if the actor drops the ack
// oneshot without responding.
let fallback_summary = build_summary(&input);
let (ack_tx, ack_rx) = tokio::sync::oneshot::channel::<UpdateGoalAck>();
// Clone the sender out of resources so we don't hold the
// resources mutex across the channel write.
let sender = {
let res = resources.lock().await;
res.get::<GoalUpdateHandle>()
.ok_or_else(|| {
kigi_tool_runtime::ToolError::custom(
"goal_not_active",
"No active goal to update (GoalUpdateHandle not registered)",
)
})?
.0
.clone()
};
sender.send((input, ack_tx)).map_err(|_| {
kigi_tool_runtime::ToolError::custom(
"goal_channel_closed",
"Goal update channel closed — the session may be shutting down",
)
})?;
// Block the tool reply on the actor's verdict-aware ack.
// An `Err` here means the actor dropped the sender without
// responding (harness bug) — surface a loud tool error
// instead of a misleading `success: true`.
let ack = match ack_rx.await {
Ok(ack) => ack,
Err(_) => {
tracing::warn!(
"update_goal: actor dropped ack oneshot without responding — surfacing as \
tool error"
);
return Err(kigi_tool_runtime::ToolError::custom(
"harness_no_ack",
format!(
"Goal-update harness dropped the response channel before producing an \
ack. This is a harness-side bug; the goal may not have been updated. \
Original intent: {fallback_summary}",
),
));
}
};
render_ack_into_output(ack)
}
}
/// Map an [`UpdateGoalAck`] to the model-facing tool result. Public
/// so host session tests can assert the same model-facing strings.
pub fn render_ack_into_output(
ack: UpdateGoalAck,
) -> Result<UpdateGoalOutput, kigi_tool_runtime::ToolError> {
match ack {
UpdateGoalAck::Accepted { summary } => Ok(UpdateGoalOutput {
success: true,
summary,
}),
UpdateGoalAck::CompletedWithoutClassifier => Ok(UpdateGoalOutput {
success: true,
summary: "Goal marked complete.".to_string(),
}),
UpdateGoalAck::ClassifierAchieved { details_path } => Ok(UpdateGoalOutput {
success: true,
summary: format!(
"Goal classifier verdict: Achieved. Goal complete. See {details_path}"
),
}),
UpdateGoalAck::ClassifierFailOpenAchieved { reason } => Ok(UpdateGoalOutput {
success: true,
summary: format!(
"Goal marked complete via fail-open (reason: {reason}). No classifier verdict \
was produced."
),
}),
UpdateGoalAck::ClassifierNotAchieved {
details_path,
attempt,
max_runs,
} => Err(kigi_tool_runtime::ToolError::custom(
"goal_classifier_not_achieved",
format!(
"Goal classifier rejected this completion attempt ({attempt}/{max_runs}). \
Review {details_path} and continue working; another attempt is available."
),
)),
UpdateGoalAck::ClassifierCapReached {
details_path,
attempt,
} => {
// Empty path ⇒ the harness wrote no synthetic details (e.g. a
// squatted scratch root); omit the "See …" pointer entirely.
let pointer = if details_path.trim().is_empty() {
String::new()
} else {
format!(" See {details_path}")
};
Err(kigi_tool_runtime::ToolError::custom(
"goal_classifier_cap_reached",
format!(
"Goal classifier rejected completion {attempt} times — goal auto-paused.{pointer}"
),
))
}
UpdateGoalAck::ClassifierStalled {
details_path,
attempt,
} => Err(kigi_tool_runtime::ToolError::custom(
"goal_classifier_stalled",
format!(
"Goal verification saw no change in the flagged gaps across {attempt} attempts \
— goal auto-paused. Review {details_path}; the user must resume."
),
)),
UpdateGoalAck::ClassifierBlocked { details_path } => {
Err(kigi_tool_runtime::ToolError::custom(
"goal_classifier_blocked",
format!(
"Goal verification found no model-fixable path (objective/plan contradiction or \
evidence that cannot be captured here) — goal paused for your decision. \
See {details_path}"
),
))
}
UpdateGoalAck::ClassifierConcurrentInFlight {
details_path,
attempt,
max_runs,
} => {
// Empty path ⇒ no harness-written details to point at; omit the
// "; see …" pointer (never reference content we didn't write).
let pointer = if details_path.trim().is_empty() {
String::new()
} else {
format!("; see {details_path}")
};
Err(kigi_tool_runtime::ToolError::custom(
"goal_classifier_in_flight",
format!(
"Goal classifier is still verifying a previous completion — do NOT call \
update_goal(completed: true) again until you receive a verdict reminder. \
This attempt was recorded as Not Achieved ({attempt}/{max_runs}){pointer}"
),
))
}
UpdateGoalAck::DeferredToTurnEnd { pending_depth } => Ok(UpdateGoalOutput {
success: true,
summary: format!(
"Goal completion queued for classifier verification at end of turn \
(pending_depth={pending_depth}). The verdict will be delivered as a \
system reminder before your next reply; do NOT call update_goal \
again until you see it."
),
}),
UpdateGoalAck::Rejected { reason, detail } => Err(kigi_tool_runtime::ToolError::custom(
reason.error_code(),
detail,
)),
}
}
fn build_summary(input: &UpdateGoalInput) -> String {
let mut parts = Vec::new();
if input.completed == Some(true) {
parts.push("Goal marked complete".to_string());
}
if let Some(ref reason) = input.blocked_reason {
parts.push(format!("Goal blocked: {reason}"));
}
if let Some(ref msg) = input.message {
parts.push(msg.clone());
}
if parts.is_empty() {
"Goal updated.".to_string()
} else {
parts.join(". ") + "."
}
}
#[cfg(test)]
mod tests {
use super::*;
fn empty_input() -> UpdateGoalInput {
UpdateGoalInput {
completed: None,
message: None,
blocked_reason: None,
}
}
#[test]
fn build_summary_empty_input() {
assert_eq!(build_summary(&empty_input()), "Goal updated.");
}
#[test]
fn build_summary_completed() {
let input = UpdateGoalInput {
completed: Some(true),
..empty_input()
};
assert_eq!(build_summary(&input), "Goal marked complete.");
}
#[test]
fn build_summary_message_only() {
let input = UpdateGoalInput {
message: Some("Working on it".into()),
..empty_input()
};
assert_eq!(build_summary(&input), "Working on it.");
}
#[test]
fn build_summary_blocked_reason_only() {
let input = UpdateGoalInput {
blocked_reason: Some("no windows sdk".into()),
..empty_input()
};
assert_eq!(build_summary(&input), "Goal blocked: no windows sdk.");
}
#[test]
fn build_summary_blocked_reason_with_message() {
let input = UpdateGoalInput {
blocked_reason: Some("X".into()),
message: Some("longer body".into()),
..empty_input()
};
let summary = build_summary(&input);
assert!(summary.contains("Goal blocked: X"));
assert!(summary.contains("longer body"));
}
#[test]
fn build_summary_completed_with_message() {
let input = UpdateGoalInput {
completed: Some(true),
message: Some("All done".into()),
..empty_input()
};
let summary = build_summary(&input);
assert!(summary.contains("Goal marked complete"));
assert!(summary.contains("All done"));
}
#[test]
fn build_summary_completed_false_treated_as_noop() {
let input = UpdateGoalInput {
completed: Some(false),
..empty_input()
};
assert_eq!(build_summary(&input), "Goal updated.");
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,187 @@
use std::io::{Read, Seek, SeekFrom, Write};
use std::path::{Path, PathBuf};
use anyhow::Context;
use fs2::FileExt;
const ARTIFACT_DIR: &str = "web_fetch";
const ALLOCATION_FILE: &str = ".allocation";
const COUNTER_WIDTH: usize = 10;
const MAX_TOTAL_BYTES: u64 = 1024 * 1024 * 1024;
#[derive(Clone, Debug, Default)]
pub(super) struct WebFetchArtifactWriter;
impl WebFetchArtifactWriter {
pub(super) async fn save(
&self,
session_folder: &Path,
bytes: &[u8],
extension: &str,
) -> anyhow::Result<PathBuf> {
let dir = session_folder.join(ARTIFACT_DIR);
tokio::fs::create_dir_all(&dir).await?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
tokio::fs::set_permissions(&dir, std::fs::Permissions::from_mode(0o700)).await?;
}
let data = bytes.to_vec();
let extension = extension.to_owned();
tokio::task::spawn_blocking(move || save_locked(&dir, &data, &extension))
.await
.context("spawn_blocking panicked")?
}
}
fn save_locked(dir: &Path, data: &[u8], extension: &str) -> anyhow::Result<PathBuf> {
let mut allocation = std::fs::OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(false)
.open(dir.join(ALLOCATION_FILE))
.context("open allocation file")?;
allocation
.lock_exclusive()
.context("lock allocation file")?;
let (max_existing, total_bytes) = scan_artifacts(dir)?;
let number = reserve_number(&mut allocation, max_existing)?;
let new_total = total_bytes
.checked_add(u64::try_from(data.len()).context("artifact size exceeds u64")?)
.context("artifact byte count overflow")?;
if new_total > MAX_TOTAL_BYTES {
anyhow::bail!(
"byte budget exceeded: {new_total}/{MAX_TOTAL_BYTES} bytes in {ARTIFACT_DIR}"
);
}
let path = dir.join(format!("{number}.{extension}"));
let mut tmp = tempfile::NamedTempFile::new_in(dir).context("create temp file")?;
tmp.write_all(data).context("write temp file")?;
tmp.as_file().sync_all().context("fsync temp file")?;
tmp.persist_noclobber(&path)
.with_context(|| format!("persist to {}", path.display()))?;
Ok(path)
}
fn reserve_number(file: &mut std::fs::File, max_existing: u32) -> anyhow::Result<u32> {
let stored = read_reserved_number(file)?.unwrap_or(max_existing);
let next = stored
.max(max_existing)
.checked_add(1)
.context("artifact number exhausted")?;
file.seek(SeekFrom::Start(0))
.context("seek allocation file")?;
write!(file, "{next:0width$}", width = COUNTER_WIDTH).context("write allocation file")?;
file.set_len(COUNTER_WIDTH as u64)
.context("truncate allocation file")?;
file.sync_all().context("fsync allocation file")?;
Ok(next)
}
fn read_reserved_number(file: &mut std::fs::File) -> anyhow::Result<Option<u32>> {
let len = file.metadata().context("stat allocation file")?.len();
if len == 0 {
return Ok(Some(0));
}
if len != COUNTER_WIDTH as u64 {
return Ok(None);
}
file.seek(SeekFrom::Start(0))
.context("seek allocation file")?;
let mut bytes = [0; COUNTER_WIDTH];
file.read_exact(&mut bytes)
.context("read allocation file")?;
Ok(std::str::from_utf8(&bytes)
.ok()
.and_then(|stored| stored.parse().ok()))
}
fn scan_artifacts(dir: &Path) -> Result<(u32, u64), std::io::Error> {
let mut max = 0u32;
let mut total_bytes = 0u64;
for entry in std::fs::read_dir(dir)? {
let entry = entry?;
let name = entry.file_name();
let name = name.to_string_lossy();
if name.starts_with(".tmp") {
let _ = std::fs::remove_file(entry.path());
continue;
}
if name.starts_with('.') {
continue;
}
if let Some(stem) = name.split_once('.').map(|(stem, _)| stem)
&& let Ok(number) = stem.parse::<u32>()
{
max = max.max(number);
}
if let Ok(metadata) = entry.metadata()
&& metadata.is_file()
{
total_bytes = total_bytes
.checked_add(metadata.len())
.ok_or_else(|| std::io::Error::other("artifact byte count overflow"))?;
}
}
Ok((max, total_bytes))
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn allocation_recovers_malformed_state_and_failed_reservation() {
let tmp = tempfile::tempdir().unwrap();
let session = tmp.path().join("session");
tokio::fs::create_dir_all(session.join(ARTIFACT_DIR))
.await
.unwrap();
tokio::fs::write(session.join(ARTIFACT_DIR).join("5.txt"), b"existing")
.await
.unwrap();
tokio::fs::write(session.join(ARTIFACT_DIR).join(ALLOCATION_FILE), b"torn")
.await
.unwrap();
let writer = WebFetchArtifactWriter;
assert!(
writer
.save(&session, b"failed", "missing/ext")
.await
.is_err()
);
let saved = writer.save(&session, b"recovered", "txt").await.unwrap();
assert_eq!(saved.file_name().unwrap(), "7.txt");
assert_eq!(
tokio::fs::read_to_string(session.join(ARTIFACT_DIR).join(ALLOCATION_FILE))
.await
.unwrap(),
"0000000007"
);
}
#[tokio::test]
async fn concurrent_writes_same_session_have_distinct_intact_files() {
let tmp = tempfile::tempdir().unwrap();
let writer = WebFetchArtifactWriter;
let payload_a: &[u8] = b"payload from writer a";
let payload_b: &[u8] = b"different payload from writer b";
let (saved_a, saved_b) = tokio::join!(
writer.save(tmp.path(), payload_a, "txt"),
writer.save(tmp.path(), payload_b, "txt"),
);
let saved_a = saved_a.unwrap();
let saved_b = saved_b.unwrap();
assert_ne!(saved_a, saved_b);
assert_eq!(tokio::fs::read(saved_a).await.unwrap(), payload_a);
assert_eq!(tokio::fs::read(saved_b).await.unwrap(), payload_b);
}
}
@@ -0,0 +1,94 @@
//! In-memory cache for self-contained text fetches with TTL expiry and eviction.
use std::collections::HashMap;
use std::time::{Duration, Instant};
use crate::types::output::WebFetchOutput;
#[derive(Clone)]
struct CachedPage {
output: WebFetchOutput,
inserted: Instant,
}
pub(crate) struct FetchCache {
entries: HashMap<String, CachedPage>,
ttl: Duration,
max_entries: usize,
}
/// Simple cache that holds N completed fetch requests on a TTL.
impl FetchCache {
pub(crate) fn new(ttl: Duration, max_entries: usize) -> Self {
Self {
entries: HashMap::new(),
ttl,
max_entries,
}
}
pub(crate) fn get(&self, url: &str) -> Option<&WebFetchOutput> {
self.entries.get(url).and_then(|entry| {
if entry.inserted.elapsed() < self.ttl {
Some(&entry.output)
} else {
None
}
})
}
/// Cache only inline text; path-bearing outputs must be materialized per call.
pub(crate) fn insert_text(&mut self, url: String, output: WebFetchOutput, was_truncated: bool) {
if was_truncated {
return;
}
if self.entries.len() >= self.max_entries {
// Evict oldest entry.
let oldest_key = self
.entries
.iter()
.max_by_key(|(_, v)| v.inserted.elapsed())
.map(|(k, _)| k.clone());
if let Some(key) = oldest_key {
self.entries.remove(&key);
}
}
self.entries.insert(
url,
CachedPage {
output,
inserted: Instant::now(),
},
);
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::output::WebFetchContent;
fn output(content: &str) -> WebFetchOutput {
WebFetchOutput::Content(WebFetchContent {
url: "https://example.com/".to_string(),
content: content.to_string(),
content_type: "markdown".to_string(),
status_code: 200,
bytes: content.len(),
source_artifact: None,
inline_fallback: None,
output_location: None,
})
}
#[test]
fn truncated_artifact_output_is_never_cached() {
let mut cache = FetchCache::new(Duration::from_secs(60), 10);
let url = "https://example.com/";
cache.insert_text(url.to_string(), output("/sessions/a/web_fetch/1.md"), true);
assert!(cache.get(url).is_none());
cache.insert_text(url.to_string(), output("fully inline"), false);
assert!(cache.get(url).is_some());
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,189 @@
//! Runtime-configurable parameters for the `web_fetch` tool.
use std::time::Duration;
use serde::{Deserialize, Serialize};
use crate::register_resource;
// Safety-boundary constants. Not configurable.
pub const MAX_URL_LENGTH: usize = 2_000;
pub const MAX_REDIRECTS: usize = 10;
pub const USER_AGENT_STRING: &str = "Mozilla/5.0 (compatible; grok-agent/1.0; +https://x.ai)";
/// Runtime-configurable parameters for the `web_fetch` tool.
///
/// Injected via `Params<WebFetchParams>` in `SharedResources`.
/// All fields are optional — `None` means "use built-in default."
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct WebFetchParams {
/// Cache time-to-live in seconds. Default: 900 (15 minutes).
pub cache_ttl_secs: Option<u64>,
/// Maximum number of cached pages. Default: 128.
pub max_cache_entries: Option<usize>,
/// HTTP request timeout in seconds. Default: 60.
pub timeout_secs: Option<u64>,
/// Maximum response body size in bytes. Default: 10 MB.
pub max_content_length: Option<usize>,
/// Maximum inline markdown output length in bytes. Default: 100,000.
pub max_markdown_length: Option<usize>,
/// Model context window size in tokens. Used to enforce 3% cap on web content.
pub context_window_tokens: Option<u64>,
/// Domains the tool is allowed to fetch. All other
/// domains are rejected before any network I/O.
/// Defaults to `DEFAULT_ALLOWED_DOMAINS` if no
/// list given.
#[serde(default)]
pub allowed_domains: Option<Vec<String>>,
/// Optional egress proxy endpoint. When set, all HTTP requests are
/// routed through this URL.
#[serde(default)]
pub proxy_endpoint: Option<String>,
}
register_resource!("grok_build", "WebFetch", WebFetchParams);
// Keep defaults here so call-sites don't have to manage unwrapping.
// Vars are still public following other conventions though.
impl WebFetchParams {
pub fn cache_ttl_secs(&self) -> Duration {
Duration::from_secs(self.cache_ttl_secs.unwrap_or(15 * 60))
}
pub fn max_cache_entries(&self) -> usize {
self.max_cache_entries.unwrap_or(128)
}
pub fn timeout_secs(&self) -> Duration {
Duration::from_secs(self.timeout_secs.unwrap_or(60))
}
pub fn max_content_length(&self) -> usize {
self.max_content_length.unwrap_or(10 * 1024 * 1024)
}
pub fn max_markdown_length(&self) -> usize {
self.max_markdown_length.unwrap_or(100_000)
}
pub fn context_window_tokens(&self) -> u64 {
self.context_window_tokens.unwrap_or(128_000)
}
pub fn allowed_domains(&self) -> Vec<String> {
match &self.allowed_domains {
Some(v) => v.clone(),
None => DEFAULT_ALLOWED_DOMAINS
.iter()
.map(|s| (*s).to_owned())
.collect(),
}
}
}
/// Default allowlist for web_fetch tool.
/// Note: GET-only preapproved domains. Path-scoped entries (e.g. vercel.com/docs) are included as-is.
pub static DEFAULT_ALLOWED_DOMAINS: &[&str] = &[
// xAI
"x.ai",
"console.x.ai",
"docs.x.ai",
"api.x.ai",
// Programming languages
"docs.python.org",
"en.cppreference.com",
"docs.oracle.com",
"learn.microsoft.com",
"developer.mozilla.org",
"go.dev",
"pkg.go.dev",
"www.php.net",
"docs.swift.org",
"kotlinlang.org",
"ruby-doc.org",
"doc.rust-lang.org",
"docs.rs",
"www.typescriptlang.org",
// Web and JS frameworks
"react.dev",
"angular.io",
"vuejs.org",
"nextjs.org",
"expressjs.com",
"nodejs.org",
"bun.sh",
"jquery.com",
"getbootstrap.com",
"tailwindcss.com",
"d3js.org",
"threejs.org",
"redux.js.org",
"webpack.js.org",
"jestjs.io",
"reactrouter.com",
// Python frameworks
"docs.djangoproject.com",
"flask.palletsprojects.com",
"fastapi.tiangolo.com",
"pandas.pydata.org",
"numpy.org",
"www.tensorflow.org",
"pytorch.org",
"scikit-learn.org",
"matplotlib.org",
"requests.readthedocs.io",
"jupyter.org",
// PHP frameworks
"laravel.com",
"symfony.com",
"wordpress.org",
// Java frameworks
"docs.spring.io",
"hibernate.org",
"tomcat.apache.org",
"gradle.org",
"maven.apache.org",
// .NET
"asp.net",
"dotnet.microsoft.com",
"nuget.org",
"blazor.net",
// Mobile
"reactnative.dev",
"docs.flutter.dev",
"developer.apple.com",
"developer.android.com",
// Data science / ML
"keras.io",
"spark.apache.org",
"huggingface.co",
"www.kaggle.com",
// Databases
"redis.io",
"www.postgresql.org",
"dev.mysql.com",
"www.sqlite.org",
"graphql.org",
"prisma.io",
// Cloud and DevOps
"docs.aws.amazon.com",
"cloud.google.com",
"kubernetes.io",
"www.docker.com",
"www.terraform.io",
"www.ansible.com",
"vercel.com/docs",
"docs.netlify.com",
"devcenter.heroku.com",
// Testing and monitoring
"cypress.io",
"selenium.dev",
// Game development
"docs.unity.com",
"docs.unrealengine.com",
// Other tools
"git-scm.com",
"nginx.org",
"httpd.apache.org",
];
@@ -0,0 +1,376 @@
//! Domain allowlist matching with precomputed host → path-prefix lookup.
//!
//! Entries are parsed once at construction into a `HashMap<host, HostEntry>`
//! giving O(1) host lookup followed by a tiny linear scan over path prefixes
//! for that host.
use std::collections::HashMap;
use url::Url;
use crate::types::output::WebFetchOutput;
// ───────────────────────────────────────────────────────────────────────────
// Domain normalization
// ───────────────────────────────────────────────────────────────────────────
/// Canonical form for domain comparison: trim whitespace, strip trailing
/// slashes and dots, remove `www.` prefix, and lowercase.
pub fn normalize_domain(raw: &str) -> String {
let s = raw.trim().trim_end_matches('/').trim_end_matches('.');
let s = s.strip_prefix("www.").unwrap_or(s);
s.to_lowercase()
}
// ───────────────────────────────────────────────────────────────────────────
// Precomputed host entry
// ───────────────────────────────────────────────────────────────────────────
/// What a single host is allowed to serve.
#[derive(Debug, Clone)]
enum HostEntry {
/// Any path on this host is allowed (host-only entry).
AnyPath,
/// Only paths matching one of these prefixes are allowed.
/// Each prefix is normalized (leading `/`, no trailing `/`, lowercased).
PathPrefixes(Vec<String>),
}
// ───────────────────────────────────────────────────────────────────────────
// DomainMatcher
// ───────────────────────────────────────────────────────────────────────────
/// Precomputed domain allowlist. Built once from the raw allowlist entries,
/// provides O(1) host lookup + small linear scan over path prefixes.
#[derive(Debug, Clone)]
pub struct DomainMatcher {
entries: HashMap<String, HostEntry>,
}
impl DomainMatcher {
/// Build from raw allowlist entries like `"docs.rs"`, `"vercel.com/docs"`.
pub fn new(raw_entries: &[String]) -> Self {
let mut entries: HashMap<String, HostEntry> = HashMap::new();
for raw in raw_entries {
let normalized = normalize_domain(raw);
if normalized.is_empty() {
continue;
}
// Split on first '/' to separate host from optional path.
let (host, path) = match normalized.find('/') {
Some(i) => (normalized[..i].to_owned(), Some(&normalized[i..])),
None => (normalized, None),
};
match path {
None => {
// Host-only → any path allowed. Overrides any existing prefixes.
entries.insert(host, HostEntry::AnyPath);
}
Some(raw_path) => {
// Don't downgrade AnyPath to PathPrefixes if a host-only
// entry was already inserted.
if matches!(entries.get(&host), Some(HostEntry::AnyPath)) {
continue;
}
// Normalize path: ensure leading '/', strip trailing '/'.
let prefix = raw_path.trim_end_matches('/');
let prefix = if prefix.is_empty() || prefix == "/" {
// Entry like "example.com/" → treat as host-only.
entries.insert(host, HostEntry::AnyPath);
continue;
} else if prefix.starts_with('/') {
prefix.to_owned()
} else {
format!("/{prefix}")
};
entries
.entry(host)
.and_modify(|e| {
if let HostEntry::PathPrefixes(v) = e
&& !v.contains(&prefix)
{
v.push(prefix.clone());
}
})
.or_insert_with(|| HostEntry::PathPrefixes(vec![prefix]));
}
}
}
Self { entries }
}
/// Returns `None` if the URL is permitted, or `Some(WebFetchOutput::DomainNotAllowed)`
/// if it should be blocked. When `entries` is empty, all fetches are blocked.
pub fn check(&self, url: &Url) -> Option<WebFetchOutput> {
let Some(raw_host) = url.host_str() else {
return Some(WebFetchOutput::DomainNotAllowed(String::new()));
};
let host = normalize_domain(raw_host);
match self.entries.get(&host) {
Some(HostEntry::AnyPath) => None,
Some(HostEntry::PathPrefixes(prefixes)) => {
let url_path = url.path().to_lowercase();
if prefixes.iter().any(|prefix| {
url_path == *prefix
|| (url_path.starts_with(prefix.as_str())
&& url_path.as_bytes().get(prefix.len()) == Some(&b'/'))
}) {
return None;
}
tracing::debug!(
domain = %host,
path = %url_path,
allowed_prefixes = ?prefixes,
"web_fetch path not in allowlist for domain"
);
Some(WebFetchOutput::DomainNotAllowed(host))
}
None => {
tracing::debug!(
domain = %host,
allowed_count = self.entries.len(),
"web_fetch domain not in allowlist"
);
Some(WebFetchOutput::DomainNotAllowed(host))
}
}
}
}
/// Extract and normalize the domain from a raw URL string.
///
/// Returns `None` for unparseable URLs or URLs with no host.
pub fn domain_from_url(raw_url: &str) -> Option<String> {
Url::parse(raw_url)
.ok()
.and_then(|u| u.host_str().map(normalize_domain))
}
// ───────────────────────────────────────────────────────────────────────────
// Tests
// ───────────────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
fn url(s: &str) -> Url {
Url::parse(s).unwrap()
}
// ── normalize_domain ─────────────────────────────────────────────────
#[test]
fn normalize_strips_www_and_trailing_dot() {
assert_eq!(normalize_domain("www.Example.COM."), "example.com");
}
#[test]
fn normalize_trims_whitespace() {
assert_eq!(normalize_domain(" docs.rs "), "docs.rs");
}
// ── Host-only entries ────────────────────────────────────────────────
#[test]
fn allows_listed_domain() {
let m = DomainMatcher::new(&["docs.rs".into(), "Example.Com".into()]);
assert!(m.check(&url("https://docs.rs/reqwest/latest")).is_none());
assert!(m.check(&url("https://example.com/page")).is_none());
}
#[test]
fn case_insensitive_host() {
let m = DomainMatcher::new(&["Example.Com".into()]);
assert!(m.check(&url("https://example.com/page")).is_none());
assert!(m.check(&url("https://EXAMPLE.COM/page")).is_none());
}
#[test]
fn rejects_unlisted_domain() {
let m = DomainMatcher::new(&["docs.rs".into()]);
let blocked = m.check(&url("https://evil.com/steal"));
assert!(matches!(blocked, Some(WebFetchOutput::DomainNotAllowed(d)) if d == "evil.com"));
}
#[test]
fn blocks_all_when_empty() {
let m = DomainMatcher::new(&[]);
assert!(m.check(&url("https://docs.python.org/3/")).is_some());
assert!(m.check(&url("https://example.com/")).is_some());
}
#[test]
fn www_prefix_stripped() {
let m = DomainMatcher::new(&["react.dev".into()]);
assert!(m.check(&url("https://www.react.dev/learn")).is_none());
}
#[test]
fn trailing_dot_stripped() {
let m = DomainMatcher::new(&["react.dev".into()]);
assert!(m.check(&url("https://react.dev./learn")).is_none());
}
// ── Path-scoped entries ──────────────────────────────────────────────
#[test]
fn path_scoped_allows_matching_path() {
let m = DomainMatcher::new(&["vercel.com/docs".into()]);
assert!(m.check(&url("https://vercel.com/docs")).is_none());
assert!(m.check(&url("https://vercel.com/docs/foo")).is_none());
assert!(
m.check(&url("https://vercel.com/docs/platform/edge-functions"))
.is_none()
);
}
#[test]
fn path_scoped_blocks_non_matching_path() {
let m = DomainMatcher::new(&["vercel.com/docs".into()]);
let blocked = m.check(&url("https://vercel.com/api"));
assert!(matches!(blocked, Some(WebFetchOutput::DomainNotAllowed(h)) if h == "vercel.com"));
assert!(m.check(&url("https://vercel.com/")).is_some());
}
#[test]
fn path_scoped_blocks_wrong_host() {
let m = DomainMatcher::new(&["vercel.com/docs".into()]);
let blocked = m.check(&url("https://netlify.com/docs"));
assert!(matches!(blocked, Some(WebFetchOutput::DomainNotAllowed(h)) if h == "netlify.com"));
}
#[test]
fn path_scoped_rejects_sibling_prefix() {
let m = DomainMatcher::new(&["vercel.com/docs".into()]);
// "/docs-internal" shares "/docs" prefix but is a sibling, not a child.
let blocked = m.check(&url("https://vercel.com/docs-internal"));
assert!(
matches!(blocked, Some(WebFetchOutput::DomainNotAllowed(h)) if h == "vercel.com"),
"expected /docs-internal to be blocked"
);
// "/documentation" — also a sibling.
let blocked2 = m.check(&url("https://vercel.com/documentation"));
assert!(
matches!(blocked2, Some(WebFetchOutput::DomainNotAllowed(h)) if h == "vercel.com"),
"expected /documentation to be blocked"
);
// Actual child path is still allowed.
assert!(
m.check(&url("https://vercel.com/docs/guide")).is_none(),
"expected /docs/guide to be allowed"
);
}
#[test]
fn path_scoped_case_insensitive() {
let m = DomainMatcher::new(&["Vercel.COM/Docs".into()]);
assert!(m.check(&url("https://vercel.com/docs")).is_none());
assert!(m.check(&url("https://VERCEL.COM/DOCS/foo")).is_none());
}
#[test]
fn root_path_entry_allows_any_path() {
// "example.com/" should behave like host-only.
let m = DomainMatcher::new(&["example.com/".into()]);
assert!(m.check(&url("https://example.com/")).is_none());
assert!(m.check(&url("https://example.com/anything")).is_none());
}
// ── Multiple path prefixes per host ──────────────────────────────────
#[test]
fn multiple_path_prefixes_per_host() {
let m = DomainMatcher::new(&["github.com/org-a".into(), "github.com/org-b".into()]);
assert!(
m.check(&url("https://github.com/org-a/project-one"))
.is_none()
);
assert!(
m.check(&url("https://github.com/org-b/project-two"))
.is_none()
);
assert!(
m.check(&url("https://github.com/evil-org/malware"))
.is_some()
);
}
#[test]
fn host_only_overrides_path_prefixes() {
// If both "github.com" (host-only) and "github.com/docs" exist,
// host-only wins — any path is allowed.
let m = DomainMatcher::new(&["github.com/docs".into(), "github.com".into()]);
assert!(m.check(&url("https://github.com/anything")).is_none());
}
// ── Model URL variants ───────────────────────────────────────────────
#[test]
fn model_url_variants() {
let m = DomainMatcher::new(&[
"docs.python.org".into(),
"developer.mozilla.org".into(),
"react.dev".into(),
"api.example.com".into(),
]);
// Should match.
assert!(m.check(&url("https://react.dev")).is_none());
assert!(
m.check(&url("https://docs.python.org/3/library/asyncio.html"))
.is_none()
);
assert!(
m.check(&url(
"https://developer.mozilla.org/en-US/docs/Web/API/fetch?v=2#syntax"
))
.is_none()
);
assert!(m.check(&url("https://react.dev/")).is_none());
assert!(m.check(&url("https://React.Dev/learn")).is_none());
assert!(m.check(&url("https://api.example.com/v1/users")).is_none());
assert!(m.check(&url("https://react.dev:443/reference")).is_none());
assert!(m.check(&url("https://www.react.dev/learn")).is_none());
// Should NOT match.
assert!(m.check(&url("https://example.com/page")).is_some());
assert!(m.check(&url("https://evil.example.com/")).is_some());
assert!(m.check(&url("https://react-dev.com/")).is_some());
assert!(
m.check(&url("https://stackoverflow.com/questions/123"))
.is_some()
);
assert!(m.check(&url("https://93.184.216.34/page")).is_some());
}
// ── domain_from_url ─────────────────────────────────────────────────
#[test]
fn domain_from_url_extracts_and_normalizes() {
assert_eq!(
domain_from_url("https://docs.python.org/3/library/asyncio.html"),
Some("docs.python.org".to_string())
);
assert_eq!(
domain_from_url("https://www.React.Dev/learn"),
Some("react.dev".to_string())
);
}
#[test]
fn domain_from_url_returns_none_for_garbage() {
assert_eq!(domain_from_url("not a url"), None);
assert_eq!(domain_from_url(""), None);
}
}
@@ -0,0 +1,157 @@
/// Structured errors for the `web_fetch` tool.
use std::net::IpAddr;
#[derive(Debug, thiserror::Error)]
pub enum WebFetchError {
#[error("URL exceeds maximum length of {max} characters")]
UrlTooLong { max: usize },
#[error("unsupported URL scheme: {scheme} (only http/https allowed)")]
UnsupportedScheme { scheme: String },
#[error("URLs with embedded credentials are not allowed")]
CredentialsInUrl,
#[error("hostname must have at least two dot-separated parts, got: {host}")]
SingleLabelHost { host: String },
#[error("invalid URL: {0}")]
InvalidUrl(#[from] url::ParseError),
#[error("SSRF blocked: {host} resolves to private/internal IP {ip}{}", ssrf_recovery_hint(.host))]
SsrfBlocked { host: String, ip: IpAddr },
#[error("DNS resolution failed for {host}: {source}")]
DnsResolution {
host: String,
source: std::io::Error,
},
#[error("DNS resolution returned no addresses for {0}")]
DnsEmpty(String),
#[error("failed to build HTTP client: {0}")]
ClientBuildError(reqwest::Error),
#[error("HTTP request failed: {0}")]
HttpRequest(#[from] reqwest::Error),
#[error("invalid redirect URL: {0}")]
InvalidRedirect(String),
#[error("too many redirects (max {max})")]
TooManyRedirects { max: usize },
#[error("response body exceeds maximum size of {max} bytes")]
ResponseTooLarge { max: usize },
#[error("invalid proxy configuration: {0}")]
ProxyConfigError(String),
#[error("failed to save downloaded file: {0}")]
IoError(#[from] std::io::Error),
#[error("unsupported content type {content_type} from {url}")]
UnsupportedContentType { content_type: String, url: String },
#[error("content body does not match claimed content type {content_type} from {url}")]
ContentTypeMismatch { content_type: String, url: String },
}
/// Extra recovery guidance appended to an [`WebFetchError::SsrfBlocked`] message.
///
/// `web_fetch` can't reach internal/private hosts, but GitHub / GitHub
/// Enterprise hosts (including internal GHE hostnames) are reachable via the
/// authenticated `gh` CLI. When the blocked host looks like GitHub **and `gh`
/// is actually installed**, point the agent at `gh` instead of letting it
/// conclude the resource is inaccessible and give up. If `gh` is not on `PATH`
/// (or the host isn't GitHub), fall back to the bare SSRF message by returning
/// an empty string.
fn ssrf_recovery_hint(host: &str) -> &'static str {
if is_github_host(host) && gh_available() {
". Use the `gh` CLI instead (e.g. `gh pr view` or `gh api`)."
} else {
""
}
}
/// Whether `host` is a GitHub / GitHub Enterprise host (one `gh` can reach).
fn is_github_host(host: &str) -> bool {
let h = host.to_ascii_lowercase();
h == "github.com" || h.ends_with(".github.com") || h.contains("github")
}
/// Whether the `gh` CLI is available on `PATH`, via the same `which` lookup the
/// rest of the codebase uses for binary discovery (e.g. `kigi-mcp`).
fn gh_available() -> bool {
which::which("gh").is_ok()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn github_host_detection() {
assert!(is_github_host("github.com"));
assert!(is_github_host("api.github.com"));
assert!(is_github_host("github.ghe.example.com")); // synthetic GHE-style
assert!(!is_github_host("ghe.example.com"));
assert!(!is_github_host("internal-wiki.corp.example.com"));
assert!(!is_github_host("gitlab.example.com"));
}
#[test]
fn which_detects_gh_in_dir() {
// Exercises the same `which` lookup `gh_available` uses, with a
// controlled search dir so it doesn't depend on the test host's PATH.
let dir = tempfile::tempdir().unwrap();
// No gh in this dir yet.
assert!(which::which_in("gh", Some(dir.path()), dir.path()).is_err());
// Create an executable `gh`.
let gh = dir.path().join("gh");
std::fs::write(&gh, b"#!/bin/sh\nexit 0\n").unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&gh, std::fs::Permissions::from_mode(0o755)).unwrap();
}
assert!(which::which_in("gh", Some(dir.path()), dir.path()).is_ok());
}
#[test]
fn ssrf_non_github_host_never_hints() {
let err = WebFetchError::SsrfBlocked {
host: "internal-wiki.corp.example.com".to_string(),
ip: "10.0.0.5".parse().unwrap(),
};
let msg = err.to_string();
assert!(msg.contains("resolves to private/internal IP 10.0.0.5"));
assert!(
!msg.contains("gh"),
"non-github host should not mention gh: {msg}"
);
}
#[test]
fn ssrf_github_host_hint_follows_gh_availability() {
let err = WebFetchError::SsrfBlocked {
// Synthetic host must contain "github" for is_github_host; IP is RFC1918 example.
host: "github.ghe.example.com".to_string(),
ip: "10.0.0.1".parse().unwrap(),
};
let msg = err.to_string();
assert!(msg.contains("resolves to private/internal IP 10.0.0.1"));
if gh_available() {
assert!(msg.contains("`gh` CLI"), "gh present -> should hint: {msg}");
assert!(msg.contains("gh pr view") && msg.contains("gh api"));
} else {
// Host names like "github…" contain the substring "gh"; assert on the
// hint markers only, not a bare "gh" contains check.
assert!(
!msg.contains("`gh` CLI") && !msg.contains("gh pr view") && !msg.contains("gh api"),
"gh absent -> previous behavior, no hint: {msg}"
);
}
}
}
@@ -0,0 +1,140 @@
//! Cached HTTP client with atomic invalidation for `web_fetch`.
//!
//! The `reqwest::Client` is held behind an `ArcSwapOption` so it can be
//! atomically invalidated on transport errors, forcing the next call to rebuild
//! with a fresh connection pool. This prevents connection pool poisoning
//! (half-read connections being returned to the pool and corrupting subsequent
//! requests).
use std::sync::Arc;
use arc_swap::ArcSwapOption;
use super::config::WebFetchParams;
use super::error::WebFetchError;
/// Cached, invalidatable HTTP client for web fetching.
///
/// - **Normal path:** `get_or_rebuild()` returns the cached client via a
/// lock-free atomic load.
/// - **On transport error:** call `invalidate()` to atomically set the
/// client to `None`. The next `get_or_rebuild()` falls through and
/// builds a fresh client with a clean connection pool.
#[derive(Clone, Debug)]
pub(crate) struct HttpClient {
inner: Arc<ArcSwapOption<reqwest::Client>>,
params: WebFetchParams,
}
impl HttpClient {
pub(crate) fn new(params: &WebFetchParams) -> Result<Self, WebFetchError> {
let client = Self::build(params)?;
Ok(Self {
inner: Arc::new(ArcSwapOption::from(Some(Arc::new(client)))),
params: params.clone(),
})
}
/// Get the current client, rebuilding if it was invalidated.
pub(crate) fn get_or_rebuild(&self) -> Result<Arc<reqwest::Client>, WebFetchError> {
// Fast path: lock-free atomic load.
if let Some(client) = self.inner.load_full() {
return Ok(client);
}
// Client was invalidated — rebuild with a fresh connection pool.
let fresh = Arc::new(Self::build(&self.params)?);
self.inner.store(Some(Arc::clone(&fresh)));
Ok(fresh)
}
/// Atomically invalidate the cached client. The next `get_or_rebuild()`
/// will construct a fresh one with a clean connection pool.
pub(crate) fn invalidate(&self) {
self.inner.store(None);
}
fn build(params: &WebFetchParams) -> Result<reqwest::Client, WebFetchError> {
let mut builder = reqwest::Client::builder()
.timeout(params.timeout_secs())
.connect_timeout(std::time::Duration::from_secs(10))
// We manage redirects for SSRF.
.redirect(reqwest::redirect::Policy::none())
.pool_max_idle_per_host(2)
.pool_idle_timeout(std::time::Duration::from_secs(30))
.tcp_nodelay(true)
// Reduce size of incoming payloads.
.gzip(true)
.brotli(true)
.deflate(true);
// Route all traffic through the egress proxy when configured.
if let Some(ref endpoint) = params.proxy_endpoint {
let proxy = reqwest::Proxy::all(endpoint)
.map_err(|e| WebFetchError::ProxyConfigError(e.to_string()))?;
builder = builder.proxy(proxy);
}
builder.build().map_err(WebFetchError::ClientBuildError)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn get_or_rebuild_returns_client() {
let client = HttpClient::new(&WebFetchParams::default()).unwrap();
let http = client.get_or_rebuild().unwrap();
assert!(Arc::strong_count(&http) >= 1);
}
#[test]
fn invalidate_forces_rebuild() {
let client = HttpClient::new(&WebFetchParams::default()).unwrap();
let first = client.get_or_rebuild().unwrap();
let first_ptr = Arc::as_ptr(&first);
client.invalidate();
let second = client.get_or_rebuild().unwrap();
let second_ptr = Arc::as_ptr(&second);
// After invalidation, we should get a different client instance.
assert_ne!(first_ptr, second_ptr);
}
#[test]
fn build_with_proxy_endpoint() {
let params = WebFetchParams {
proxy_endpoint: Some("https://proxy.corp.example.com".into()),
..Default::default()
};
// Should succeed — reqwest accepts the proxy URL.
let client = HttpClient::new(&params);
assert!(client.is_ok());
}
#[test]
fn build_without_proxy_is_default() {
let params = WebFetchParams::default();
assert!(params.proxy_endpoint.is_none());
let client = HttpClient::new(&params);
assert!(client.is_ok());
}
#[test]
fn build_with_invalid_proxy_endpoint() {
let params = WebFetchParams {
proxy_endpoint: Some("not a valid url".into()),
..Default::default()
};
let result = HttpClient::new(&params);
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(
err.contains("proxy"),
"Expected proxy-related error, got: {err}"
);
}
}
@@ -0,0 +1,234 @@
//! `web_fetch` tool — client-side URL fetching with improved HTML-to-markdown
//! conversion and SSRF protection.
//!
//! Fetches a URL via `reqwest`, converts HTML to markdown via `htmd` (with
//! `<script>`/`<style>`/etc. stripped), and returns content to the model.
//! Prefers `text/markdown` in the Accept header so doc sites that serve markdown
//! directly bypass the HTML conversion entirely.
mod artifact;
mod cache;
pub mod client;
pub mod config;
pub mod domain;
pub mod error;
mod http;
pub(crate) mod overflow;
mod ssrf;
pub use client::WebFetchClient;
pub use config::WebFetchParams;
pub use domain::{DomainMatcher, domain_from_url};
pub use error::WebFetchError;
// ───────────────────────────────────────────────────────────────────────────
// Config enum (feature flag gating)
// ───────────────────────────────────────────────────────────────────────────
/// Configuration for the `web_fetch` tool.
///
/// When `Enabled`, the tool is registered and a `WebFetchClient` is injected
/// into `Resources`. When `Disabled` (default), the tool is not registered.
#[derive(Debug, Clone, Default)]
pub enum WebFetchConfig {
#[default]
Disabled,
Enabled {
/// Runtime parameters (allowed_domains, proxy_endpoint, timeouts, etc.)
params: WebFetchParams,
},
}
impl WebFetchConfig {
/// Returns `true` when the config is the `Enabled` variant.
pub fn is_enabled(&self) -> bool {
matches!(self, Self::Enabled { .. })
}
}
use crate::types::output::WebFetchOutput;
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::resources::SessionFolder;
use crate::types::tool::{ToolKind, ToolNamespace};
// ───────────────────────────────────────────────────────────────────────────
// Input
// ───────────────────────────────────────────────────────────────────────────
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct WebFetchInput {
/// The URL to fetch content from.
#[schemars(description = "The URL to fetch content from.")]
pub url: String,
}
// ───────────────────────────────────────────────────────────────────────────
// Tool
// ───────────────────────────────────────────────────────────────────────────
#[derive(Debug, Default)]
pub struct WebFetchTool;
impl crate::types::tool_metadata::ToolMetadata for WebFetchTool {
fn kind(&self) -> ToolKind {
ToolKind::WebFetch
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
r#"Fetch the content of a specific URL and return it as markdown.
IMPORTANT: ${{ tools.by_kind.web_fetch }} WILL FAIL for authenticated or private URLs (e.g. Google Docs, Confluence, Jira, GitHub private repos). Use specialized MCP tools for those instead.
Usage notes:
- HTTP URLs will be automatically upgraded to HTTPS
- Long pages will be truncated to fit your context window"#
}
fn versioned_definition(
&self,
_contract_version: Option<&str>,
client_name: &str,
description_override: Option<&str>,
renderer: &crate::types::template_renderer::TemplateRenderer,
param_map: &std::collections::HashMap<String, String>,
input_schema: &serde_json::Value,
effective_params: &serde_json::Value,
) -> crate::types::definition::ToolDefinition {
let params: WebFetchParams = serde_json::from_value(effective_params.clone())
.unwrap_or_else(|e| {
tracing::warn!("Failed to deserialize WebFetchParams: {e}");
WebFetchParams::default()
});
let raw_desc = description_override.unwrap_or_else(|| {
crate::types::tool_metadata::ToolMetadata::description_template(self)
});
let extras = serde_json::json!({
"proxy_enabled": params.proxy_endpoint.is_some(),
});
let description = renderer
.render_with_extra(raw_desc, &extras)
.unwrap_or_else(|e| {
tracing::warn!("Description template render failed, using raw: {e}");
raw_desc.to_string()
});
let remapped_schema = if param_map.is_empty() {
input_schema.clone()
} else {
crate::util::remap::remap_schema_properties(input_schema, param_map)
};
crate::types::definition::ToolDefinition::function(
client_name,
Some(&description),
remapped_schema,
)
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for WebFetchTool {
type Args = WebFetchInput;
type Output = WebFetchOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("web_fetch").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"web_fetch",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(name = "tool.web_fetch", skip_all, fields(url = %input.url))]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: WebFetchInput,
) -> Result<WebFetchOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let (client, session_folder, read_tool_name, execute_tool_name) = {
let res = resources.lock().await;
let client = res.require::<WebFetchClient>()?.clone();
let session_folder = res.get::<SessionFolder>().map(|folder| folder.0.clone());
let renderer = res.get::<crate::types::template_renderer::TemplateRenderer>();
let read_tool_name = renderer
.and_then(|renderer| renderer.tool_for_kind(ToolKind::Read))
.map(str::to_owned);
let execute_tool_name = renderer
.and_then(|renderer| renderer.tool_for_kind(ToolKind::Execute))
.map(str::to_owned);
(client, session_folder, read_tool_name, execute_tool_name)
};
let output = client
.fetch(
&input.url,
session_folder.as_deref(),
read_tool_name.as_deref(),
execute_tool_name.as_deref(),
)
.await?;
Ok(output)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::tool_metadata::test_ctx_with_call_id;
#[test]
fn tool_name_and_description() {
let tool = WebFetchTool;
assert_eq!(kigi_tool_runtime::Tool::id(&tool).as_str(), "web_fetch");
assert_eq!(
crate::types::tool_metadata::ToolMetadata::kind(&tool),
ToolKind::WebFetch
);
assert!(
crate::types::tool_metadata::ToolMetadata::description_template(&tool)
.contains("Fetch the content of a specific URL")
);
}
#[tokio::test]
async fn errors_when_client_not_in_resources() {
let resources = crate::types::resources::Resources::new();
let tool = WebFetchTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "test-call"),
WebFetchInput {
url: "https://example.com".into(),
},
)
.await;
assert!(result.is_err());
let err_msg = result.unwrap_err().to_string();
assert!(
err_msg.contains("missing required resource"),
"Expected 'missing required resource' error, got: {err_msg}"
);
}
}
@@ -0,0 +1,845 @@
//! Bounded inline previews with recoverable session artifacts and tool-aware hints.
use std::path::{Path, PathBuf};
use super::artifact::WebFetchArtifactWriter;
use crate::util::query_tools::{QueryTools, examples_clause};
use crate::util::truncate;
const WEB_FETCH_CONTEXT_PERCENT: f64 = 0.03;
const RECOVERY_FOOTER_PREFIX: &str = "\n\n[web_fetch content truncated:";
const LONG_LINE_BYTES: usize = 2_000;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum PayloadFormat {
Markdown,
Json,
JsonLines,
Text,
}
impl PayloadFormat {
fn extension(self) -> &'static str {
match self {
Self::Markdown => "md",
Self::Json => "json",
Self::JsonLines => "jsonl",
Self::Text => "txt",
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct PayloadClassification {
format: PayloadFormat,
has_long_line: bool,
}
impl PayloadClassification {
fn classify(content_type: &str, text: &str) -> Self {
let mime = content_type
.split(';')
.next()
.unwrap_or(content_type)
.trim()
.to_ascii_lowercase();
let format = if mime == "markdown" || mime == "text/markdown" {
PayloadFormat::Markdown
} else if matches!(
mime.as_str(),
"application/x-ndjson"
| "application/ndjson"
| "application/jsonl"
| "text/jsonl"
| "text/x-jsonl"
) {
PayloadFormat::JsonLines
} else if mime == "application/json"
|| mime == "text/json"
|| mime.ends_with("+json")
|| serde_json::from_str::<serde::de::IgnoredAny>(text.trim()).is_ok()
{
PayloadFormat::Json
} else {
PayloadFormat::Text
};
let has_long_line = text.lines().map(str::len).max().unwrap_or(0) > LONG_LINE_BYTES;
Self {
format,
has_long_line,
}
}
}
#[derive(Clone, Debug)]
pub(super) struct OverflowHandler {
writer: WebFetchArtifactWriter,
}
#[derive(Clone, Copy)]
pub(super) struct RecoveryTools<'a> {
pub(super) read: Option<&'a str>,
pub(super) execute: Option<&'a str>,
}
pub(super) struct OverflowResult {
pub(super) content: String,
pub(super) was_truncated: bool,
pub(super) artifact_path: Option<PathBuf>,
pub(super) path_free_content: Option<String>,
}
#[derive(Clone, Copy)]
pub(super) struct InlineBudget {
preview_bytes: usize,
output_bytes: usize,
}
impl OverflowHandler {
pub(super) fn new() -> Self {
Self {
writer: WebFetchArtifactWriter,
}
}
pub(super) async fn process(
&self,
content: String,
budget: InlineBudget,
session_folder: Option<&Path>,
content_type: &str,
tools: RecoveryTools<'_>,
) -> OverflowResult {
if content.len() <= budget.preview_bytes.min(budget.output_bytes) {
return OverflowResult {
content,
was_truncated: false,
artifact_path: None,
path_free_content: None,
};
}
let total_bytes = content.len();
let classification = PayloadClassification::classify(content_type, &content);
let extension = classification.format.extension();
let saved_path = match session_folder {
Some(folder) => match self
.writer
.save(folder, content.as_bytes(), extension)
.await
{
Ok(path) => Some(path),
Err(error) => {
tracing::warn!(
session_folder = %folder.display(),
total_bytes,
error = %error,
"Failed to persist full web_fetch content"
);
None
}
},
None => {
tracing::warn!(
total_bytes,
"Cannot persist full web_fetch content without a session folder"
);
None
}
};
let output = bounded_output(
&content,
budget,
total_bytes,
saved_path.as_deref(),
classification,
tools,
);
let path_free_content =
bounded_output(&content, budget, total_bytes, None, classification, tools);
OverflowResult {
content: output,
was_truncated: true,
artifact_path: saved_path,
path_free_content: Some(path_free_content),
}
}
}
pub(super) fn inline_budget(
context_window_tokens: u64,
max_markdown_length: usize,
) -> InlineBudget {
let context_budget = (truncate::estimate_chars(context_window_tokens) as f64
* WEB_FETCH_CONTEXT_PERCENT) as usize;
InlineBudget {
preview_bytes: context_budget.min(max_markdown_length),
output_bytes: max_markdown_length,
}
}
fn recovery_footer(shown_bytes: usize, total_bytes: usize, file_hint: &str) -> String {
format!(
"{RECOVERY_FOOTER_PREFIX} showing first {shown_bytes} of \
{total_bytes} bytes.{file_hint}]"
)
}
fn bounded_output(
content: &str,
budget: InlineBudget,
total_bytes: usize,
saved_path: Option<&Path>,
classification: PayloadClassification,
tools: RecoveryTools<'_>,
) -> String {
if let Some(path) = saved_path {
let path_hint = format!(" Full content saved to: {}.", path.display());
let steer = web_fetch_steer(classification, tools, QueryTools::detect());
if !steer.is_empty() {
let full_hint = format!("{path_hint}{steer}");
if let Some(output) = render_with_hint(content, budget, total_bytes, &full_hint) {
return output;
}
}
if let Some(output) = render_with_hint(content, budget, total_bytes, &path_hint) {
return output;
}
} else if let Some(output) = render_with_hint(content, budget, total_bytes, "") {
return output;
}
bounded_generic_marker(content, budget)
}
fn render_with_hint(
content: &str,
budget: InlineBudget,
total_bytes: usize,
file_hint: &str,
) -> Option<String> {
let provisional_footer = recovery_footer(budget.preview_bytes, total_bytes, file_hint);
if provisional_footer.len() > budget.output_bytes {
return None;
}
let preview_bytes = budget
.preview_bytes
.min(budget.output_bytes - provisional_footer.len());
let preview = truncate::truncate_str(content, preview_bytes);
let footer = recovery_footer(preview.len(), total_bytes, file_hint);
let output = format!("{preview}{footer}");
(output.len() <= budget.output_bytes).then_some(output)
}
fn bounded_generic_marker(content: &str, budget: InlineBudget) -> String {
let Some(marker) = ["\n\n[web_fetch output truncated]", "[truncated]", "..."]
.into_iter()
.find(|marker| marker.len() <= budget.output_bytes)
else {
return String::new();
};
let preview_bytes = budget.preview_bytes.min(budget.output_bytes - marker.len());
format!("{}{marker}", truncate::truncate_str(content, preview_bytes))
}
fn read_steer(read_tool: Option<&str>) -> String {
read_tool.map_or_else(String::new, |read_tool| {
format!(" Use `{read_tool}` with offsets and limits to read it in chunks.")
})
}
fn web_fetch_steer(
classification: PayloadClassification,
tools: RecoveryTools<'_>,
query_tools: QueryTools,
) -> String {
if classification.has_long_line {
return tools.execute.map_or_else(String::new, |execute| {
let (format, action, examples) = match classification.format {
PayloadFormat::Json => ("valid JSON", "query", query_tools.json_tools()),
PayloadFormat::JsonLines => ("JSON Lines", "query", query_tools.json_tools()),
PayloadFormat::Markdown => {
("Markdown", "slice or search", query_tools.text_tools())
}
PayloadFormat::Text => ("text", "slice or search", query_tools.text_tools()),
};
format!(
" The saved file is {format} with a very long line, so \
line-oriented read and search tools are ineffective on it — use \
`{execute}` to {action} it{eg}.",
eg = examples_clause(&examples),
)
});
}
match classification.format {
PayloadFormat::Json => tools.execute.map_or_else(
|| read_steer(tools.read),
|execute| {
format!(
" The saved file is valid JSON; use `{execute}` to query it{eg}.",
eg = examples_clause(&query_tools.json_tools()),
)
},
),
PayloadFormat::JsonLines => tools.execute.map_or_else(
|| read_steer(tools.read),
|execute| {
format!(
" The saved file is JSON Lines; use `{execute}` to query it{eg}.",
eg = examples_clause(&query_tools.json_tools()),
)
},
),
PayloadFormat::Markdown | PayloadFormat::Text => {
let read = read_steer(tools.read);
if !read.is_empty() {
return read;
}
tools.execute.map_or_else(String::new, |execute| {
format!(
" Use `{execute}` to inspect it{}.",
examples_clause(&query_tools.text_tools())
)
})
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn all_query_tools() -> QueryTools {
QueryTools {
jq: Some("jq"),
python: Some("python3"),
sed: Some("sed"),
cut: Some("cut"),
}
}
fn tools<'a>(read: Option<&'a str>, execute: Option<&'a str>) -> RecoveryTools<'a> {
RecoveryTools { read, execute }
}
fn budget(preview_bytes: usize) -> InlineBudget {
InlineBudget {
preview_bytes,
output_bytes: 512,
}
}
#[tokio::test]
async fn oversized_markdown_persists_exact_content_and_omitted_tail() {
let tmp = tempfile::tempdir().unwrap();
let handler = OverflowHandler::new();
let tail = "TAIL-MUST-REMAIN-RECOVERABLE";
let full = format!("# Heading\n{}\n{tail}", "body line\n".repeat(1_000));
let result = handler
.process(
full.clone(),
budget(100),
Some(tmp.path()),
"markdown",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert!(result.was_truncated);
assert!(result.content.len() <= 512);
assert!(!result.content.contains(tail));
assert_eq!(
result
.content
.matches("web_fetch content truncated")
.count(),
1
);
assert!(result.content.contains("showing first 100 of"));
assert!(result.content.contains("ReadAsset"));
let dump = tmp.path().join("web_fetch/1.md");
assert!(result.content.contains(dump.to_string_lossy().as_ref()));
assert_eq!(tokio::fs::read_to_string(dump).await.unwrap(), full);
}
#[tokio::test]
async fn exact_limit_stays_inline_and_one_over_is_recoverable() {
let tmp = tempfile::tempdir().unwrap();
let handler = OverflowHandler::new();
let exact = handler
.process(
"a".repeat(100),
budget(100),
Some(tmp.path()),
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert!(!exact.was_truncated);
assert_eq!(exact.content, "a".repeat(100));
assert!(!tmp.path().join("web_fetch").exists());
let one_over = handler
.process(
"b".repeat(101),
budget(100),
Some(tmp.path()),
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert!(one_over.was_truncated);
assert!(one_over.content.len() <= 512);
assert!(one_over.content.contains("showing first 100 of 101 bytes"));
assert_eq!(
tokio::fs::read_to_string(tmp.path().join("web_fetch/1.txt"))
.await
.unwrap(),
"b".repeat(101)
);
}
#[tokio::test]
async fn utf8_preview_uses_one_safe_byte_boundary() {
let tmp = tempfile::tempdir().unwrap();
let result = OverflowHandler::new()
.process(
"ééé".to_string(),
budget(5),
Some(tmp.path()),
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert!(result.was_truncated);
assert!(result.content.starts_with("éé\n\n[web_fetch"));
assert!(result.content.contains("showing first 4 of 6 bytes"));
assert_eq!(
tokio::fs::read_to_string(tmp.path().join("web_fetch/1.txt"))
.await
.unwrap(),
"ééé"
);
}
#[test]
fn classification_uses_mime_and_keeps_layout_independent() {
let cases = [
("markdown", "plain", PayloadFormat::Markdown, "md"),
(
"text/markdown; charset=utf-8",
"plain",
PayloadFormat::Markdown,
"md",
),
("application/json", "not valid", PayloadFormat::Json, "json"),
("text/json", "not valid", PayloadFormat::Json, "json"),
(
"application/problem+json",
"not valid",
PayloadFormat::Json,
"json",
),
(
"application/x-ndjson",
"{\"a\":1}\n",
PayloadFormat::JsonLines,
"jsonl",
),
(
"application/ndjson",
"{\"a\":1}\n",
PayloadFormat::JsonLines,
"jsonl",
),
(
"application/jsonl",
"{\"a\":1}\n",
PayloadFormat::JsonLines,
"jsonl",
),
(
"text/jsonl",
"{\"a\":1}\n",
PayloadFormat::JsonLines,
"jsonl",
),
(
"text/x-jsonl",
"{\"a\":1}\n",
PayloadFormat::JsonLines,
"jsonl",
),
("text/plain", "{\"a\":1}", PayloadFormat::Json, "json"),
("text/plain", "plain", PayloadFormat::Text, "txt"),
];
for (content_type, content, format, extension) in cases {
let classification = PayloadClassification::classify(content_type, content);
assert_eq!(classification.format, format);
assert_eq!(classification.format.extension(), extension);
}
let long_markdown =
PayloadClassification::classify("markdown", &"x".repeat(LONG_LINE_BYTES + 1));
assert_eq!(long_markdown.format, PayloadFormat::Markdown);
assert!(long_markdown.has_long_line);
let normal_json_lines =
PayloadClassification::classify("application/x-ndjson", "{\"a\":1}\n{\"b\":2}\n");
assert_eq!(normal_json_lines.format, PayloadFormat::JsonLines);
assert!(!normal_json_lines.has_long_line);
}
#[tokio::test]
async fn json_json_lines_and_text_use_matching_extensions() {
let tmp = tempfile::tempdir().unwrap();
let handler = OverflowHandler::new();
let json = r#"{"items":["one","two","three"]}"#.to_string();
let json_lines = "{\"item\":1}\n{\"item\":2}\n".to_string();
let text = "line one\nline two\nline three".to_string();
handler
.process(
json.clone(),
budget(5),
Some(tmp.path()),
"application/json",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
handler
.process(
json_lines.clone(),
budget(5),
Some(tmp.path()),
"application/x-ndjson",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
handler
.process(
text.clone(),
budget(5),
Some(tmp.path()),
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert_eq!(
tokio::fs::read_to_string(tmp.path().join("web_fetch/1.json"))
.await
.unwrap(),
json
);
assert_eq!(
tokio::fs::read_to_string(tmp.path().join("web_fetch/2.jsonl"))
.await
.unwrap(),
json_lines
);
assert_eq!(
tokio::fs::read_to_string(tmp.path().join("web_fetch/3.txt"))
.await
.unwrap(),
text
);
}
#[test]
fn steering_prefers_execute_but_pretty_json_falls_back_to_read() {
let pretty_json = PayloadClassification {
format: PayloadFormat::Json,
has_long_line: false,
};
let execute = web_fetch_steer(
pretty_json,
tools(Some("ReadAsset"), Some("ExecuteAsset")),
all_query_tools(),
);
assert!(execute.contains("ExecuteAsset"));
assert!(!execute.contains("ReadAsset"));
let read = web_fetch_steer(
pretty_json,
tools(Some("ReadAsset"), None),
all_query_tools(),
);
assert!(read.contains("ReadAsset"));
assert!(read.contains("offsets and limits"));
let json_lines = web_fetch_steer(
PayloadClassification {
format: PayloadFormat::JsonLines,
has_long_line: false,
},
tools(Some("ReadAsset"), None),
all_query_tools(),
);
assert!(json_lines.contains("ReadAsset"));
for format in [PayloadFormat::Json, PayloadFormat::Markdown] {
assert!(
web_fetch_steer(
PayloadClassification {
format,
has_long_line: true,
},
tools(Some("ReadAsset"), None),
all_query_tools(),
)
.is_empty()
);
}
}
#[test]
fn steering_names_only_detected_query_utilities() {
let detected = QueryTools {
jq: None,
python: Some("python3"),
sed: None,
cut: None,
};
let steer = web_fetch_steer(
PayloadClassification {
format: PayloadFormat::Json,
has_long_line: true,
},
tools(Some("ReadAsset"), Some("ExecuteAsset")),
detected,
);
assert!(steer.contains("ExecuteAsset"));
assert!(steer.contains("python3"));
assert!(!steer.contains("ReadAsset"));
assert!(!steer.contains("jq"));
let markdown = web_fetch_steer(
PayloadClassification {
format: PayloadFormat::Markdown,
has_long_line: true,
},
tools(Some("ReadAsset"), Some("ExecuteAsset")),
all_query_tools(),
);
assert!(markdown.contains("ExecuteAsset"));
assert!(markdown.contains("sed"));
assert!(!markdown.contains("ReadAsset"));
assert!(!markdown.contains("jq"));
}
#[tokio::test]
async fn missing_or_failed_session_storage_never_claims_an_artifact() {
let tmp = tempfile::tempdir().unwrap();
let handler = OverflowHandler::new();
let full = "line\n".repeat(1_000);
let no_session = handler
.process(
full.clone(),
budget(100),
None,
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert!(no_session.was_truncated);
assert!(!no_session.content.contains("Full content saved"));
assert!(!no_session.content.contains("ReadAsset"));
let blocker = tmp.path().join("not-a-directory");
tokio::fs::write(&blocker, b"file").await.unwrap();
let failed = handler
.process(
full.clone(),
budget(100),
Some(&blocker),
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert!(failed.was_truncated);
assert!(!failed.content.contains("Full content saved"));
assert!(!failed.content.contains("ReadAsset"));
tokio::fs::remove_file(&blocker).await.unwrap();
let retry = handler
.process(
full.clone(),
budget(100),
Some(&blocker),
"text/plain",
tools(Some("RetryRead"), Some("ExecuteAsset")),
)
.await;
assert!(retry.content.contains("RetryRead"));
assert!(retry.content.contains("Full content saved"));
assert_eq!(
tokio::fs::read_to_string(blocker.join("web_fetch/1.txt"))
.await
.unwrap(),
full
);
}
#[tokio::test]
async fn deleted_artifact_and_changed_tool_name_are_rematerialized() {
let tmp = tempfile::tempdir().unwrap();
let handler = OverflowHandler::new();
let full = "line\n".repeat(1_000);
let first = handler
.process(
full.clone(),
budget(100),
Some(tmp.path()),
"text/plain",
tools(Some("FirstRead"), Some("ExecuteAsset")),
)
.await;
let first_path = tmp.path().join("web_fetch/1.txt");
assert!(first.content.contains("FirstRead"));
tokio::fs::remove_file(first_path).await.unwrap();
let second = handler
.process(
full.clone(),
budget(100),
Some(tmp.path()),
"text/plain",
tools(Some("SecondRead"), Some("ExecuteAsset")),
)
.await;
let second_path = tmp.path().join("web_fetch/2.txt");
assert!(second.content.contains("SecondRead"));
assert!(!second.content.contains("FirstRead"));
assert_eq!(tokio::fs::read_to_string(second_path).await.unwrap(), full);
}
#[tokio::test]
async fn one_million_token_context_keeps_footer_inside_output_budget() {
let tmp = tempfile::tempdir().unwrap();
let budget = inline_budget(1_000_000, 100_000);
assert_eq!(budget.preview_bytes, 100_000);
assert_eq!(budget.output_bytes, 100_000);
assert_eq!(inline_budget(1_000_000, 20_000).preview_bytes, 20_000);
let result = OverflowHandler::new()
.process(
"line\n".repeat(21_000),
budget,
Some(tmp.path()),
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert!(result.was_truncated);
assert!(result.content.len() <= 100_000);
assert!(result.content.contains("Full content saved to:"));
assert!(result.content.contains("ReadAsset"));
let footer_start = result.content.rfind(RECOVERY_FOOTER_PREFIX).unwrap();
assert!(result.content[footer_start..].starts_with(&format!(
"{RECOVERY_FOOTER_PREFIX} showing first {footer_start} of "
)));
}
#[tokio::test]
async fn oversized_tool_name_drops_guidance_but_keeps_actionable_path() {
let tmp = tempfile::tempdir().unwrap();
let read_tool = "ReadAsset".repeat(200);
let result = OverflowHandler::new()
.process(
"line\n".repeat(1_000),
InlineBudget {
preview_bytes: 100,
output_bytes: 300,
},
Some(tmp.path()),
"text/plain",
tools(Some(&read_tool), Some("ExecuteAsset")),
)
.await;
let artifact = result.artifact_path.unwrap();
assert!(result.content.len() <= 300);
assert!(result.content.contains(artifact.to_string_lossy().as_ref()));
assert!(!result.content.contains(read_tool.as_str()));
assert!(tokio::fs::try_exists(artifact).await.unwrap());
}
#[tokio::test]
async fn oversized_path_falls_back_to_bounded_generic_marker() {
let tmp = tempfile::tempdir().unwrap();
let session = tmp.path().join("p".repeat(180));
let result = OverflowHandler::new()
.process(
"line\n".repeat(1_000),
InlineBudget {
preview_bytes: 100,
output_bytes: 128,
},
Some(&session),
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert!(result.content.len() <= 128);
assert!(result.content.contains("truncated"));
assert!(!result.content.contains("Full content saved"));
assert!(
tokio::fs::try_exists(result.artifact_path.unwrap())
.await
.unwrap()
);
}
#[tokio::test]
async fn tiny_budget_returns_only_bounded_generic_marker() {
let tmp = tempfile::tempdir().unwrap();
let result = OverflowHandler::new()
.process(
"short".to_string(),
InlineBudget {
preview_bytes: 10,
output_bytes: 3,
},
Some(tmp.path()),
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert_eq!(result.content, "...");
assert!(
tokio::fs::try_exists(result.artifact_path.unwrap())
.await
.unwrap()
);
}
#[tokio::test]
async fn budgets_below_marker_width_return_empty_output() {
let handler = OverflowHandler::new();
for output_bytes in 0..=2 {
let result = handler
.process(
"short".to_string(),
InlineBudget {
preview_bytes: 10,
output_bytes,
},
None,
"text/plain",
tools(Some("ReadAsset"), Some("ExecuteAsset")),
)
.await;
assert!(result.was_truncated);
assert!(result.content.is_empty());
}
}
}
@@ -0,0 +1,230 @@
//! SSRF (Server-Side Request Forgery) protection for `web_fetch`.
//!
//! Validates that resolved IP addresses are not in private, link-local, or
//! cloud metadata ranges before allowing outbound HTTP requests.
//!
//! Reference: [IANA IPv4 Special-Purpose Address Registry](https://www.iana.org/assignments/iana-ipv4-special-registry/)
use std::net::IpAddr;
use url::Url;
use super::error::WebFetchError;
/// Returns `true` if an IP address is in a private, link-local, or cloud
/// metadata range that should be blocked to prevent SSRF attacks.
///
/// **Allowed:** loopback (`127.x` / `::1`) for local development.
/// **Blocked:** RFC 1918, link-local, CGNAT/cloud metadata, unspecified.
pub(crate) fn is_blocked_ip(ip: &IpAddr) -> bool {
match ip {
IpAddr::V4(v4) => {
let octets = v4.octets();
// Loopback (127.0.0.0/8) — allowed for local dev servers.
if octets[0] == 127 {
return false;
}
// RFC 1918: 10.0.0.0/8 — private network.
if octets[0] == 10 {
return true;
}
// RFC 1918: 172.16.0.0/12 — private network.
if octets[0] == 172 && (16..=31).contains(&octets[1]) {
return true;
}
// RFC 1918: 192.168.0.0/16 — private network.
if octets[0] == 192 && octets[1] == 168 {
return true;
}
// RFC 3927: 169.254.0.0/16 — link-local.
// Includes AWS/GCP/Azure metadata endpoint 169.254.169.254.
if octets[0] == 169 && octets[1] == 254 {
return true;
}
// RFC 6598: 100.64.0.0/10 — CGNAT / shared address space.
// Used by some cloud providers for internal metadata services.
if octets[0] == 100 && (64..=127).contains(&octets[1]) {
return true;
}
// 0.0.0.0 — unspecified address.
if v4.is_unspecified() {
return true;
}
false
}
IpAddr::V6(v6) => {
// ::1 — loopback, allowed for local dev.
if v6.is_loopback() {
return false;
}
// :: — unspecified.
if v6.is_unspecified() {
return true;
}
// IPv4-mapped IPv6 (::ffff:x.x.x.x) — delegate to v4 checks.
if let Some(v4) = v6.to_ipv4_mapped() {
return is_blocked_ip(&IpAddr::V4(v4));
}
let segments = v6.segments();
// RFC 4291: fe80::/10 — link-local unicast.
if segments[0] & 0xffc0 == 0xfe80 {
return true;
}
// RFC 4193: fc00::/7 — unique local address (ULA).
if segments[0] & 0xfe00 == 0xfc00 {
return true;
}
false
}
}
}
/// Resolve hostname via DNS and verify none of the resolved addresses are
/// in blocked private/link-local ranges.
pub(crate) async fn check_ssrf(url: &Url) -> Result<(), WebFetchError> {
let host = url
.host_str()
.ok_or_else(|| WebFetchError::SingleLabelHost {
host: String::new(),
})?;
// If the host is already a literal IP, check it directly.
if let Ok(ip) = host.parse::<IpAddr>() {
if is_blocked_ip(&ip) {
return Err(WebFetchError::SsrfBlocked {
host: host.to_string(),
ip,
});
}
return Ok(());
}
// DNS resolution.
let port = url.port_or_known_default().unwrap_or(443);
let addr_str = format!("{host}:{port}");
let addrs: Vec<std::net::SocketAddr> = tokio::net::lookup_host(&addr_str)
.await
.map_err(|e| WebFetchError::DnsResolution {
host: host.to_string(),
source: e,
})?
.collect();
if addrs.is_empty() {
return Err(WebFetchError::DnsEmpty(host.to_string()));
}
addrs
.iter()
.find(|addr| is_blocked_ip(&addr.ip()))
.map_or(Ok(()), |addr| {
Err(WebFetchError::SsrfBlocked {
host: host.to_string(),
ip: addr.ip(),
})
})
}
#[cfg(test)]
mod tests {
use super::*;
// ── IPv4 blocking ───────────────────────────────────────────────────
#[test]
fn blocks_rfc1918_10x() {
assert!(is_blocked_ip(&"10.0.0.1".parse().unwrap()));
assert!(is_blocked_ip(&"10.255.255.255".parse().unwrap()));
}
#[test]
fn blocks_rfc1918_172x() {
assert!(is_blocked_ip(&"172.16.0.1".parse().unwrap()));
assert!(is_blocked_ip(&"172.31.255.255".parse().unwrap()));
assert!(!is_blocked_ip(&"172.15.0.1".parse().unwrap()));
assert!(!is_blocked_ip(&"172.32.0.1".parse().unwrap()));
}
#[test]
fn blocks_rfc1918_192168() {
assert!(is_blocked_ip(&"192.168.0.1".parse().unwrap()));
assert!(is_blocked_ip(&"192.168.255.255".parse().unwrap()));
}
#[test]
fn blocks_link_local() {
assert!(is_blocked_ip(&"169.254.0.1".parse().unwrap()));
assert!(is_blocked_ip(&"169.254.169.254".parse().unwrap()));
}
#[test]
fn blocks_cgnat_cloud_metadata() {
assert!(is_blocked_ip(&"100.64.0.1".parse().unwrap()));
assert!(is_blocked_ip(&"100.127.255.255".parse().unwrap()));
assert!(!is_blocked_ip(&"100.63.0.1".parse().unwrap()));
assert!(!is_blocked_ip(&"100.128.0.1".parse().unwrap()));
}
#[test]
fn blocks_unspecified() {
assert!(is_blocked_ip(&"0.0.0.0".parse().unwrap()));
assert!(is_blocked_ip(&"::".parse().unwrap()));
}
#[test]
fn allows_loopback() {
assert!(!is_blocked_ip(&"127.0.0.1".parse().unwrap()));
assert!(!is_blocked_ip(&"127.0.0.2".parse().unwrap()));
assert!(!is_blocked_ip(&"::1".parse().unwrap()));
}
#[test]
fn allows_public_ips() {
assert!(!is_blocked_ip(&"1.1.1.1".parse().unwrap()));
assert!(!is_blocked_ip(&"8.8.8.8".parse().unwrap()));
assert!(!is_blocked_ip(&"142.250.80.46".parse().unwrap()));
}
// ── IPv6 ────────────────────────────────────────────────────────────
#[test]
fn blocks_ipv6_link_local() {
assert!(is_blocked_ip(&"fe80::1".parse().unwrap()));
}
#[test]
fn blocks_ipv6_unique_local() {
assert!(is_blocked_ip(&"fc00::1".parse().unwrap()));
assert!(is_blocked_ip(&"fd00::1".parse().unwrap()));
}
#[test]
fn blocks_ipv4_mapped_ipv6_private() {
assert!(is_blocked_ip(&"::ffff:10.0.0.1".parse::<IpAddr>().unwrap()));
assert!(is_blocked_ip(
&"::ffff:192.168.1.1".parse::<IpAddr>().unwrap()
));
}
#[test]
fn allows_ipv4_mapped_ipv6_public() {
assert!(!is_blocked_ip(&"::ffff:8.8.8.8".parse::<IpAddr>().unwrap()));
}
// ── check_ssrf integration ──────────────────────────────────────────
#[tokio::test]
async fn ssrf_blocks_ip_literal_private() {
let url = Url::parse("https://10.0.0.1/secret").unwrap();
let result = check_ssrf(&url).await;
assert!(result.is_err());
assert!(result.unwrap_err().to_string().contains("private"));
}
#[tokio::test]
async fn ssrf_allows_ip_literal_public() {
let url = Url::parse("https://1.1.1.1/").unwrap();
let result = check_ssrf(&url).await;
assert!(result.is_ok());
}
}
@@ -0,0 +1,151 @@
//! `web_search` tool — new architecture (`Tool` trait).
//!
//! Calls the Responses API with web search capability. Reads the
//! pre-constructed `WebSearchClient` from Resources (inserted by
//! `with_backend()` when the config is `Enabled`).
use crate::implementations::web_search::client::WebSearchClient;
use crate::types::output::WebSearchOutput;
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
// ───────────────────────────────────────────────────────────────────────────
// Input
// ───────────────────────────────────────────────────────────────────────────
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
pub struct WebSearchInput {
#[schemars(description = "The search query to perform.")]
pub query: String,
#[schemars(description = "Optional list of domains to restrict search to.")]
pub allowed_domains: Option<Vec<String>>,
}
// ───────────────────────────────────────────────────────────────────────────
// Tool implementation
// ───────────────────────────────────────────────────────────────────────────
#[derive(Debug, Default)]
pub struct WebSearchTool;
impl crate::types::tool_metadata::ToolMetadata for WebSearchTool {
fn kind(&self) -> ToolKind {
ToolKind::WebSearch
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuild
}
fn description_template(&self) -> &str {
"Search the web for up-to-date information, tailored for coding and software development tasks."
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for WebSearchTool {
type Args = WebSearchInput;
type Output = WebSearchOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("web_search").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"web_search",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(name = "tool.web_search", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: WebSearchInput,
) -> Result<WebSearchOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let client;
{
let res = resources.lock().await;
client = res.require::<WebSearchClient>()?.clone();
}
let (content, citations) = client
.search(&input.query, input.allowed_domains.clone())
.await
.map_err(|e| {
kigi_tool_runtime::ToolError::execution(
kigi_tool_protocol::ToolId::new("web_search").expect("valid"),
e.to_string(),
)
})?;
Ok(WebSearchOutput {
query: input.query.clone(),
content,
citations,
allowed_domains: input.allowed_domains.clone(),
pre_formatted: None,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::resources::Resources;
use crate::types::tool_metadata::test_ctx_with_call_id;
#[test]
fn tool_name_and_description() {
let tool = WebSearchTool;
assert_eq!(kigi_tool_runtime::Tool::id(&tool).as_str(), "web_search");
assert!(
crate::types::tool_metadata::ToolMetadata::description_template(&tool)
.contains("Search the web")
);
assert!(
crate::types::tool_metadata::ToolMetadata::description_template(&tool)
.contains("coding")
);
}
#[tokio::test]
async fn errors_when_client_not_in_resources() {
let resources = Resources::new();
let tool = WebSearchTool;
let result = kigi_tool_runtime::Tool::run(
&tool,
test_ctx_with_call_id(resources.into_shared(), "test-call"),
WebSearchInput {
query: "test".into(),
allowed_domains: None,
},
)
.await;
assert!(result.is_err());
let err_msg = result.unwrap_err().to_string();
assert!(
err_msg.contains("missing required resource"),
"Expected 'missing required resource' error, got: {err_msg}"
);
}
}
@@ -0,0 +1,287 @@
//! Concise variant of the `run_terminal_cmd` (bash) tool.
use strip_ansi_escapes::strip_str;
use crate::implementations::grok_build::bash::{
BashTool, BashToolInput, BashToolOutput, KillReason,
};
use crate::types::output::BashOutput;
use crate::types::requirements::{Expr, ToolParamsRequirement, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
use crate::util::truncate::format_bytes;
fn annotations(bash: &BashOutput) -> String {
let mut s = String::new();
if bash.truncated {
let shown = format_bytes(bash.output.len());
let total = format_bytes(bash.total_bytes);
s.push_str(&format!(
" [truncated: showing last {} of {} - full output at: {}]",
shown, total, bash.output_file
));
}
if let Some(signal) = &bash.signal {
// Synthetic kill reasons are conveyed by the `Exit code: killed (reason)`
// header — suppress the redundant `[signal=…]` / `[timeout]` here.
if signal.parse::<KillReason>().is_err() {
s.push_str(&format!(" [signal={}]", signal));
}
}
s
}
/// CONCISE foreground format: `Exit code: N [annotations]\n\nCommand output:\n\n```...```\n\nCommand completed.\n...`
///
/// When the process was killed by the harness or a kernel signal
/// (see [`KillReason`]), the header reads
/// `Exit code: killed (reason)` instead of `Exit code: -1 [signal=…]`.
fn format_concise_foreground_prompt(bash: &BashOutput) -> String {
let raw = String::from_utf8_lossy(&bash.output);
let output_str = strip_str(&raw).to_string();
let header = match bash
.signal
.as_deref()
.and_then(|s| s.parse::<KillReason>().ok())
{
Some(reason) => format!("Exit code: killed ({}){}", reason, annotations(bash)),
None => format!("Exit code: {}{}", bash.exit_code, annotations(bash)),
};
format!(
"{}\n\n\
Command output:\n\n\
```\n{}\n```\n\n\
Command completed.\n\n\
The previous shell command ended, so on the next invocation of this tool, \
you will be using a new shell session.\n\n\
On the next terminal tool call, the directory of the shell will be {}.",
header, output_str, bash.current_dir
)
}
/// CONCISE backgrounded format: same as DEFAULT backgrounded (code-fenced partial output).
fn format_concise_background_prompt(bash: &BashOutput) -> String {
let raw = String::from_utf8_lossy(&bash.output);
let output_str = strip_str(&raw).to_string();
let shown = format_bytes(bash.output.len());
let total = format_bytes(bash.total_bytes);
format!(
"[Command moved to background]\n\n\
Partial output ({} of {} total):\n\n\
```\n{}\n```\n\n\
The command is still running in the background. You can continue with other tasks.\n\
Full output is being written to: {}\n\n\
On the next terminal tool call, the directory of the shell will be {}.",
shown, total, output_str, bash.output_file, bash.current_dir
)
}
/// Concise variant of `BashTool`.
///
/// Delegates to `BashTool::run()`, then overwrites `output_for_prompt` with
/// the concise format. The `concise` concept lives entirely in this file.
#[derive(Debug, Default)]
pub struct BashConciseTool;
impl crate::types::tool_metadata::ToolMetadata for BashConciseTool {
fn kind(&self) -> ToolKind {
ToolKind::Execute
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuildConcise
}
fn description_template(&self) -> &str {
crate::types::tool_metadata::ToolMetadata::description_template(&BashTool)
}
fn emitted_notifications(&self) -> &'static [&'static str] {
crate::types::tool_metadata::ToolMetadata::emitted_notifications(&BashTool)
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
Expr::And(vec![
Expr::Value(ToolRequirement::if_params(
ToolParamsRequirement::new("enabled_background", true),
ToolRequirement::tool_kind(ToolKind::BackgroundTaskAction),
)),
Expr::Value(ToolRequirement::if_params(
ToolParamsRequirement::new("enabled_background", true),
ToolRequirement::tool_kind(ToolKind::KillTaskAction),
)),
])
}
}
impl kigi_tool_runtime::Tool for BashConciseTool {
type Args = BashToolInput;
type Output = BashToolOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("run_terminal_cmd").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"run_terminal_cmd",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(name = "tool.run_terminal_cmd_concise", skip_all)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: BashToolInput,
) -> Result<BashToolOutput, kigi_tool_runtime::ToolError> {
let result = kigi_tool_runtime::Tool::run(&BashTool, ctx, input).await?;
match result {
// TODO: Add different concise message for auto backgrounded terminal task
BashToolOutput::Foreground(mut bash) => {
let is_backgrounded = bash.signal.as_deref() == Some("backgrounded");
bash.output_for_prompt = if is_backgrounded {
format_concise_background_prompt(&bash)
} else {
format_concise_foreground_prompt(&bash)
};
Ok(BashToolOutput::Foreground(bash))
}
bg @ BashToolOutput::Background(_) => Ok(bg),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_bash(exit_code: i32, output: &str) -> BashOutput {
BashOutput {
output: output.as_bytes().to_vec(),
output_for_prompt: BashOutput::make_output_for_prompt(output),
exit_code,
command: "echo test".to_string(),
truncated: false,
signal: None,
timed_out: false,
description: None,
current_dir: "/tmp".to_string(),
output_file: String::new(),
total_bytes: output.len(),
output_delta: None,
was_bare_echo: false,
}
}
#[test]
fn concise_prompt_starts_with_exit_code() {
let bash = make_bash(0, "hello world\n");
let prompt = format_concise_foreground_prompt(&bash);
assert!(
prompt.starts_with("Exit code: 0"),
"CONCISE should start with 'Exit code: 0', got: {prompt}"
);
assert!(prompt.contains("```"), "CONCISE should have code fences");
assert!(prompt.contains("Command completed."));
assert!(!prompt.contains("exit: 0"));
}
#[test]
fn concise_prompt_no_double_header() {
let bash = make_bash(0, "hello\n");
let prompt = format_concise_foreground_prompt(&bash);
assert_eq!(
prompt.matches("Exit code:").count(),
1,
"Should have exactly one 'Exit code:' header, got: {prompt}"
);
assert!(
!prompt.contains("exit: "),
"Should NOT contain DEFAULT 'exit: ' header, got: {prompt}"
);
}
#[test]
fn concise_timeout_annotations() {
let mut bash = make_bash(-1, "partial\n");
bash.signal = Some("timeout".to_string());
bash.timed_out = true;
let prompt = format_concise_foreground_prompt(&bash);
// Synthetic kill reasons render as `Exit code: killed (reason)` —
// no redundant `[signal=…]` / `[timeout]` annotation.
assert!(
prompt.starts_with("Exit code: killed (timeout)"),
"expected `Exit code: killed (timeout)` header, got: {prompt}"
);
assert!(!prompt.contains("[signal=timeout]"));
assert!(!prompt.contains("[timeout]"));
assert!(!prompt.starts_with("Exit code: -1"));
}
#[test]
fn concise_killed_reasons() {
for reason in ["timeout", "max_runtime", "cancelled", "killed", "signal 15"] {
let mut bash = make_bash(-1, "partial\n");
bash.signal = Some(reason.to_string());
let prompt = format_concise_foreground_prompt(&bash);
let expected = format!("Exit code: killed ({})", reason);
assert!(
prompt.starts_with(&expected),
"expected header `{expected}`, got: {prompt}"
);
assert!(!prompt.contains("[signal="));
}
}
#[test]
fn concise_backgrounded() {
let mut bash = make_bash(0, "partial output\n");
bash.signal = Some("backgrounded".to_string());
bash.output_file = "/tmp/bg.log".to_string();
bash.total_bytes = 10000;
let prompt = format_concise_background_prompt(&bash);
assert!(prompt.starts_with("[Command moved to background]"));
assert!(prompt.contains("```"));
assert!(prompt.contains("still running in the background"));
}
#[test]
fn no_double_header_after_default_prebake() {
// Simulate the real flow: BashTool pre-bakes DEFAULT prompt into
// output_for_prompt, then BashConciseTool overwrites with concise.
// to_prompt_format() is a passthrough — it must NOT add another header.
let mut bash = make_bash(0, "hello world\n");
// Pre-bake DEFAULT (what BashTool::run() does)
bash.output_for_prompt =
crate::implementations::grok_build::bash::format_default_prompt(&bash);
assert!(bash.output_for_prompt.starts_with("exit: 0"));
// Concise post-processing (what BashConciseTool::run() does)
bash.output_for_prompt = format_concise_foreground_prompt(&bash);
// to_prompt_format() is a passthrough
let prompt = crate::types::output::ToolOutput::Bash(bash).to_prompt_format();
assert!(
prompt.starts_with("Exit code: 0"),
"Should start with concise header, got: {prompt}"
);
assert_eq!(
prompt.matches("exit:").count() + prompt.matches("Exit code:").count(),
1,
"Should have exactly one header, got: {prompt}"
);
}
}
@@ -0,0 +1,13 @@
//! `GrokBuildConcise` namespace — concise variants of core GrokBuild tools.
//!
//! These tools share implementation with `grok_build` via `pub(crate)` helpers
//! but produce concise output (compact line numbers, shorter messages,
//! concise bash formatting).
pub mod bash;
pub mod read_file;
pub mod search_replace;
pub use bash::BashConciseTool;
pub use read_file::ReadFileConciseTool;
pub use search_replace::SearchReplaceConciseTool;
@@ -0,0 +1,162 @@
//! Concise variant of the `read_file` tool.
use crate::implementations::grok_build::read_file::{ReadFileInput, run_read_file};
const DESCRIPTION_CONCISE: &str = r#"Reads a file from the computer's filesystem. You can access any file directly by using this tool.
It is okay to read a file that does not exist; an error will be returned.
Usage:
- You can optionally specify a line offset and limit (especially handy for long files).
- Lines in the output are numbered starting at 1, using following format: LINE_NUMBER→LINE_CONTENT.
- You have the capability to call multiple tools in a single response. It is always better to speculatively read multiple files as a batch that are potentially useful."#;
use crate::types::output::ReadFileOutput;
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
/// Concise variant of `ReadFileTool`.
///
/// Delegates to `run_read_file()`, then swaps `content_concise` into `content`
/// (no line-number padding).
#[derive(Debug, Default)]
pub struct ReadFileConciseTool;
impl crate::types::tool_metadata::ToolMetadata for ReadFileConciseTool {
fn kind(&self) -> ToolKind {
ToolKind::Read
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuildConcise
}
fn description_template(&self) -> &str {
DESCRIPTION_CONCISE
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
Expr::True
}
}
impl kigi_tool_runtime::Tool for ReadFileConciseTool {
type Args = ReadFileInput;
type Output = ReadFileOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("read_file").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"read_file",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: true,
tool_scope: Some(kigi_tool_protocol::ToolScope::Read),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.read_file_concise",
skip_all,
fields(path = %input.path)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: ReadFileInput,
) -> Result<ReadFileOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
// GrokBuildConcise is not version-managed — always pass None.
let cwd_override = ctx
.extensions
.get::<kigi_tool_runtime::Cwd>()
.map(|c| c.0.clone());
// `None`: the concise tool does not stream, so it needs no
// text-path streamability signal (see `run_read_file`).
let result = run_read_file(input, cwd_override, None, resources, None).await?;
match result {
ReadFileOutput::FileContent(mut fc) => {
if let Some(concise) = fc.content_concise.take() {
fc.content = concise;
}
Ok(ReadFileOutput::FileContent(fc))
}
other => Ok(other),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::tool_metadata::test_ctx;
use crate::computer::local::LocalFs;
use crate::implementations::grok_build::read_file::ReadFileTool;
use crate::notification::types::ToolNotificationHandle;
use crate::types::resources::{Cwd, FileSystem, NotificationHandle, Resources};
use std::sync::Arc;
use tempfile::TempDir;
fn test_resources(cwd: &std::path::Path) -> Resources {
let mut resources = Resources::new();
resources.insert(Cwd(cwd.to_path_buf()));
resources.insert(FileSystem(Arc::new(LocalFs)));
resources.insert(NotificationHandle(ToolNotificationHandle::noop()));
resources
}
#[test]
fn description_template_selects_concise() {
use crate::types::tool_metadata::ToolMetadata;
let default_tool = ReadFileTool;
let concise_tool = ReadFileConciseTool;
assert_ne!(
ToolMetadata::description_template(&default_tool),
ToolMetadata::description_template(&concise_tool),
);
assert_eq!(
ToolMetadata::description_template(&concise_tool),
super::DESCRIPTION_CONCISE
);
}
#[tokio::test]
async fn concise_mode_uses_concise_content() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("test.txt"), "hello\nworld\n").unwrap();
let tool = ReadFileConciseTool;
let resources = test_resources(tmp.path());
let input = ReadFileInput {
path: "test.txt".to_string(),
offset: None,
limit: None,
pages: None,
format: None,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(resources.into_shared()), input)
.await
.unwrap();
match result {
ReadFileOutput::FileContent(content) => {
assert_eq!(content.content, "1→hello\nworld\n");
}
other => panic!("Expected FileContent, got {:?}", other),
}
}
}
@@ -0,0 +1,202 @@
//! Concise variant of the `search_replace` tool.
use crate::implementations::grok_build::search_replace::{SearchReplaceInput, run_search_replace};
/// Concise description — no read-before-edit enforcement, simplified formatting guidance.
const DESCRIPTION_CONCISE: &str = r#"Replace an exact string in a file.
- Do not include the "LINE_NUMBER→" prefixes from file reads in ${{ params.edit.old_string }} or ${{ params.edit.new_string }}; keep the exact indentation.
- ${{ params.edit.old_string }} must match exactly one place in the file. If it appears more than once, add surrounding lines to make it unique, or set ${{ params.edit.replace_all }} to change every occurrence (handy for renaming an identifier).
- To create a new file, set ${{ params.edit.old_string }} to an empty string."#;
use crate::types::output::SearchReplaceOutput;
use crate::types::requirements::{Expr, ToolRequirement};
use crate::types::tool::{ToolKind, ToolNamespace};
/// Concise variant of `SearchReplaceTool`.
///
/// Differences from `SearchReplaceTool`:
/// - Always skips the read-before-edit guard.
/// - Uses the shorter `tool_output_for_prompt_concise` as prompt output.
/// - No `IfParams` requirement for a Read tool (guard is always skipped).
#[derive(Debug, Default)]
pub struct SearchReplaceConciseTool;
impl crate::types::tool_metadata::ToolMetadata for SearchReplaceConciseTool {
fn kind(&self) -> ToolKind {
ToolKind::Edit
}
fn tool_namespace(&self) -> ToolNamespace {
ToolNamespace::GrokBuildConcise
}
fn description_template(&self) -> &str {
DESCRIPTION_CONCISE
}
fn emitted_notifications(&self) -> &'static [&'static str] {
&["FileWritten"]
}
fn requires_expr(&self) -> Expr<ToolRequirement> {
Expr::And(vec![
Expr::Value(ToolRequirement::input_param(ToolKind::Edit, "old_string")),
Expr::Value(ToolRequirement::input_param(ToolKind::Edit, "new_string")),
Expr::Value(ToolRequirement::input_param(ToolKind::Edit, "replace_all")),
])
}
}
impl kigi_tool_runtime::Tool for SearchReplaceConciseTool {
type Args = SearchReplaceInput;
type Output = SearchReplaceOutput;
fn id(&self) -> kigi_tool_protocol::ToolId {
kigi_tool_protocol::ToolId::new("search_replace").expect("valid tool id")
}
fn description(
&self,
_ctx: &::kigi_tool_runtime::ListToolsContext,
) -> kigi_tool_types::ToolDescription {
kigi_tool_types::ToolDescription::new(
"search_replace",
crate::types::tool_metadata::ToolMetadata::description_template(self),
)
}
fn capabilities(&self) -> kigi_tool_protocol::ToolCapabilities {
kigi_tool_protocol::ToolCapabilities {
is_read_only: false,
tool_scope: Some(kigi_tool_protocol::ToolScope::Write),
..Default::default()
}
}
#[tracing::instrument(
name = "tool.search_replace_concise",
skip_all,
fields(
file_path = %input.file_path,
replace_all = %input.replace_all,
)
)]
async fn run(
&self,
ctx: kigi_tool_runtime::ToolCallContext,
input: SearchReplaceInput,
) -> Result<SearchReplaceOutput, kigi_tool_runtime::ToolError> {
use crate::types::tool_metadata::shared_resources;
let resources = shared_resources(&ctx)?;
let mut result = run_search_replace(input, &ctx, resources).await?;
if let SearchReplaceOutput::EditsApplied(ref mut applied) = result
&& let Some(concise_text) = applied.tool_output_for_prompt_concise.take()
{
applied.tool_output_for_prompt = concise_text;
}
Ok(result)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::tool_metadata::test_ctx;
use crate::computer::local::LocalFs;
use crate::notification::types::ToolNotificationHandle;
use crate::types::resources::{Cwd, FileSystem, NotificationHandle, Resources};
use std::sync::Arc;
use tempfile::TempDir;
fn test_resources(cwd: &std::path::Path) -> Resources {
let mut resources = Resources::new();
resources.insert(Cwd(cwd.to_path_buf()));
resources.insert(FileSystem(Arc::new(LocalFs)));
resources.insert(NotificationHandle(ToolNotificationHandle::noop()));
let edit_params = std::collections::HashMap::from([
("old_string".to_string(), "old_string".to_string()),
("new_string".to_string(), "new_string".to_string()),
("replace_all".to_string(), "replace_all".to_string()),
]);
let kind_map = std::collections::HashMap::from([(ToolKind::Edit, edit_params)]);
let tool_map = std::collections::HashMap::from([
(ToolKind::Read, "read_file".to_string()),
(ToolKind::Edit, "search_replace".to_string()),
]);
resources.insert(crate::types::template_renderer::TemplateRenderer::new(
tool_map, kind_map,
));
resources
}
fn make_input(file: &str, old: &str, new: &str) -> SearchReplaceInput {
SearchReplaceInput {
file_path: file.to_string(),
old_string: old.to_string(),
new_string: new.to_string(),
replace_all: false,
}
}
#[tokio::test]
async fn concise_tool_skips_read_guard() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("test.txt"), "hello world\n").unwrap();
let tool = SearchReplaceConciseTool;
let resources = test_resources(tmp.path());
let input = make_input("test.txt", "hello", "goodbye");
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(resources.into_shared()), input)
.await
.unwrap();
match result {
SearchReplaceOutput::EditsApplied(applied) => {
let content = std::fs::read_to_string(tmp.path().join("test.txt")).unwrap();
assert_eq!(content, "goodbye world\n");
assert_eq!(
applied.tool_output_for_prompt,
"The file test.txt has been updated."
);
}
other => panic!("Expected EditsApplied, got {:?}", other),
}
}
#[tokio::test]
async fn concise_replace_all_output() {
let tmp = TempDir::new().unwrap();
std::fs::write(tmp.path().join("test.txt"), "aaa bbb aaa bbb aaa\n").unwrap();
let tool = SearchReplaceConciseTool;
let resources = test_resources(tmp.path());
let input = SearchReplaceInput {
file_path: "test.txt".to_string(),
old_string: "aaa".to_string(),
new_string: "ccc".to_string(),
replace_all: true,
};
let result = kigi_tool_runtime::Tool::run(&tool, test_ctx(resources.into_shared()), input)
.await
.unwrap();
match result {
SearchReplaceOutput::EditsApplied(applied) => {
let content = std::fs::read_to_string(tmp.path().join("test.txt")).unwrap();
assert_eq!(content, "ccc bbb ccc bbb ccc\n");
assert_eq!(
applied.tool_output_for_prompt,
"The file test.txt has been updated. All occurrences were replaced."
);
}
other => panic!("Expected EditsApplied, got {:?}", other),
}
}
}
@@ -0,0 +1,157 @@
//! Anchor convenience helpers and re-exports.
//!
//! This module re-exports the core types from [`super::scheme`] and provides
//! helper functions for common anchor operations.
pub use super::scheme::{
Anchor, AnchorScheme, CheckpointChain, ChunkFingerprint, ContentOnly, DEFAULT_SEARCH_RADIUS,
ParsedAnchor, ShiftResult, ValidationResult,
};
/// Split file content into lines suitable for anchor generation.
///
/// Strips trailing newlines from each line (matching the convention used by
/// `AnchorScheme::generate_anchors`). The returned `Vec<&str>` has one entry
/// per logical line.
pub fn split_lines(content: &str) -> Vec<&str> {
if content.is_empty() {
return vec![""];
}
let mut lines: Vec<&str> = content.lines().collect();
// `str::lines()` does not yield a trailing empty entry for content ending
// with '\n'. Add one to match the 1-based line numbering convention where
// "hello\n" has 2 lines (line 1 = "hello", line 2 = "").
if content.ends_with('\n') {
lines.push("");
}
lines
}
/// Generate anchors for file content using the given scheme.
///
/// Convenience wrapper: splits `content` into lines and calls
/// `scheme.generate_anchors()`.
pub fn generate_for_content(scheme: &dyn AnchorScheme, content: &str) -> Vec<Anchor> {
let lines = split_lines(content);
scheme.generate_anchors(&lines)
}
/// Validate a parsed anchor against file content.
///
/// Convenience wrapper: splits `content` into lines and calls
/// `scheme.validate()`.
pub fn validate_against_content(
scheme: &dyn AnchorScheme,
anchor: &ParsedAnchor,
content: &str,
) -> ValidationResult {
let lines = split_lines(content);
scheme.validate(anchor, &lines)
}
/// Search for a shifted anchor in file content.
///
/// Convenience wrapper: splits `content` into lines and calls
/// `scheme.find_shifted()`.
pub fn find_shifted_in_content(
scheme: &dyn AnchorScheme,
anchor: &ParsedAnchor,
content: &str,
search_radius: usize,
) -> ShiftResult {
let lines = split_lines(content);
scheme.find_shifted(anchor, &lines, search_radius)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn split_lines_basic() {
assert_eq!(split_lines("a\nb\nc"), vec!["a", "b", "c"]);
}
#[test]
fn split_lines_trailing_newline() {
assert_eq!(split_lines("a\nb\n"), vec!["a", "b", ""]);
}
#[test]
fn split_lines_empty() {
assert_eq!(split_lines(""), vec![""]);
}
#[test]
fn split_lines_single_newline() {
assert_eq!(split_lines("\n"), vec!["", ""]);
}
#[test]
fn generate_for_content_roundtrip() {
let content = "line one\nline two\nline three\n";
let scheme = ContentOnly::new();
let anchors = generate_for_content(&scheme, content);
assert_eq!(anchors.len(), 4); // 3 content lines + trailing empty
assert_eq!(anchors[0].line, 1);
assert_eq!(anchors[3].line, 4);
}
#[test]
fn validate_against_content_valid() {
let content = "let x = 1;\nlet y = 2;\n";
let scheme = ContentOnly::new();
let anchors = generate_for_content(&scheme, content);
let parsed = ParsedAnchor {
line: anchors[0].line,
local: anchors[0].local.clone(),
context: None,
};
assert_eq!(
validate_against_content(&scheme, &parsed, content),
ValidationResult::Valid
);
}
#[test]
fn validate_against_content_stale() {
let original = "let x = 1;\nlet y = 2;\n";
let scheme = ContentOnly::new();
let anchors = generate_for_content(&scheme, original);
let modified = "let x = 999;\nlet y = 2;\n";
let parsed = ParsedAnchor {
line: anchors[0].line,
local: anchors[0].local.clone(),
context: None,
};
assert_eq!(
validate_against_content(&scheme, &parsed, modified),
ValidationResult::Stale
);
}
#[test]
fn find_shifted_in_content_found() {
let original = "a\nb\nc\n";
let scheme = ContentOnly::new();
let anchors = generate_for_content(&scheme, original);
// Insert a line at the top → "b" shifts from line 2 to line 3.
let modified = "new\na\nb\nc\n";
let parsed = ParsedAnchor {
line: anchors[1].line, // originally line 2 ("b")
local: anchors[1].local.clone(),
context: None,
};
match find_shifted_in_content(&scheme, &parsed, modified, 5) {
ShiftResult::Found { new_line } => assert_eq!(new_line, 3),
other => panic!("Expected Found, got {:?}", other),
}
}
}
@@ -0,0 +1,887 @@
//! Offline benchmark harness for comparing anchor schemes.
//!
//! This module implements the Phase 1 (non-LLM microbenchmarks) and Phase 2
//! (deterministic edit-trace simulation) for the hashline anchor schemes.
//!
//! ## Usage
//!
//! ```rust,ignore
//! use kigi_tools::implementations::grok_build_hashline::benchmark::*;
//!
//! let corpus = vec![
//! ("small.rs", "fn main() {}\n"),
//! ("medium.rs", include_str!("fixtures/medium.rs.txt")),
//! ];
//! let report = run_benchmark(&corpus, &BenchmarkConfig::default());
//! println!("{report}");
//! ```
use std::fmt;
use std::time::Instant;
use super::anchor::split_lines;
use super::mutate::{self, LineOutcome, Mutation, apply_mutation};
use super::scheme::{
AnchorScheme, CheckpointChain, ChunkFingerprint, ContentOnly, DEFAULT_SEARCH_RADIUS,
ParsedAnchor, ShiftResult, ValidationResult,
};
/// Configuration for the benchmark harness.
#[derive(Debug, Clone)]
pub struct BenchmarkConfig {
/// Hash lengths to test (default: [2, 3]).
pub hash_lengths: Vec<usize>,
/// Chunk sizes to test for Candidate B (default: [8, 16, 32]).
pub chunk_sizes: Vec<usize>,
/// Checkpoint intervals to test for Candidate C (default: [16, 32, 64]).
pub checkpoint_intervals: Vec<usize>,
/// Search radius for shifted-anchor recovery (default: 15).
pub search_radius: usize,
}
impl Default for BenchmarkConfig {
fn default() -> Self {
Self {
hash_lengths: vec![2, 3],
chunk_sizes: vec![8, 16, 32],
checkpoint_intervals: vec![16, 32, 64],
search_radius: DEFAULT_SEARCH_RADIUS,
}
}
}
/// Aggregated metrics for one scheme configuration across all corpus files
/// and all mutation scenarios.
#[derive(Debug, Clone)]
pub struct SchemeMetrics {
/// Human-readable scheme description (e.g. "chunk_v1 hash=3 chunk=16").
pub label: String,
/// Total anchors validated across all scenarios.
pub total_validations: usize,
/// Anchors that correctly reported Valid (true positives for "unchanged").
pub true_valid: usize,
/// Anchors that correctly reported Stale (true positives for "changed").
pub true_stale: usize,
/// Anchors that reported Valid but should have been Stale (false acceptance).
pub false_valid: usize,
/// Anchors that reported Stale but should have been Valid (false rejection).
pub false_stale: usize,
/// Shifted-anchor recovery attempts.
pub recovery_attempts: usize,
/// Shifted-anchor recovery: found the correct shifted target.
pub recovery_correct: usize,
/// Shifted-anchor recovery: found a line, but not the correct target.
pub recovery_wrong: usize,
/// Shifted-anchor recovery: ambiguous (multiple candidates).
pub recovery_ambiguous: usize,
/// Shifted-anchor recovery: not found.
pub recovery_not_found: usize,
/// Collision count: distinct lines that produced the same anchor in the
/// same file (local hash only).
pub collision_count: usize,
/// Total lines across all corpus files.
pub total_lines: usize,
/// Total validation time in microseconds.
pub validation_us: u128,
/// Number of edit-trace steps completed.
pub trace_steps: usize,
/// Edit-trace: steps where post-edit anchors remained valid.
pub trace_anchors_survived: usize,
/// Edit-trace: steps that required re-read (anchor stale after edit).
pub trace_reread_required: usize,
/// Estimated total read-amplification lines across all validations.
/// Candidate A: 1 line per validation.
/// Candidate B: chunk_size lines per validation.
/// Candidate C: (line_idx - checkpoint_start + 1) lines per validation.
pub read_amp_lines: usize,
}
impl SchemeMetrics {
fn new(label: String) -> Self {
Self {
label,
total_validations: 0,
true_valid: 0,
true_stale: 0,
false_valid: 0,
false_stale: 0,
recovery_attempts: 0,
recovery_correct: 0,
recovery_wrong: 0,
recovery_ambiguous: 0,
recovery_not_found: 0,
collision_count: 0,
total_lines: 0,
validation_us: 0,
trace_steps: 0,
trace_anchors_survived: 0,
trace_reread_required: 0,
read_amp_lines: 0,
}
}
/// Stale detection precision: of anchors reported Stale, fraction that
/// were truly changed.
pub fn stale_precision(&self) -> f64 {
let reported_stale = self.true_stale + self.false_stale;
if reported_stale == 0 {
return 1.0;
}
self.true_stale as f64 / reported_stale as f64
}
/// Stale detection recall: of anchors that were truly changed, fraction
/// correctly detected as Stale.
pub fn stale_recall(&self) -> f64 {
let truly_changed = self.true_stale + self.false_valid;
if truly_changed == 0 {
return 1.0;
}
self.true_stale as f64 / truly_changed as f64
}
/// Collision rate: fraction of total lines that share an anchor with
/// another line in the same file.
pub fn collision_rate(&self) -> f64 {
if self.total_lines == 0 {
return 0.0;
}
self.collision_count as f64 / self.total_lines as f64
}
/// Average validation latency in microseconds.
pub fn avg_validation_us(&self) -> f64 {
if self.total_validations == 0 {
return 0.0;
}
self.validation_us as f64 / self.total_validations as f64
}
/// Average read-amplification lines per validation.
pub fn avg_read_amp_lines(&self) -> f64 {
if self.total_validations == 0 {
return 0.0;
}
self.read_amp_lines as f64 / self.total_validations as f64
}
/// Edit-trace anchor survival rate.
pub fn trace_survival_rate(&self) -> f64 {
if self.trace_steps == 0 {
return 0.0;
}
self.trace_anchors_survived as f64 / self.trace_steps as f64
}
}
/// Complete benchmark report across all scheme configurations.
#[derive(Debug, Clone)]
pub struct BenchmarkReport {
/// Per-scheme metrics, one entry per configuration tested.
pub schemes: Vec<SchemeMetrics>,
/// Number of corpus files used.
pub corpus_files: usize,
/// Total lines across all corpus files.
pub total_corpus_lines: usize,
}
impl fmt::Display for BenchmarkReport {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
writeln!(f, "=== Hashline Anchor Benchmark Report ===")?;
writeln!(
f,
"Corpus: {} files, {} total lines\n",
self.corpus_files, self.total_corpus_lines
)?;
writeln!(
f,
"{:<30} {:>6} {:>8} {:>8} {:>7} {:>7} {:>8} {:>8} {:>7} {:>10}",
"Scheme",
"Lines",
"Collis%",
"FalseOK",
"Prec",
"Recall",
"Recov%",
"Surv%",
"RdAmp",
"Avg µs"
)?;
writeln!(f, "{}", "-".repeat(115))?;
for m in &self.schemes {
let recovery_rate = if m.recovery_attempts > 0 {
m.recovery_correct as f64 / m.recovery_attempts as f64 * 100.0
} else {
0.0
};
writeln!(
f,
"{:<30} {:>6} {:>7.3}% {:>8} {:>6.3} {:>6.3} {:>7.1}% {:>7.1}% {:>6.1} {:>10.2}",
m.label,
m.total_lines,
m.collision_rate() * 100.0,
m.false_valid,
m.stale_precision(),
m.stale_recall(),
recovery_rate,
m.trace_survival_rate() * 100.0,
m.avg_read_amp_lines(),
m.avg_validation_us(),
)?;
}
Ok(())
}
}
/// Run the full benchmark across all scheme configurations.
///
/// `corpus` is a slice of `(filename, file_content)` pairs.
pub fn run_benchmark(corpus: &[(&str, &str)], config: &BenchmarkConfig) -> BenchmarkReport {
let mut all_metrics: Vec<SchemeMetrics> = Vec::new();
let total_corpus_lines: usize = corpus
.iter()
.map(|(_, content)| split_lines(content).len())
.sum();
// Build scheme configurations to test.
let schemes = build_scheme_configs(config);
for (label, scheme) in &schemes {
let mut metrics = SchemeMetrics::new(label.clone());
for (name, content) in corpus {
run_phase1_for_file(&**scheme, name, content, config, &mut metrics);
run_phase2_for_file(&**scheme, name, content, config, &mut metrics);
}
all_metrics.push(metrics);
}
BenchmarkReport {
schemes: all_metrics,
corpus_files: corpus.len(),
total_corpus_lines,
}
}
/// Build all scheme configurations to benchmark from the config.
fn build_scheme_configs(config: &BenchmarkConfig) -> Vec<(String, Box<dyn AnchorScheme>)> {
let mut schemes: Vec<(String, Box<dyn AnchorScheme>)> = Vec::new();
for &hl in &config.hash_lengths {
// Candidate A
schemes.push((
format!("content_only h={hl}"),
Box::new(ContentOnly::with_hash_len(hl)),
));
// Candidate B — vary chunk size
for &cs in &config.chunk_sizes {
schemes.push((
format!("chunk h={hl} c={cs}"),
Box::new(ChunkFingerprint::with_params(hl, cs)),
));
}
// Candidate C — vary checkpoint interval
for &ci in &config.checkpoint_intervals {
schemes.push((
format!("checkpoint h={hl} i={ci}"),
Box::new(CheckpointChain::with_params(hl, ci)),
));
}
}
schemes
}
/// Standard set of mutations to apply per file.
fn standard_mutations(line_count: usize) -> Vec<(&'static str, Mutation)> {
if line_count < 3 {
return vec![];
}
let mid = line_count / 2;
let mut mutations = vec![
("insert_above_mid", mutate::gen_insert_above(mid, 3)),
("delete_at_mid", mutate::gen_delete(mid, 2)),
(
"token_edit_mid",
mutate::gen_token_edit(mid, "// EDITED LINE"),
),
("reindent_mid", mutate::gen_reindent(mid, " ")),
(
"boilerplate_top",
mutate::gen_boilerplate_insert(0, "// boilerplate", 5),
),
];
if line_count >= 6 {
mutations.push((
"range_rewrite",
mutate::gen_range_rewrite(mid, (mid + 3).min(line_count), &["// replaced"]),
));
}
mutations
}
/// Estimate the read-amplification cost for a single validation under the
/// given scheme, using the scheme's own `validation_window_lines()` method.
fn estimate_read_amp_lines(scheme: &dyn AnchorScheme, line_count: usize, line_idx: usize) -> usize {
scheme.validation_window_lines(line_idx, line_count)
}
/// Run Phase 1 (single-mutation microbenchmarks) for one file.
fn run_phase1_for_file(
scheme: &dyn AnchorScheme,
_file_name: &str,
content: &str,
config: &BenchmarkConfig,
metrics: &mut SchemeMetrics,
) {
let original_lines = split_lines(content);
let line_count = original_lines.len();
metrics.total_lines += line_count;
// --- Collision measurement ---
let anchors = scheme.generate_anchors(&original_lines);
let mut seen = std::collections::HashSet::new();
for a in &anchors {
let key = match &a.context {
Some(ctx) => format!("{}:{}", a.local, ctx),
None => a.local.clone(),
};
if !seen.insert(key) {
metrics.collision_count += 1;
}
}
// --- Mutation scenarios ---
let mutations = standard_mutations(line_count);
for (_mutation_name, mutation) in &mutations {
let mut mutated_lines: Vec<String> = original_lines.iter().map(|s| s.to_string()).collect();
let mutation_result = apply_mutation(&mut mutated_lines, mutation);
let mutated_refs: Vec<&str> = mutated_lines.iter().map(|s| s.as_str()).collect();
// For each original anchor, validate against the mutated file.
for (orig_idx, anchor) in anchors.iter().enumerate() {
let parsed = ParsedAnchor {
line: anchor.line,
local: anchor.local.clone(),
context: anchor.context.clone(),
};
// Ground truth: determine expected validity based on LineOutcome.
// An anchor should be Valid if the line is Unchanged or Reindented
// (whitespace-normalized hashing preserves anchors across
// indentation changes). Shifted, Modified, and Deleted anchors
// should all be detected as invalid (Stale or OutOfRange).
let outcome = &mutation_result.outcomes[orig_idx];
let should_be_valid =
matches!(outcome, LineOutcome::Unchanged | LineOutcome::Reindented);
let t0 = Instant::now();
let result = scheme.validate(&parsed, &mutated_refs);
metrics.validation_us += t0.elapsed().as_micros();
metrics.total_validations += 1;
metrics.read_amp_lines += estimate_read_amp_lines(scheme, line_count, orig_idx);
let reported_valid = result == ValidationResult::Valid;
if reported_valid && should_be_valid {
metrics.true_valid += 1;
} else if reported_valid && !should_be_valid {
metrics.false_valid += 1;
} else if !reported_valid && !should_be_valid {
metrics.true_stale += 1;
} else {
// !reported_valid && should_be_valid
metrics.false_stale += 1;
}
// Shifted recovery: attempt when validation failed and the
// line was shifted (not modified or deleted).
if !reported_valid && let LineOutcome::Shifted { new_idx } = outcome {
metrics.recovery_attempts += 1;
let expected_line = new_idx + 1; // 1-based
match scheme.find_shifted(&parsed, &mutated_refs, config.search_radius) {
ShiftResult::Found { new_line } => {
if new_line == expected_line {
metrics.recovery_correct += 1;
} else {
metrics.recovery_wrong += 1;
}
}
ShiftResult::Ambiguous { .. } => metrics.recovery_ambiguous += 1,
ShiftResult::NotFound => metrics.recovery_not_found += 1,
}
}
}
}
}
/// A single step in a deterministic edit trace.
struct TraceStep {
mutation: Mutation,
/// 0-based index of the anchor to probe after the edit.
probe_anchor_idx: usize,
}
/// Standard edit traces to run per file.
fn standard_traces(line_count: usize) -> Vec<Vec<TraceStep>> {
if line_count < 6 {
return vec![];
}
let mid = line_count / 2;
vec![
// Trace 1: point edit followed by nearby point edit
vec![
TraceStep {
mutation: mutate::gen_token_edit(mid, "// step1 edit"),
probe_anchor_idx: mid + 1,
},
TraceStep {
mutation: mutate::gen_token_edit(mid + 1, "// step2 edit"),
probe_anchor_idx: mid + 2,
},
],
// Trace 2: insert above then probe below
vec![
TraceStep {
mutation: mutate::gen_insert_above(mid, 2),
probe_anchor_idx: mid + 1,
},
TraceStep {
mutation: mutate::gen_token_edit(mid + 3, "// post-insert edit"),
probe_anchor_idx: mid + 4,
},
],
// Trace 3: reindent (formatter pass) then edit
vec![
TraceStep {
mutation: mutate::gen_reindent(mid, " "),
probe_anchor_idx: mid,
},
TraceStep {
mutation: mutate::gen_token_edit(mid + 1, "// after reindent"),
probe_anchor_idx: mid + 2,
},
],
]
}
/// Run Phase 2 (edit-trace simulation) for one file.
///
/// The simulation keeps using the existing anchor set as long as the probed
/// anchor survives. Anchors are only regenerated (simulating a re-read) when
/// the probed anchor is stale. This measures how often each scheme forces a
/// re-read in sequential editing workflows.
fn run_phase2_for_file(
scheme: &dyn AnchorScheme,
_file_name: &str,
content: &str,
_config: &BenchmarkConfig,
metrics: &mut SchemeMetrics,
) {
let original_lines = split_lines(content);
let line_count = original_lines.len();
let traces = standard_traces(line_count);
for trace in &traces {
// Start with the original file and its anchors.
let mut current_lines: Vec<String> = original_lines.iter().map(|s| s.to_string()).collect();
let mut current_anchors = scheme.generate_anchors(&original_lines);
let mut needs_refresh = false;
for step in trace {
// If the previous step required a re-read, regenerate anchors now.
if needs_refresh {
let refs: Vec<&str> = current_lines.iter().map(|s| s.as_str()).collect();
current_anchors = scheme.generate_anchors(&refs);
needs_refresh = false;
}
let probe_idx = step.probe_anchor_idx;
if probe_idx >= current_anchors.len() {
continue;
}
// Snapshot the anchor we want to probe.
let probe_anchor = ParsedAnchor {
line: current_anchors[probe_idx].line,
local: current_anchors[probe_idx].local.clone(),
context: current_anchors[probe_idx].context.clone(),
};
// Apply the mutation.
apply_mutation(&mut current_lines, &step.mutation);
// Validate the probed anchor against the mutated file.
let refs: Vec<&str> = current_lines.iter().map(|s| s.as_str()).collect();
let result = scheme.validate(&probe_anchor, &refs);
metrics.trace_steps += 1;
if result == ValidationResult::Valid {
metrics.trace_anchors_survived += 1;
// Keep using existing anchors — no refresh needed.
} else {
metrics.trace_reread_required += 1;
// Mark for refresh at the start of the next step.
needs_refresh = true;
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
const SMALL_FILE: &str =
"fn main() {\n let x = 1;\n let y = 2;\n println!(\"{x} {y}\");\n}\n";
const MEDIUM_FILE: &str = "\
use std::collections::HashMap;
fn process(items: &[String]) -> HashMap<String, usize> {
let mut counts = HashMap::new();
for item in items {
*counts.entry(item.clone()).or_insert(0) += 1;
}
counts
}
fn format_counts(counts: &HashMap<String, usize>) -> String {
let mut result = String::new();
for (key, value) in counts {
result.push_str(&format!(\"{key}: {value}\\n\"));
}
result
}
fn main() {
let items = vec![
\"apple\".to_string(),
\"banana\".to_string(),
\"apple\".to_string(),
\"cherry\".to_string(),
];
let counts = process(&items);
let formatted = format_counts(&counts);
println!(\"{formatted}\");
}
";
const REPETITIVE_FILE: &str = "\
use serde::Deserialize;
#[derive(Debug, Deserialize)]
struct Config {
name: String,
value: String,
}
#[derive(Debug, Deserialize)]
struct Config2 {
name: String,
value: String,
}
#[derive(Debug, Deserialize)]
struct Config3 {
name: String,
value: String,
}
";
fn test_corpus() -> Vec<(&'static str, &'static str)> {
vec![
("small.rs", SMALL_FILE),
("medium.rs", MEDIUM_FILE),
("repetitive.rs", REPETITIVE_FILE),
]
}
#[test]
fn benchmark_runs_without_panic() {
let corpus = test_corpus();
let report = run_benchmark(&corpus, &BenchmarkConfig::default());
assert!(!report.schemes.is_empty());
assert_eq!(report.corpus_files, 3);
assert!(report.total_corpus_lines > 0);
}
#[test]
fn all_schemes_produce_metrics() {
let corpus = test_corpus();
let config = BenchmarkConfig::default();
let report = run_benchmark(&corpus, &config);
// Expected: hash_lengths.len() * (1 + chunk_sizes.len() + checkpoint_intervals.len())
let expected = config.hash_lengths.len()
* (1 + config.chunk_sizes.len() + config.checkpoint_intervals.len());
assert_eq!(report.schemes.len(), expected);
for m in &report.schemes {
assert!(m.total_validations > 0, "no validations for {}", m.label);
assert!(m.total_lines > 0, "no lines counted for {}", m.label);
}
}
#[test]
fn content_only_has_zero_false_stale() {
// With proper ground truth (Unchanged = should be Valid, Shifted =
// should be Stale), Candidate A should have zero false_stale:
// it never reports Stale for a truly unchanged-at-same-position line
// because it has no contextual component.
let corpus = test_corpus();
let config = BenchmarkConfig {
hash_lengths: vec![3],
chunk_sizes: vec![],
checkpoint_intervals: vec![],
search_radius: DEFAULT_SEARCH_RADIUS,
};
let report = run_benchmark(&corpus, &config);
assert_eq!(report.schemes.len(), 1);
assert_eq!(
report.schemes[0].false_stale, 0,
"content_only should have zero false_stale with correct ground truth"
);
}
#[test]
fn chunk_has_nonzero_false_stale() {
// Candidate B reports Stale for unchanged lines when a nearby line
// in the same chunk changed (chunk context invalidation). These are
// false_stale: the line is unchanged but the scheme conservatively
// rejects it.
let corpus = test_corpus();
let config = BenchmarkConfig {
hash_lengths: vec![3],
chunk_sizes: vec![16],
checkpoint_intervals: vec![],
search_radius: DEFAULT_SEARCH_RADIUS,
};
let report = run_benchmark(&corpus, &config);
assert_eq!(report.schemes.len(), 2); // A + B
let b = &report.schemes[1];
assert!(
b.false_stale > 0,
"chunk scheme should have some false_stale from chunk invalidation"
);
}
#[test]
fn chunk_has_higher_stale_recall_than_content_only() {
// Candidate B should detect more staleness than A because it also
// invalidates when the chunk changes.
let corpus = test_corpus();
let config = BenchmarkConfig {
hash_lengths: vec![3],
chunk_sizes: vec![16],
checkpoint_intervals: vec![],
search_radius: DEFAULT_SEARCH_RADIUS,
};
let report = run_benchmark(&corpus, &config);
assert_eq!(report.schemes.len(), 2); // A + B
let a = &report.schemes[0];
let b = &report.schemes[1];
// B should report at least as many stale results as A.
let a_stale = a.true_stale + a.false_stale;
let b_stale = b.true_stale + b.false_stale;
assert!(
b_stale >= a_stale,
"chunk should detect at least as many stale as content_only: B={b_stale} A={a_stale}"
);
}
#[test]
fn repetitive_file_shows_collisions() {
let corpus = vec![("repetitive.rs", REPETITIVE_FILE)];
let config = BenchmarkConfig {
hash_lengths: vec![3],
chunk_sizes: vec![],
checkpoint_intervals: vec![],
search_radius: DEFAULT_SEARCH_RADIUS,
};
let report = run_benchmark(&corpus, &config);
// Repetitive file has identical struct fields ("name: String," etc.)
// so content-only should show collisions.
assert!(
report.schemes[0].collision_count > 0,
"repetitive file should produce collisions"
);
}
#[test]
fn edit_trace_metrics_populated() {
let corpus = test_corpus();
let config = BenchmarkConfig {
hash_lengths: vec![3],
chunk_sizes: vec![16],
checkpoint_intervals: vec![],
search_radius: DEFAULT_SEARCH_RADIUS,
};
let report = run_benchmark(&corpus, &config);
for m in &report.schemes {
assert!(
m.trace_steps > 0,
"trace_steps should be > 0 for {}",
m.label
);
}
}
#[test]
fn report_display_does_not_panic() {
let corpus = test_corpus();
let report = run_benchmark(&corpus, &BenchmarkConfig::default());
let output = format!("{report}");
assert!(output.contains("Hashline Anchor Benchmark Report"));
assert!(output.contains("content_only"));
assert!(output.contains("chunk"));
assert!(output.contains("checkpoint"));
}
#[test]
fn stale_precision_recall_bounds() {
let corpus = test_corpus();
let report = run_benchmark(&corpus, &BenchmarkConfig::default());
for m in &report.schemes {
let p = m.stale_precision();
let r = m.stale_recall();
assert!(
(0.0..=1.0).contains(&p),
"precision out of bounds for {}: {p}",
m.label
);
assert!(
(0.0..=1.0).contains(&r),
"recall out of bounds for {}: {r}",
m.label
);
}
}
#[test]
fn empty_corpus_produces_empty_report() {
let report = run_benchmark(&[], &BenchmarkConfig::default());
assert_eq!(report.corpus_files, 0);
assert_eq!(report.total_corpus_lines, 0);
for m in &report.schemes {
assert_eq!(m.total_validations, 0);
}
}
#[test]
fn single_scheme_config() {
let corpus = test_corpus();
let config = BenchmarkConfig {
hash_lengths: vec![3],
chunk_sizes: vec![],
checkpoint_intervals: vec![],
search_radius: 5,
};
let report = run_benchmark(&corpus, &config);
assert_eq!(report.schemes.len(), 1);
assert!(report.schemes[0].label.contains("content_only"));
}
#[test]
fn read_amplification_content_only_is_one() {
let corpus = test_corpus();
let config = BenchmarkConfig {
hash_lengths: vec![3],
chunk_sizes: vec![],
checkpoint_intervals: vec![],
search_radius: DEFAULT_SEARCH_RADIUS,
};
let report = run_benchmark(&corpus, &config);
let a = &report.schemes[0];
// Content-only reads 1 line per validation → avg should be 1.0.
let avg = a.avg_read_amp_lines();
assert!(
(avg - 1.0).abs() < f64::EPSILON,
"content_only avg read amp should be 1.0, got {avg}"
);
}
#[test]
fn read_amplification_chunk_higher_than_content_only() {
let corpus = test_corpus();
let config = BenchmarkConfig {
hash_lengths: vec![3],
chunk_sizes: vec![16],
checkpoint_intervals: vec![],
search_radius: DEFAULT_SEARCH_RADIUS,
};
let report = run_benchmark(&corpus, &config);
let a = &report.schemes[0];
let b = &report.schemes[1];
assert!(
b.avg_read_amp_lines() > a.avg_read_amp_lines(),
"chunk read amp ({}) should be > content_only ({})",
b.avg_read_amp_lines(),
a.avg_read_amp_lines()
);
}
#[test]
fn recovery_correctness_tracked() {
// Verify that recovery_correct + recovery_wrong + recovery_ambiguous
// + recovery_not_found == recovery_attempts.
let corpus = test_corpus();
let config = BenchmarkConfig {
hash_lengths: vec![3],
chunk_sizes: vec![],
checkpoint_intervals: vec![],
search_radius: DEFAULT_SEARCH_RADIUS,
};
let report = run_benchmark(&corpus, &config);
for m in &report.schemes {
let total =
m.recovery_correct + m.recovery_wrong + m.recovery_ambiguous + m.recovery_not_found;
assert_eq!(
total, m.recovery_attempts,
"recovery outcomes should sum to attempts for {}",
m.label
);
}
}
}
@@ -0,0 +1,390 @@
//! Hashline scheme configuration — shared across all hashline tools.
use super::scheme::{AnchorScheme, ChunkFingerprint, ContentOnly};
/// Example anchor strings for tool descriptions.
#[derive(Debug, Clone)]
pub struct ExampleAnchors {
pub anchor: String,
pub read_line1: String,
pub read_line2: String,
pub grep_match: String,
pub grep_context: String,
}
/// Configurable parameters for the hashline anchor scheme.
///
/// Stored as a resource (`Params<HashlineSchemeParams>`) so all three
/// hashline tools use the same scheme within a session.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)]
#[serde(default, deny_unknown_fields)]
pub struct HashlineSchemeParams {
/// Active scheme: `"chunk"` (default) or `"content_only"`.
#[serde(default = "default_scheme_name")]
pub scheme: String,
/// Anchor hash length in characters (1–4).
#[serde(default = "default_hash_len")]
pub hash_len: usize,
/// Chunk size for the chunk scheme.
#[serde(default = "default_chunk_size")]
pub chunk_size: usize,
}
fn default_scheme_name() -> String {
"chunk".to_owned()
}
fn default_hash_len() -> usize {
3
}
fn default_chunk_size() -> usize {
8
}
impl crate::types::resources::ResourceType for HashlineSchemeParams {
const ID: &'static str = "hashline_scheme_params";
fn validate_params_value(
value: &Self,
) -> Result<(), crate::types::params_validation::ParamValidationError> {
match value.validate() {
Ok(()) => Ok(()),
Err(message) if message.contains("unknown scheme") => {
Err(crate::types::params_validation::ParamValidationError::new(
message,
"params_constraint",
)
.with_field_path("scheme")
.with_expected("\"chunk\" or \"content_only\""))
}
Err(message) if message.contains("hash_len") => {
Err(crate::types::params_validation::ParamValidationError::new(
message,
"params_constraint",
)
.with_field_path("hash_len")
.with_expected("1..=4"))
}
Err(message) if message.contains("chunk_size") => {
Err(crate::types::params_validation::ParamValidationError::new(
message,
"params_constraint",
)
.with_field_path("chunk_size")
.with_expected("> 0 when scheme is \"chunk\""))
}
Err(message) => Err(crate::types::params_validation::ParamValidationError::new(
message,
"params_constraint",
)),
}
}
}
impl Default for HashlineSchemeParams {
fn default() -> Self {
Self {
scheme: default_scheme_name(),
hash_len: default_hash_len(),
chunk_size: default_chunk_size(),
}
}
}
impl HashlineSchemeParams {
/// Validate the parameters. Returns an error message if invalid.
pub fn validate(&self) -> Result<(), String> {
match self.scheme.as_str() {
"chunk" | "content_only" => {}
other => {
return Err(format!(
"unknown scheme \"{other}\": expected \"chunk\" or \"content_only\""
));
}
}
if self.hash_len == 0 || self.hash_len > 4 {
return Err(format!("hash_len must be 1..=4, got {}", self.hash_len));
}
if self.scheme == "chunk" && self.chunk_size == 0 {
return Err("chunk_size must be > 0".to_owned());
}
Ok(())
}
/// Generate example anchor strings for use in tool descriptions.
/// Returns (single_anchor, line_with_anchor) based on the configured scheme.
/// Returns `(anchor, read_line1, read_line2, grep_match, grep_context)`.
pub fn example_anchors(&self) -> ExampleAnchors {
let len = self.hash_len.clamp(1, 4);
let hash = &"abcd"[..len];
let ctx = &"rstu"[..len];
match self.scheme.as_str() {
"content_only" => ExampleAnchors {
anchor: format!("22:{hash}"),
read_line1: format!(" 1:{hash}→fn main() {{"),
read_line2: format!(" 2:{hash}→ let x = 1;"),
grep_match: format!("2:{hash}: let x = 1;"),
grep_context: format!("3:{hash}- let y = 2;"),
},
_ => ExampleAnchors {
anchor: format!("22:{hash}:{ctx}"),
read_line1: format!(" 1:{hash}:{ctx}→fn main() {{"),
read_line2: format!(" 2:{hash}:{ctx}→ let x = 1;"),
grep_match: format!("2:{hash}:{ctx}: let x = 1;"),
grep_context: format!("3:{hash}:{ctx}- let y = 2;"),
},
}
}
/// Replace description placeholders with scheme-appropriate examples.
pub fn render_description(&self, template: &str) -> String {
let ex = self.example_anchors();
template
.replace("{example_anchor}", &ex.anchor)
.replace("{example_line1}", &ex.read_line1)
.replace("{example_line2}", &ex.read_line2)
.replace("{grep_match}", &ex.grep_match)
.replace("{grep_context}", &ex.grep_context)
}
/// Build a `ToolDefinition` with scheme-aware description rendering.
///
/// Shared by all 3 hashline tools' `versioned_definition` overrides.
pub fn build_tool_definition(
&self,
template: &str,
client_name: &str,
description_override: Option<&str>,
renderer: &crate::types::template_renderer::TemplateRenderer,
param_map: &std::collections::HashMap<String, String>,
input_schema: &serde_json::Value,
) -> crate::types::definition::ToolDefinition {
let raw = description_override.unwrap_or(template);
let with_examples = self.render_description(raw);
let description = renderer.render(&with_examples).unwrap_or(with_examples);
let schema = if param_map.is_empty() {
input_schema.clone()
} else {
crate::util::remap::remap_schema_properties(input_schema, param_map)
};
crate::types::definition::ToolDefinition::function(client_name, Some(description), schema)
}
/// Validate and build the anchor scheme. Returns an error if params are invalid.
pub fn build_scheme(&self) -> Result<Box<dyn AnchorScheme>, String> {
self.validate()?;
Ok(match self.scheme.as_str() {
"content_only" => Box::new(ContentOnly::with_hash_len(self.hash_len)),
_ => Box::new(ChunkFingerprint::with_params(
self.hash_len,
self.chunk_size,
)),
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn default_builds_chunk_scheme() {
let scheme = HashlineSchemeParams::default().build_scheme().unwrap();
assert_eq!(scheme.name(), "chunk_v1");
assert_eq!(scheme.hash_len(), 3);
}
#[test]
fn content_only_builds() {
let params = HashlineSchemeParams {
scheme: "content_only".to_owned(),
hash_len: 2,
chunk_size: 8,
};
let scheme = params.build_scheme().unwrap();
assert_eq!(scheme.name(), "content_only_v1");
assert_eq!(scheme.hash_len(), 2);
}
#[test]
fn custom_chunk_params() {
let params = HashlineSchemeParams {
scheme: "chunk".to_owned(),
hash_len: 4,
chunk_size: 16,
};
let scheme = params.build_scheme().unwrap();
assert_eq!(scheme.name(), "chunk_v1");
assert_eq!(scheme.hash_len(), 4);
}
#[test]
fn unknown_scheme_rejected() {
let params = HashlineSchemeParams {
scheme: "bogus".to_owned(),
hash_len: 3,
chunk_size: 8,
};
assert!(params.build_scheme().is_err());
assert!(params.validate().unwrap_err().contains("unknown scheme"));
}
#[test]
fn hash_len_zero_rejected() {
let params = HashlineSchemeParams {
scheme: "chunk".to_owned(),
hash_len: 0,
chunk_size: 8,
};
assert!(params.build_scheme().is_err());
}
#[test]
fn hash_len_five_rejected() {
let params = HashlineSchemeParams {
scheme: "chunk".to_owned(),
hash_len: 5,
chunk_size: 8,
};
assert!(params.build_scheme().is_err());
}
#[test]
fn chunk_size_zero_rejected() {
let params = HashlineSchemeParams {
scheme: "chunk".to_owned(),
hash_len: 3,
chunk_size: 0,
};
assert!(params.build_scheme().is_err());
}
#[test]
fn content_only_ignores_chunk_size() {
// chunk_size doesn't matter for content_only scheme.
let params = HashlineSchemeParams {
scheme: "content_only".to_owned(),
hash_len: 3,
chunk_size: 0,
};
assert!(params.build_scheme().is_ok());
}
#[test]
fn deserializes_from_json() {
let json = r#"{"scheme":"content_only","hash_len":2,"chunk_size":4}"#;
let params: HashlineSchemeParams = serde_json::from_str(json).unwrap();
assert_eq!(params.scheme, "content_only");
assert_eq!(params.hash_len, 2);
}
#[test]
fn render_description_chunk_3() {
let params = HashlineSchemeParams::default();
let rendered = params.render_description("anchor={example_anchor}");
assert_eq!(rendered, "anchor=22:abc:rst");
}
#[test]
fn render_description_content_only_2() {
let params = HashlineSchemeParams {
scheme: "content_only".to_owned(),
hash_len: 2,
chunk_size: 8,
};
let rendered = params.render_description("anchor={example_anchor}");
assert_eq!(rendered, "anchor=22:ab");
assert!(!rendered.contains(":rs"));
}
#[test]
fn render_grep_examples_chunk() {
let params = HashlineSchemeParams::default();
let rendered = params.render_description("{grep_match} / {grep_context}");
assert!(rendered.contains("2:abc:rst:"), "match: {rendered}");
assert!(rendered.contains("3:abc:rst-"), "context: {rendered}");
}
#[test]
fn render_grep_examples_content_only() {
let params = HashlineSchemeParams {
scheme: "content_only".to_owned(),
hash_len: 2,
chunk_size: 8,
};
let rendered = params.render_description("{grep_match} / {grep_context}");
assert!(rendered.contains("2:ab:"), "match: {rendered}");
assert!(rendered.contains("3:ab-"), "context: {rendered}");
}
#[test]
fn render_does_not_panic_on_invalid_hash_len() {
let params = HashlineSchemeParams {
scheme: "chunk".to_owned(),
hash_len: 100, // invalid but clamped
chunk_size: 8,
};
let rendered = params.render_description("{example_anchor}");
assert_eq!(rendered, "22:abcd:rstu"); // clamped to 4
}
#[test]
fn render_does_not_panic_on_zero_hash_len() {
let params = HashlineSchemeParams {
scheme: "chunk".to_owned(),
hash_len: 0, // invalid but clamped
chunk_size: 8,
};
let rendered = params.render_description("{example_anchor}");
assert_eq!(rendered, "22:a:r"); // clamped to 1
}
#[test]
fn build_tool_definition_renders_examples() {
use crate::types::template_renderer::TemplateRenderer;
let params = HashlineSchemeParams::default();
let renderer = TemplateRenderer::new(
std::collections::HashMap::new(),
std::collections::HashMap::new(),
);
let schema = serde_json::json!({"type": "object"});
let def = params.build_tool_definition(
"Read {example_anchor} and {grep_match}",
"test_tool",
None,
&renderer,
&std::collections::HashMap::new(),
&schema,
);
let desc = def.function.description.as_deref().unwrap();
assert!(desc.contains("22:abc:rst"), "anchor: {desc}");
assert!(desc.contains("2:abc:rst:"), "grep match: {desc}");
assert!(
!desc.contains("{example_anchor}"),
"placeholder not resolved: {desc}"
);
}
#[test]
fn build_tool_definition_invalid_params_no_panic() {
use crate::types::template_renderer::TemplateRenderer;
let params = HashlineSchemeParams {
scheme: "bogus".to_owned(),
hash_len: 999,
chunk_size: 0,
};
let renderer = TemplateRenderer::new(
std::collections::HashMap::new(),
std::collections::HashMap::new(),
);
let schema = serde_json::json!({"type": "object"});
let def = params.build_tool_definition(
"{example_anchor}",
"test",
None,
&renderer,
&std::collections::HashMap::new(),
&schema,
);
assert!(def.function.description.is_some());
}
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,88 @@
//! Stateless range-size policy for hashline edit safety.
//!
//! Classifies edit ranges by size and produces tiered warnings:
//! - Small (≤5 lines): no warning
//! - Medium (6–20 lines): caution
//! - Large (>20 lines): stronger caution
//!
//! No session state — purely a function of the requested range.
const SMALL_MAX: usize = 5;
const MEDIUM_MAX: usize = 20;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum RangeSize {
Small,
Medium,
Large,
}
impl RangeSize {
pub fn classify(line_count: usize) -> Self {
if line_count <= SMALL_MAX {
Self::Small
} else if line_count <= MEDIUM_MAX {
Self::Medium
} else {
Self::Large
}
}
}
/// Evaluate a range edit and return a warning for medium or large ranges.
pub fn range_warning(start: usize, end: usize) -> Option<String> {
let count = end.saturating_sub(start);
match RangeSize::classify(count) {
RangeSize::Large => Some(format!(
"Caution: large range edit ({count} lines, lines {}-{}). \
Verify the target range is correct.",
start + 1,
end,
)),
RangeSize::Medium => Some(format!(
"Note: medium range edit ({count} lines, lines {}-{}).",
start + 1,
end,
)),
RangeSize::Small => None,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn classification() {
assert_eq!(RangeSize::classify(1), RangeSize::Small);
assert_eq!(RangeSize::classify(5), RangeSize::Small);
assert_eq!(RangeSize::classify(6), RangeSize::Medium);
assert_eq!(RangeSize::classify(20), RangeSize::Medium);
assert_eq!(RangeSize::classify(21), RangeSize::Large);
assert_eq!(RangeSize::classify(100), RangeSize::Large);
}
#[test]
fn small_range_no_warning() {
assert!(range_warning(0, 3).is_none());
}
#[test]
fn medium_range_warns() {
let w = range_warning(0, 10).unwrap();
assert!(w.contains("10 lines"));
assert!(w.contains("medium range"));
}
#[test]
fn large_range_warns() {
let w = range_warning(0, 30).unwrap();
assert!(w.contains("30 lines"));
assert!(w.contains("large range"));
}
#[test]
fn empty_range_no_warning() {
assert!(range_warning(5, 5).is_none());
}
}
Loaded 100 of 2612 files, more files were not shown because too many files have changed in this diff. Show more