docs(comments): rewrite comments across all crates to the guidelines
Sweep every first-party crate source (1956 .rs files) to the project comment guidelines: delete redundant restatements, decorative banners, change narration, and end-of-line comments; keep and tighten the crucial ones (invariants, bug rationale, SAFETY blocks, ported-source attribution). No functional code changed. Every edit is proven comment-only against the prior tree by a comment-stripping lexer (string/char/raw-string aware) plus a separate doctest-fence check. Where removing a comment made rustfmt or clippy want to re-lay-out adjacent code, the minimal triggering comment is restored so code tokens stay byte-identical. Gates green: cargo fmt --all --check (0 diffs), cargo check and cargo clippy --workspace --all-targets (0 warnings). Adds scripts/check_codegen_comment_guidelines.py — the enforcement gate for these guidelines (flags banners, end-of-line comments, change narration, and commented-out code).
This commit is contained in:
@@ -1,7 +1,6 @@
|
||||
//! Compacted-history assembly (kigi's rebuild structure, generic).
|
||||
//!
|
||||
//! Moved from `kigi-chat-state::compaction_utils::build_compacted_history` and
|
||||
//! made generic over a write-side item factory so any harness can assemble
|
||||
//! Generic over a write-side item factory so any harness can assemble
|
||||
//! the canonical post-compaction history:
|
||||
//!
|
||||
//! ```text
|
||||
@@ -25,7 +24,6 @@ use super::summary::{format_compact_summary_content, wrap_user_query};
|
||||
/// - Providing the `user_message_prefix` (e.g. `<user_info>` block).
|
||||
/// - Extracting `last_user_query` / `recent_messages` from its own state.
|
||||
pub struct CompactedHistoryParts<T> {
|
||||
/// The original system message from the conversation.
|
||||
pub system_message: T,
|
||||
/// The user-info / project-layout prefix (not wrapped in `<user_query>`).
|
||||
pub user_message_prefix: String,
|
||||
@@ -36,7 +34,6 @@ pub struct CompactedHistoryParts<T> {
|
||||
pub last_user_query: Option<String>,
|
||||
/// Messages retained verbatim from after the last real user turn.
|
||||
pub recent_messages: Vec<T>,
|
||||
/// The LLM-generated compaction summary text.
|
||||
pub compaction_summary: String,
|
||||
/// An optional pre-rendered `<system-reminder>` block to append after the
|
||||
/// summary. `None` means no state reminder is appended.
|
||||
@@ -75,7 +72,6 @@ pub fn assemble_compacted_history<T: CompactionItemFactory>(
|
||||
compacted.push(T::new_project_instructions(reminder.clone()));
|
||||
}
|
||||
|
||||
// Last user query wrapped in <user_query> tags for consistency.
|
||||
if let Some(ref last_query) = parts.last_user_query {
|
||||
compacted.push(T::new_user(wrap_user_query(last_query.as_str())));
|
||||
}
|
||||
@@ -88,7 +84,6 @@ pub fn assemble_compacted_history<T: CompactionItemFactory>(
|
||||
}
|
||||
let summary_item = T::new_user_meta(formatted_summary);
|
||||
|
||||
// Recent messages come first, then the summary.
|
||||
for msg in parts.recent_messages {
|
||||
compacted.push(msg);
|
||||
}
|
||||
|
||||
@@ -8,12 +8,12 @@
|
||||
//! build prompt → sample (retry + classify) → clean → assemble
|
||||
//! ```
|
||||
//!
|
||||
//! Per-harness concerns stay in the product host (for example `kigi-shell`): the triggers, the
|
||||
//! conversation *gathering / sanitization* that produces `llm_turns`, the
|
||||
//! verbatim→fitted→lossy input ladder, the live LLM transport (the
|
||||
//! [`CompactionSampler`] impl), persistence/replay, and the rendering of
|
||||
//! `system_reminder`. This function takes those as inputs and returns the
|
||||
//! rebuilt history; it never commits or persists.
|
||||
//! Per-harness concerns stay in the product host (for example `kigi-shell`):
|
||||
//! the triggers, the conversation gathering/sanitization that produces
|
||||
//! `llm_turns`, the verbatim→fitted→lossy input ladder, the live LLM transport
|
||||
//! (the [`CompactionSampler`] impl), persistence/replay, and the rendering of
|
||||
//! `system_reminder`. This module takes those as inputs and returns the rebuilt
|
||||
//! history; it never commits or persists.
|
||||
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
@@ -30,37 +30,33 @@ use super::prompt::build_summary_prompt;
|
||||
use super::sample::{SampleRetryError, SampledSummary, sample_summary_with_retries};
|
||||
|
||||
/// Everything the assembler needs that the harness extracts from its own
|
||||
/// state (separate from the conversation that gets summarized).
|
||||
/// state (separate from the conversation that gets summarized). Every field is
|
||||
/// carried through verbatim; nothing here is re-derived.
|
||||
pub struct FullReplaceContext<T> {
|
||||
/// The original system message, carried over verbatim.
|
||||
pub system_message: T,
|
||||
/// The user-info / project-layout prefix (no `<user_query>` tags).
|
||||
pub user_message_prefix: String,
|
||||
/// Pre-rendered AGENTS.md block to re-inject, if any.
|
||||
pub agents_md_reminder: Option<String>,
|
||||
/// The last real user query (raw), kept verbatim post-compaction.
|
||||
/// Raw, not wrapped in `<user_query>` tags yet.
|
||||
pub last_user_query: Option<String>,
|
||||
/// Working tail retained verbatim (tool/subagent results from the current
|
||||
/// turn). kigi keeps this; pass empty to drop it.
|
||||
/// Working tail (tool/subagent results from the current turn). Pass empty
|
||||
/// to drop it.
|
||||
pub recent_messages: Vec<T>,
|
||||
/// Pre-rendered `<system-reminder>` (edited files, running tasks,
|
||||
/// subagents, MCP, …). The harness builds this; we only carry it.
|
||||
/// subagents, MCP, …).
|
||||
pub system_reminder: Option<String>,
|
||||
/// Optional transcript-pointer block appended to the summary.
|
||||
/// Transcript-pointer block appended to the summary.
|
||||
pub transcript_hint: Option<String>,
|
||||
}
|
||||
|
||||
/// Outcome of a failed full-replace pass.
|
||||
#[derive(Debug)]
|
||||
pub enum FullReplaceError {
|
||||
/// No turns were supplied to summarize.
|
||||
NothingToCompact,
|
||||
/// The model returned no usable summary text after all attempts.
|
||||
EmptyResponse,
|
||||
/// The sampler failed deterministically (re-sending can't help), or all
|
||||
/// transient retries were exhausted.
|
||||
Sampler {
|
||||
/// The rendered upstream error.
|
||||
message: String,
|
||||
/// Whether re-sending the *same* input cannot help. The product host
|
||||
/// uses this to decide whether to suppress auto-compaction.
|
||||
@@ -84,46 +80,40 @@ impl std::fmt::Display for FullReplaceError {
|
||||
|
||||
impl std::error::Error for FullReplaceError {}
|
||||
|
||||
/// A successful full-replace pass.
|
||||
pub struct FullReplaceOutput<T> {
|
||||
/// The rebuilt, compacted history (`[SP, UP', AGENTS_MD?, UQ_last?,
|
||||
/// recent…, summary, reminder?]`).
|
||||
/// Shape: `[SP, UP', AGENTS_MD?, UQ_last?, recent…, summary, reminder?]`.
|
||||
pub history: Vec<T>,
|
||||
/// The **raw** model summary (pre-clean), so the product host can persist
|
||||
/// it (request artifact, compaction segment) exactly as the model emitted
|
||||
/// it. The cleaned form is already embedded in `history` by the assembler.
|
||||
pub summary: String,
|
||||
/// Total sample attempts made (first try + retries).
|
||||
/// First try + retries.
|
||||
pub attempts: u32,
|
||||
}
|
||||
|
||||
/// A successful full-replace **sampling** pass (summary only, no assembly).
|
||||
///
|
||||
/// Returned by [`sample_full_replace_summary`] for harnesses (kigi's
|
||||
/// shell) that drive the input ladder and assemble the history themselves —
|
||||
/// they build the assembly inputs (state-context system-reminder, AGENTS.md,
|
||||
/// plan-mode) *after* the LLM call, so they cannot use the bundled
|
||||
/// Summary-only result of [`sample_full_replace_summary`], for harnesses
|
||||
/// (kigi's shell) that drive the input ladder and assemble the history
|
||||
/// themselves — they build the assembly inputs (state-context system-reminder,
|
||||
/// AGENTS.md, plan-mode) *after* the LLM call, so they cannot use the bundled
|
||||
/// [`apply_full_replace_compaction`].
|
||||
pub struct FullReplaceSummary {
|
||||
/// The **raw** model summary (pre-clean).
|
||||
pub summary: String,
|
||||
/// Total sample attempts made (first try + retries).
|
||||
/// First try + retries.
|
||||
pub attempts: u32,
|
||||
}
|
||||
|
||||
/// Run kigi's full-replace compaction pass and return the rebuilt
|
||||
/// history. Pure orchestration: no triggers, no persistence, no commit.
|
||||
///
|
||||
/// - `llm_turns` — the (harness-prepared/sanitized) conversation the model
|
||||
/// summarizes. Empty ⇒ [`FullReplaceError::NothingToCompact`].
|
||||
/// - `user_context` — optional `/compact <text>` context spliced into the prompt.
|
||||
/// - `ctx` — the assembly inputs the harness extracted from its state.
|
||||
/// - `observer` — per-attempt + terminal telemetry seam (pass `&()` for none).
|
||||
/// Empty `llm_turns` ⇒ [`FullReplaceError::NothingToCompact`]. `user_context`
|
||||
/// is the optional `/compact <text>` context spliced into the prompt; pass
|
||||
/// `&()` as `observer` for no telemetry.
|
||||
///
|
||||
/// The **input ladder** (verbatim → fitted → lossy) stays in the product host: on a
|
||||
/// context-length overflow this returns
|
||||
/// [`FullReplaceError::Sampler`] with `context_overflow = true`, and the
|
||||
/// harness rebuilds a smaller input and calls this pass again.
|
||||
/// The **input ladder** (verbatim → fitted → lossy) stays in the product host:
|
||||
/// on a context-length overflow this returns [`FullReplaceError::Sampler`] with
|
||||
/// `context_overflow = true`, and the harness rebuilds a smaller input and
|
||||
/// calls this pass again.
|
||||
pub async fn apply_full_replace_compaction<T, S, O>(
|
||||
sampler: &S,
|
||||
llm_turns: &[T],
|
||||
@@ -147,9 +137,8 @@ where
|
||||
"[FullReplaceCompaction] sampled summary; assembling history"
|
||||
);
|
||||
|
||||
// Clean (inside the assembler via `format_compact_summary_content`) and
|
||||
// rebuild the compacted history. `compaction_summary` is the raw model
|
||||
// output; the assembler strips scratchpad / control tokens.
|
||||
// `compaction_summary` goes in raw; the assembler strips scratchpad /
|
||||
// control tokens via `format_compact_summary_content`.
|
||||
let parts = CompactedHistoryParts {
|
||||
system_message: ctx.system_message,
|
||||
user_message_prefix: ctx.user_message_prefix,
|
||||
@@ -170,12 +159,8 @@ where
|
||||
/// Run only the **sampling** half of the full-replace pass: build the prompt,
|
||||
/// sample with bounded retries (transient + degenerate), classify failures,
|
||||
/// and report every attempt through `observer`. Returns the raw summary; the
|
||||
/// caller assembles the history (and owns the input ladder).
|
||||
///
|
||||
/// This is the seam kigi's shell uses: it drives the verbatim → fitted →
|
||||
/// lossy input ladder around this call (stepping on a
|
||||
/// [`FullReplaceError::Sampler`] with `context_overflow = true`) and assembles
|
||||
/// the compacted history afterward from inputs it gathers post-sampling.
|
||||
/// caller assembles the history and owns the input ladder, stepping it on a
|
||||
/// [`FullReplaceError::Sampler`] with `context_overflow = true`.
|
||||
pub async fn sample_full_replace_summary<T, S, O>(
|
||||
sampler: &S,
|
||||
llm_turns: &[T],
|
||||
@@ -365,10 +350,7 @@ mod tests {
|
||||
)
|
||||
}
|
||||
|
||||
/// Golden end-to-end test: a realistic conversation + a mock sampler that
|
||||
/// returns a structured summary must produce kigi's exact compacted
|
||||
/// history shape, with the LLM output cleaned and the agent-state reminder
|
||||
/// carried through as the final item.
|
||||
/// Golden test pinning kigi's exact compacted history shape end to end.
|
||||
#[tokio::test]
|
||||
async fn full_replace_produces_kigi_history_shape() {
|
||||
let llm_turns = vec![
|
||||
@@ -408,7 +390,7 @@ mod tests {
|
||||
MockItem::Tail("tool: read_file(auth.rs) -> ...".into())
|
||||
);
|
||||
|
||||
// Summary carrier: cleaned (no <analysis>/<summary> tags), with preamble.
|
||||
// Summary carrier: cleaned, with preamble.
|
||||
let MockItem::UserMeta(summary) = &out[5] else {
|
||||
panic!("expected UserMeta summary at [5], got {:?}", out[5]);
|
||||
};
|
||||
@@ -421,7 +403,6 @@ mod tests {
|
||||
assert!(!summary.contains("<summary>"), "live tag leaked: {summary}");
|
||||
assert!(!summary.contains("thinking about it"));
|
||||
|
||||
// Agent-state reminder carried through verbatim as the final item.
|
||||
assert_eq!(
|
||||
out[6],
|
||||
MockItem::SystemReminder(
|
||||
@@ -553,8 +534,6 @@ mod tests {
|
||||
assert_eq!(sampler.call_count(), 1, "overflow must not retry");
|
||||
}
|
||||
|
||||
/// The observer sees one terminal `on_success` and the right per-attempt
|
||||
/// outcomes (a degenerate retry then a success).
|
||||
#[tokio::test]
|
||||
async fn observer_receives_attempt_and_success_callbacks() {
|
||||
use std::sync::Mutex;
|
||||
|
||||
@@ -1,32 +1,26 @@
|
||||
//! kigi compaction configuration.
|
||||
//!
|
||||
//! Holds the [`FullReplaceConfig`] tunables struct (mirroring
|
||||
//! Tunables and defaults only; trigger *wiring* (pre-sampling checks,
|
||||
//! preflight overflow, model-switch, suppression) stays per-host. Mirrors
|
||||
//! [`IntraCompactionConfig`](crate::intra_compaction::IntraCompactionConfig) /
|
||||
//! [`InterCompactionConfig`](crate::inter_compaction::InterCompactionConfig),
|
||||
//! which also live in their module's `config.rs`) plus the shared default
|
||||
//! values. Trigger *wiring* (pre-sampling checks, preflight overflow,
|
||||
//! model-switch, suppression) stays per-host.
|
||||
//! which also live in their module's `config.rs`.
|
||||
|
||||
/// Default auto-compact threshold (% of context window) when no other source
|
||||
/// (env var, user config, remote per-model/global flags) sets it. Shared by
|
||||
/// kigi and Kigi chat (~85% trigger on both sides).
|
||||
/// Applies only when no other source (env var, user config, remote
|
||||
/// per-model/global flags) sets it. Shared by kigi and Kigi chat.
|
||||
pub const DEFAULT_AUTO_COMPACT_THRESHOLD_PERCENT: u8 = 85;
|
||||
|
||||
/// Minimum character count for a cleaned summary seed.
|
||||
///
|
||||
/// kigi retries when the cleaned summary is shorter than this — the
|
||||
/// smallest healthy prod summary observed was ~3,242 chars; anything under
|
||||
/// 500 is treated as degenerate and retried like a transient failure.
|
||||
/// A cleaned summary shorter than this is treated as degenerate and retried
|
||||
/// like a transient failure. The smallest healthy prod summary observed was
|
||||
/// ~3,242 chars, so 500 is a wide margin below anything legitimate.
|
||||
pub const MIN_SUMMARY_SEED_CHARS: usize = 500;
|
||||
|
||||
/// Tunables for the full-replace pass.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct FullReplaceConfig {
|
||||
/// Total LLM attempts (first try + retries) on transient failures.
|
||||
/// First try + retries, counted together.
|
||||
pub max_attempts: u32,
|
||||
/// Delay between transient retries.
|
||||
pub retry_delay_secs: u64,
|
||||
/// End-to-end timeout for each compaction LLM call.
|
||||
/// Applies per attempt, not to the whole retry loop.
|
||||
pub sampling_timeout_secs: u64,
|
||||
}
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
//! Deterministic-vs-transient failure classification for compaction
|
||||
//! LLM calls.
|
||||
//!
|
||||
//! The *policy* lives here (shared across harnesses); the per-harness error
|
||||
//! types and their wrapping (e.g. kigi's `SamplingError` →
|
||||
//! The *policy* lives here, shared across harnesses; per-harness error types
|
||||
//! and their wrapping (e.g. kigi's `SamplingError` →
|
||||
//! `CompactFailure(acp::Error)`) stay in thin host wrappers that delegate the
|
||||
//! status/message decisions to these functions.
|
||||
|
||||
@@ -17,15 +17,16 @@ pub enum FailureKind {
|
||||
}
|
||||
|
||||
impl FailureKind {
|
||||
/// `true` for [`FailureKind::Deterministic`].
|
||||
pub fn is_deterministic(self) -> bool {
|
||||
matches!(self, Self::Deterministic)
|
||||
}
|
||||
}
|
||||
|
||||
/// True when an error message indicates a context-window overflow. Backends report
|
||||
/// this inconsistently with no stable error code, so we match the message text; it's
|
||||
/// deterministic (re-sending the same payload always fails), so callers must not retry.
|
||||
/// True when an error message indicates a context-window overflow.
|
||||
///
|
||||
/// Backends report this inconsistently with no stable error code, so the match
|
||||
/// is on message text. Re-sending the same payload always fails, so callers
|
||||
/// must not retry.
|
||||
pub fn is_context_length_error(message: &str) -> bool {
|
||||
let m = message.to_ascii_lowercase();
|
||||
m.contains("too long for this model")
|
||||
@@ -38,10 +39,8 @@ pub fn is_context_length_error(message: &str) -> bool {
|
||||
/// Classify an HTTP API failure (status + message) for the compaction retry
|
||||
/// loop.
|
||||
///
|
||||
/// 4xx responses other than 408 (timeout) and 429 (rate limit) are
|
||||
/// deterministic; a context-length overflow message is deterministic
|
||||
/// regardless of status (backends sometimes dress it as a synthesized 500).
|
||||
/// Everything else (5xx, 408, 429) is transient.
|
||||
/// A context-length overflow message is deterministic regardless of status,
|
||||
/// because backends sometimes dress it as a synthesized 500.
|
||||
pub fn classify_http_status(status: u16, message: &str) -> FailureKind {
|
||||
if is_context_length_error(message)
|
||||
|| ((400..500).contains(&status) && status != 408 && status != 429)
|
||||
@@ -55,15 +54,11 @@ pub fn classify_http_status(status: u16, message: &str) -> FailureKind {
|
||||
/// Classify a provider-style stream error event (`ResponseError` /
|
||||
/// `ResponseFailed.error`) for the compaction retry loop.
|
||||
///
|
||||
/// `code` is the structured `code` field on the event (typically a numeric
|
||||
/// HTTP status as a string, but some providers also use error-type strings like
|
||||
/// `"invalid_request_error"`). `message` is the human-readable detail.
|
||||
///
|
||||
/// Numeric codes are classified by HTTP-status range. The
|
||||
/// `invalid_request_error` marker, which can appear in either field, always
|
||||
/// maps to `Deterministic` (schema violations cannot be fixed by re-sending
|
||||
/// the same payload). The check order is semantic — marker, then numeric
|
||||
/// code, then context-length message, then default-to-transient.
|
||||
/// `code` is the structured `code` field on the event: typically a numeric HTTP
|
||||
/// status as a string, but some providers send error-type strings like
|
||||
/// `"invalid_request_error"` instead, and that marker can also arrive buried in
|
||||
/// `message` alone. It always means a schema violation, which re-sending the
|
||||
/// same payload cannot fix.
|
||||
pub fn classify_stream_event_error(code: Option<&str>, message: &str) -> FailureKind {
|
||||
if matches!(code, Some("invalid_request_error")) || message.contains("invalid_request_error") {
|
||||
return FailureKind::Deterministic;
|
||||
|
||||
@@ -9,17 +9,6 @@
|
||||
//! [`intra_compaction`](crate::intra_compaction) (tail-keep, per-step) and
|
||||
//! [`inter_compaction`](crate::inter_compaction) (chunked, between-turn).
|
||||
//!
|
||||
//! Layout (mirroring
|
||||
//! [`intra_compaction`](crate::intra_compaction) /
|
||||
//! [`inter_compaction`](crate::inter_compaction)):
|
||||
//!
|
||||
//! - **Policy & content**: [`prompt`] (summarization prompt), [`summary`]
|
||||
//! (summary cleaning + carrier), [`failure`] (deterministic-vs-transient
|
||||
//! classification), [`config`] (tunables + trigger/seed defaults).
|
||||
//! - **Algorithm**: [`assemble`] (full-replace history rebuild).
|
||||
//! - **Orchestration**: [`compact`]
|
||||
//! (`build prompt → sample → clean → assemble`).
|
||||
//!
|
||||
//! Host-specific concerns (triggers, transport, persistence/replay, state
|
||||
//! commit, metrics observer) stay in the product host (for example
|
||||
//! `kigi-shell`).
|
||||
|
||||
@@ -9,8 +9,7 @@
|
||||
//! / [`InterCompactionObserver`](crate::inter_compaction::InterCompactionObserver).
|
||||
//!
|
||||
//! Emission points are part of the behavior contract: the kigi observer
|
||||
//! preserves the pre-migration `CompactionAttempt`/`CompactionRetryDegraded`
|
||||
//! semantics byte-for-byte.
|
||||
//! preserves `CompactionAttempt` / `CompactionRetryDegraded` semantics.
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
@@ -34,14 +33,13 @@ pub enum FullReplaceAttemptOutcome<'a> {
|
||||
/// The cleaned summary seed was too short to carry the conversation's task
|
||||
/// state; retried like a transient failure.
|
||||
Degenerate {
|
||||
/// Raw model summary text (still captured for offline inspection).
|
||||
/// Captured for offline inspection even when too short to keep.
|
||||
summary: &'a str,
|
||||
/// Whether the orchestrator will retry after this attempt.
|
||||
will_retry: bool,
|
||||
},
|
||||
/// The sampler returned an error.
|
||||
Failure {
|
||||
/// Rendered error message.
|
||||
message: &'a str,
|
||||
/// Whether re-sending the *same* input cannot help (auth / schema /
|
||||
/// size). Transient failures (timeout / stream blip / 5xx) are `false`.
|
||||
|
||||
@@ -4,8 +4,6 @@
|
||||
//! full-replace pass ([`sample_full_replace_summary`](super::sample_full_replace_summary))
|
||||
//! and Kigi chat's intra `Shared` summarizer
|
||||
//! ([`apply_intra_compaction`](crate::intra_compaction::apply_intra_compaction)).
|
||||
//! Centralising it here removes the two near-identical copies that previously
|
||||
//! lived in `code_compaction::compact` and `intra_compaction::compact`.
|
||||
//!
|
||||
//! Classification is uniform:
|
||||
//! - a usable, non-degenerate response wins immediately;
|
||||
@@ -38,7 +36,7 @@ use super::summary::is_degenerate_summary;
|
||||
/// plus the total number of attempts made (first try + retries).
|
||||
#[derive(Debug)]
|
||||
pub struct SampledSummary {
|
||||
/// Raw model summary text, exactly as emitted. Callers clean it as needed.
|
||||
/// Exactly as emitted; callers clean before use.
|
||||
pub summary: String,
|
||||
/// Total sample attempts made (1-based).
|
||||
pub attempts: u32,
|
||||
@@ -58,14 +56,12 @@ pub enum SampleRetryError {
|
||||
/// input cannot help — auth / schema / context overflow), or transient but
|
||||
/// retries were exhausted.
|
||||
Failure {
|
||||
/// Rendered upstream error message.
|
||||
message: String,
|
||||
/// Whether re-sending the same input cannot help.
|
||||
deterministic: bool,
|
||||
/// Whether the failure was a context-length overflow (a deterministic
|
||||
/// signal the kigi host uses to step down its input size).
|
||||
context_overflow: bool,
|
||||
/// Total attempts made.
|
||||
attempts: u32,
|
||||
},
|
||||
}
|
||||
@@ -344,7 +340,7 @@ mod tests {
|
||||
|
||||
#[tokio::test]
|
||||
async fn empty_then_degenerate_exhausts_to_empty() {
|
||||
let short = "<summary>\n1. Primary Request: q\n</summary>"; // degenerate
|
||||
let short = "<summary>\n1. Primary Request: q\n</summary>";
|
||||
let sampler = MockSampler::scripted(vec![Ok(String::new()), Ok(short.into())]);
|
||||
let err = run(&sampler, 2).await.expect_err("should fail");
|
||||
assert!(matches!(err, SampleRetryError::Empty { attempts: 2 }));
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
//! Summary output cleaning and carrier formatting.
|
||||
//!
|
||||
//! Moved verbatim from `kigi-chat-state`'s `compaction_utils`. Covers:
|
||||
//!
|
||||
//! - cleaning the compaction model's raw output ([`format_compact_summary`]),
|
||||
//! - the kigi continuation carrier ([`format_compact_summary_content`]),
|
||||
//! - the canonical `<user_query>` wrapping ([`wrap_user_query`]).
|
||||
@@ -19,15 +17,15 @@
|
||||
pub fn format_compact_summary(summary: &str) -> String {
|
||||
let mut result = summary.to_string();
|
||||
|
||||
// 1. Remove leading <analysis>…</analysis> drafting block(s). A block is
|
||||
// only stripped when it is a genuinely LEADING scratchpad: top-level
|
||||
// (before any <summary>) or immediately after the <summary> open modulo
|
||||
// whitespace (nested). An <analysis> quoted mid-body — after real
|
||||
// sections, e.g. a section-6 instruction echo — is NOT leading and is
|
||||
// left for step 3 to neutralize, so neither a balanced body quote
|
||||
// spanning sections nor an unclosed one ever deletes real content. The
|
||||
// loop peels successive leading blocks should the model emit more than
|
||||
// one.
|
||||
// Strip leading <analysis>…</analysis> drafting blocks only when they are
|
||||
// genuinely LEADING scratchpad: top-level (before any <summary>) or
|
||||
// immediately after the <summary> open modulo whitespace (nested). An
|
||||
// <analysis> quoted mid-body — after real sections, e.g. a section-6
|
||||
// instruction echo — is NOT leading and is left for
|
||||
// `neutralize_compaction_control_tokens` to defuse, so neither a balanced
|
||||
// body quote spanning sections nor an unclosed one ever deletes real
|
||||
// content. The loop peels successive leading blocks if the model emits more
|
||||
// than one.
|
||||
while let Some(start) = result.find("<analysis>") {
|
||||
let is_leading = match result.find("<summary>") {
|
||||
Some(sp) => start < sp || result[sp + "<summary>".len()..start].trim().is_empty(),
|
||||
@@ -53,12 +51,13 @@ pub fn format_compact_summary(summary: &str) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Convert the outer <summary>…</summary> to "Summary:\n{inner}", keeping
|
||||
// any text outside the wrapper. `rfind` matches the outer close, so a
|
||||
// literal "</summary>" echoed in the body does not truncate the summary;
|
||||
// `end > start` guards a malformed "</summary> … <summary>" order. Leading
|
||||
// scratchpad inside the block is peeled (see `strip_leading_scratchpad`);
|
||||
// a body echo that quotes the instruction is left for step 3 to defuse.
|
||||
// Convert outer <summary>…</summary> to "Summary:\n{inner}", keeping text
|
||||
// outside the wrapper. `rfind` matches the outer close so a literal
|
||||
// "</summary>" echoed in the body does not truncate; `end > start` guards a
|
||||
// malformed "</summary> … <summary>" order. Leading scratchpad inside the
|
||||
// block is peeled (see `strip_leading_scratchpad`); a body echo that quotes
|
||||
// the instruction is left for `neutralize_compaction_control_tokens` to
|
||||
// defuse.
|
||||
if let Some(start) = result.find("<summary>")
|
||||
&& let Some(end) = result.rfind("</summary>")
|
||||
&& end > start
|
||||
@@ -69,8 +68,8 @@ pub fn format_compact_summary(summary: &str) -> String {
|
||||
result = format!("{before}Summary:\n{inner}{after}");
|
||||
}
|
||||
|
||||
// 3. Defuse any compaction-control tokens still echoed inside the body so the
|
||||
// seed can't prime the next turn to re-emit a <summary> block.
|
||||
// Defuse any compaction-control tokens still echoed inside the body so the
|
||||
// seed can't prime the next turn to re-emit a <summary> block.
|
||||
result = neutralize_compaction_control_tokens(&result);
|
||||
|
||||
// Collapse excessive blank lines (3+ newlines → 2)
|
||||
@@ -84,13 +83,14 @@ pub fn format_compact_summary(summary: &str) -> String {
|
||||
/// Peel leading drafting scratchpad off an extracted `<summary>` block.
|
||||
///
|
||||
/// A markdown "**Analysis**"-style header has no opening `<analysis>` tag for
|
||||
/// step 1 to catch; it ends at an orphan `</analysis>`. Everything up to and
|
||||
/// including the *last* `</analysis>` is dropped, so a scratchpad that itself
|
||||
/// quotes `</analysis>` mid-reasoning is still removed whole. The peel is
|
||||
/// skipped when the block already starts with a numbered section — including a
|
||||
/// markdown-decorated one like `## 1.` or `**1.**` — so a `</analysis>` merely
|
||||
/// echoed inside a real section never truncates the summary. Any leftover
|
||||
/// leading `<summary>` wrapper is then unwrapped.
|
||||
/// the leading-block peel in [`format_compact_summary`] to catch; it ends at an
|
||||
/// orphan `</analysis>`. Everything up to and including the *last*
|
||||
/// `</analysis>` is dropped, so a scratchpad that itself quotes `</analysis>`
|
||||
/// mid-reasoning is still removed whole. The peel is skipped when the block
|
||||
/// already starts with a numbered section — including a markdown-decorated one
|
||||
/// like `## 1.` or `**1.**` — so a `</analysis>` merely echoed inside a real
|
||||
/// section never truncates the summary. Any leftover leading `<summary>`
|
||||
/// wrapper is then unwrapped.
|
||||
fn strip_leading_scratchpad(inner: &str) -> String {
|
||||
let mut s = inner.trim();
|
||||
let lead = s.trim_start_matches(['#', '*', '-', '>', ' ', '\t']);
|
||||
|
||||
Reference in New Issue
Block a user