M0: compilable skeleton — Kigi 0.1.0 fork surgery
Hard fork of xai-org/grok-build (Apache-2.0) re-targeted as Kigi, an
unofficial Kimi Code CLI community build.
Rename & identity
- 72 xai-*/xai-grok-* crates -> kigi-* (explicit: xai-grok-pager-bin ->
kigi-bin [binary `kigi`], xai-grok-pager -> kigi-tui; rest mechanical);
ptyctl, ptyctl-cli, third_party/ unchanged; proto package
xai.grok.tools.v1 -> kigi.tools.v1
- Config home ~/.kigi (KIGI_SHARE_DIR override), env prefix GROK_* ->
KIGI_*, `kigi --version` carries the unofficial-community-build notice
- clap identity, help text, startup banner, prompt templates rebranded
(templates re-encrypted)
Deletions (PRD removal list #5/#6/#7/#9/#10)
- voice input (xai-grok-voice) and all TUI wiring
- telemetry: Mixpanel client, external OTel stream, Sentry, OTLP layers,
trace/GCS/S3 upload queues (kigi-file-utils halved), workspace upload
module & dc_log, heap-profile uploader, auth-diagnostics uploader,
session-analytics halves of feedback; local zero-egress observability
preserved in new kigi-log crate (unified log, --debug firehose,
subsystem file logs, opt-in instrumentation)
- announcements (crate, remote-settings fields, TUI surfaces)
- plugin marketplace (crate, sources/browse/CTA/extensions-modal tab);
direct plugin install/uninstall/update via kigi-agent git_install kept
- relay/gateway/assets endpoints and features (agent relay, headless
relay transport, gateway bridge, LeaderEnvUrls); leader IPC socket now
~/.kigi/leader.sock + KIGI_LEADER_SOCKET, no ws-url derivation
- functional types rehomed instead of deleted: PermissionMode ->
kigi-config-types, McpInitStrategy -> kigi-mcp, PrCreationSource ->
session signals, TerminalDiagnostics -> kigi-pager-render, agent_id ->
shell util
Endpoints
- kigi-env rewritten: single production KigiEndpoints {coding_api_base_url
https://api.kimi.com/coding/v1 (KIGI_CODE_BASE_URL), oauth_host
https://auth.kimi.com (KIGI_OAUTH_HOST), update_base_url (GitHub
Releases API), upgrade_page_url}; GrokBuildEnvironment enum deleted
Toolchain & workspace hygiene
- Rust 1.97.0 pinned; edition 2024; full cargo update; git2 hoisted to
workspace at 0.21 (Option->Result API migration), quick-xml 0.41
- Root Cargo.toml hand-maintained (PRD §8.1): version 0.1.0 inherited by
all members, members sorted, unused deps pruned
- cargo-deny advisories gate (deny.toml with documented transitive
exceptions); CI workflow (check/clippy/fmt/deny/test, macOS+Linux)
- cross-crate test seams re-gated behind `test-support` cargo feature;
insta snapshot baselines renamed to the kigi_tui prefix
- clippy --workspace --all-targets: zero warnings; fmt clean
Fixes surfaced by the port
- updater probe/installer divergence (bin/kigi vs bin/grok symlink set)
- idle model-metadata refresh dead under KIGI_CODE_BASE_URL override
(new is_effective_coding_endpoint_url, loopback+override aware)
- macOS symlinked-TMPDIR fixture canonicalization (foreign_sessions,
fast-worktree); RSS measurement tests serialized via serial_test
Docs & legal (Apache §4)
- NOTICE added (upstream attribution + change statement); THIRD-PARTY
notices sustained; kigi-tools ported-code notices extended; README,
CONTRIBUTING, SECURITY, AGENTS.md rewritten
Out of scope for M0 (tracked): Kimi auth/inference (M1), search/fetch,
command parity, config import (M2), Computer Hub excision & final
brand-token sweep (M2), distribution & self-update rewrite (M3).
This commit is contained in:
@@ -0,0 +1,23 @@
|
||||
[package]
|
||||
license = "Apache-2.0"
|
||||
name = "kigi-sampling-types"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
description = "Pure data types for the xAI sampling / chat-completion API layer"
|
||||
|
||||
[dependencies]
|
||||
async-openai = { workspace = true }
|
||||
indexmap = { workspace = true, features = ["serde"] }
|
||||
reqwest = { workspace = true }
|
||||
serde = { workspace = true, features = ["derive"] }
|
||||
serde_json = { workspace = true, features = ["preserve_order"] }
|
||||
thiserror = { workspace = true }
|
||||
tracing = { workspace = true }
|
||||
kigi-compaction = { path = "../../common/kigi-compaction" }
|
||||
kigi-tools = { path = "../kigi-tools" }
|
||||
|
||||
[dev-dependencies]
|
||||
assert_matches = { workspace = true }
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,527 @@
|
||||
//! Server-side doom-loop check: wire contract types and tolerant parsers.
|
||||
//!
|
||||
//! When the client opts in via the `x-grok-doom-loop-check` request header,
|
||||
//! the inference API reports detected generation loops on streaming
|
||||
//! `/v1/responses` requests in two places:
|
||||
//!
|
||||
//! * a non-standard mid-stream SSE event (`response.doom_loop_check`)
|
||||
//! emitted as new triggers appear, carrying the **cumulative** trigger set:
|
||||
//! `{"type": "response.doom_loop_check", "doom_loop_check": {"triggers": ["…"]}}`
|
||||
//! * a `doom_loop_check: {"triggers": ["…"]}` field on the terminal response
|
||||
//! object (`response.completed` / `response.incomplete`).
|
||||
//!
|
||||
//! Triggers are opaque labels with the grammar
|
||||
//! `tail_repetition:{threshold}@{channel}` or `low_logprob@{channel}`.
|
||||
//! Presence is itself the detection signal; the set is non-empty when present.
|
||||
//!
|
||||
//! This module is the single home for that wire shape: if the server contract
|
||||
//! changes, only this file (and its tests) should need to change. Everything
|
||||
//! here is best-effort by design — malformed payloads yield `Unknown` kinds or
|
||||
//! empty trigger sets, never an error, so the feature can never fail a stream.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Request header whose presence enables the server-side check.
|
||||
pub const DOOM_LOOP_CHECK_HEADER: &str = "x-grok-doom-loop-check";
|
||||
|
||||
/// `type` of the non-standard mid-stream SSE event — also its SSE `event:`
|
||||
/// name. async-openai's typed `rs::ResponseStreamEvent` does not know this
|
||||
/// variant, so raw payloads carrying this name or type must be intercepted
|
||||
/// before typed deserialization.
|
||||
pub const DOOM_LOOP_CHECK_EVENT_TYPE: &str = "response.doom_loop_check";
|
||||
|
||||
/// Byte-exact `data:` payload of a check-event frame as emitted by the
|
||||
/// server (verbatim from the server's wire sample; the
|
||||
/// frame's SSE `event:` name is [`DOOM_LOOP_CHECK_EVENT_TYPE`]). Exported as
|
||||
/// a fixture so transport tests pin the real bytes, not a paraphrase.
|
||||
pub const SAMPLE_CHECK_EVENT_DATA: &str = r#"{"sequence_number":4176,"type":"response.doom_loop_check","doom_loop_check":{"triggers":["tail_repetition:4@response"]}}"#;
|
||||
|
||||
/// Companion fixture to [`SAMPLE_CHECK_EVENT_DATA`]: the follow-up frame from
|
||||
/// the same wire sample, carrying the grown **cumulative** trigger set.
|
||||
pub const SAMPLE_CHECK_EVENT_DATA_CUMULATIVE: &str = r#"{"sequence_number":4178,"type":"response.doom_loop_check","doom_loop_check":{"triggers":["tail_repetition:4@response","tail_repetition:2@response"]}}"#;
|
||||
|
||||
/// Resolved runtime tunables for doom-loop recovery.
|
||||
///
|
||||
/// Produced once per session by the shell's config resolver
|
||||
/// (env > config.toml > remote settings > default), which returns `None` when
|
||||
/// the check is disabled — absence IS the off state, so there is no separate
|
||||
/// enabled flag to keep in sync. When present on `SamplerConfig`, the sampler
|
||||
/// both sends the opt-in request header and parses the reported triggers; the
|
||||
/// tunables are consumed by the recovery decision logic.
|
||||
///
|
||||
/// Per-field serde defaults keep configs persisted by older versions
|
||||
/// deserializing when future fields are added.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct DoomLoopRecoveryPolicy {
|
||||
/// Act only on `tail_repetition:{t}@thinking` triggers with `t` at or
|
||||
/// below this value (lower thresholds indicate tighter, more confident
|
||||
/// loops).
|
||||
#[serde(default = "default_max_threshold")]
|
||||
pub max_threshold: u32,
|
||||
/// Resample budget per turn before accepting the response as-is.
|
||||
#[serde(default = "default_max_retries")]
|
||||
pub max_retries: u32,
|
||||
}
|
||||
|
||||
fn default_max_threshold() -> u32 {
|
||||
DoomLoopRecoveryPolicy::DEFAULT_MAX_THRESHOLD
|
||||
}
|
||||
|
||||
fn default_max_retries() -> u32 {
|
||||
DoomLoopRecoveryPolicy::DEFAULT_MAX_RETRIES
|
||||
}
|
||||
|
||||
/// Channel label of the model's thinking stream — the only channel recovery
|
||||
/// acts on (loops in visible output are the user's to judge).
|
||||
pub const THINKING_CHANNEL: &str = "thinking";
|
||||
|
||||
impl DoomLoopRecoveryPolicy {
|
||||
/// Clamp range for `max_threshold`.
|
||||
pub const MAX_THRESHOLD_RANGE: std::ops::RangeInclusive<u32> = 2..=64;
|
||||
/// Clamp range for `max_retries`.
|
||||
pub const MAX_RETRIES_RANGE: std::ops::RangeInclusive<u32> = 0..=5;
|
||||
/// Default `max_threshold` (lowest common threshold across the backtest
|
||||
/// corpus of confirmed loops).
|
||||
pub const DEFAULT_MAX_THRESHOLD: u32 = 8;
|
||||
/// Default `max_retries`.
|
||||
pub const DEFAULT_MAX_RETRIES: u32 = 2;
|
||||
|
||||
/// Clamp a configured `max_threshold` into [`Self::MAX_THRESHOLD_RANGE`].
|
||||
pub fn clamp_max_threshold(value: u32) -> u32 {
|
||||
value.clamp(
|
||||
*Self::MAX_THRESHOLD_RANGE.start(),
|
||||
*Self::MAX_THRESHOLD_RANGE.end(),
|
||||
)
|
||||
}
|
||||
|
||||
/// Clamp a configured `max_retries` into [`Self::MAX_RETRIES_RANGE`].
|
||||
pub fn clamp_max_retries(value: u32) -> u32 {
|
||||
value.clamp(
|
||||
*Self::MAX_RETRIES_RANGE.start(),
|
||||
*Self::MAX_RETRIES_RANGE.end(),
|
||||
)
|
||||
}
|
||||
|
||||
/// A signal this policy treats as a real loop worth acting on: tail
|
||||
/// repetition in the thinking channel, at or below the confidence
|
||||
/// threshold (lower detector thresholds mean tighter repetition).
|
||||
/// Everything else — other channels, `low_logprob`, unknown kinds,
|
||||
/// looser thresholds — is warn-only.
|
||||
pub fn is_confident(&self, signal: &DoomLoopSignal) -> bool {
|
||||
let tight = |t: u32| t <= self.max_threshold;
|
||||
signal.channel == THINKING_CHANNEL
|
||||
&& matches!(signal.kind, DoomLoopSignalKind::TailRepetition(t) if tight(t))
|
||||
}
|
||||
|
||||
/// Raw labels of the confident signals in `signals`; empty when none.
|
||||
pub fn confident_triggers(&self, signals: &[DoomLoopSignal]) -> Vec<String> {
|
||||
signals
|
||||
.iter()
|
||||
.filter(|s| self.is_confident(s))
|
||||
.map(|s| s.raw.clone())
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for DoomLoopRecoveryPolicy {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
max_threshold: Self::DEFAULT_MAX_THRESHOLD,
|
||||
max_retries: Self::DEFAULT_MAX_RETRIES,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Parsed classification of a single trigger label.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum DoomLoopSignalKind {
|
||||
/// `tail_repetition:{threshold}@{channel}` — a repeating tail was found
|
||||
/// at the given detector threshold.
|
||||
TailRepetition(u32),
|
||||
/// `low_logprob@{channel}` — degenerate low-entropy generation.
|
||||
LowLogprob,
|
||||
/// Any label this client version cannot classify; the unparsed kind
|
||||
/// segment is preserved verbatim.
|
||||
Unknown(String),
|
||||
}
|
||||
|
||||
/// One doom-loop trigger reported by the server.
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct DoomLoopSignal {
|
||||
pub kind: DoomLoopSignalKind,
|
||||
/// Channel the loop was detected on (e.g. `thinking`, `response`).
|
||||
/// Empty when the label carries no `@channel` suffix.
|
||||
pub channel: String,
|
||||
/// The verbatim label; the stable identity used for deduplication and
|
||||
/// logging.
|
||||
pub raw: String,
|
||||
}
|
||||
|
||||
impl DoomLoopSignal {
|
||||
/// Parse a trigger label. Never fails: any grammar mismatch yields
|
||||
/// `DoomLoopSignalKind::Unknown` with the raw label preserved.
|
||||
pub fn parse(raw: &str) -> Self {
|
||||
let (head, channel) = match raw.split_once('@') {
|
||||
Some((head, channel)) => (head, channel),
|
||||
None => (raw, ""),
|
||||
};
|
||||
let kind = match head.split_once(':') {
|
||||
Some(("tail_repetition", threshold)) => match threshold.parse::<u32>() {
|
||||
Ok(t) => DoomLoopSignalKind::TailRepetition(t),
|
||||
Err(_) => DoomLoopSignalKind::Unknown(head.to_string()),
|
||||
},
|
||||
None if head == "low_logprob" => DoomLoopSignalKind::LowLogprob,
|
||||
_ => DoomLoopSignalKind::Unknown(head.to_string()),
|
||||
};
|
||||
Self {
|
||||
kind,
|
||||
channel: channel.to_string(),
|
||||
raw: raw.to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// The tightest label among `raws`: the `tail_repetition` trigger with
|
||||
/// the LOWEST threshold (tighter repetition = stronger evidence), falling
|
||||
/// back to the first label when none parse as `tail_repetition`. Raw
|
||||
/// labels only — telemetry-safe.
|
||||
pub fn tightest(raws: impl IntoIterator<Item = impl AsRef<str>>) -> Option<String> {
|
||||
let mut first: Option<String> = None;
|
||||
let mut best: Option<(u32, String)> = None;
|
||||
for raw in raws {
|
||||
let raw = raw.as_ref();
|
||||
if first.is_none() {
|
||||
first = Some(raw.to_string());
|
||||
}
|
||||
if let DoomLoopSignalKind::TailRepetition(t) = Self::parse(raw).kind
|
||||
&& best.as_ref().is_none_or(|(bt, _)| t < *bt)
|
||||
{
|
||||
best = Some((t, raw.to_string()));
|
||||
}
|
||||
}
|
||||
best.map(|(_, raw)| raw).or(first)
|
||||
}
|
||||
}
|
||||
|
||||
/// Result of peeking a raw SSE `data:` payload for doom-loop content.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum DoomLoopPeek {
|
||||
/// The payload is the non-standard `response.doom_loop_check` event.
|
||||
/// The caller must swallow it (never forward to the typed event parser);
|
||||
/// the vec is empty when the payload is malformed.
|
||||
CheckEvent(Vec<DoomLoopSignal>),
|
||||
/// The payload is an ordinary event whose `response` object carries a
|
||||
/// `doom_loop_check` field (the terminal belt-and-braces copy). Forward
|
||||
/// the event as usual after recording the signals.
|
||||
ResponseField(Vec<DoomLoopSignal>),
|
||||
/// Nothing doom-loop related; forward untouched.
|
||||
None,
|
||||
}
|
||||
|
||||
/// Tolerantly peek a raw SSE `data:` JSON payload for doom-loop content.
|
||||
///
|
||||
/// Cheap for the common case: payloads that don't mention `doom_loop_check`
|
||||
/// return [`DoomLoopPeek::None`] without a JSON parse. Anything malformed
|
||||
/// (non-JSON, wrong types, missing keys) degrades to `None` or an empty
|
||||
/// trigger vec — never an error.
|
||||
pub fn peek_doom_loop(data: &str) -> DoomLoopPeek {
|
||||
if !data.contains("doom_loop_check") {
|
||||
return DoomLoopPeek::None;
|
||||
}
|
||||
let Ok(value) = serde_json::from_str::<serde_json::Value>(data) else {
|
||||
return DoomLoopPeek::None;
|
||||
};
|
||||
if value.get("type").and_then(|t| t.as_str()) == Some(DOOM_LOOP_CHECK_EVENT_TYPE) {
|
||||
let triggers = value.pointer("/doom_loop_check/triggers");
|
||||
return DoomLoopPeek::CheckEvent(parse_triggers(triggers));
|
||||
}
|
||||
match value.pointer("/response/doom_loop_check/triggers") {
|
||||
Some(triggers) => DoomLoopPeek::ResponseField(parse_triggers(Some(triggers))),
|
||||
None => DoomLoopPeek::None,
|
||||
}
|
||||
}
|
||||
|
||||
/// True when an SSE frame IS the doom-loop check event — by its SSE `event:`
|
||||
/// name, or (for servers that omit the name) by a tolerant peek of the
|
||||
/// payload's `"type"` tag, gated on a cheap substring precheck so normal
|
||||
/// traffic never pays a JSON parse. The type confirmation prevents
|
||||
/// false-swallowing a legitimate event whose content text merely quotes the
|
||||
/// event-type string. An unnamed frame with an unparseable payload is NOT
|
||||
/// the check event — a real server frame always carries the name or a
|
||||
/// parseable `type` tag, so forwarding preserves today's behavior for
|
||||
/// non-check traffic.
|
||||
pub fn is_check_event(event_name: &str, data: &str) -> bool {
|
||||
if event_name == DOOM_LOOP_CHECK_EVENT_TYPE {
|
||||
return true;
|
||||
}
|
||||
data.contains(DOOM_LOOP_CHECK_EVENT_TYPE)
|
||||
&& serde_json::from_str::<serde_json::Value>(data).is_ok_and(|v| {
|
||||
v.get("type").and_then(|t| t.as_str()) == Some(DOOM_LOOP_CHECK_EVENT_TYPE)
|
||||
})
|
||||
}
|
||||
|
||||
/// Parse a `triggers` JSON value into signals, skipping non-string entries.
|
||||
/// A missing or non-array value yields an empty vec.
|
||||
fn parse_triggers(triggers: Option<&serde_json::Value>) -> Vec<DoomLoopSignal> {
|
||||
triggers
|
||||
.and_then(|t| t.as_array())
|
||||
.map(|arr| {
|
||||
arr.iter()
|
||||
.filter_map(|v| v.as_str())
|
||||
.map(DoomLoopSignal::parse)
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn parse_tail_repetition_label() {
|
||||
let s = DoomLoopSignal::parse("tail_repetition:8@thinking");
|
||||
assert_eq!(s.kind, DoomLoopSignalKind::TailRepetition(8));
|
||||
assert_eq!(s.channel, "thinking");
|
||||
assert_eq!(s.raw, "tail_repetition:8@thinking");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_low_logprob_label() {
|
||||
let s = DoomLoopSignal::parse("low_logprob@response");
|
||||
assert_eq!(s.kind, DoomLoopSignalKind::LowLogprob);
|
||||
assert_eq!(s.channel, "response");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_unknown_kind_preserved() {
|
||||
let s = DoomLoopSignal::parse("novel_detector:3@thinking");
|
||||
assert_eq!(
|
||||
s.kind,
|
||||
DoomLoopSignalKind::Unknown("novel_detector:3".to_string())
|
||||
);
|
||||
assert_eq!(s.channel, "thinking");
|
||||
assert_eq!(s.raw, "novel_detector:3@thinking");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_grammar_mismatches_are_unknown_never_error() {
|
||||
// Non-numeric threshold.
|
||||
assert!(matches!(
|
||||
DoomLoopSignal::parse("tail_repetition:huge@thinking").kind,
|
||||
DoomLoopSignalKind::Unknown(_)
|
||||
));
|
||||
// low_logprob must not carry a threshold segment.
|
||||
assert!(matches!(
|
||||
DoomLoopSignal::parse("low_logprob:3@thinking").kind,
|
||||
DoomLoopSignalKind::Unknown(_)
|
||||
));
|
||||
// Missing channel: kind still parses, channel empty.
|
||||
let s = DoomLoopSignal::parse("tail_repetition:4");
|
||||
assert_eq!(s.kind, DoomLoopSignalKind::TailRepetition(4));
|
||||
assert_eq!(s.channel, "");
|
||||
// Empty label.
|
||||
assert!(matches!(
|
||||
DoomLoopSignal::parse("").kind,
|
||||
DoomLoopSignalKind::Unknown(_)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn signal_serde_round_trip() {
|
||||
let s = DoomLoopSignal::parse("tail_repetition:8@thinking");
|
||||
let json = serde_json::to_string(&s).unwrap();
|
||||
let back: DoomLoopSignal = serde_json::from_str(&json).unwrap();
|
||||
assert_eq!(back, s);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn policy_default_matches_documented_tunables() {
|
||||
let p = DoomLoopRecoveryPolicy::default();
|
||||
assert_eq!(p.max_threshold, 8);
|
||||
assert_eq!(p.max_retries, 2);
|
||||
}
|
||||
|
||||
/// Per-field serde defaults: payloads written before a field existed (or
|
||||
/// with fields yet to exist) must keep deserializing.
|
||||
#[test]
|
||||
fn policy_deserializes_with_missing_or_extra_fields() {
|
||||
let p: DoomLoopRecoveryPolicy = serde_json::from_str("{}").unwrap();
|
||||
assert_eq!(p, DoomLoopRecoveryPolicy::default());
|
||||
let p: DoomLoopRecoveryPolicy = serde_json::from_str(r#"{"max_threshold":4}"#).unwrap();
|
||||
assert_eq!(p.max_threshold, 4);
|
||||
assert_eq!(p.max_retries, DoomLoopRecoveryPolicy::DEFAULT_MAX_RETRIES);
|
||||
let p: DoomLoopRecoveryPolicy =
|
||||
serde_json::from_str(r#"{"max_retries":1,"future_knob":true}"#).unwrap();
|
||||
assert_eq!(p.max_retries, 1);
|
||||
}
|
||||
|
||||
/// Pin the server's exact wire bytes (not a paraphrase): the
|
||||
/// first frame and its cumulative follow-up both classify as check
|
||||
/// events with fully parsed labels.
|
||||
#[test]
|
||||
fn sample_wire_frames_parse_byte_exactly() {
|
||||
match peek_doom_loop(SAMPLE_CHECK_EVENT_DATA) {
|
||||
DoomLoopPeek::CheckEvent(signals) => {
|
||||
assert_eq!(signals.len(), 1);
|
||||
assert_eq!(signals[0].kind, DoomLoopSignalKind::TailRepetition(4));
|
||||
assert_eq!(signals[0].channel, "response");
|
||||
assert_eq!(signals[0].raw, "tail_repetition:4@response");
|
||||
}
|
||||
other => panic!("expected CheckEvent, got {other:?}"),
|
||||
}
|
||||
match peek_doom_loop(SAMPLE_CHECK_EVENT_DATA_CUMULATIVE) {
|
||||
DoomLoopPeek::CheckEvent(signals) => {
|
||||
assert_eq!(signals.len(), 2);
|
||||
assert_eq!(signals[0].raw, "tail_repetition:4@response");
|
||||
assert_eq!(signals[1].kind, DoomLoopSignalKind::TailRepetition(2));
|
||||
}
|
||||
other => panic!("expected CheckEvent, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn peek_check_event_parses_cumulative_triggers() {
|
||||
let data = r#"{"type":"response.doom_loop_check","doom_loop_check":{"triggers":["tail_repetition:8@thinking","low_logprob@thinking"]}}"#;
|
||||
match peek_doom_loop(data) {
|
||||
DoomLoopPeek::CheckEvent(signals) => {
|
||||
assert_eq!(signals.len(), 2);
|
||||
assert_eq!(signals[0].kind, DoomLoopSignalKind::TailRepetition(8));
|
||||
assert_eq!(signals[1].kind, DoomLoopSignalKind::LowLogprob);
|
||||
}
|
||||
other => panic!("expected CheckEvent, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn peek_check_event_swallowed_even_when_malformed() {
|
||||
// The event type alone must classify as CheckEvent so the caller
|
||||
// never forwards it to the typed parser, whatever the payload.
|
||||
for data in [
|
||||
r#"{"type":"response.doom_loop_check"}"#,
|
||||
r#"{"type":"response.doom_loop_check","doom_loop_check":{}}"#,
|
||||
r#"{"type":"response.doom_loop_check","doom_loop_check":{"triggers":"oops"}}"#,
|
||||
r#"{"type":"response.doom_loop_check","doom_loop_check":{"triggers":42}}"#,
|
||||
r#"{"type":"response.doom_loop_check","doom_loop_check":{"triggers":[1,{"a":2}]}}"#,
|
||||
r#"{"type":"response.doom_loop_check","doom_loop_check":null,"extra":true}"#,
|
||||
] {
|
||||
match peek_doom_loop(data) {
|
||||
DoomLoopPeek::CheckEvent(signals) => assert!(signals.is_empty(), "{data}"),
|
||||
other => panic!("expected CheckEvent for {data}, got {other:?}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn peek_check_event_skips_non_string_entries() {
|
||||
let data = r#"{"type":"response.doom_loop_check","doom_loop_check":{"triggers":[7,"tail_repetition:8@thinking",null]}}"#;
|
||||
match peek_doom_loop(data) {
|
||||
DoomLoopPeek::CheckEvent(signals) => {
|
||||
assert_eq!(signals.len(), 1);
|
||||
assert_eq!(signals[0].raw, "tail_repetition:8@thinking");
|
||||
}
|
||||
other => panic!("expected CheckEvent, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn peek_terminal_response_field() {
|
||||
let data = r#"{"type":"response.completed","response":{"id":"r1","doom_loop_check":{"triggers":["tail_repetition:16@thinking"]}}}"#;
|
||||
match peek_doom_loop(data) {
|
||||
DoomLoopPeek::ResponseField(signals) => {
|
||||
assert_eq!(signals.len(), 1);
|
||||
assert_eq!(signals[0].kind, DoomLoopSignalKind::TailRepetition(16));
|
||||
}
|
||||
other => panic!("expected ResponseField, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// Confidence is the conjunction kind = TailRepetition AND channel =
|
||||
/// thinking AND threshold <= max_threshold (boundary inclusive); each
|
||||
/// factor is falsified independently.
|
||||
#[test]
|
||||
fn confidence_requires_kind_channel_and_threshold() {
|
||||
let policy = DoomLoopRecoveryPolicy::default();
|
||||
assert!(policy.is_confident(&DoomLoopSignal::parse("tail_repetition:8@thinking")));
|
||||
assert!(policy.is_confident(&DoomLoopSignal::parse("tail_repetition:2@thinking")));
|
||||
assert!(!policy.is_confident(&DoomLoopSignal::parse("tail_repetition:9@thinking")));
|
||||
assert!(!policy.is_confident(&DoomLoopSignal::parse("tail_repetition:2@response")));
|
||||
assert!(!policy.is_confident(&DoomLoopSignal::parse("low_logprob@thinking")));
|
||||
assert!(!policy.is_confident(&DoomLoopSignal::parse("novel_detector:2@thinking")));
|
||||
|
||||
let signals = vec![
|
||||
DoomLoopSignal::parse("tail_repetition:4@response"),
|
||||
DoomLoopSignal::parse("tail_repetition:4@thinking"),
|
||||
DoomLoopSignal::parse("low_logprob@thinking"),
|
||||
];
|
||||
assert_eq!(
|
||||
policy.confident_triggers(&signals),
|
||||
vec!["tail_repetition:4@thinking".to_string()]
|
||||
);
|
||||
assert!(policy.confident_triggers(&[]).is_empty());
|
||||
}
|
||||
|
||||
/// Tightest = lowest tail-repetition threshold; non-tail labels only win
|
||||
/// when nothing parses as tail_repetition.
|
||||
#[test]
|
||||
fn tightest_prefers_lowest_tail_repetition_threshold() {
|
||||
assert_eq!(
|
||||
DoomLoopSignal::tightest([
|
||||
"tail_repetition:64@thinking",
|
||||
"tail_repetition:4@thinking",
|
||||
"tail_repetition:16@thinking",
|
||||
]),
|
||||
Some("tail_repetition:4@thinking".to_string())
|
||||
);
|
||||
assert_eq!(
|
||||
DoomLoopSignal::tightest(["low_logprob@thinking", "tail_repetition:8@thinking"]),
|
||||
Some("tail_repetition:8@thinking".to_string())
|
||||
);
|
||||
assert_eq!(
|
||||
DoomLoopSignal::tightest(["low_logprob@thinking", "novel:2@thinking"]),
|
||||
Some("low_logprob@thinking".to_string()),
|
||||
"no tail_repetition label: fall back to the first"
|
||||
);
|
||||
assert_eq!(DoomLoopSignal::tightest(Vec::<String>::new()), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn is_check_event_matches_name_or_payload_type() {
|
||||
// Named frame: payload validity is irrelevant.
|
||||
assert!(is_check_event(DOOM_LOOP_CHECK_EVENT_TYPE, "not json"));
|
||||
// Unnamed frame identified by its payload `type` tag.
|
||||
assert!(is_check_event("message", SAMPLE_CHECK_EVENT_DATA));
|
||||
// A normal delta QUOTING the event-type string is not the check
|
||||
// event: the substring precheck hits but the type confirm fails.
|
||||
let quoting = r#"{"type":"response.output_text.delta","delta":"response.doom_loop_check"}"#;
|
||||
assert!(!is_check_event("response.output_text.delta", quoting));
|
||||
// Unnamed + unparseable payload: forwarded, not swallowed — a real
|
||||
// server frame carries the name or a parseable `type` tag.
|
||||
assert!(!is_check_event(
|
||||
"message",
|
||||
"garbage response.doom_loop_check garbage"
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn peek_none_for_ordinary_and_malformed_payloads() {
|
||||
assert_eq!(
|
||||
peek_doom_loop(r#"{"type":"response.output_text.delta","delta":"hi"}"#),
|
||||
DoomLoopPeek::None
|
||||
);
|
||||
// Terminal event without the field.
|
||||
assert_eq!(
|
||||
peek_doom_loop(r#"{"type":"response.completed","response":{"id":"r1"}}"#),
|
||||
DoomLoopPeek::None
|
||||
);
|
||||
// Non-JSON mentioning the key still degrades to None.
|
||||
assert_eq!(
|
||||
peek_doom_loop("doom_loop_check garbage"),
|
||||
DoomLoopPeek::None
|
||||
);
|
||||
// The key appearing in unrelated positions is ignored.
|
||||
assert_eq!(
|
||||
peek_doom_loop(r#"{"type":"response.output_text.delta","delta":"doom_loop_check"}"#),
|
||||
DoomLoopPeek::None
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,765 @@
|
||||
//! Sampling error types.
|
||||
//!
|
||||
//! TODO: Move from kigi-shell/src/sampling/error.rs
|
||||
|
||||
use std::fmt;
|
||||
|
||||
use reqwest::StatusCode;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use thiserror::Error;
|
||||
|
||||
pub type Result<T> = std::result::Result<T, SamplingError>;
|
||||
|
||||
/// Why the model's response was classified as "empty" by [`ConversationResponse::empty_reason`].
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum EmptyReason {
|
||||
/// The model emitted reasoning tokens but produced no visible content
|
||||
/// and no tool calls. The stream completed normally (has `finish_reason`).
|
||||
ReasoningOnly,
|
||||
/// The stream carried at least one `choice` but the final assistant
|
||||
/// message has empty `content` and no tool calls (and no reasoning).
|
||||
NoVisibleContent,
|
||||
}
|
||||
|
||||
impl EmptyReason {
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
EmptyReason::ReasoningOnly => "reasoning_only",
|
||||
EmptyReason::NoVisibleContent => "no_visible_content",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Display for EmptyReason {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.write_str(self.as_str())
|
||||
}
|
||||
}
|
||||
|
||||
/// Structured context captured at L2 stream completion time when the
|
||||
/// response is classified as empty. Carries everything needed to
|
||||
/// root-cause the issue from a single log line or error payload.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct EmptyResponseContext {
|
||||
pub reason: EmptyReason,
|
||||
/// Whether the response contained reasoning tokens.
|
||||
pub had_reasoning: bool,
|
||||
/// Byte length of the accumulated `content` string (0 for truly empty).
|
||||
pub content_len: usize,
|
||||
/// Number of tool calls in the final response.
|
||||
pub tool_call_count: usize,
|
||||
/// The `finish_reason` from the stream, if any.
|
||||
pub finish_reason: Option<String>,
|
||||
/// Token usage from the response (when available).
|
||||
pub completion_tokens: Option<u32>,
|
||||
pub reasoning_tokens: Option<u32>,
|
||||
pub prompt_tokens: Option<u32>,
|
||||
/// Model that produced the response.
|
||||
pub model: String,
|
||||
/// Whether at least one `choice` was seen in the stream.
|
||||
pub first_choice_seen: bool,
|
||||
}
|
||||
|
||||
impl EmptyResponseContext {
|
||||
pub fn finish_reason_str(&self) -> &str {
|
||||
self.finish_reason.as_deref().unwrap_or("none")
|
||||
}
|
||||
}
|
||||
|
||||
/// Model metadata from response headers.
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct ResponseModelMetadata {
|
||||
pub context_window: Option<u64>,
|
||||
pub max_completion_tokens: Option<u32>,
|
||||
/// `x-models-etag` — triggers model catalog refresh when changed.
|
||||
pub models_etag: Option<String>,
|
||||
}
|
||||
|
||||
/// Display prefix of [`SamplingError::Serialization`]. Shared with the
|
||||
/// variant's `#[error(...)]` template so [`SamplingError::serialization_from_rendered`]
|
||||
/// can never drift from what Display actually emits.
|
||||
const SERIALIZATION_DISPLAY_PREFIX: &str = "serialization error: ";
|
||||
|
||||
#[derive(Debug, Error)]
|
||||
pub enum SamplingError {
|
||||
#[error("{0}")]
|
||||
Auth(String),
|
||||
#[error("invalid client configuration: {0}")]
|
||||
InvalidConfiguration(&'static str),
|
||||
#[error("request error: {0}")]
|
||||
Http(reqwest::Error),
|
||||
#[error("{prefix}{0}", prefix = SERIALIZATION_DISPLAY_PREFIX)]
|
||||
Serialization(serde_json::Error),
|
||||
#[error("API error (status {status}): {message}")]
|
||||
Api {
|
||||
status: StatusCode,
|
||||
message: String,
|
||||
model_metadata: Option<ResponseModelMetadata>,
|
||||
/// Parsed from the `Retry-After` response header (seconds).
|
||||
retry_after_secs: Option<u64>,
|
||||
/// Parsed from the `x-should-retry` response header.
|
||||
/// `Some(true)` = transient, retry may help.
|
||||
/// `Some(false)` = request-content error, don't retry.
|
||||
/// `None` = header absent (old server or non-proxy origin).
|
||||
should_retry: Option<bool>,
|
||||
},
|
||||
#[error("reqwest error stream: {0}")]
|
||||
EventStreamError(String),
|
||||
/// Server-side stream error (sent as JSON within the SSE stream)
|
||||
#[error("stream error ({error_type}): {message}")]
|
||||
StreamError { error_type: String, message: String },
|
||||
/// Per-chunk idle timeout — no SSE chunk received from the model within the
|
||||
/// configured deadline. NOT retryable: the model (or network path) is stuck,
|
||||
/// and replaying the same request would likely stall again.
|
||||
#[error("inference idle timeout after {elapsed_secs}s with no chunks")]
|
||||
IdleTimeout { elapsed_secs: u64 },
|
||||
#[error("empty response from model ({})", context.reason)]
|
||||
EmptyResponse { context: EmptyResponseContext },
|
||||
#[error("response truncated by max_tokens")]
|
||||
MaxTokensTruncation,
|
||||
/// A confident server-reported doom loop on the attempt (mid-stream or
|
||||
/// on the completed response). Retryable on the recovery loop's own
|
||||
/// budget, separate from the transport budget. Carries the raw trigger
|
||||
/// labels (never generation content) plus, for telemetry only, the
|
||||
/// stream chunk index the mid-stream abort fired at (`None` when the
|
||||
/// signal was only seen on the completed response).
|
||||
#[error("doom loop detected: {}", triggers.join(", "))]
|
||||
DoomLoopDetected {
|
||||
triggers: Vec<String>,
|
||||
aborted_at_chunk: Option<u64>,
|
||||
},
|
||||
}
|
||||
|
||||
impl SamplingError {
|
||||
/// Rebuild a `Serialization` error from a rendered message for non-`Clone`
|
||||
/// contexts; it must stay `Serialization` so it remains non-retryable.
|
||||
pub fn serialization_message(msg: impl fmt::Display) -> Self {
|
||||
Self::Serialization(serde::de::Error::custom(msg))
|
||||
}
|
||||
|
||||
/// Rebuild from this variant's full rendered Display (e.g. a round-tripped
|
||||
/// `SamplingErrorInfo` message), stripping the Display prefix so the
|
||||
/// rebuilt error does not render it twice.
|
||||
pub fn serialization_from_rendered(rendered: &str) -> Self {
|
||||
Self::serialization_message(
|
||||
rendered
|
||||
.strip_prefix(SERIALIZATION_DISPLAY_PREFIX)
|
||||
.unwrap_or(rendered),
|
||||
)
|
||||
}
|
||||
|
||||
pub fn is_auth_error(&self) -> bool {
|
||||
// Only 401 Unauthorized means the credentials themselves were rejected
|
||||
// and warrant a token refresh / re-auth. 403 Forbidden means the
|
||||
// request was authenticated successfully but the action is not
|
||||
// permitted (e.g. content-safety blocks, ZDR-blocked operations,
|
||||
// or other policy denials unrelated to credentials). Treating 403
|
||||
// as an auth error triggers a pointless
|
||||
// OIDC refresh and then surfaces as acp::Error::auth_required on
|
||||
// the client, which in the desktop app tears down the session and
|
||||
// can race with invalid_grant_threshold to wipe auth.json.
|
||||
matches!(
|
||||
self,
|
||||
SamplingError::Auth(_)
|
||||
| SamplingError::Api {
|
||||
status: StatusCode::UNAUTHORIZED,
|
||||
..
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
pub fn is_rate_limited(&self) -> bool {
|
||||
matches!(
|
||||
self,
|
||||
SamplingError::Api {
|
||||
status: StatusCode::TOO_MANY_REQUESTS,
|
||||
..
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
pub fn is_payload_too_large(&self) -> bool {
|
||||
matches!(
|
||||
self,
|
||||
SamplingError::Api {
|
||||
status: StatusCode::PAYLOAD_TOO_LARGE,
|
||||
..
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
/// `true` when the error looks like a connection reset or broken pipe
|
||||
/// during request upload — the pattern nginx produces when it rejects an
|
||||
/// oversized payload by closing the connection instead of responding 413.
|
||||
///
|
||||
/// Timeouts and connect failures are excluded: those are unrelated to
|
||||
/// payload size and stripping images on them would lose context for no
|
||||
/// reason.
|
||||
pub fn is_likely_body_rejected(&self) -> bool {
|
||||
match self {
|
||||
SamplingError::Http(err) => {
|
||||
// `is_request()` covers broken-pipe / connection-reset during
|
||||
// body upload. `is_body()` covers stream-write failures.
|
||||
// Exclude timeouts and connect errors — those are unrelated.
|
||||
(err.is_request() || err.is_body()) && !err.is_timeout() && !err.is_connect()
|
||||
}
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// The server rejected the request because the conversation history
|
||||
/// contains `encrypted_content` from a different model family that the
|
||||
/// current model cannot decrypt. Never retryable — the user must start
|
||||
/// a new session.
|
||||
pub fn is_encrypted_content_error(&self) -> bool {
|
||||
matches!(
|
||||
self,
|
||||
SamplingError::Api {
|
||||
status: StatusCode::BAD_REQUEST,
|
||||
message,
|
||||
..
|
||||
} if message.contains("encrypted_content")
|
||||
)
|
||||
}
|
||||
|
||||
/// The API rejected the request because an inline image could not be
|
||||
/// processed. Matches both direct 400 and proxy-wrapped 500 responses.
|
||||
/// Exact-case match — consistent with `is_encrypted_content_error`.
|
||||
pub fn is_image_processing_error(&self) -> bool {
|
||||
matches!(
|
||||
self,
|
||||
SamplingError::Api {
|
||||
status,
|
||||
message,
|
||||
..
|
||||
} if matches!(status.as_u16(), 400 | 500) && message.contains("Could not process image")
|
||||
)
|
||||
}
|
||||
|
||||
pub fn is_retryable(&self) -> bool {
|
||||
match self {
|
||||
SamplingError::Auth(_) => false,
|
||||
SamplingError::InvalidConfiguration(_) => false,
|
||||
SamplingError::Http(err) => is_retryable_reqwest(err),
|
||||
SamplingError::Serialization(_) => false,
|
||||
SamplingError::Api { status, .. } => {
|
||||
matches!(status.as_u16(), 429 | 500 | 502 | 503 | 504 | 520)
|
||||
}
|
||||
SamplingError::EventStreamError(_) => true,
|
||||
SamplingError::StreamError { .. } => true,
|
||||
SamplingError::IdleTimeout { .. } => false,
|
||||
SamplingError::EmptyResponse { .. } => true,
|
||||
SamplingError::MaxTokensTruncation => false,
|
||||
SamplingError::DoomLoopDetected { .. } => true,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn model_metadata(&self) -> Option<&ResponseModelMetadata> {
|
||||
match self {
|
||||
SamplingError::Api { model_metadata, .. } => model_metadata.as_ref(),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn retry_after(&self) -> Option<u64> {
|
||||
match self {
|
||||
SamplingError::Api {
|
||||
retry_after_secs, ..
|
||||
} => *retry_after_secs,
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Server hint on whether this error is worth retrying.
|
||||
pub fn should_retry_header(&self) -> Option<bool> {
|
||||
match self {
|
||||
SamplingError::Api { should_retry, .. } => *should_retry,
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// True when this error is a context-window/size overflow — deterministic,
|
||||
/// so retrying the same payload can't help. See [`is_context_length_error`].
|
||||
pub fn is_context_length_error(&self) -> bool {
|
||||
match self {
|
||||
SamplingError::Api { message, .. } | SamplingError::StreamError { message, .. } => {
|
||||
is_context_length_error(message)
|
||||
}
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl From<reqwest::Error> for SamplingError {
|
||||
fn from(value: reqwest::Error) -> Self {
|
||||
Self::Http(value)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<serde_json::Error> for SamplingError {
|
||||
fn from(value: serde_json::Error) -> Self {
|
||||
tracing::debug!("Serde deserialization error: {:?}", &value);
|
||||
Self::Serialization(value)
|
||||
}
|
||||
}
|
||||
|
||||
/// OpenAI-standard provider error format: `{"error": {"message": "...", "type": "..."}}`.
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct ErrorResponse {
|
||||
error: ErrorBody,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct ErrorBody {
|
||||
message: Option<String>,
|
||||
#[serde(rename = "type")]
|
||||
kind: Option<String>,
|
||||
}
|
||||
|
||||
/// Flat error from the Grok proxy/gateway: `{"code": "...", "error": "..."}`.
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct FlatErrorResponse {
|
||||
error: String,
|
||||
#[serde(default)]
|
||||
code: Option<String>,
|
||||
}
|
||||
|
||||
/// Extract `(error_type, message)` from either error format.
|
||||
fn try_parse_error(data: &str) -> Option<(String, String)> {
|
||||
if let Ok(resp) = serde_json::from_str::<ErrorResponse>(data) {
|
||||
return Some((
|
||||
resp.error.kind.unwrap_or_else(|| "unknown".to_string()),
|
||||
resp.error
|
||||
.message
|
||||
.unwrap_or_else(|| "unknown error".to_string()),
|
||||
));
|
||||
}
|
||||
if let Ok(flat) = serde_json::from_str::<FlatErrorResponse>(data) {
|
||||
return Some((
|
||||
flat.code.unwrap_or_else(|| "server_error".to_string()),
|
||||
flat.error,
|
||||
));
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
pub fn parse_error_bytes(bytes: &[u8]) -> String {
|
||||
if let Some((error_type, message)) = std::str::from_utf8(bytes).ok().and_then(try_parse_error) {
|
||||
if error_type == "unknown" || error_type == "server_error" {
|
||||
return message;
|
||||
}
|
||||
return format!("{error_type}: {message}");
|
||||
}
|
||||
String::from_utf8_lossy(bytes).trim().to_owned()
|
||||
}
|
||||
|
||||
pub fn try_parse_stream_error(data: &str) -> Option<SamplingError> {
|
||||
let (error_type, message) = try_parse_error(data)?;
|
||||
tracing::warn!(error_type, message, "Server-side stream error");
|
||||
Some(SamplingError::StreamError {
|
||||
error_type,
|
||||
message,
|
||||
})
|
||||
}
|
||||
|
||||
/// True when an error message indicates a context-window overflow. Backends report
|
||||
/// this inconsistently with no stable error code, so we match the message text; it's
|
||||
/// deterministic (re-sending the same payload always fails), so callers must not retry.
|
||||
pub fn is_context_length_error(message: &str) -> bool {
|
||||
let m = message.to_ascii_lowercase();
|
||||
m.contains("too long for this model")
|
||||
|| m.contains("prompt is too long")
|
||||
|| m.contains("maximum prompt length")
|
||||
|| m.contains("maximum context length")
|
||||
|| m.contains("context_length_exceeded")
|
||||
}
|
||||
|
||||
/// Decide whether a [`reqwest::Error`] is worth retrying.
|
||||
pub fn is_retryable_reqwest(err: &reqwest::Error) -> bool {
|
||||
if err.is_timeout() || err.is_connect() {
|
||||
return true;
|
||||
}
|
||||
|
||||
if err.is_status() {
|
||||
return matches!(
|
||||
err.status(),
|
||||
Some(status) if status.is_server_error() || status == StatusCode::TOO_MANY_REQUESTS
|
||||
);
|
||||
}
|
||||
|
||||
if err.is_request() || err.is_body() {
|
||||
return true;
|
||||
}
|
||||
|
||||
false
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn context_length_error_matches_backend_variants() {
|
||||
for msg in [
|
||||
"This model's maximum prompt length is 256000 but the request contains 1500000",
|
||||
"The prompt is too long for this model's context window.",
|
||||
"none: The prompt is too long for this model's context window.",
|
||||
"This model's maximum context length is 200000 tokens",
|
||||
"invalid_request_error: prompt is too long: 300000 tokens > 200000 maximum",
|
||||
"error type: context_length_exceeded",
|
||||
] {
|
||||
assert!(is_context_length_error(msg), "should match: {msg}");
|
||||
}
|
||||
for msg in ["rate limited", "internal server error", "connection reset"] {
|
||||
assert!(!is_context_length_error(msg), "should not match: {msg}");
|
||||
}
|
||||
// The method delegates for the Api/StreamError variants.
|
||||
let api = SamplingError::Api {
|
||||
status: StatusCode::INTERNAL_SERVER_ERROR,
|
||||
message: "none: The prompt is too long for this model's context window.".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(api.is_context_length_error());
|
||||
assert!(
|
||||
SamplingError::StreamError {
|
||||
error_type: "overloaded_error".into(),
|
||||
message: "prompt is too long".into(),
|
||||
}
|
||||
.is_context_length_error()
|
||||
);
|
||||
assert!(!SamplingError::Auth("nope".into()).is_context_length_error());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serialization_message_stays_serialization_and_non_retryable() {
|
||||
let err = SamplingError::serialization_message("bad payload at line 1 column 7");
|
||||
assert!(matches!(err, SamplingError::Serialization(_)));
|
||||
assert!(!err.is_retryable());
|
||||
assert!(err.to_string().contains("bad payload at line 1 column 7"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn serialization_from_rendered_round_trips_display() {
|
||||
// Derived from a REAL error's Display so a template rewording cannot
|
||||
// silently desynchronize the strip from the prefix it mirrors.
|
||||
let original =
|
||||
SamplingError::Serialization(serde_json::from_str::<i32>("not a number").unwrap_err());
|
||||
let rendered = original.to_string();
|
||||
let rebuilt = SamplingError::serialization_from_rendered(&rendered);
|
||||
assert!(matches!(rebuilt, SamplingError::Serialization(_)));
|
||||
assert!(!rebuilt.is_retryable());
|
||||
assert_eq!(
|
||||
rebuilt.to_string(),
|
||||
rendered,
|
||||
"rendered Display must round-trip without double-prefixing"
|
||||
);
|
||||
// Bare (non-rendered) input gains the prefix exactly once.
|
||||
assert_eq!(
|
||||
SamplingError::serialization_from_rendered("bare message").to_string(),
|
||||
format!("{SERIALIZATION_DISPLAY_PREFIX}bare message"),
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn idle_timeout_is_not_retryable() {
|
||||
let err = SamplingError::IdleTimeout { elapsed_secs: 300 };
|
||||
assert!(
|
||||
!err.is_retryable(),
|
||||
"IdleTimeout must not be retried — would cause 3× amplification"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn event_stream_error_is_retryable() {
|
||||
// Verify the existing contract hasn't changed — EventStreamError is retryable.
|
||||
let err = SamplingError::EventStreamError("connection reset".into());
|
||||
assert!(err.is_retryable());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn idle_timeout_display() {
|
||||
let err = SamplingError::IdleTimeout { elapsed_secs: 120 };
|
||||
let msg = err.to_string();
|
||||
assert!(
|
||||
msg.contains("120s"),
|
||||
"Display should include elapsed_secs: {msg}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn try_parse_stream_error_flat_format() {
|
||||
let data = r#"{"code":"The service is currently unavailable","error":"Service temporarily unavailable. The model did not respond to this request."}"#;
|
||||
let err = try_parse_stream_error(data).expect("should parse flat error");
|
||||
match err {
|
||||
SamplingError::StreamError {
|
||||
error_type,
|
||||
message,
|
||||
} => {
|
||||
assert_eq!(error_type, "The service is currently unavailable");
|
||||
assert_eq!(
|
||||
message,
|
||||
"Service temporarily unavailable. The model did not respond to this request."
|
||||
);
|
||||
}
|
||||
other => panic!("expected StreamError, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn try_parse_stream_error_valid_chunk_returns_none() {
|
||||
let data = r#"{"id":"abc","object":"chat.completion.chunk","created":0,"model":"test","choices":[]}"#;
|
||||
assert!(
|
||||
try_parse_stream_error(data).is_none(),
|
||||
"valid chunk should not be parsed as error"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_error_bytes_flat_format() {
|
||||
let bytes =
|
||||
br#"{"code":"The service is currently unavailable","error":"Service temporarily unavailable."}"#;
|
||||
let msg = parse_error_bytes(bytes);
|
||||
assert_eq!(
|
||||
msg,
|
||||
"The service is currently unavailable: Service temporarily unavailable."
|
||||
);
|
||||
}
|
||||
|
||||
/// Regression test: 403 Forbidden must NOT be classified as an auth
|
||||
/// error. The proxy returns 403 for policy denials that are unrelated
|
||||
/// to the caller's credentials (content-safety blocks, ZDR-gated
|
||||
/// operations, or other usage-policy blocks). Misclassifying these as
|
||||
/// auth errors triggers a pointless OIDC
|
||||
/// refresh and surfaces as acp::Error::auth_required on the client,
|
||||
/// tearing down the session and risking an
|
||||
/// `invalid_grant_threshold`-triggered wipe of auth.json.
|
||||
#[test]
|
||||
fn forbidden_is_not_auth_error() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::FORBIDDEN,
|
||||
message: "Content violates usage guidelines.".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(
|
||||
!err.is_auth_error(),
|
||||
"403 Forbidden must not be treated as an auth error"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unauthorized_is_auth_error() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::UNAUTHORIZED,
|
||||
message: "Invalid or expired credentials".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(
|
||||
err.is_auth_error(),
|
||||
"401 Unauthorized must be an auth error"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn auth_variant_is_auth_error() {
|
||||
let err = SamplingError::Auth("bad key".into());
|
||||
assert!(err.is_auth_error());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rate_limited_api_error_is_detected() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::TOO_MANY_REQUESTS,
|
||||
message: "Rate limit exceeded".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(err.is_rate_limited());
|
||||
assert!(err.is_retryable(), "429 should be retryable");
|
||||
assert!(!err.is_auth_error());
|
||||
assert!(!err.is_payload_too_large());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_rate_limit_errors_are_not_rate_limited() {
|
||||
let server_error = SamplingError::Api {
|
||||
status: StatusCode::INTERNAL_SERVER_ERROR,
|
||||
message: "internal".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(!server_error.is_rate_limited());
|
||||
|
||||
let auth_error = SamplingError::Auth("bad key".into());
|
||||
assert!(!auth_error.is_rate_limited());
|
||||
|
||||
let timeout = SamplingError::IdleTimeout { elapsed_secs: 30 };
|
||||
assert!(!timeout.is_rate_limited());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn retry_after_returns_header_value() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::TOO_MANY_REQUESTS,
|
||||
message: "slow down".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: Some(42),
|
||||
should_retry: None,
|
||||
};
|
||||
assert_eq!(err.retry_after(), Some(42));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn retry_after_returns_none_when_absent() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::TOO_MANY_REQUESTS,
|
||||
message: "slow down".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert_eq!(err.retry_after(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn retry_after_returns_none_for_non_api_errors() {
|
||||
assert_eq!(SamplingError::Auth("x".into()).retry_after(), None);
|
||||
assert_eq!(
|
||||
SamplingError::IdleTimeout { elapsed_secs: 10 }.retry_after(),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encrypted_content_400_is_detected() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::BAD_REQUEST,
|
||||
message: "Could not decrypt the provided encrypted_content. Ensure the value is the unmodified encrypted_content from a previous response.".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(err.is_encrypted_content_error());
|
||||
assert!(
|
||||
!err.is_retryable(),
|
||||
"encrypted_content errors must not be retried"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encrypted_content_wrong_status_not_detected() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::INTERNAL_SERVER_ERROR,
|
||||
message: "encrypted_content decryption failed".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(
|
||||
!err.is_encrypted_content_error(),
|
||||
"only 400 should match, not 500"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encrypted_content_unrelated_400_not_detected() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::BAD_REQUEST,
|
||||
message: "Invalid model parameter".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(
|
||||
!err.is_encrypted_content_error(),
|
||||
"unrelated 400 errors must not match"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn image_processing_error_direct_400_detected() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::BAD_REQUEST,
|
||||
message: "Could not process image: unsupported format".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(err.is_image_processing_error());
|
||||
assert!(!err.is_encrypted_content_error());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn image_processing_error_500_wrapped_detected() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::INTERNAL_SERVER_ERROR,
|
||||
message: "upstream error: 400 Bad Request: Could not process image".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(err.is_image_processing_error());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn image_processing_error_unrelated_400_not_detected() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::BAD_REQUEST,
|
||||
message: "Invalid model parameter".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(!err.is_image_processing_error());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn image_processing_error_unrelated_500_not_detected() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::INTERNAL_SERVER_ERROR,
|
||||
message: "internal server error".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(!err.is_image_processing_error());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn image_processing_error_wrong_status_not_detected() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::BAD_GATEWAY,
|
||||
message: "Could not process image".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(
|
||||
!err.is_image_processing_error(),
|
||||
"only 400 and 500 should match"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn image_processing_error_400_is_not_retryable_standalone() {
|
||||
let err = SamplingError::Api {
|
||||
status: StatusCode::BAD_REQUEST,
|
||||
message: "Could not process image".into(),
|
||||
model_metadata: None,
|
||||
retry_after_secs: None,
|
||||
should_retry: None,
|
||||
};
|
||||
assert!(
|
||||
!err.is_retryable(),
|
||||
"direct 400 must not be retryable by is_retryable()"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
//! Pure data types for the xAI sampling / chat-completion API layer.
|
||||
//!
|
||||
//! This crate contains the API-agnostic conversation types, chat completion
|
||||
//! request/response types, streaming types, and error types used across the
|
||||
//! xAI agent stack. It intentionally contains **no I/O** (no HTTP clients,
|
||||
//! no file system access) so it can be depended on by downstream crates
|
||||
//! (e.g., `kigi-chat-state`) without pulling in the full `kigi-shell`.
|
||||
|
||||
pub mod conversation;
|
||||
pub mod doom_loop;
|
||||
pub mod error;
|
||||
pub mod messages;
|
||||
pub mod serde_helpers;
|
||||
pub mod types;
|
||||
|
||||
pub use self::conversation::*;
|
||||
pub use self::doom_loop::{
|
||||
DOOM_LOOP_CHECK_EVENT_TYPE, DOOM_LOOP_CHECK_HEADER, DoomLoopPeek, DoomLoopRecoveryPolicy,
|
||||
DoomLoopSignal, DoomLoopSignalKind, is_check_event, peek_doom_loop,
|
||||
};
|
||||
pub use self::error::{
|
||||
EmptyReason, EmptyResponseContext, ResponseModelMetadata, Result, SamplingError,
|
||||
is_context_length_error,
|
||||
};
|
||||
pub use self::types::*;
|
||||
|
||||
// Re-export async-openai crate Responses API types under `rs` namespace
|
||||
pub use async_openai::types::responses as rs;
|
||||
@@ -0,0 +1,421 @@
|
||||
//! Anthropic Messages API (`/v1/messages`) wire types.
|
||||
//!
|
||||
//! These types represent the request/response format for the `/v1/messages` API.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
// ============================================================================
|
||||
// Request Types
|
||||
// ============================================================================
|
||||
|
||||
/// POST /v1/messages request body
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct MessagesRequest {
|
||||
pub model: String,
|
||||
pub messages: Vec<Message>,
|
||||
pub max_tokens: u32,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub system: Option<SystemParam>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub tools: Option<Vec<ToolParam>>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub tool_choice: Option<ToolChoiceParam>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub temperature: Option<f32>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub top_p: Option<f32>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub top_k: Option<u32>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub stream: Option<bool>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub stop_sequences: Option<Vec<String>>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub thinking: Option<ThinkingConfig>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub output_config: Option<OutputConfig>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub metadata: Option<Metadata>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct OutputConfig {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub effort: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub format: Option<OutputFormat>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "type", rename_all = "snake_case")]
|
||||
pub enum OutputFormat {
|
||||
JsonSchema { schema: serde_json::Value },
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Message {
|
||||
pub role: MessageRole,
|
||||
pub content: MessageContent,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "lowercase")]
|
||||
pub enum MessageRole {
|
||||
User,
|
||||
Assistant,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(untagged)]
|
||||
pub enum MessageContent {
|
||||
Text(String),
|
||||
Blocks(Vec<ContentBlock>),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(untagged)]
|
||||
pub enum SystemParam {
|
||||
Text(String),
|
||||
Blocks(Vec<TextBlock>),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct TextBlock {
|
||||
#[serde(rename = "type")]
|
||||
pub r#type: String, // always "text"
|
||||
pub text: String,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub cache_control: Option<CacheControl>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct CacheControl {
|
||||
#[serde(rename = "type")]
|
||||
pub r#type: String, // "ephemeral"
|
||||
}
|
||||
|
||||
/// Content blocks used in both requests and responses
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "type", rename_all = "snake_case")]
|
||||
pub enum ContentBlock {
|
||||
Text {
|
||||
text: String,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
cache_control: Option<CacheControl>,
|
||||
},
|
||||
Image {
|
||||
source: ImageSource,
|
||||
},
|
||||
ToolUse {
|
||||
id: String,
|
||||
name: String,
|
||||
input: serde_json::Value,
|
||||
},
|
||||
ToolResult {
|
||||
tool_use_id: String,
|
||||
content: ToolResultContent,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
cache_control: Option<CacheControl>,
|
||||
},
|
||||
Thinking {
|
||||
thinking: String,
|
||||
signature: String,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "type", rename_all = "snake_case")]
|
||||
pub enum ImageSource {
|
||||
Base64 { media_type: String, data: String },
|
||||
Url { url: String },
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(untagged)]
|
||||
pub enum ToolResultContent {
|
||||
Text(String),
|
||||
Blocks(Vec<ContentBlock>),
|
||||
}
|
||||
|
||||
/// Tool definition (Anthropic Messages API format)
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ToolParam {
|
||||
pub name: String,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub description: Option<String>,
|
||||
pub input_schema: serde_json::Value,
|
||||
}
|
||||
|
||||
/// Tool choice (Anthropic Messages API format)
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "type", rename_all = "snake_case")]
|
||||
pub enum ToolChoiceParam {
|
||||
Auto,
|
||||
Any,
|
||||
Tool { name: String },
|
||||
}
|
||||
|
||||
/// Extended thinking configuration
|
||||
///
|
||||
/// Three modes per the Anthropic Messages API:
|
||||
/// - Adaptive: 4.6+ models, API decides budget
|
||||
/// - Enabled: 4.0-4.5 models, explicit budget_tokens
|
||||
/// - Disabled: pre-thinking models or thinking_budget=0
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum ThinkingDisplay {
|
||||
Omitted,
|
||||
Summarized,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "type", rename_all = "snake_case")]
|
||||
pub enum ThinkingConfig {
|
||||
Enabled {
|
||||
budget_tokens: u32,
|
||||
},
|
||||
Adaptive {
|
||||
// Newer thinking-capable models omit thinking content unless display = "summarized".
|
||||
// Older models ignore this field. Skip when None to stay back-compat.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
display: Option<ThinkingDisplay>,
|
||||
},
|
||||
Disabled,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Metadata {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub user_id: Option<String>,
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Response Types
|
||||
// ============================================================================
|
||||
|
||||
/// Non-streaming response from POST /v1/messages
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct MessagesResponse {
|
||||
pub id: String,
|
||||
#[serde(rename = "type")]
|
||||
pub r#type: String, // "message"
|
||||
pub role: String, // "assistant"
|
||||
pub content: Vec<ContentBlock>,
|
||||
pub model: String,
|
||||
pub stop_reason: Option<StopReason>,
|
||||
pub usage: MessagesUsage,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum StopReason {
|
||||
EndTurn,
|
||||
MaxTokens,
|
||||
ToolUse,
|
||||
StopSequence,
|
||||
Refusal,
|
||||
PauseTurn,
|
||||
ModelContextWindowExceeded,
|
||||
/// Catch-all for stop reasons this client does not know yet, so a new
|
||||
/// server-side value can never fail the terminal `message_delta` parse
|
||||
/// and discard an already-streamed response. Preserves the wire string
|
||||
/// for logging and faithful re-serialization; must stay the LAST variant
|
||||
/// (serde tries the tagged variants above first).
|
||||
#[serde(untagged)]
|
||||
Unknown(String),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct MessagesUsage {
|
||||
pub input_tokens: u32,
|
||||
pub output_tokens: u32,
|
||||
#[serde(default)]
|
||||
pub cache_creation_input_tokens: u32,
|
||||
#[serde(default)]
|
||||
pub cache_read_input_tokens: u32,
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Streaming Event Types
|
||||
// ============================================================================
|
||||
|
||||
/// Top-level streaming event (SSE `type` field determines variant)
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "type", rename_all = "snake_case")]
|
||||
pub enum MessageStreamEvent {
|
||||
MessageStart {
|
||||
message: MessagesResponse,
|
||||
},
|
||||
MessageDelta {
|
||||
delta: MessageDeltaBody,
|
||||
usage: MessageDeltaUsage,
|
||||
},
|
||||
MessageStop,
|
||||
ContentBlockStart {
|
||||
index: u32,
|
||||
content_block: ContentBlock,
|
||||
},
|
||||
ContentBlockDelta {
|
||||
index: u32,
|
||||
delta: StreamDelta,
|
||||
},
|
||||
ContentBlockStop {
|
||||
index: u32,
|
||||
},
|
||||
Ping,
|
||||
Error {
|
||||
error: StreamError,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct MessageDeltaBody {
|
||||
pub stop_reason: Option<StopReason>,
|
||||
/// Provider detail for the stop; on `refusal`, `explanation` carries the
|
||||
/// reason the request was blocked (e.g. an Anthropic ToS auto-refusal).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub stop_details: Option<StopDetails>,
|
||||
}
|
||||
|
||||
/// Detail for a terminal `message_delta`, e.g.
|
||||
/// `{"type":"refusal","category":"frontier_llm","explanation":"..."}`.
|
||||
/// All fields optional so an unknown shape never fails the terminal parse.
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct StopDetails {
|
||||
#[serde(rename = "type", default)]
|
||||
pub r#type: Option<String>,
|
||||
#[serde(default)]
|
||||
pub category: Option<String>,
|
||||
#[serde(default)]
|
||||
pub explanation: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct MessageDeltaUsage {
|
||||
pub output_tokens: u32,
|
||||
#[serde(default)]
|
||||
pub input_tokens: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub cache_read_input_tokens: Option<u32>,
|
||||
#[serde(default)]
|
||||
pub cache_creation_input_tokens: Option<u32>,
|
||||
}
|
||||
|
||||
/// Content delta within a content_block_delta event
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "type", rename_all = "snake_case")]
|
||||
pub enum StreamDelta {
|
||||
TextDelta { text: String },
|
||||
InputJsonDelta { partial_json: String },
|
||||
ThinkingDelta { thinking: String },
|
||||
SignatureDelta { signature: String },
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct StreamError {
|
||||
#[serde(rename = "type")]
|
||||
pub r#type: String,
|
||||
pub message: String,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn stop_reason_deserializes_all_known_values_and_catches_unknown() {
|
||||
let parse = |raw: &str| -> StopReason {
|
||||
serde_json::from_str(&format!("\"{raw}\""))
|
||||
.unwrap_or_else(|e| panic!("stop_reason {raw:?} must parse: {e}"))
|
||||
};
|
||||
assert!(matches!(parse("end_turn"), StopReason::EndTurn));
|
||||
assert!(matches!(parse("max_tokens"), StopReason::MaxTokens));
|
||||
assert!(matches!(parse("tool_use"), StopReason::ToolUse));
|
||||
assert!(matches!(parse("stop_sequence"), StopReason::StopSequence));
|
||||
assert!(matches!(parse("refusal"), StopReason::Refusal));
|
||||
assert!(matches!(parse("pause_turn"), StopReason::PauseTurn));
|
||||
assert!(matches!(
|
||||
parse("model_context_window_exceeded"),
|
||||
StopReason::ModelContextWindowExceeded
|
||||
));
|
||||
match parse("some_future_stop_reason") {
|
||||
StopReason::Unknown(s) => assert_eq!(s, "some_future_stop_reason"),
|
||||
other => panic!("unknown value must preserve the wire string, got {other:?}"),
|
||||
}
|
||||
assert_eq!(
|
||||
serde_json::to_string(&StopReason::Unknown("some_future_stop_reason".into())).unwrap(),
|
||||
"\"some_future_stop_reason\"",
|
||||
"catch-all must re-serialize the wire string faithfully"
|
||||
);
|
||||
// The catch-all must also work through the Option<StopReason> field
|
||||
// it is parsed from in production.
|
||||
let delta: MessageDeltaBody =
|
||||
serde_json::from_str(r#"{"stop_reason":"mystery_reason"}"#).unwrap();
|
||||
match delta.stop_reason {
|
||||
Some(StopReason::Unknown(s)) => assert_eq!(s, "mystery_reason"),
|
||||
other => panic!("expected Unknown through Option, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// The terminal `message_delta` of a refusal-terminated stream must parse
|
||||
/// (the internally-tagged `MessageStreamEvent` wrapper is the actual
|
||||
/// production parse site, hence the full-event fixture).
|
||||
#[test]
|
||||
fn message_delta_with_refusal_stop_reason_parses() {
|
||||
let event: MessageStreamEvent = serde_json::from_str(
|
||||
r#"{"type":"message_delta","delta":{"stop_reason":"refusal"},"usage":{"output_tokens":5,"input_tokens":10}}"#,
|
||||
)
|
||||
.expect("refusal message_delta must deserialize");
|
||||
match event {
|
||||
MessageStreamEvent::MessageDelta { delta, usage } => {
|
||||
assert!(matches!(delta.stop_reason, Some(StopReason::Refusal)));
|
||||
assert!(delta.stop_details.is_none(), "no stop_details on the wire");
|
||||
assert_eq!(usage.output_tokens, 5);
|
||||
}
|
||||
other => panic!("expected MessageDelta, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// A refusal `message_delta` carrying `stop_details` (as emitted by
|
||||
/// Anthropic ToS auto-refusals) must parse and preserve the explanation,
|
||||
/// and unknown keys inside `stop_details` must not fail the parse.
|
||||
#[test]
|
||||
fn message_delta_with_refusal_stop_details_parses() {
|
||||
let event: MessageStreamEvent = serde_json::from_str(
|
||||
r#"{"type":"message_delta","delta":{"stop_reason":"refusal","stop_sequence":null,"stop_details":{"type":"refusal","category":"frontier_llm","explanation":"This request was blocked.","future_key":42}},"usage":{"output_tokens":0}}"#,
|
||||
)
|
||||
.expect("refusal message_delta with stop_details must deserialize");
|
||||
match event {
|
||||
MessageStreamEvent::MessageDelta { delta, .. } => {
|
||||
assert!(matches!(delta.stop_reason, Some(StopReason::Refusal)));
|
||||
let details = delta.stop_details.expect("stop_details must be captured");
|
||||
assert_eq!(details.r#type.as_deref(), Some("refusal"));
|
||||
assert_eq!(details.category.as_deref(), Some("frontier_llm"));
|
||||
assert_eq!(
|
||||
details.explanation.as_deref(),
|
||||
Some("This request was blocked.")
|
||||
);
|
||||
}
|
||||
other => panic!("expected MessageDelta, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn output_format_json_schema_wire_shape() {
|
||||
let fmt = OutputFormat::JsonSchema {
|
||||
schema: serde_json::json!({"type": "object", "properties": {"x": {"type": "string"}}}),
|
||||
};
|
||||
let json = serde_json::to_value(&fmt).unwrap();
|
||||
assert_eq!(json["type"], "json_schema");
|
||||
assert_eq!(json["schema"]["type"], "object");
|
||||
assert!(json.get("name").is_none());
|
||||
|
||||
let config = OutputConfig {
|
||||
effort: None,
|
||||
format: Some(fmt),
|
||||
};
|
||||
let json = serde_json::to_value(&config).unwrap();
|
||||
assert!(json.get("effort").is_none(), "effort omitted when None");
|
||||
assert_eq!(json["format"]["type"], "json_schema");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
use serde::{Deserialize, Deserializer};
|
||||
|
||||
pub fn empty_string_as_none<'de, D>(deserializer: D) -> Result<Option<String>, D::Error>
|
||||
where
|
||||
D: Deserializer<'de>,
|
||||
{
|
||||
let opt = Option::<String>::deserialize(deserializer)?;
|
||||
Ok(opt.filter(|s| !s.is_empty()))
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user