ZacharyZhang-NY/Kigi-CLI
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

M2 audit: excise the Computer Hub stack — Kigi's last remote-cloud surface

Browse Source 
Removed root-and-branch for the zero-egress guarantee (the hub was xAI's
remote-workspace/cloud-sandbox service):

- Crates deleted: kigi-computer-hub-core, kigi-computer-hub-sdk,
  kigi-computer-hub-mcp-adapter, kigi-workspace-client (hub-proxied
  workspace RPC client), and kigi-tracing (its sole network path was the
  OTLP gRPC exporter; zero consumers remained). kigi-tracing-macros
  (purely local) stays.
- kigi-workspace: every hub surface deleted — hub server/channel/auth,
  HITL-over-hub permissions, donation/metrics pumps, file upload RPCs,
  hub tool-snapshot merge (resolve pipeline is MCP-only now),
  WorkspaceOps::Proxy. Local worktrees, sessions, leader IPC, MCP, and
  the ACP permission prompt path are untouched; LocalRegistry re-homed
  into kigi-tool-runtime on the existing ToolDyn types so in-process
  tool dispatch is unchanged.
- kigi-shell: leader workspace-exposure control surface (incl. the
  wss://computer-hub... URL), [hub] config, ObservabilityBridge, hub
  WebSocket proxy, dead OTLP config knobs. ClientMode::Headless (never
  constructed) removed.
- kigi-tui/bin: hidden `kigi workspace` command removed (`kigi
  worktree` stays).
- Renames: --xai-api-base-url → --api-base-url / KIGI_API_BASE_URL /
  [endpoints] api_base_url (serde alias keeps old configs working; the
  flag feeds BYOK/custom-endpoint routing, not main inference);
  grok_version → kigi_version in inspect/models-cache/trace metadata
  (old caches self-heal via version-mismatch refetch).
- Dependency tree: dropped fastrace*, opentelemetry-otlp/http/proto,
  tokio-tungstenite from the workspace; fixed the 4 real useless_format
  violations the fastrace lint allowance was masking and removed the
  allowance.
- marketplaceAllowlist kept: it gates the LOCAL plugin-marketplace
  feature, not an xAI service.

Known §9 leftover (deliberate, for the M3 sweep): the BYOK default base
URL string. Gates: workspace check/clippy 0/0, fmt, deny ok; suites
green (workspace 1042, shell 4918, tui 6634, tools 2608, tool-runtime
47, mcp 154).
This commit is contained in:
ZacharyZhang-NY
2026-07-17 22:34:10 -04:00
parent 5919526e91
commit fa75eb139a
90 changed files with 452 additions and 9702 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/codegen/kigi-shell/src/agent/app.rs
+1 -7
View File
@@ -518,11 +518,7 @@ pub async fn run_leader(
lock_path: lock.lock_path().clone(),
socket_suffix: socket_suffix_from_paths(lock.lock_path(), &socket_path).unwrap_or_default(),
leader_binary_version: kigi_version::VERSION.to_string(),
})
.with_default_hub_url(agent_config.hub.url.clone());
// Cloned before control_state moves into the IPC server; auth wired below.
let workspace_control = control_state.workspace.clone();
});
// ── Phase 3: Bind socket and start IPC server (BEFORE auth/prefetch) ──────
//
@@ -654,8 +650,6 @@ pub async fn run_leader(
// process so a refresh can't straddle a suspend.
shared_auth_manager.start_system_power_listener();
// Same manager as the leader, so the exposure never writes auth.json itself.
workspace_control.set_auth_manager(shared_auth_manager.clone());
let auth_manager_for_agent = shared_auth_manager.clone();
let auth_manager_for_config = shared_auth_manager;
crates/codegen/kigi-shell/src/agent/config.rs
+19 -588
View File
@@ -43,7 +43,7 @@ pub fn default_agent_type() -> String {
DEFAULT_AGENT_TYPE.to_owned()
}
/// Default base URL for the public xAI API.
pub const XAI_API_BASE_URL_DEFAULT: &str = "https://api.x.ai/v1";
pub const API_BASE_URL_DEFAULT: &str = "https://api.x.ai/v1";
/// One or more environment variable names that may hold a model API key.
///
/// Serde `untagged`: accepts a string or an array in TOML/JSON.
@@ -142,8 +142,10 @@ pub struct EndpointsConfig {
/// default value) lets an org pin the proxy to the default on purpose.
#[serde(skip_serializing_if = "Option::is_none")]
pub coding_api_base_url: Option<String>,
/// Base URL for the public xAI API.
pub xai_api_base_url: String,
/// Base URL for direct (BYOK / external-API-key) API calls.
/// Accepts the legacy `xai_api_base_url` config key.
#[serde(alias = "xai_api_base_url")]
pub api_base_url: String,
/// Optional extra access-header value (applied only with the optional
/// non-production feature, and only for matching first-party hosts).
#[serde(skip_serializing_if = "Option::is_none")]
@@ -166,54 +168,6 @@ pub struct EndpointsConfig {
/// Defaults to `{proxy_url()}/deployment/config`.
#[serde(skip_serializing_if = "Option::is_none")]
pub managed_config_url: Option<String>,
/// Env: `OTEL_EXPORTER_OTLP_ENDPOINT`. OTLP collector base; `/v1/traces` is
/// appended. Legacy repoint of the INTERNAL trace pipeline — deprecated in
/// favor of `KIGI_INTERNAL_OTLP_TRACES_ENDPOINT`, and ignored by the internal
/// pipeline when `KIGI_EXTERNAL_OTEL` is set (the standard `OTEL_*` vars then
/// route the external stream only).
#[serde(skip_serializing_if = "Option::is_none")]
pub otel_exporter_otlp_endpoint: Option<String>,
/// Env: `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`. Full traces endpoint, used
/// verbatim; overrides `otel_exporter_otlp_endpoint`. Same legacy/deprecation
/// semantics as `otel_exporter_otlp_endpoint`.
#[serde(skip_serializing_if = "Option::is_none")]
pub otel_exporter_otlp_traces_endpoint: Option<String>,
/// Env: `OTEL_EXPORTER_OTLP_HEADERS`. `k=v,k2=v2`; merged onto export headers.
/// Same legacy/deprecation semantics as `otel_exporter_otlp_endpoint`.
#[serde(skip_serializing_if = "Option::is_none")]
pub otel_exporter_otlp_headers: Option<String>,
/// Env: `KIGI_INTERNAL_OTLP_TRACES_ENDPOINT`. Full INTERNAL traces endpoint,
/// used verbatim. Dev/debug repoint of the internal span firehose (replaces
/// the legacy `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` behavior; used by
/// local-ic-testing / internal dev flows). Wins over the legacy `OTEL_*` vars.
#[serde(skip_serializing_if = "Option::is_none")]
pub grok_internal_otlp_traces_endpoint: Option<String>,
/// Env: `KIGI_INTERNAL_OTLP_HEADERS`. `k=v,k2=v2` extra headers for the
/// internal export (debug). Wins over the legacy `OTEL_EXPORTER_OTLP_HEADERS`.
#[serde(skip_serializing_if = "Option::is_none")]
pub grok_internal_otlp_headers: Option<String>,
/// External-OTEL master switch, captured at construction via
/// [`external_otel_master_switch_resolved`] — the same layered resolution
/// (requirement pin > `KIGI_EXTERNAL_OTEL` env > `[telemetry].otel_enabled`
/// config, managed layers included) that activates the external stream.
/// When set, the standard `OTEL_EXPORTER_OTLP_*` vars are reserved for the
/// external OTEL stream and the internal trace pipeline ignores them
/// entirely — an admin who opts in (by *any* layer, including an org
/// enable distributed via managed config with no env var) never receives
/// the internally-authed firehose. Held as a field (not re-read in the
/// resolvers) so the resolvers stay pure and testable without env races.
#[serde(skip)]
pub external_otel_master_switch: bool,
/// Env: `OTEL_TRACES_EXPORTER`. `otlp` (default) or `none` to disable spans.
#[serde(skip_serializing_if = "Option::is_none")]
pub otel_traces_exporter: Option<String>,
/// Env: `OTEL_BSP_SCHEDULE_DELAY` (OTel) or `OTEL_TRACES_EXPORT_INTERVAL`
/// (Claude alias). Batch flush interval (ms).
#[serde(skip_serializing_if = "Option::is_none")]
pub otel_traces_export_interval: Option<u64>,
/// Env: `OTEL_EXPORTER_OTLP_TIMEOUT`. Export HTTP timeout (ms).
#[serde(skip_serializing_if = "Option::is_none")]
pub otel_exporter_otlp_timeout: Option<u64>,
/// Read by `load_management_api_key_sync()`. Declared for `serde_ignored`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub management_api_key: Option<String>,
@@ -228,18 +182,6 @@ fn blank_as_unset(opt: &Option<String>) -> Option<String> {
.filter(|s| !s.trim().is_empty())
.map(str::to_owned)
}
/// Parse a `k=v,k2=v2` OTLP header list (the `OTEL_EXPORTER_OTLP_HEADERS`
/// format, shared with `KIGI_INTERNAL_OTLP_HEADERS`): split on `,`,
/// `split_once('=')`, trim key/value, skip blank keys, keep empty values.
fn parse_otlp_header_list(raw: &str) -> Vec<(String, String)> {
raw.split(',')
.filter_map(|kv| {
let (k, v) = kv.split_once('=')?;
let k = k.trim();
(!k.is_empty()).then(|| (k.to_string(), v.trim().to_string()))
})
.collect()
}
impl EndpointsConfig {
pub fn has_custom_endpoint(&self) -> bool {
self.models_base_url.is_some() || self.models_list_url.is_some()
@@ -257,22 +199,18 @@ impl EndpointsConfig {
/// Layer the `[endpoints]` table from `config` over the env/default base.
/// No field is derived from another — defaulting is done by the resolvers.
pub(crate) fn from_config_value(config: &toml::Value) -> Self {
let default = Self::default();
let external_otel_master_switch = default.external_otel_master_switch;
let mut base = match toml::Value::try_from(default) {
let mut base = match toml::Value::try_from(Self::default()) {
Ok(v) => v,
Err(_) => return Self::default(),
};
if let Some(endpoints) = config.get("endpoints") {
crate::config::deep_merge_toml(&mut base, endpoints);
}
let mut resolved: Self = base.try_into().unwrap_or_default();
resolved.external_otel_master_switch = external_otel_master_switch;
resolved
base.try_into().unwrap_or_default()
}
/// The subscription proxy base URL through which all auxiliary services (and
/// OAuth/session inference) resolve: explicit `coding_api_base_url`, else
/// [`kigi_env::coding_api_base_url`]. NEVER falls back to `xai_api_base_url` —
/// [`kigi_env::coding_api_base_url`]. NEVER falls back to `api_base_url` —
/// that is the inference endpoint (API-key auth) only.
pub fn proxy_url(&self) -> String {
blank_as_unset(&self.coding_api_base_url).unwrap_or_else(kigi_env::coding_api_base_url)
@@ -283,12 +221,12 @@ impl EndpointsConfig {
.unwrap_or_else(|| self.proxy_url())
}
/// Feedback endpoint — an auxiliary service, so it defaults to the
/// cli-chat-proxy, never `xai_api_base_url`.
/// cli-chat-proxy, never `api_base_url`.
pub fn resolve_feedback_base_url(&self) -> String {
blank_as_unset(&self.feedback_base_url).unwrap_or_else(|| self.proxy_url())
}
/// Managed deployment-config URL (`grok setup`): explicit `managed_config_url`,
/// else `proxy_url` + `/deployment/config`. Never `xai_api_base_url`, so the
/// else `proxy_url` + `/deployment/config`. Never `api_base_url`, so the
/// deployment key reaches the proxy, not the inference host.
pub fn resolve_managed_config_url(&self) -> String {
blank_as_unset(&self.managed_config_url).unwrap_or_else(|| {
@@ -298,99 +236,6 @@ impl EndpointsConfig {
)
})
}
/// INTERNAL OTLP traces endpoint. Precedence:
/// 1. `grok_internal_otlp_traces_endpoint` (verbatim)
/// 2. legacy `otel_exporter_otlp_traces_endpoint` (verbatim) >
/// `otel_exporter_otlp_endpoint` + `/v1/traces` — ONLY when the
/// external-OTEL master switch is unset (back-compat; deprecated)
/// 3. `proxy_url` + `/traces`.
/// Uses the proxy default (not the `xai_api_base_url` fallback) so
/// telemetry reports to xAI even when inference is overridden. When the
/// master switch IS set, the standard `OTEL_EXPORTER_OTLP_*` values are
/// completely ignored here so the internally-authed firehose never lands
/// at an external collector.
pub fn resolve_otlp_traces_endpoint(&self) -> String {
if let Some(full) = blank_as_unset(&self.grok_internal_otlp_traces_endpoint) {
return full.trim_end_matches('/').to_string();
}
if !self.external_otel_master_switch
&& let Some(legacy) = self.legacy_internal_otlp_traces_endpoint()
{
tracing::warn!(
"Repointing the internal trace pipeline via OTEL_EXPORTER_OTLP_ENDPOINT / \
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT is deprecated; use \
KIGI_INTERNAL_OTLP_TRACES_ENDPOINT instead — the standard OTEL_* vars will \
route the external OTEL stream only in a future release"
);
return legacy;
}
format!("{}/traces", self.proxy_url().trim_end_matches('/'))
}
/// Legacy (standard-OTEL-var) internal traces endpoint, if any:
/// `otel_exporter_otlp_traces_endpoint` verbatim, else
/// `otel_exporter_otlp_endpoint` + `/v1/traces`. Ignores the master switch.
fn legacy_internal_otlp_traces_endpoint(&self) -> Option<String> {
if let Some(full) = blank_as_unset(&self.otel_exporter_otlp_traces_endpoint) {
return Some(full.trim_end_matches('/').to_string());
}
blank_as_unset(&self.otel_exporter_otlp_endpoint)
.map(|base| format!("{}/v1/traces", base.trim_end_matches('/')))
}
/// Extra headers for the INTERNAL export: `grok_internal_otlp_headers`
/// first; legacy fallback to `otel_exporter_otlp_headers` ONLY when the
/// external-OTEL master switch is unset (back-compat for existing users).
pub fn resolve_otlp_headers(&self) -> Vec<(String, String)> {
if let Some(headers) = blank_as_unset(&self.grok_internal_otlp_headers) {
return parse_otlp_header_list(&headers);
}
if !self.external_otel_master_switch {
return parse_otlp_header_list(
self.otel_exporter_otlp_headers.as_deref().unwrap_or(""),
);
}
Vec::new()
}
/// Whether the legacy fallback actually supplied the internal endpoint OR
/// internal headers from the standard `OTEL_EXPORTER_OTLP_*` vars — i.e.
/// the master switch is unset AND (`otel_exporter_otlp_traces_endpoint` /
/// `otel_exporter_otlp_endpoint` is non-blank for the endpoint, or
/// `otel_exporter_otlp_headers` is non-blank for headers) AND no
/// `grok_internal_otlp_*` override shadowed that half.
///
/// CONTRACT: this flag is passed to the external OTEL stream's init, which
/// MUST refuse to activate when it is true — the same standard vars cannot
/// feed both pipelines (no-double-send invariant, enforced in code).
pub fn internal_otlp_consumed_standard_vars(&self) -> bool {
if self.external_otel_master_switch {
return false;
}
let endpoint_consumed = blank_as_unset(&self.grok_internal_otlp_traces_endpoint).is_none()
&& self.legacy_internal_otlp_traces_endpoint().is_some();
let headers_consumed = blank_as_unset(&self.grok_internal_otlp_headers).is_none()
&& blank_as_unset(&self.otel_exporter_otlp_headers).is_some();
endpoint_consumed || headers_consumed
}
/// Trace export enabled unless `OTEL_TRACES_EXPORTER=none`. Deliberately
/// still honored by the internal pipeline even with `KIGI_EXTERNAL_OTEL`
/// set: disabling internal span export is the safe direction.
pub fn resolve_traces_export_enabled(&self) -> bool {
!matches!(
self.otel_traces_exporter.as_deref().map(str::trim),
Some("none")
)
}
/// `OTEL_BSP_SCHEDULE_DELAY` / `OTEL_TRACES_EXPORT_INTERVAL` — tuning-only,
/// deliberately shared between the internal and external pipelines.
pub fn resolve_otlp_export_interval(&self) -> Option<std::time::Duration> {
self.otel_traces_export_interval
.map(std::time::Duration::from_millis)
}
/// `OTEL_EXPORTER_OTLP_TIMEOUT` — tuning-only, deliberately shared between
/// the internal and external pipelines.
pub fn resolve_otlp_timeout(&self) -> Option<std::time::Duration> {
self.otel_exporter_otlp_timeout
.map(std::time::Duration::from_millis)
}
/// `models_list_url` > `{models_base_url}/models` > `{proxy_base_url}/models`.
pub fn resolve_models_list_url(&self) -> String {
if let Some(ref url) = self.models_list_url {
@@ -407,26 +252,14 @@ impl Default for EndpointsConfig {
fn default() -> Self {
Self {
coding_api_base_url: std::env::var("KIGI_CODE_BASE_URL").ok(),
xai_api_base_url: std::env::var("KIGI_XAI_API_BASE_URL")
.unwrap_or_else(|_| XAI_API_BASE_URL_DEFAULT.to_owned()),
api_base_url: std::env::var("KIGI_API_BASE_URL")
.unwrap_or_else(|_| API_BASE_URL_DEFAULT.to_owned()),
alpha_test_key: None,
models_base_url: env_string("KIGI_MODELS_BASE_URL"),
models_list_url: env_string("KIGI_MODELS_LIST_URL"),
feedback_base_url: env_string("KIGI_FEEDBACK_BASE_URL"),
deployment_key: env_string("KIGI_DEPLOYMENT_KEY"),
managed_config_url: env_string("KIGI_MANAGED_CONFIG_URL"),
otel_exporter_otlp_endpoint: env_string("OTEL_EXPORTER_OTLP_ENDPOINT"),
otel_exporter_otlp_traces_endpoint: env_string("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT"),
otel_exporter_otlp_headers: env_string("OTEL_EXPORTER_OTLP_HEADERS"),
grok_internal_otlp_traces_endpoint: env_string("KIGI_INTERNAL_OTLP_TRACES_ENDPOINT"),
grok_internal_otlp_headers: env_string("KIGI_INTERNAL_OTLP_HEADERS"),
external_otel_master_switch: external_otel_master_switch_resolved(),
otel_traces_exporter: env_string("OTEL_TRACES_EXPORTER"),
otel_traces_export_interval: env_string("OTEL_BSP_SCHEDULE_DELAY")
.or_else(|| env_string("OTEL_TRACES_EXPORT_INTERVAL"))
.and_then(|s| s.parse().ok()),
otel_exporter_otlp_timeout: env_string("OTEL_EXPORTER_OTLP_TIMEOUT")
.and_then(|s| s.parse().ok()),
management_api_key: None,
gcs_service_account_key: None,
}
@@ -1005,30 +838,6 @@ pub struct RemoteConfig {
#[serde(skip_serializing_if = "Option::is_none")]
pub secret: Option<String>,
}
/// `[hub]` section from config.toml.
///
/// Optional default Computer Hub URL for **workspace provider** exposure
/// (`grok workspace` / leader `with_default_hub_url`). Does **not** enable
/// agent-side harness/client connections or alter local session behavior.
///
/// ```toml
/// [hub]
/// url = "wss://hub.x.ai/ws"
/// ```
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct HubConfig {
/// Hub WebSocket URL (`ws://` or `wss://`) used as the leader default for
/// `grok workspace start` when the CLI does not pass `--hub-url`.
#[serde(skip_serializing_if = "Option::is_none")]
pub url: Option<String>,
}
impl HubConfig {
/// Whether a non-empty hub URL is configured (workspace default only).
pub fn is_enabled(&self) -> bool {
self.url.as_ref().is_some_and(|u| !u.trim().is_empty())
}
}
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
#[serde(default)]
pub struct WorktreePoolConfig {
@@ -1250,9 +1059,6 @@ pub struct Config {
pub harness: HarnessConfig,
#[serde(default, skip_serializing)]
pub remote: RemoteConfig,
/// Computer Hub configuration (`[hub]` in config.toml).
#[serde(default, skip_serializing)]
pub hub: HubConfig,
#[serde(default, skip_serializing)]
pub worktree_pool: WorktreePoolConfig,
#[serde(default, skip_serializing)]
@@ -1632,7 +1438,6 @@ impl Default for Config {
platforms: PlatformsConfig::default(),
harness: HarnessConfig::default(),
remote: RemoteConfig::default(),
hub: HubConfig::default(),
worktree_pool: WorktreePoolConfig::default(),
sandbox: SandboxSettingsConfig::default(),
mcp_servers: std::collections::HashMap::new(),
@@ -2703,41 +2508,6 @@ pub(crate) fn read_requirements_toml() -> Option<toml::Value> {
pub fn deployment_id_from_key(key: &str) -> String {
uuid::Uuid::new_v5(&uuid::Uuid::NAMESPACE_OID, key.as_bytes()).to_string()
}
/// Resolve the external-OTEL master switch exactly the way the external
/// stream's activation does: **requirement pin > `KIGI_EXTERNAL_OTEL` env >
/// `[telemetry].otel_enabled` config layer (managed config included) > off**.
///
/// The internal trace pipeline keys its "ignore `OTEL_EXPORTER_OTLP_*`"
/// behavior off this value ([`EndpointsConfig::external_otel_master_switch`]),
/// so an org enable distributed via managed config / requirements (no env
/// var) flips **both** sides together. A desync here would leave the
/// internally-authed firehose honoring legacy `OTEL_*` repointing while
/// `internal_pipeline_consumed_otel_vars` simultaneously blocks the external
/// stream — exactly the split this design forbids.
pub(crate) fn external_otel_master_switch_resolved() -> bool {
external_otel_master_switch_from(
kigi_config::load_merged_requirements().as_ref(),
env_bool("KIGI_EXTERNAL_OTEL"),
crate::config::load_effective_config().ok().as_ref(),
)
}
/// Testable core of [`external_otel_master_switch_resolved`].
pub(crate) fn external_otel_master_switch_from(
requirements: Option<&toml::Value>,
env_switch: Option<bool>,
effective_config: Option<&toml::Value>,
) -> bool {
let table_enabled = |v: Option<&toml::Value>| -> Option<bool> {
v?.get("telemetry")?.get("otel_enabled")?.as_bool()
};
if let Some(pinned) = table_enabled(requirements) {
return pinned;
}
if let Some(env) = env_switch {
return env;
}
table_enabled(effective_config).unwrap_or(false)
}
/// Seed free-function remote caches after writing `Config.remote_settings`.
pub fn apply_remote_settings_side_effects(settings: Option<&crate::util::config::RemoteSettings>) {
crate::util::config::cache_remote_mcp_startup_timeout_secs(
@@ -4983,7 +4753,7 @@ reasoning_effort = "low"
auth_scheme: AuthScheme::Bearer,
};
assert_eq!(
api_key_creds.base_url, endpoints.xai_api_base_url,
api_key_creds.base_url, endpoints.api_base_url,
"{model_id}: ExternalApiKey must route to api.x.ai"
);
}
@@ -6890,24 +6660,18 @@ reasoning_effort = "low"
for k in [
"KIGI_CODE_BASE_URL",
kigi_env::CODE_BASE_URL_ENV,
"KIGI_XAI_API_BASE_URL",
"KIGI_API_BASE_URL",
"KIGI_FEEDBACK_BASE_URL",
"KIGI_TRACE_UPLOAD_URL",
"KIGI_MANAGED_CONFIG_URL",
"KIGI_MODELS_BASE_URL",
"KIGI_MODELS_LIST_URL",
"OTEL_EXPORTER_OTLP_ENDPOINT",
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
"OTEL_EXPORTER_OTLP_HEADERS",
"KIGI_INTERNAL_OTLP_TRACES_ENDPOINT",
"KIGI_INTERNAL_OTLP_HEADERS",
"KIGI_EXTERNAL_OTEL",
] {
unsafe { std::env::remove_var(k) };
}
}
/// INVARIANT: auxiliary-service resolvers resolve to the cli-chat-proxy, never
/// `xai_api_base_url` — overriding ONLY inference keeps every aux endpoint on
/// `api_base_url` — overriding ONLY inference keeps every aux endpoint on
/// the proxy; explicit per-service overrides win verbatim.
#[test]
#[serial]
@@ -6915,7 +6679,7 @@ reasoning_effort = "low"
unset_endpoint_env_vars();
let inference = "https://inference.acme-corp.example/xai/v1";
let cfg = EndpointsConfig {
xai_api_base_url: inference.to_string(),
api_base_url: inference.to_string(),
coding_api_base_url: None,
..Default::default()
};
@@ -6928,11 +6692,7 @@ reasoning_effort = "low"
format!("{proxy}/deployment/config")
);
assert_eq!(cfg.resolve_feedback_base_url(), proxy);
assert_eq!(
cfg.resolve_otlp_traces_endpoint(),
format!("{proxy}/traces")
);
assert_eq!(cfg.xai_api_base_url, inference);
assert_eq!(cfg.api_base_url, inference);
let overridden = EndpointsConfig {
coding_api_base_url: Some("https://proxy.enterprise.example/v1".to_string()),
managed_config_url: Some(
@@ -6945,10 +6705,6 @@ reasoning_effort = "low"
overridden.proxy_url(),
"https://proxy.enterprise.example/v1"
);
assert_eq!(
overridden.resolve_otlp_traces_endpoint(),
"https://proxy.enterprise.example/v1/traces"
);
assert_eq!(
overridden.resolve_managed_config_url(),
"https://control.enterprise.example/deployment/config"
@@ -6958,7 +6714,7 @@ reasoning_effort = "low"
"https://feedback.enterprise.example"
);
}
/// REGRESSION: the managed-config URL never follows `xai_api_base_url`
/// REGRESSION: the managed-config URL never follows `api_base_url`
/// through the full loader `Config::new_from_toml_cfg` — a distinct construction
/// path from `from_config_value`, so the deployment key never reaches the
/// inference host on either.
@@ -6969,7 +6725,7 @@ reasoning_effort = "low"
let cfg = Config::new_from_toml_cfg(
&toml::from_str(
r#"[endpoints]
xai_api_base_url = "https://inference.acme-corp.example/xai/v1""#,
api_base_url = "https://inference.acme-corp.example/xai/v1""#,
)
.unwrap(),
)
@@ -8467,311 +8223,6 @@ agent_type = "cursor"
"exactly the typo'd key must be flagged"
);
}
#[test]
fn otlp_traces_endpoint_precedence() {
let proxy = "https://inference.acme.com/v1".to_string();
let derived = EndpointsConfig {
coding_api_base_url: Some(proxy.clone()),
..Default::default()
};
assert_eq!(
derived.resolve_otlp_traces_endpoint(),
"https://inference.acme.com/v1/traces"
);
let base = EndpointsConfig {
coding_api_base_url: Some(proxy.clone()),
otel_exporter_otlp_endpoint: Some("https://otel.acme.com".to_string()),
..Default::default()
};
assert_eq!(
base.resolve_otlp_traces_endpoint(),
"https://otel.acme.com/v1/traces"
);
let full = EndpointsConfig {
coding_api_base_url: Some(proxy),
otel_exporter_otlp_endpoint: Some("https://ignored.example".to_string()),
otel_exporter_otlp_traces_endpoint: Some("https://otel.acme.com/v1/traces".to_string()),
..Default::default()
};
assert_eq!(
full.resolve_otlp_traces_endpoint(),
"https://otel.acme.com/v1/traces"
);
}
#[test]
fn otlp_headers_parse() {
let cfg = EndpointsConfig {
otel_exporter_otlp_headers: Some("a=1, b = 2 ,=skip,c=".to_string()),
..Default::default()
};
assert_eq!(
cfg.resolve_otlp_headers(),
vec![
("a".to_string(), "1".to_string()),
("b".to_string(), "2".to_string()),
("c".to_string(), String::new()),
]
);
}
/// Base config for the internal-OTLP tests: pinned proxy, every OTLP knob
/// explicitly unset so ambient env (via `Default`) can't leak in.
fn internal_otlp_test_config() -> EndpointsConfig {
EndpointsConfig {
coding_api_base_url: Some("https://proxy.example/v1".to_string()),
otel_exporter_otlp_endpoint: None,
otel_exporter_otlp_traces_endpoint: None,
otel_exporter_otlp_headers: None,
grok_internal_otlp_traces_endpoint: None,
grok_internal_otlp_headers: None,
external_otel_master_switch: false,
..Default::default()
}
}
/// `grok_internal_otlp_traces_endpoint` wins over the legacy `OTEL_*`
/// fields regardless of the master switch.
#[test]
fn internal_otlp_endpoint_grok_internal_wins_regardless_of_switch() {
for switch in [false, true] {
let cfg = EndpointsConfig {
grok_internal_otlp_traces_endpoint: Some(
"https://internal.example/traces/".to_string(),
),
otel_exporter_otlp_traces_endpoint: Some(
"https://legacy.example/v1/traces".to_string(),
),
otel_exporter_otlp_endpoint: Some("https://legacy-base.example".to_string()),
external_otel_master_switch: switch,
..internal_otlp_test_config()
};
assert_eq!(
cfg.resolve_otlp_traces_endpoint(),
"https://internal.example/traces",
"switch={switch}: KIGI_INTERNAL_OTLP_TRACES_ENDPOINT must win verbatim (trailing / trimmed)"
);
}
}
/// Master switch unset → legacy fallback preserved (back-compat).
#[test]
fn internal_otlp_endpoint_legacy_fallback_when_switch_unset() {
let traces = EndpointsConfig {
otel_exporter_otlp_traces_endpoint: Some(
"https://legacy.example/v1/traces".to_string(),
),
..internal_otlp_test_config()
};
assert_eq!(
traces.resolve_otlp_traces_endpoint(),
"https://legacy.example/v1/traces"
);
let base = EndpointsConfig {
otel_exporter_otlp_endpoint: Some("https://legacy-base.example/".to_string()),
..internal_otlp_test_config()
};
assert_eq!(
base.resolve_otlp_traces_endpoint(),
"https://legacy-base.example/v1/traces"
);
}
/// Master switch SET → legacy `OTEL_*` endpoint/headers are completely
/// ignored by the internal pipeline (the external stream owns them); the
/// internal pipeline falls back to the proxy default and
/// `internal_otlp_consumed_standard_vars()` is false.
#[test]
fn internal_otlp_ignores_legacy_vars_when_switch_set() {
let cfg = EndpointsConfig {
otel_exporter_otlp_traces_endpoint: Some(
"https://admin-collector.example/v1/traces".to_string(),
),
otel_exporter_otlp_endpoint: Some("https://admin-collector.example".to_string()),
otel_exporter_otlp_headers: Some("authorization=Bearer admin".to_string()),
external_otel_master_switch: true,
..internal_otlp_test_config()
};
assert_eq!(
cfg.resolve_otlp_traces_endpoint(),
"https://proxy.example/v1/traces",
"internal firehose must never follow OTEL_* to the external collector"
);
assert_eq!(cfg.resolve_otlp_headers(), Vec::<(String, String)>::new());
assert!(!cfg.internal_otlp_consumed_standard_vars());
}
/// `internal_otlp_consumed_standard_vars()` truth table.
#[test]
fn internal_otlp_consumed_standard_vars_cases() {
struct Case {
switch: bool,
legacy_traces_ep: bool,
legacy_base_ep: bool,
legacy_headers: bool,
internal_ep: bool,
internal_headers: bool,
expected: bool,
why: &'static str,
}
let unset = Case {
switch: false,
legacy_traces_ep: false,
legacy_base_ep: false,
legacy_headers: false,
internal_ep: false,
internal_headers: false,
expected: false,
why: "nothing set",
};
let cases = [
Case { ..unset },
Case {
legacy_traces_ep: true,
expected: true,
why: "legacy traces endpoint consumed",
..unset
},
Case {
legacy_base_ep: true,
expected: true,
why: "legacy base endpoint consumed",
..unset
},
Case {
legacy_headers: true,
expected: true,
why: "legacy headers consumed",
..unset
},
Case {
legacy_traces_ep: true,
internal_ep: true,
expected: false,
why: "internal endpoint shadows legacy",
..unset
},
Case {
legacy_headers: true,
internal_headers: true,
expected: false,
why: "internal headers shadow legacy",
..unset
},
Case {
legacy_traces_ep: true,
legacy_headers: true,
internal_ep: true,
expected: true,
why: "endpoint shadowed but legacy headers still consumed (headers half)",
..unset
},
Case {
switch: true,
legacy_traces_ep: true,
legacy_base_ep: true,
legacy_headers: true,
expected: false,
why: "switch set: legacy vars ignored",
..unset
},
];
for case in cases {
let cfg = EndpointsConfig {
external_otel_master_switch: case.switch,
otel_exporter_otlp_traces_endpoint: case
.legacy_traces_ep
.then(|| "https://legacy.example/v1/traces".to_string()),
otel_exporter_otlp_endpoint: case
.legacy_base_ep
.then(|| "https://legacy-base.example".to_string()),
otel_exporter_otlp_headers: case.legacy_headers.then(|| "k=v".to_string()),
grok_internal_otlp_traces_endpoint: case
.internal_ep
.then(|| "https://internal.example/traces".to_string()),
grok_internal_otlp_headers: case.internal_headers.then(|| "ik=iv".to_string()),
..internal_otlp_test_config()
};
assert_eq!(
cfg.internal_otlp_consumed_standard_vars(),
case.expected,
"case: {}",
case.why
);
}
}
/// Headers precedence: `grok_internal_otlp_headers` wins; legacy
/// `otel_exporter_otlp_headers` only when the master switch is unset.
#[test]
fn internal_otlp_headers_precedence() {
for switch in [false, true] {
let cfg = EndpointsConfig {
grok_internal_otlp_headers: Some("x-debug=1".to_string()),
otel_exporter_otlp_headers: Some("legacy=1".to_string()),
external_otel_master_switch: switch,
..internal_otlp_test_config()
};
assert_eq!(
cfg.resolve_otlp_headers(),
vec![("x-debug".to_string(), "1".to_string())],
"switch={switch}"
);
}
let legacy = EndpointsConfig {
otel_exporter_otlp_headers: Some("legacy=1".to_string()),
..internal_otlp_test_config()
};
assert_eq!(
legacy.resolve_otlp_headers(),
vec![("legacy".to_string(), "1".to_string())]
);
}
/// Regression: an org enable via `[telemetry].otel_enabled`
/// (managed config / requirements — no `KIGI_EXTERNAL_OTEL` env var) must
/// flip the master switch the *internal* pipeline keys off, so legacy
/// `OTEL_EXPORTER_OTLP_*` repointing shuts off in lockstep with the
/// external stream activating. A desync would point the internally-authed
/// firehose at the customer collector while
/// `internal_pipeline_consumed_otel_vars` blocks the external stream.
#[test]
fn external_otel_master_switch_resolves_from_all_layers() {
let enabled_table: toml::Value =
toml::from_str("[telemetry]\notel_enabled = true").unwrap();
let disabled_table: toml::Value =
toml::from_str("[telemetry]\notel_enabled = false").unwrap();
assert!(external_otel_master_switch_from(
None,
None,
Some(&enabled_table)
));
assert!(!external_otel_master_switch_from(None, None, None));
assert!(!external_otel_master_switch_from(
None,
Some(false),
Some(&enabled_table)
));
assert!(external_otel_master_switch_from(
None,
Some(true),
Some(&disabled_table)
));
assert!(!external_otel_master_switch_from(
Some(&disabled_table),
Some(true),
Some(&enabled_table)
));
assert!(external_otel_master_switch_from(
Some(&enabled_table),
Some(false),
None
));
let cfg = EndpointsConfig {
otel_exporter_otlp_traces_endpoint: Some(
"https://collector.corp:4318/v1/traces".into(),
),
external_otel_master_switch: true,
..internal_otlp_test_config()
};
assert!(!cfg.internal_otlp_consumed_standard_vars());
assert!(
!cfg.resolve_otlp_traces_endpoint()
.contains("collector.corp")
);
}
fn empty_config() -> toml::Value {
toml::Value::Table(toml::map::Map::new())
}
@@ -9820,26 +9271,6 @@ default = "grok-4.5"
}
}
#[test]
fn hub_config_default_has_no_url() {
assert!(HubConfig::default().url.is_none());
assert!(!HubConfig::default().is_enabled());
}
#[test]
fn hub_config_is_enabled_only_for_nonempty_url() {
assert!(
HubConfig {
url: Some("wss://hub.example/ws".into()),
}
.is_enabled()
);
assert!(
!HubConfig {
url: Some(" ".into()),
}
.is_enabled()
);
}
#[test]
fn resolve_model_list_prunes_bundled_entries_not_in_prefetch() {
let cfg = Config::default();
let mut defs = default_model_entries(&EndpointsConfig::default());
crates/codegen/kigi-shell/src/agent/mod.rs
-1
View File
@@ -12,7 +12,6 @@ pub mod init;
pub mod models;
pub(crate) mod models_fetch;
pub mod mvp_agent;
pub(crate) mod proxy;
pub(crate) mod restore_code;
pub mod roster;
pub mod server;
crates/codegen/kigi-shell/src/agent/models.rs
+7 -7
View File
@@ -1323,7 +1323,7 @@ const CACHE_TTL: std::time::Duration = std::time::Duration::from_secs(300);
struct ModelsCache {
fetched_at: DateTime<Utc>,
#[serde(default, skip_serializing_if = "Option::is_none")]
grok_version: Option<String>,
kigi_version: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
auth_method: Option<CacheAuthMethod>,
/// Models-list URL this catalog was fetched from
@@ -1418,7 +1418,7 @@ impl ModelsCacheManager {
) -> Option<ModelsCache> {
let data = std::fs::read(&self.path).ok()?;
let cache: ModelsCache = serde_json::from_slice(&data).ok()?;
if cache.grok_version.as_deref() != Some(kigi_version::VERSION) {
if cache.kigi_version.as_deref() != Some(kigi_version::VERSION) {
tracing::debug!("models cache version mismatch");
return None;
}
@@ -1447,7 +1447,7 @@ impl ModelsCacheManager {
) {
let cache = ModelsCache {
fetched_at: Utc::now(),
grok_version: Some(kigi_version::VERSION.to_string()),
kigi_version: Some(kigi_version::VERSION.to_string()),
auth_method: Some(auth_method),
origin: Some(origin.to_string()),
etag: etag.map(|s| s.to_string()),
@@ -3064,7 +3064,7 @@ mod tests {
let auth_method = mgr.inner.fetch_auth.read().cache_auth_method();
let stale = ModelsCache {
fetched_at: Utc::now() - ChronoDuration::seconds(3600),
grok_version: Some(kigi_version::VERSION.to_string()),
kigi_version: Some(kigi_version::VERSION.to_string()),
auth_method: Some(auth_method),
origin: Some(mgr.cache_origin()),
etag: Some("etag-stale".into()),
@@ -3140,7 +3140,7 @@ mod tests {
let auth_method = mgr.inner.fetch_auth.read().cache_auth_method();
let legacy = ModelsCache {
fetched_at: Utc::now(),
grok_version: Some(kigi_version::VERSION.to_string()),
kigi_version: Some(kigi_version::VERSION.to_string()),
auth_method: Some(auth_method),
origin: None,
etag: Some("etag-legacy".into()),
@@ -4100,7 +4100,7 @@ mod tests {
let cache = ModelsCacheManager::new();
let stale = ModelsCache {
fetched_at: Utc::now() - ChronoDuration::seconds(86_400),
grok_version: Some(kigi_version::VERSION.to_string()),
kigi_version: Some(kigi_version::VERSION.to_string()),
auth_method: Some(CacheAuthMethod::Platforms),
origin: Some(origin),
etag: None,
@@ -4187,7 +4187,7 @@ mod tests {
let cache = ModelsCacheManager::new();
cache.atomic_write(&ModelsCache {
fetched_at: Utc::now() - ChronoDuration::seconds(86_400),
grok_version: Some(kigi_version::VERSION.to_string()),
kigi_version: Some(kigi_version::VERSION.to_string()),
auth_method: Some(CacheAuthMethod::Platforms),
origin: Some(with_key_origin),
etag: None,
crates/codegen/kigi-shell/src/agent/mvp_agent/agent_ops.rs
+3 -3
View File
@@ -762,7 +762,7 @@ impl MvpAgent {
}
/// Build image generation config.
///
/// Both BYOK and session (OAuth) users go direct to `xai_api_base_url`.
/// Both BYOK and session (OAuth) users go direct to `api_base_url`.
/// `sampling_config.api_key` carries the OAuth bearer for session users (the
/// `api_key_provider` refreshes it per request), so IC authenticates and
/// meters Imagine usage per-user.
@@ -775,7 +775,7 @@ impl MvpAgent {
return ImageGenConfig::Disabled;
};
let cfg = self.cfg.borrow();
let base_url = cfg.endpoints.xai_api_base_url.clone();
let base_url = cfg.endpoints.api_base_url.clone();
let version = cfg
.client_version
.clone()
@@ -818,7 +818,7 @@ impl MvpAgent {
tracing::info!("video_gen disabled by tools.disable_zdr_incompatible_tools");
return VideoGenConfig::Disabled;
}
let base_url = cfg.endpoints.xai_api_base_url.clone();
let base_url = cfg.endpoints.api_base_url.clone();
let version = cfg
.client_version
.clone()
crates/codegen/kigi-shell/src/agent/mvp_agent/mod.rs
-2
View File
@@ -664,8 +664,6 @@ pub struct MvpAgent {
plugin_registry_initialized: std::cell::Cell<bool>,
persona_io_summaries: Vec<String>,
/// Local workspace ops, built lazily via [`Self::ensure_local_workspace_ops`].
/// The agent never opens Computer Hub as a harness/client; remote cloud
/// sandboxes are gateway-owned (`gateway_bridge` / `computer_sessions`).
workspace_ops: RefCell<Option<kigi_workspace::WorkspaceOps>>,
/// Sessions opened with `require_gateway` / chat light-frontend (K13).
/// Prompt-time guard consults this when the bridge map entry is missing,
crates/codegen/kigi-shell/src/agent/mvp_agent/tests.rs
-34
View File
@@ -652,7 +652,6 @@ async fn file_toolset_override_e2e_to_finalized_toolset() {
video_gen_config: kigi_tools::implementations::grok_build::video_gen::VideoGenConfig::default(),
app_builder_deployer_config: kigi_tools::implementations::grok_build::deploy_app::AppBuilderDeployerConfig::default(),
api_key_provider: None,
auth_provider: None,
attribution_callback: None,
system_reminder_tag: kigi_tools::reminders::DEFAULT_REMINDER_TAG,
};
@@ -3183,7 +3182,6 @@ fn interactive_trust_prompt_reprompts_after_untrust() {
}
mod direct_hub_cloud_removed {
use super::super::{DIRECT_HUB_CLOUD_REMOVED_MSG, reject_direct_hub_cloud_meta};
use crate::agent::config::HubConfig;
fn assert_direct_hub_error(err: agent_client_protocol::Error) {
assert_eq!(
err.data.as_ref(),
@@ -3238,38 +3236,6 @@ mod direct_hub_cloud_removed {
.is_ok()
);
}
#[test]
fn hub_url_gating_matrix() {
let with_url = HubConfig {
url: Some("wss://hub.example/ws".into()),
};
let without_url = HubConfig { url: None };
let blank = HubConfig {
url: Some(" ".into()),
};
assert!(with_url.is_enabled());
assert!(!without_url.is_enabled());
assert!(!blank.is_enabled());
}
#[test]
fn hub_config_is_url_only_workspace_default() {
let json = serde_json::to_value(HubConfig {
url: Some("wss://hub.example/ws".into()),
})
.expect("serialize");
let obj = json.as_object().expect("object");
assert_eq!(
obj.keys().collect::<Vec<_>>(),
vec!["url"],
"HubConfig must only serialize url (no proxy-mode fields)"
);
let from_legacy: HubConfig = serde_json::from_value(serde_json::json!(
{ "url" : "wss://hub.example/ws", "workspace_mode" : "remote",
"send_turn_hooks" : false, }
))
.expect("ignore unknown fields");
assert_eq!(from_legacy.url.as_deref(), Some("wss://hub.example/ws"));
}
}
mod soft_default_settings_emit {
use super::*;
crates/codegen/kigi-shell/src/agent/mvp_agent/tests/subagent_spawn_context_tests.rs
+1
View File
@@ -39,6 +39,7 @@ async fn subagent_spawn_context_inherits_parent_permission_handle() {
Vec::new(),
false,
None,
true,
);
let mut handle = make_test_handle("test-model", false, None);
crates/codegen/kigi-shell/src/agent/proxy.rs
-619
View File
@@ -1,619 +0,0 @@
//! HTTP CONNECT proxy support for WebSocket connections.
//!
//! When running behind a corporate egress proxy,
//! `tokio-tungstenite`'s `connect_async` cannot reach external
//! hosts directly because it does not read the standard `HTTPS_PROXY` /
//! `HTTP_PROXY` environment variables.
//!
//! This module provides:
//! - [`resolve_proxy_for_host`]: reads proxy env vars and `NO_PROXY`, returning
//! the proxy URL to use for a given target host (or `None` for direct).
//! - [`connect_via_proxy`]: opens a TCP connection to the proxy, sends an HTTP
//! CONNECT request to create a tunnel, wraps the result in TLS, and returns a
//! stream suitable for `tokio_tungstenite::client_async`.
use std::sync::{Arc, OnceLock};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::TcpStream;
use tokio_tungstenite::MaybeTlsStream;
use tracing::debug;
// ---------------------------------------------------------------------------
// Environment-variable resolution
// ---------------------------------------------------------------------------
/// Read proxy configuration from the environment and decide whether `target_host`
/// should be connected through a proxy.
///
/// Resolution order (matches `curl` / `reqwest` behaviour):
/// 1. If `NO_PROXY` contains `target_host` (or a matching domain suffix / CIDR),
/// return `None`.
/// 2. If `HTTPS_PROXY` (or `https_proxy`) is set, return its value.
/// 3. If `HTTP_PROXY` (or `http_proxy`) is set, return its value.
/// 4. Otherwise return `None`.
pub fn resolve_proxy_for_host(target_host: &str) -> Option<String> {
resolve_proxy_for_host_with(target_host, |key| std::env::var(key))
}
/// Testable inner implementation that accepts a custom env-var reader.
fn resolve_proxy_for_host_with<F>(target_host: &str, env: F) -> Option<String>
where
F: for<'a> Fn(&'a str) -> Result<String, std::env::VarError>,
{
// Check NO_PROXY / no_proxy.
let no_proxy = env("NO_PROXY")
.or_else(|_| env("no_proxy"))
.unwrap_or_default();
if is_host_bypassed(target_host, &no_proxy) {
return None;
}
// HTTPS_PROXY takes precedence (our target is always wss://).
if let Ok(url) = env("HTTPS_PROXY").or_else(|_| env("https_proxy")) {
let url = url.trim().to_string();
if !url.is_empty() {
return Some(url);
}
}
// Fall back to HTTP_PROXY.
if let Ok(url) = env("HTTP_PROXY").or_else(|_| env("http_proxy")) {
let url = url.trim().to_string();
if !url.is_empty() {
return Some(url);
}
}
None
}
/// Check whether `host` is in the `no_proxy` list.
///
/// The `no_proxy` value is a comma-separated list of hostnames, domain
/// suffixes (with or without a leading dot), IP addresses, or CIDR ranges.
/// The special value `*` matches everything.
fn is_host_bypassed(host: &str, no_proxy: &str) -> bool {
let host_lower = host.to_ascii_lowercase();
for entry in no_proxy.split(',') {
let entry = entry.trim().to_ascii_lowercase();
if entry.is_empty() {
continue;
}
// Wildcard — bypass all hosts.
if entry == "*" {
return true;
}
// Exact match.
if host_lower == entry {
return true;
}
// Domain suffix match: ".example.com" matches "foo.example.com".
// Also handle the common convention of omitting the leading dot:
// "example.com" in NO_PROXY should match "sub.example.com".
let matches_suffix = if entry.starts_with('.') {
host_lower.ends_with(entry.as_str())
} else {
host_lower.len() > entry.len()
&& host_lower.ends_with(entry.as_str())
&& host_lower.as_bytes()[host_lower.len() - entry.len() - 1] == b'.'
};
if matches_suffix {
return true;
}
// CIDR / IP matching is intentionally omitted here — our target host
// is always a DNS name, not an IP literal. Keeping this simple avoids
// pulling in a CIDR parsing dependency.
}
false
}
// ---------------------------------------------------------------------------
// HTTP CONNECT tunnel
// ---------------------------------------------------------------------------
/// Establish a TLS-wrapped TCP stream through an HTTP CONNECT proxy.
///
/// Steps:
/// 1. Parse the proxy URL to get host + port.
/// 2. Open a TCP connection to the proxy and perform the CONNECT handshake.
/// 3. Wrap the tunnel in TLS (using rustls with native root certificates).
/// 4. Return the stream as `MaybeTlsStream<TcpStream>` so it is compatible
/// with `tokio_tungstenite::client_async`.
pub async fn connect_via_proxy(
proxy_url: &str,
target_host: &str,
target_port: u16,
) -> anyhow::Result<MaybeTlsStream<TcpStream>> {
let stream = open_connect_tunnel(proxy_url, target_host, target_port).await?;
let tls_stream = tls_wrap(stream, target_host).await?;
Ok(MaybeTlsStream::Rustls(tls_stream))
}
/// Open a raw TCP tunnel through an HTTP CONNECT proxy (no TLS).
///
/// 1. Parse the proxy URL to get host + port.
/// 2. Open a plain TCP connection to the proxy.
/// 3. Send `CONNECT target_host:target_port HTTP/1.1\r\n\r\n`.
/// 4. Read the proxy's response; expect `HTTP/1.x 200 …`.
/// 5. Return the raw `TcpStream` positioned after the CONNECT response.
async fn open_connect_tunnel(
proxy_url: &str,
target_host: &str,
target_port: u16,
) -> anyhow::Result<TcpStream> {
// 1. Parse proxy URL.
let (proxy_host, proxy_port) = parse_proxy_url(proxy_url)?;
// 2. TCP connect to proxy.
let proxy_addr = format!("{proxy_host}:{proxy_port}");
debug!(proxy_addr = %proxy_addr, "Opening TCP to proxy");
let stream = TcpStream::connect(&proxy_addr)
.await
.map_err(|e| anyhow::anyhow!("Failed to connect to proxy at {proxy_addr}: {e}"))?;
// 3. Send HTTP CONNECT.
let connect_req = format!(
"CONNECT {target_host}:{target_port} HTTP/1.1\r\n\
Host: {target_host}:{target_port}\r\n\
\r\n"
);
let (reader_half, mut writer_half) = stream.into_split();
writer_half.write_all(connect_req.as_bytes()).await?;
writer_half.flush().await?;
// 4. Read the status line from the proxy.
let mut reader = BufReader::new(reader_half);
let mut status_line = String::new();
reader.read_line(&mut status_line).await?;
debug!(status_line = %status_line.trim(), "Proxy CONNECT response");
if !status_line.starts_with("HTTP/1.1 200") && !status_line.starts_with("HTTP/1.0 200") {
anyhow::bail!("Proxy CONNECT failed: {}", status_line.trim());
}
// Consume remaining response headers (until empty line).
loop {
let mut line = String::new();
reader.read_line(&mut line).await?;
if line.trim().is_empty() {
break;
}
}
// 5. Assert the BufReader's internal buffer is empty before reuniting.
// BufReader::read_line may have read ahead into its buffer. If extra
// bytes were consumed beyond the HTTP headers (e.g., from a proxy that
// eagerly forwards data or coalesced TCP segments), dropping them would
// corrupt the subsequent TLS handshake.
let remaining = reader.buffer();
if !remaining.is_empty() {
anyhow::bail!(
"Proxy sent {} unexpected byte(s) after CONNECT response headers",
remaining.len()
);
}
// 6. Reunite the split halves back into a TcpStream.
let stream = reader.into_inner().reunite(writer_half)?;
Ok(stream)
}
/// Lazily-initialized TLS client configuration.
///
/// Loading native root certificates involves syscalls (reading `/etc/ssl/certs/`
/// or the macOS Keychain) and the cert store never changes at runtime. We build
/// the `ClientConfig` once and reuse it across all proxy connections / reconnects.
///
/// Stores `Ok(config)` on success or `Err(message)` if cert loading fails.
static TLS_CONFIG: OnceLock<Result<Arc<rustls::ClientConfig>, String>> = OnceLock::new();
/// Build (or return the cached) TLS client configuration.
fn get_tls_config() -> anyhow::Result<Arc<rustls::ClientConfig>> {
let result = TLS_CONFIG.get_or_init(|| {
let mut root_store = rustls::RootCertStore::empty();
let cert_result = rustls_native_certs::load_native_certs();
if cert_result.certs.is_empty() {
let errors: Vec<_> = cert_result.errors.iter().map(|e| e.to_string()).collect();
return Err(format!(
"No native root certificates found. Errors: {}",
if errors.is_empty() {
"(none)".to_string()
} else {
errors.join("; ")
}
));
}
for cert in cert_result.certs {
if let Err(e) = root_store.add(cert) {
tracing::warn!(error = %e, "Skipping unparseable native root certificate");
}
}
let config = rustls::ClientConfig::builder()
.with_root_certificates(root_store)
.with_no_client_auth();
Ok(Arc::new(config))
});
match result {
Ok(config) => Ok(config.clone()),
Err(msg) => anyhow::bail!("{msg}"),
}
}
/// Perform a TLS handshake over an existing TCP stream using rustls with
/// native root certificates (cached via [`TLS_CONFIG`]).
async fn tls_wrap(
stream: TcpStream,
server_name: &str,
) -> anyhow::Result<tokio_rustls::client::TlsStream<TcpStream>> {
let tls_config = get_tls_config()?;
let connector = tokio_rustls::TlsConnector::from(tls_config);
let dns_name = rustls::pki_types::ServerName::try_from(server_name.to_string())
.map_err(|e| anyhow::anyhow!("Invalid TLS server name '{server_name}': {e}"))?;
let tls_stream = connector
.connect(dns_name, stream)
.await
.map_err(|e| anyhow::anyhow!("TLS handshake through proxy failed: {e}"))?;
Ok(tls_stream)
}
/// Parse a proxy URL into (host, port).
///
/// Accepted formats:
/// - `http://host:port`
/// - `http://host` (defaults to port 80)
/// - `host:port`
fn parse_proxy_url(url: &str) -> anyhow::Result<(String, u16)> {
// Strip scheme if present.
let without_scheme = url
.strip_prefix("http://")
.or_else(|| url.strip_prefix("https://"))
.unwrap_or(url);
// Strip trailing path/slash.
let authority = without_scheme.split('/').next().unwrap_or(without_scheme);
if let Some((host, port_str)) = authority.rsplit_once(':') {
let port: u16 = port_str
.parse()
.map_err(|_| anyhow::anyhow!("Invalid proxy port in '{url}'"))?;
Ok((host.to_string(), port))
} else {
// No port — default to 80 for HTTP proxies.
Ok((authority.to_string(), 80))
}
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
#[cfg(test)]
mod tests {
use super::*;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
// ===== parse_proxy_url =====
#[test]
fn test_parse_proxy_url_with_scheme_and_port() {
let (host, port) = parse_proxy_url("http://proxy.example.com:3140").unwrap();
assert_eq!(host, "proxy.example.com");
assert_eq!(port, 3140);
}
#[test]
fn test_parse_proxy_url_without_scheme() {
let (host, port) = parse_proxy_url("proxy.example.com:8080").unwrap();
assert_eq!(host, "proxy.example.com");
assert_eq!(port, 8080);
}
#[test]
fn test_parse_proxy_url_without_port() {
let (host, port) = parse_proxy_url("http://proxy.example.com").unwrap();
assert_eq!(host, "proxy.example.com");
assert_eq!(port, 80);
}
#[test]
fn test_parse_proxy_url_with_trailing_slash() {
let (host, port) = parse_proxy_url("http://proxy.example.com:3140/").unwrap();
assert_eq!(host, "proxy.example.com");
assert_eq!(port, 3140);
}
#[test]
fn test_parse_proxy_url_https_scheme() {
let (host, port) = parse_proxy_url("https://secure-proxy:443").unwrap();
assert_eq!(host, "secure-proxy");
assert_eq!(port, 443);
}
#[test]
fn test_parse_proxy_url_multi_label_host() {
let (host, port) =
parse_proxy_url("http://http-proxy.services.internal.example:3128").unwrap();
assert_eq!(host, "http-proxy.services.internal.example");
assert_eq!(port, 3128);
}
#[test]
fn test_parse_proxy_url_invalid_port() {
assert!(parse_proxy_url("http://proxy:notaport").is_err());
}
// ===== is_host_bypassed =====
#[test]
fn test_bypass_exact_match() {
assert!(is_host_bypassed("localhost", "localhost,127.0.0.1"));
}
#[test]
fn test_bypass_domain_suffix_with_dot() {
assert!(is_host_bypassed(
"api.corp.example",
"localhost,.corp.example"
));
}
#[test]
fn test_bypass_domain_suffix_without_dot() {
// Common convention: "example.com" in NO_PROXY matches "api.example.com".
assert!(is_host_bypassed("api.example.com", "localhost,example.com"));
}
#[test]
fn test_bypass_wildcard() {
assert!(is_host_bypassed("anything.example.com", "*"));
}
#[test]
fn test_no_bypass_when_not_listed() {
assert!(!is_host_bypassed(
"api.external.example",
"localhost,127.0.0.1,.corp.example,.internal.example"
));
}
#[test]
fn test_bypass_case_insensitive() {
assert!(is_host_bypassed("API.Corp.EXAMPLE", ".corp.example"));
}
#[test]
fn test_bypass_empty_no_proxy() {
assert!(!is_host_bypassed("api.external.example", ""));
}
#[test]
fn test_bypass_spaces_in_entries() {
assert!(is_host_bypassed(
"foo.example.com",
" localhost , .example.com , .other.com "
));
}
#[test]
fn test_bypass_cidr_not_matched_for_dns_names() {
// CIDR entries like 10.0.0.0/8 should not match DNS names.
assert!(!is_host_bypassed("api.external.example", "10.0.0.0/8"));
}
#[test]
fn test_bypass_combined_no_proxy_list() {
// A typical corporate NO_PROXY mixes loopback, private CIDRs, and domain suffixes.
let no_proxy = "localhost,127.0.0.1,10.0.0.0/8,.internal.example,.corp.example";
assert!(!is_host_bypassed("api.external.example", no_proxy));
assert!(is_host_bypassed("db.internal.example", no_proxy));
assert!(is_host_bypassed("git.corp.example", no_proxy));
assert!(is_host_bypassed("localhost", no_proxy));
}
// ===== resolve_proxy_for_host_with =====
#[test]
fn test_resolve_no_proxy_vars_set() {
let result = resolve_proxy_for_host_with("api.external.example", |_| {
Err(std::env::VarError::NotPresent)
});
assert_eq!(result, None);
}
#[test]
fn test_resolve_https_proxy_used() {
let result = resolve_proxy_for_host_with("api.external.example", |key| match key {
"HTTPS_PROXY" => Ok("http://proxy.example.com:3128".to_string()),
"NO_PROXY" => Err(std::env::VarError::NotPresent),
_ => Err(std::env::VarError::NotPresent),
});
assert_eq!(result, Some("http://proxy.example.com:3128".to_string()));
}
#[test]
fn test_resolve_http_proxy_fallback() {
let result = resolve_proxy_for_host_with("api.external.example", |key| match key {
"HTTP_PROXY" => Ok("http://proxy.example.com:8080".to_string()),
"NO_PROXY" => Err(std::env::VarError::NotPresent),
_ => Err(std::env::VarError::NotPresent),
});
assert_eq!(result, Some("http://proxy.example.com:8080".to_string()));
}
#[test]
fn test_resolve_no_proxy_bypasses() {
let result = resolve_proxy_for_host_with("api.corp.example", |key| match key {
"HTTPS_PROXY" => Ok("http://proxy.example.com:3128".to_string()),
"NO_PROXY" => Ok("localhost,.corp.example".to_string()),
_ => Err(std::env::VarError::NotPresent),
});
assert_eq!(result, None);
}
#[test]
fn test_resolve_https_proxy_takes_precedence() {
let result = resolve_proxy_for_host_with("api.external.example", |key| match key {
"HTTPS_PROXY" => Ok("http://https-proxy.example.com:443".to_string()),
"HTTP_PROXY" => Ok("http://http-proxy.example.com:80".to_string()),
"NO_PROXY" => Err(std::env::VarError::NotPresent),
_ => Err(std::env::VarError::NotPresent),
});
assert_eq!(
result,
Some("http://https-proxy.example.com:443".to_string())
);
}
#[test]
fn test_resolve_lowercase_env_vars() {
let result = resolve_proxy_for_host_with("api.external.example", |key| match key {
"https_proxy" => Ok("http://proxy.example.com:3128".to_string()),
"no_proxy" => Err(std::env::VarError::NotPresent),
_ => Err(std::env::VarError::NotPresent),
});
assert_eq!(result, Some("http://proxy.example.com:3128".to_string()));
}
#[test]
fn test_resolve_empty_proxy_ignored() {
let result = resolve_proxy_for_host_with("api.external.example", |key| match key {
"HTTPS_PROXY" => Ok(" ".to_string()),
"HTTP_PROXY" => Ok("http://proxy.example.com:8080".to_string()),
_ => Err(std::env::VarError::NotPresent),
});
assert_eq!(result, Some("http://proxy.example.com:8080".to_string()));
}
#[test]
fn test_resolve_respects_no_proxy_when_proxy_set() {
let result = resolve_proxy_for_host_with("api.external.example", |key| match key {
"HTTPS_PROXY" | "HTTP_PROXY" => Ok("http://proxy.example.com:3128".to_string()),
"NO_PROXY" => {
Ok("localhost,127.0.0.1,10.0.0.0/8,.internal.example,.corp.example".to_string())
}
_ => Err(std::env::VarError::NotPresent),
});
assert_eq!(result, Some("http://proxy.example.com:3128".to_string()));
}
// ===== HTTP CONNECT tunnel (integration-style) =====
/// Helper: spawn a mock HTTP CONNECT proxy that accepts one connection.
///
/// On receiving a CONNECT request, it validates the request format,
/// replies with `status_line`, and then echoes data (simulating a tunnel).
/// Returns the proxy's listen address.
async fn spawn_mock_proxy(status_line: &'static str) -> std::net::SocketAddr {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
tokio::spawn(async move {
let (mut stream, _) = listener.accept().await.unwrap();
// Read CONNECT request (read until \r\n\r\n).
let mut buf = vec![0u8; 4096];
let mut total = 0;
loop {
let n = stream.read(&mut buf[total..]).await.unwrap();
if n == 0 {
return;
}
total += n;
let so_far = std::str::from_utf8(&buf[..total]).unwrap_or("");
if so_far.contains("\r\n\r\n") {
break;
}
}
let request = std::str::from_utf8(&buf[..total]).unwrap().to_string();
assert!(
request.contains("CONNECT ") && request.contains(" HTTP/1.1"),
"Expected CONNECT request, got: {request}"
);
// Reply with the provided status line.
stream.write_all(status_line.as_bytes()).await.unwrap();
// Echo loop (simulates the transparent tunnel).
let mut echo_buf = [0u8; 1024];
loop {
let n = match stream.read(&mut echo_buf).await {
Ok(0) | Err(_) => break,
Ok(n) => n,
};
if stream.write_all(&echo_buf[..n]).await.is_err() {
break;
}
}
});
addr
}
/// Tests that `open_connect_tunnel` sends a correct CONNECT request,
/// parses the proxy's 200 response, and returns a usable tunnel stream.
#[tokio::test]
async fn test_open_connect_tunnel_success() {
let addr =
spawn_mock_proxy("HTTP/1.1 200 Connection Established\r\nServer: mock\r\n\r\n").await;
let proxy_url = format!("http://{addr}");
// Call the real function under test.
let mut stream = open_connect_tunnel(&proxy_url, "example.com", 443)
.await
.expect("tunnel should succeed");
// Verify the tunnel works by echoing data through it.
stream.write_all(b"hello tunnel").await.unwrap();
stream.flush().await.unwrap();
let mut response = vec![0u8; 12];
stream.read_exact(&mut response).await.unwrap();
assert_eq!(&response, b"hello tunnel");
}
/// Tests that `open_connect_tunnel` with a non-default port sends the
/// correct CONNECT target.
#[tokio::test]
async fn test_open_connect_tunnel_custom_port() {
let addr = spawn_mock_proxy("HTTP/1.1 200 OK\r\n\r\n").await;
let proxy_url = format!("http://{addr}");
let stream = open_connect_tunnel(&proxy_url, "internal.example.com", 8443).await;
assert!(stream.is_ok(), "tunnel should succeed for custom port");
}
/// Tests that `open_connect_tunnel` returns an error when the proxy
/// rejects the CONNECT request with a non-200 status.
#[tokio::test]
async fn test_open_connect_tunnel_proxy_rejects() {
let addr = spawn_mock_proxy("HTTP/1.1 403 Forbidden\r\n\r\n").await;
let proxy_url = format!("http://{addr}");
let result = open_connect_tunnel(&proxy_url, "blocked.example.com", 443).await;
assert!(result.is_err());
let err_msg = result.unwrap_err().to_string();
assert!(
err_msg.contains("403"),
"Error should mention 403: {err_msg}"
);
}
/// Tests that `open_connect_tunnel` returns an error when connecting
/// to a proxy that isn't listening.
#[tokio::test]
async fn test_open_connect_tunnel_proxy_unreachable() {
let result = open_connect_tunnel("http://127.0.0.1:1", "example.com", 443).await;
assert!(result.is_err());
let err_msg = result.unwrap_err().to_string();
assert!(
err_msg.contains("Failed to connect to proxy"),
"Error should mention proxy connection failure: {err_msg}"
);
}
}