fix(startup): thread-exhaustion resilience — cap workers, never panic

Port of upstream grok-build 0.2.113/0.2.114 fixes, extended to Kigi's
startup order.

- kigi-tty-utils runtime module: worker-cap policy (8 max) — tokio's
  per-core default pins 100+ thread slots on many-core shared hosts
  against RLIMIT_NPROC / pids.max, and later spawns die with EAGAIN.
- kigi-bin builds the main runtime with capped workers and reports an
  out-of-threads host with a diagnostic + exit(1) instead of a panic.
- The pre-runtime optimization threads (HTTP warmup, early model
  prefetch) spawn recoverably and degrade — their panic would have
  preempted the runtime's own diagnostic.
- The per-session actor thread and its runtime report spawn/build
  failure through the existing acp::Error / AgentBuildError boundaries
  (new RuntimeBuild variant) instead of panicking the agent host.
- fsnotify's watcher thread spawn maps into FsNotifyError::WatcherStart
  so callers degrade instead of crashing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KSy9S31pZNWXtRZNVJ1sDf
This commit is contained in:
ZacharyZhang-NYandClaude Fable 5 committed 2026-08-27 11:28:17 -04:00
1 parent bf93091416
commit 97de661554
8 files changed
+144 -17

No files matched your search

@@ -1639,10 +1639,23 @@ pub(crate) async fn spawn_session_on_thread(
};
(initial_last_compaction, initial_prompt_texts)
};
let rt = tokio::runtime::Builder::new_current_thread()
// Runtime construction acquires fds (epoll/kqueue, waker) and
// fails with EMFILE/EAGAIN under resource pressure — report
// through the init channel instead of panicking the thread.
let rt = match tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("session runtime");
{
Ok(rt) => rt,
Err(e) => {
tracing::error!(
error = %e,
"failed to build session runtime (resource exhaustion?)"
);
let _ = init_tx.send(Err(kigi_agent::AgentBuildError::RuntimeBuild(e)));
return;
}
};
let local = tokio::task::LocalSet::new();
local.block_on(&rt, async move {
let _trace_span = parent_traceparent.as_ref().map(|tp| {
@@ -1763,8 +1776,19 @@ pub(crate) async fn spawn_session_on_thread(
}));
let _ = session_done_rx.await;
});
})
.expect("spawn session thread");
});
let join_handle = match join_handle {
Ok(handle) => handle,
Err(e) => {
tracing::error!(
error = %e,
"failed to spawn session thread (thread/PID limit or memory pressure?)"
);
return Err(
acp::Error::internal_error().data(format!("failed to spawn session thread: {e}"))
);
}
};
let init = init_rx
.await
.map_err(|_| {