feat(session): auto-compact long transcripts via transformContext

pi's `transformContext` hook fired on every LLM call but returned
the transcript unchanged, so a long-running agent paid per-token
for its entire history on every turn. The README promised "sessions
persist + auto-compact" but the main loop had no compaction wired
in at all — only rotation at the long-cycle boundary.

`context-compactor` closes that gap. A pure, provider-free function
estimates tokens (~4 chars/token), short-circuits below a 120K
threshold, and on overflow keeps the last 20 messages verbatim
while collapsing older ones into a single "context summary" user
turn. The summary preserves the original task and the first twelve
assistant highlights so the agent has continuity without forwarding
every historical turn.

Compaction is non-destructive: the Agent's in-memory transcript
and the SQLite session store still hold the full history, feeding
UI timelines and audit. Only the LLM payload shrinks. Rotation
remains the long-cycle boundary (separate commit) — compaction is
what keeps each individual LLM call within budget.
This commit is contained in:
ZacharyZhang-NY committed 2026-04-19 01:14:48 -04:00
1 parent 7f23f40e87
commit e53ac36176
2 files changed
+166 -6

No files matched your search

+8 -6
View File
@@ -29,6 +29,7 @@ import {
projectToolExecutionToWorkProduct,
type ArtifactProjectorRunContext,
} from "./artifact-projector";
import { compactMessagesIfNeeded } from "./context-compactor";
export interface ActiveRunContext {
runId: string;
@@ -174,15 +175,16 @@ export function handleAfterToolCall(
}
/**
* Context-transform hook. Placeholder pass-through; auto-compaction
* wiring lives in session-management-service and plugs in here once
* the compactor is available (tracked separately from the contract
* wiring).
* Context-transform hook. Delegates to `compactMessagesIfNeeded`,
* which is a pure, provider-free deterministic compactor. Runs before
* every LLM call; short transcripts short-circuit on the token-budget
* check so the steady-state cost is negligible.
*/
export function handleTransformContext(
_deps: HookHandlerDeps,
_ctx: ActiveRunContext,
_payload: TransformContextRpcPayload,
payload: TransformContextRpcPayload,
): TransformContextRpcResult {
return { changed: false, messages: null };
const outcome = compactMessagesIfNeeded(payload.messages);
return { changed: outcome.changed, messages: outcome.messages };
}