Chat-completions turns stream: reasoning models no longer die at the 60s header wall
Root cause of the stuck kimi-for-coding kickoff (task running forever, no
team spawning): runKimiCodeTurn posted stream:false, and a non-streaming
completion returns headers only after the whole generation — once the CEO's
context grew past ~150k chars, a reasoning turn needed >60s and provider-http's
header deadline aborted it ("Provider request timed out."), failing the run;
every heartbeat retry re-read the workspace, regrew the context, and died at
the same wall until the 3-run budget was gone.
Fix at the family level: accumulateChatStream (chat-format.ts) consumes the
standard OpenAI chunk deltas — content appends, indexed tool_call fragments,
reasoning_content keeping the 60s idle deadline fed — and a shared
runChatCompletionsStream drives makeChatCompletionsTurn (kimi, kimi-cn, xai,
groq, mistral, cerebras, openrouter) and runKimiCodeTurn.
Wire shape verified against MoonshotAI/kimi-cli (kosong streams by default)
and a live streamed probe of api.kimi.com/coding/v1 with the stored plan
credential: role opener, reasoning deltas, tool_call fragments (id+name then
argument text), finish_reason, [DONE] — all reassembled correctly.
verify:ci green including e2e.
This commit is contained in:
1 parent
3a7b267ead
commit
0fa99d94a7
3 files changed
+104
-29
No files matched your search
@@ -5,7 +5,7 @@ import { describe, expect, test } from "vitest";
|
||||
import type { JsonRecord, ProviderEventStreamMessage } from "../../src/main/services/provider-http";
|
||||
import type { AgentTurnMessage } from "../../src/main/services/provider-tool-turn";
|
||||
import { accumulateResponsesStream, parseResponsesOutput, responsesToolDefs, toResponsesInput } from "../../src/main/services/provider-tool-turns/responses-format";
|
||||
import { parseChatChoice, toChatMessages } from "../../src/main/services/provider-tool-turns/chat-format";
|
||||
import { accumulateChatStream, parseChatChoice, toChatMessages } from "../../src/main/services/provider-tool-turns/chat-format";
|
||||
import { parseAnthropicContent, toAnthropicMessages } from "../../src/main/services/provider-tool-turns/anthropic-format";
|
||||
import { accumulateGeminiStream, antigravityRequestBody } from "../../src/main/services/provider-tool-turns/gemini-format";
|
||||
import { bedrockToolConfig, parseBedrockOutput, toBedrockMessages } from "../../src/main/services/provider-tool-turns/bedrock-format";
|
||||
@@ -90,6 +90,33 @@ describe("openai chat completions encoding", () => {
|
||||
const data = { choices: [{ message: { content: "hi", tool_calls: [{ id: "c9", function: { name: "f", arguments: '{"a":1}' } }] } }] };
|
||||
expect(parseChatChoice(data)).toEqual({ text: "hi", toolCalls: [{ id: "c9", name: "f", arguments: { a: 1 } }] });
|
||||
});
|
||||
|
||||
test("accumulates streamed content deltas and indexed tool-call fragments", async () => {
|
||||
// The Kimi coding endpoint streams reasoning_content first (which must be
|
||||
// ignored), then content deltas, then tool calls split across chunks: the
|
||||
// opening fragment carries index/id/name, later fragments append arguments.
|
||||
const result = await accumulateChatStream(stream([
|
||||
{ id: "cmpl-1", choices: [{ delta: { role: "assistant", reasoning_content: "thinking..." } }] },
|
||||
{ choices: [{ delta: { content: "Staffing " } }] },
|
||||
{ choices: [{ delta: { content: "the team." } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, id: "c1", type: "function", function: { name: "hire_agent", arguments: "" } }] } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, function: { arguments: '{"name":' } }, { index: 1, id: "c2", type: "function", function: { name: "create_task", arguments: '{"title":"x"}' } }] } }] },
|
||||
{ choices: [{ delta: { tool_calls: [{ index: 0, function: { arguments: '"Sam"}' } }] }, finish_reason: "tool_calls" }] },
|
||||
{ choices: [], usage: { prompt_tokens: 10, completion_tokens: 5 } },
|
||||
]));
|
||||
expect(result.text).toBe("Staffing the team.");
|
||||
expect(result.toolCalls).toEqual([
|
||||
{ id: "c1", name: "hire_agent", arguments: { name: "Sam" } },
|
||||
{ id: "c2", name: "create_task", arguments: { title: "x" } },
|
||||
]);
|
||||
});
|
||||
|
||||
test("fails the turn on a streamed error event and on an empty stream", async () => {
|
||||
await expect(accumulateChatStream(stream([
|
||||
{ error: { message: "The usage limit has been reached" } },
|
||||
]))).rejects.toThrow("The usage limit has been reached");
|
||||
await expect(accumulateChatStream(stream([]))).rejects.toThrow("Chat completions stream ended without any events.");
|
||||
});
|
||||
});
|
||||
|
||||
describe("anthropic encoding", () => {
|
||||
|
||||
Reference in new issue
Block a user