Force agents to act so weak models can't complete a no-op
The whole premise is cheap models + skills + a team — so the control plane, not the model, must guarantee action. Every adapter hardcoded tool_choice:"auto", letting a weak model answer the kickoff with prose and "succeed" having hired no one, delegated nothing, and said nothing. Now the agent loop compels a tool call on the opening turn (tool_choice plumbed through every backend: OpenAI/Responses/Chat, Anthropic, Codex, Copilot, Azure, Custom, Gemini ANY mode, Bedrock any). If the forced turn still yields no tool call, the run fails loudly instead of recording a deceptive success — surfacing the problem (let it crash) rather than a silent dead company. After it has acted, the model chooses freely. Tested: a stub that rambles on "auto" but acts when forced now posts a kickoff message; a prose-only first turn fails the run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
fa01aa682f
commit
ef9ee463c9
9 files changed
+107
-35
No files matched your search
@@ -37,7 +37,10 @@ describe("task execution service", () => {
|
||||
const fakeToolTurn = async (input: ProviderToolTurnInput): Promise<AgentTurnResult> => {
|
||||
const first = input.messages[0];
|
||||
if (first?.role === "user") capturedPrompt = first.text;
|
||||
return { text: "executed:gpt-5.4", toolCalls: [], rawResponseJson: JSON.stringify({ id: "resp-task-1" }) };
|
||||
if (input.toolChoice === "required") {
|
||||
return { text: "executed:gpt-5.4", toolCalls: [{ id: `call_${randomUUID()}`, name: "create_document", arguments: { title: "Run note", doc_type: "memo", content: "Executed." } }], rawResponseJson: JSON.stringify({ id: "resp-task-1a" }) };
|
||||
}
|
||||
return { text: "executed:gpt-5.4", toolCalls: [], rawResponseJson: JSON.stringify({ id: "resp-task-1b" }) };
|
||||
};
|
||||
const providers = new ProviderService(connection.db, providerAdapters, testVault, undefined, fakeToolTurn);
|
||||
const skills = new SkillService(connection.db, join(tmpdir(), `agentcompany-task-skills-${randomUUID()}`));
|
||||
@@ -141,11 +144,12 @@ describe("task execution service", () => {
|
||||
const organization = new OrganizationService(connection.db);
|
||||
const approvals = new ApprovalService(connection.db);
|
||||
const snapshot = new SnapshotService(connection.db);
|
||||
const fakeToolTurn = async (): Promise<AgentTurnResult> => ({
|
||||
text: "executed:gpt-5.4",
|
||||
toolCalls: [],
|
||||
rawResponseJson: JSON.stringify({ id: "resp-task-approved" }),
|
||||
});
|
||||
const fakeToolTurn = async (input: ProviderToolTurnInput): Promise<AgentTurnResult> => {
|
||||
if (input.toolChoice === "required") {
|
||||
return { text: "executed:gpt-5.4", toolCalls: [{ id: `call_${randomUUID()}`, name: "create_document", arguments: { title: "Run note", doc_type: "memo", content: "Executed." } }], rawResponseJson: JSON.stringify({ id: "resp-task-approved-a" }) };
|
||||
}
|
||||
return { text: "executed:gpt-5.4", toolCalls: [], rawResponseJson: JSON.stringify({ id: "resp-task-approved-b" }) };
|
||||
};
|
||||
const providers = new ProviderService(connection.db, providerAdapters, testVault, undefined, fakeToolTurn);
|
||||
const execution = new TaskExecutionService(connection.db, providers);
|
||||
let orgId = "";
|
||||
|
||||
Reference in new issue
Block a user