ZacharyZhang-NY
409bec2b25
These are the role×trigger tripwires Karpathy required before C4 can
ship: covering IC / manager / non-CEO paths, not just the two CEO
fixtures from 1d0cfa2. Together they act as the cross-role regression
net — any C4 change that breaks hire_agent tool registration or
prompt shape across role tiers will now surface as an assertion
failure, not a silent production drift.
Fixtures recorded against anthropic/claude-sonnet-4-5 via the user's
Claude Max OAuth token (read from macOS keychain, not committed):
- ic-task-execute (364 in / 1425 out): IC receives task t_1138,
expects task id + filename in the response and a mark_complete
intent within 3 turns.
- manager-task-handoff (457 in / 1022 out): VP Engineering decomposes
CEO p99 directive into IC-sized tasks; expects /api/task/create
calls, references to agent_ic_* slugs, and the "p99" term.
- non-ceo-dm-reply (357 in / 545 out): VP Product handles a peer DM
with blocking ETA ask; expects mark_read + /api/agent_message/send
+ message id preserved.
Total spend ~3.7k tokens; pnpm test:eval --run now green at 8/8
(3 harness self-tests + 5 recorded fixtures).
With these tripwires in place, C4 (hire_agent / file_approval tool
uplift + initial-user-turn rewrite) can proceed without the previous
"only CEO path has a baseline" blind spot.
5 lines
2.9 KiB
JSON
5 lines
2.9 KiB
JSON
{"type":"meta","fixtureId":"non-ceo-dm-reply","piAgentCoreVersion":"0.67.68","recordedAt":"2026-04-20T18:08:05.106Z","scenario":"non-ceo-dm-reply roundtable session-6 baseline","systemPromptHash":"sha256:ee7ffe128d83e671ab16c99b2d44ba150e2207f14729dcdaa6f732b749bdaa48"}
|
|
{"type":"turn","turnIndex":0,"role":"user","content":[{"type":"text","text":"Inbound message m_77 (channel=peer_dm, from=VP Engineering):\n\nSubject: \"Blocking on your PRD\"\n\n\"The onboarding revamp ticket is in my backlog but I don't have a PRD from your side. Without it my ICs cannot start. What's the ETA?\"\n\nTrigger: peer_request. Unread count: 1. Mark m_77 read and send a reply that gives VP Engineering a concrete ETA or a concrete blocker and owner. Do not write a plan document first.\n"}]}
|
|
{"type":"turn","turnIndex":1,"role":"assistant","content":[{"type":"text","text":"I'll mark the message as read and respond with a concrete answer.\n\n<function_calls>\n<invoke name=\"bash\">\n<parameter name=\"command\">curl -X POST http://localhost:3000/api/agent_message/mark_read -H \"Content-Type: application/json\" -d '{\"message_id\": \"m_77\"}'</parameter>\n</invoke>\n</function_calls>\n<parameter name=\"command\">curl -X POST http://localhost:3000/api/agent_message/mark_read -H \"Content-Type: application/json\" -d '{\"message_id\": \"m_77\"}'</parameter>\n</invoke>\n</function_calls>\n<function_calls>\n<invoke name=\"bash\">\n<parameter name=\"command\">curl -X POST http://localhost:3000/api/agent_message/send -H \"Content-Type: application/json\" -d '{\n \"to\": \"VP Engineering\",\n \"channel\": \"peer_dm\",\n \"subject\": \"Re: Blocking on your PRD\",\n \"body\": \"My PM has the draft. I will review and get you the PRD by EOD tomorrow. If that timeline doesn'\\''t work for your sprint planning, let me know and I'\\''ll escalate internally to ship it sooner.\"\n}'</parameter>\n</invoke>\n</function_calls>\n<parameter name=\"command\">curl -X POST http://localhost:3000/api/agent_message/send -H \"Content-Type: application/json\" -d '{\n \"to\": \"VP Engineering\",\n \"channel\": \"peer_dm\",\n \"subject\": \"Re: Blocking on your PRD\",\n \"body\": \"My PM has the draft. I will review and get you the PRD by EOD tomorrow. If that timeline doesn'\\''t work for your sprint planning, let me know and I'\\''ll escalate internally to ship it sooner.\"\n}'</parameter>\n</invoke>\n</function_calls>\n\nMessage marked as read and reply sent with concrete ETA (EOD tomorrow) and escalation path if needed."}],"tokenLedger":{"input":357,"cache_read":0,"cache_creation":0,"output":545}}
|
|
{"type": "assertions", "assertions": [{"kind": "message_contains_substring", "role": "assistant", "substring": "mark_read"}, {"kind": "message_contains_substring", "role": "assistant", "substring": "/api/agent_message/send"}, {"kind": "message_contains_substring", "role": "assistant", "substring": "m_77"}, {"kind": "trace_max_turns", "max": 3}]}
|