ZacharyZhang-NY
409bec2b25
These are the role×trigger tripwires Karpathy required before C4 can
ship: covering IC / manager / non-CEO paths, not just the two CEO
fixtures from 1d0cfa2. Together they act as the cross-role regression
net — any C4 change that breaks hire_agent tool registration or
prompt shape across role tiers will now surface as an assertion
failure, not a silent production drift.
Fixtures recorded against anthropic/claude-sonnet-4-5 via the user's
Claude Max OAuth token (read from macOS keychain, not committed):
- ic-task-execute (364 in / 1425 out): IC receives task t_1138,
expects task id + filename in the response and a mark_complete
intent within 3 turns.
- manager-task-handoff (457 in / 1022 out): VP Engineering decomposes
CEO p99 directive into IC-sized tasks; expects /api/task/create
calls, references to agent_ic_* slugs, and the "p99" term.
- non-ceo-dm-reply (357 in / 545 out): VP Product handles a peer DM
with blocking ETA ask; expects mark_read + /api/agent_message/send
+ message id preserved.
Total spend ~3.7k tokens; pnpm test:eval --run now green at 8/8
(3 harness self-tests + 5 recorded fixtures).
With these tripwires in place, C4 (hire_agent / file_approval tool
uplift + initial-user-turn rewrite) can proceed without the previous
"only CEO path has a baseline" blind spot.
5 lines
4.8 KiB
JSON
5 lines
4.8 KiB
JSON
{"type":"meta","fixtureId":"manager-task-handoff","piAgentCoreVersion":"0.67.68","recordedAt":"2026-04-20T18:07:56.403Z","scenario":"manager-task-handoff roundtable session-6 baseline","systemPromptHash":"sha256:cafd79817ccfbbf043bced12899f06c995b0f745200facbbc46c94484eee7da6"}
|
|
{"type":"turn","turnIndex":0,"role":"user","content":[{"type":"text","text":"Inbound from CEO (channel=eng_direct): \"We are getting p99 latency spikes on the /api/companies endpoint after the v48 migration. Figure out what broke and fix it by end of week.\"\n\nYour ICs:\n- agent_ic_be1 (senior backend IC, SQL + Node)\n- agent_ic_be2 (mid backend IC, API + schema)\n- agent_ic_fe (frontend, React)\n- agent_ic_sre (SRE, metrics + profiling)\n\nTrigger: message. Decompose the CEO directive into 2-4 IC-sized tasks now. Use /api/task/create for each. Do not write a plan document.\n"}]}
|
|
{"type":"turn","turnIndex":1,"role":"assistant","content":[{"type":"text","text":"I'll decompose this latency investigation into tasks for the appropriate ICs.\n\n<bash>\ncurl -X POST http://127.0.0.1:${AGENTCOMPANY_API_PORT}/api/task/create \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"title\": \"Instrument /api/companies endpoint with p99 latency metrics and identify spike pattern\",\n \"description\": \"Set up detailed monitoring for /api/companies to capture p99 latency spikes. Identify:\\n- When spikes occur (time of day, load pattern)\\n- Baseline vs spike latency values\\n- Any correlation with request parameters\\n- CPU/memory/DB connection pool metrics during spikes\\n\\nDeliver a summary doc with graphs showing the spike pattern and potential bottleneck areas (DB query, serialization, etc).\",\n \"assigneeAgentId\": \"agent_ic_sre\",\n \"dueAt\": \"2024-01-10T17:00:00Z\"\n }'\n</bash>\n\n<bash>\ncurl -X POST http://127.0.0.1:${AGENTCOMPANY_API_PORT}/api/task/create \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"title\": \"Profile /api/companies SQL queries and compare v47 vs v48 execution plans\",\n \"description\": \"Investigate database-level changes from v48 migration affecting /api/companies:\\n- Review migration scripts for schema changes to companies table or related tables\\n- Run EXPLAIN ANALYZE on all queries in the endpoint handler\\n- Compare query plans with v47 (check for missing indexes, inefficient joins)\\n- Check for N+1 query patterns introduced in v48\\n- Identify the slowest query contributing to p99 spikes\\n\\nDeliver findings with specific query performance metrics and recommendations.\",\n \"assigneeAgentId\": \"agent_ic_be1\",\n \"dueAt\": \"2024-01-10T17:00:00Z\"\n }'\n</bash>\n\n<bash>\ncurl -X POST http://127.0.0.1:${AGENTCOMPANY_API_PORT}/api/task/create \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"title\": \"Review /api/companies API handler code changes in v48 for performance regressions\",\n \"description\": \"Code-level investigation of /api/companies endpoint changes in v48:\\n- Git diff the endpoint handler between v47 and v48\\n- Look for synchronous operations that should be async\\n- Check for added loops, redundant serialization, or blocking calls\\n- Review any new middleware or validation logic in the request path\\n- Check response payload size changes\\n\\nDeliver a code review doc highlighting any performance anti-patterns introduced in v48.\",\n \"assigneeAgentId\": \"agent_ic_be2\",\n \"dueAt\": \"2024-01-10T17:00:00Z\"\n }'\n</bash>\n\n<bash>\ncurl -X POST http://127.0.0.1:${AGENTCOMPANY_API_PORT}/api/task/create \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"title\": \"Implement fixes for /api/companies p99 latency and deploy to staging\",\n \"description\": \"Based on findings from SRE profiling and BE1/BE2 investigations, implement the fix:\\n- Apply recommended index additions or query optimizations\\n- Refactor any synchronous code to async if identified\\n- Add caching if appropriate\\n- Run load tests in staging to verify p99 drops to acceptable levels (<200ms target)\\n- Prepare production deploy plan with rollback strategy\\n\\nDeliver: fixed code in PR, staging test results, deploy runbook.\",\n \"assigneeAgentId\": \"agent_ic_be1\",\n \"dueAt\": \"2024-01-12T17:00:00Z\"\n }'\n</bash>\n\nTasks created. I've assigned the investigation phase to SRE (metrics), BE1 (SQL), and BE2 (API code), all due Wed EOD. BE1 will then implement the fix by Fri EOD based on the combined findings."}],"tokenLedger":{"input":457,"cache_read":0,"cache_creation":0,"output":1022}}
|
|
{"type": "assertions", "assertions": [{"kind": "message_contains_substring", "role": "assistant", "substring": "/api/task/create"}, {"kind": "message_contains_substring", "role": "assistant", "substring": "agent_ic_"}, {"kind": "message_contains_substring", "role": "assistant", "substring": "p99"}, {"kind": "trace_max_turns", "max": 3}]}
|