Files
AgentCompany/tests/eval/scenarios/manager-task-handoff/user.txt
T
ZacharyZhang-NY 409bec2b25 Record 3 new B5 fixtures: IC task exec, manager handoff, non-CEO DM
These are the role×trigger tripwires Karpathy required before C4 can
ship: covering IC / manager / non-CEO paths, not just the two CEO
fixtures from 1d0cfa2. Together they act as the cross-role regression
net — any C4 change that breaks hire_agent tool registration or
prompt shape across role tiers will now surface as an assertion
failure, not a silent production drift.

Fixtures recorded against anthropic/claude-sonnet-4-5 via the user's
Claude Max OAuth token (read from macOS keychain, not committed):

- ic-task-execute (364 in / 1425 out): IC receives task t_1138,
  expects task id + filename in the response and a mark_complete
  intent within 3 turns.
- manager-task-handoff (457 in / 1022 out): VP Engineering decomposes
  CEO p99 directive into IC-sized tasks; expects /api/task/create
  calls, references to agent_ic_* slugs, and the "p99" term.
- non-ceo-dm-reply (357 in / 545 out): VP Product handles a peer DM
  with blocking ETA ask; expects mark_read + /api/agent_message/send
  + message id preserved.

Total spend ~3.7k tokens; pnpm test:eval --run now green at 8/8
(3 harness self-tests + 5 recorded fixtures).

With these tripwires in place, C4 (hire_agent / file_approval tool
uplift + initial-user-turn rewrite) can proceed without the previous
"only CEO path has a baseline" blind spot.
2026-04-20 14:08:35 -04:00

10 lines
495 B
Plaintext

Inbound from CEO (channel=eng_direct): "We are getting p99 latency spikes on the /api/companies endpoint after the v48 migration. Figure out what broke and fix it by end of week."
Your ICs:
- agent_ic_be1 (senior backend IC, SQL + Node)
- agent_ic_be2 (mid backend IC, API + schema)
- agent_ic_fe (frontend, React)
- agent_ic_sre (SRE, metrics + profiling)
Trigger: message. Decompose the CEO directive into 2-4 IC-sized tasks now. Use /api/task/create for each. Do not write a plan document.