This is the verification artefact for the C4 refactor (e2c839b through
03864f3). The old fixture (recorded at 1d0cfa2, scenario asked the model
to POST hire_agent requests via bash+node+fetch) captured the model
emitting `node -e 'fetch(\`http://...\`)'` — exactly the antipattern
Jobs called out.
Post-C4 scenario was updated to reflect the new world: the CEO now has
explicit hire_agent and file_approval tools. Re-recording against
anthropic/claude-sonnet-4-5 produced 509 in / 731 out tokens and the
assistant response:
- Contains `hire_agent` (pseudo-XML tool invocations, one per hire)
- Contains `file_approval` with type=approve_ceo_strategy
- No `node -e` / bash+fetch / HTTP POST pattern anywhere
- 3-5 hires as requested, single approval block
Assertions updated to pin the post-C4 tool names as positive signals.
Absence of bash/fetch is demonstrated by diff against 1d0cfa2 (see that
commit's recorded JSONL for the before state).
Eval harness now green at 8/8 (3 self-tests + 5 recorded fixtures) with
this new baseline. Any future regression that reverts CEO bootstrap to
HTTP-path prompts will fail all three hire_agent / approve_ceo_strategy /
file_approval assertions.
ZacharyZhang-NY
8e99e0119f