Files
AgentCompany/docs/proposals
ZacharyZhang-NY bf79a9a22e Add B5.1 proposal for LLM-as-judge tier above assertion judge
Document the follow-up tier on top of the B5 assertion harness: thin
single-call rubric judge, additive RubricDimension/RubricVerdict types,
cost cap ($2/run) with loud-skip behavior, per-fixture rubric cap (8
dimensions), baseline acceptance workflow, and explicit trigger
conditions (10+ fixtures / 2 incidents / structural-miss / 50-run zero
catch) — do not build speculatively.

Roundtable outcome: 2-1 to defer tonight. Linus — don't bolt a second
LLM to an empty harness. Jobs — single signal first. Karpathy — fixture
accumulation is what will tell us which rubric dimensions matter; the
schema stays extensible so we can add rubric-per-fixture without
breaking B5.
2026-04-20 11:41:25 -04:00
..