ZacharyZhang-NY
e7f879bed0
Land the B5 harness skeleton: fixture loader (strict JSONL contract with meta header + ordered turn records + assertions tail), replay-client that swaps in for pi-ai's streamSimple by replaying recorded assistant turns, structural assertion judge (tool_called / no_tool_called / message_contains_substring / tool_call_arg_matches / trace_max_turns), an eval-runner that sums the token ledger and produces an EvalResult, a vitest driver (tests/eval/harness.test.ts) that discovers *.jsonl fixtures, and a record-fixture CLI that writes fixture meta headers. Hard contracts: - No fallback: replay-client and fixture-loader throw on exhaustion / malformed input rather than synthesizing defaults. - No mock: types reuse @mariozechner/pi-ai AgentMessage / ToolCall verbatim; shadow types are forbidden. - No fake fixture data: tests/eval/fixtures/ ships only README + .gitkeep. Real traces are recorded by the user against live providers. package.json adds test:eval (vitest --passWithNoTests so an empty fixture dir is green) and eval:record (CLI entrypoint). electron-builder already excludes tests/ via its files: glob — fixtures never enter the shipped .asar.
90 lines
1.4 KiB
Plaintext
90 lines
1.4 KiB
Plaintext
# Dependencies
|
|
node_modules/
|
|
|
|
# Build outputs
|
|
out/
|
|
mac-arm64/
|
|
dist/
|
|
coverage/
|
|
*.tsbuildinfo
|
|
win-unpacked
|
|
.blockmap
|
|
.claude
|
|
/build
|
|
.omc
|
|
.omx
|
|
.planning/*
|
|
!.planning/**/
|
|
paperclip-reference
|
|
Reference
|
|
.pnpm-store
|
|
# Test artifacts
|
|
test-results/
|
|
playwright-report/
|
|
|
|
# Signing certificates (NEVER commit these)
|
|
*.p12
|
|
*.cer
|
|
*.pem
|
|
*.key
|
|
*.p8
|
|
*.csr
|
|
*.certSigningRequest
|
|
*.provisionprofile
|
|
*.mobileprovision
|
|
|
|
# Environment files
|
|
.env
|
|
.env.*
|
|
.env.local
|
|
.env.*.local
|
|
!.env.example
|
|
!.env.release.example
|
|
|
|
# Editor and OS files
|
|
.DS_Store
|
|
Thumbs.db
|
|
.idea/
|
|
.vscode/
|
|
|
|
# Local reference material
|
|
oldreference/
|
|
|
|
# Markdown docs
|
|
*.md
|
|
!/README.md
|
|
!/docs/*.md
|
|
!/docs/proposals/*.md
|
|
!/CLAUDE.md
|
|
!/AGENTS.md
|
|
!/.planning/**/*.md
|
|
!/src/main/prompt-builder/**/*.md
|
|
# Bundled Agent Skills ship their SKILL.md / README.md as source of truth.
|
|
# The Pi skills loader rejects entries whose frontmatter `name` diverges from
|
|
# the parent directory; losing SKILL.md to the blanket *.md ignore meant a
|
|
# fresh checkout silently dropped every skill from the catalogue.
|
|
!/builtin-skills/**/*.md
|
|
!/tests/eval/**/*.md
|
|
*.zip
|
|
*.blockmap
|
|
*.exe
|
|
|
|
# Local design scratch files
|
|
homepagedesign.html
|
|
|
|
# Landing page is part of the repo and should be versioned
|
|
|
|
# Boss edition build (contains credentials — never commit)
|
|
boss-build/
|
|
|
|
# Package manager logs
|
|
npm-debug.log*
|
|
pnpm-debug.log*
|
|
yarn-debug.log*
|
|
yarn-error.log*
|
|
local-release/
|
|
.omx/
|
|
|
|
# Runtime state (written by app at runtime)
|
|
state/
|