test(runtime): guard pi package contracts

This commit is contained in:
ZacharyZhang-NY committed 2026-04-23 19:47:55 -04:00
1 parent c26e7420fb
commit 245586a9aa
6 files changed
+293 -26

No files matched your search

@@ -1,12 +1,13 @@
# Proposal B1.1: pi-agent-core 升级到 in-tree reference 的决策记录
> 来源:问题.md B1(pi-mono-reference 与实际依赖 62 patch 漂移)
> 状态:executed(已 commit `35c1229`)+ open(需 eval 监控)
> 状态:executed(已 commit `35c1229`)+ guarded(2026-04-23 runtime contract tests)
> 决策时间:2026-04-20 Roundtable
> Owner: TBD(建议项目 maintainer)
> 关联 commit:`35c1229 Pin pi-agent-core/pi-ai/pi-coding-agent to exact 0.67.68`
> Blocking dependencies for full close-out:B5 LLM eval harness
> Runtime guardrail:`tests/unit/pi-runtime-dependency-contract.test.ts` + `tests/integration/pi-runtime-contract.spec.ts` + `tests/eval/harness.test.ts`
> Superseded note(2026-04-22):in-tree `pi-mono-reference/` snapshot has been removed. Runtime truth now lives in root `package.json` and `pnpm-lock.yaml`.
> Guardrail note(2026-04-23):runtime truth is now executable: package pins, lockfile entries, installed package versions, exported API surface, Agent factory wiring, coding tools, skill loader, hook bundle, and eval fixture `piAgentCoreVersion` are covered by automated tests.
---
@@ -54,9 +55,27 @@ caret range `^0.67.6` 在 pnpm semver 里允许 resolve 到 `>=0.67.6 <0.68.0`
4. **transitive dep 跃迁**:`@mistralai/mistralai` 在该版本范围内做过 1.x → 2.x major bump(pnpm-lock diff 可见),如果使用 Mistral provider 行为可能变化
5. **hook 签名兼容性**:本地 `hook-handlers.ts` / `context-compactor.ts` / `cost-writer.ts` 的 hook 注入接口若有变化,会在运行时才暴露
## 自动化护栏
`pi-mono-reference/` 删除后,源级 review 能力转为本项目内的运行时契约。当前护栏分三层:
1. `tests/unit/pi-runtime-dependency-contract.test.ts`
- `package.json` 三个 pi runtime 包必须精确 pin 到 `0.67.68`
- `pnpm-lock.yaml` importer / resolution / snapshot 必须指向同一版本
- `node_modules` 中实际安装包版本必须一致
- AgentCompany 使用的 pi API surface 必须存在
2. `tests/integration/pi-runtime-contract.spec.ts`
- `createPiAgent` 必须创建真实 `Agent`,保持 sequential tool execution 和三类 hook
- `createRunScopedCodingTools` 必须输出当前 worker 依赖的 coding-tool surface
- bash spawn hook 必须保留 run-scoped env 注入
- `loadSkills` 必须能读取 progressive-disclosure `SKILL.md`
3. `tests/eval/harness.test.ts`
- 每个 fixture 的 `meta.piAgentCoreVersion` 必须等于 shipping runtime version
- 真实 fixture 继续穿过 `pi-agent-core` Agent loop replay
## 监控与回滚计划
### 监控指标(手动观察直到 B5 落地)
### 监控指标
- 启动主进程后第一个 agent run 是否能跑完一轮 tool call
- compactor 触发时是否有 tool_use/tool_result 报错
- 长会话(>50 轮)是否出现非预期 cost / token 偏差
@@ -85,7 +104,7 @@ git commit -m "Revert pi-agent-core pin to 0.67.6 due to behavior regression"
## 完成定义
本 proposal 在以下条件全部满足时关闭:
1. B5 LLM eval harness 落地,能跑一组 agent 行为 fixture
2. 在该 harness 上对比 `0.67.6` ↔ `0.67.68` 的行为 diff
3. 行为 diff 报告附 commit,确认无意外回归
4. 关闭 B1.1,写入到 changelog
1. `package.json` / `pnpm-lock.yaml` / installed package versions 三者一致
2. pi exported API surface、Agent factory、coding tools、skill loader、hook bundle 都有自动化契约
3. eval fixtures 的 `piAgentCoreVersion` 与 shipping runtime version 一致
4. `pnpm verify` 覆盖 unit / integration / eval / e2e