Commit Graph
60 Commits
Author SHA1 Message Date
ZacharyZhang-NY d1a3d23c70 fix(security): scope skill assignments by company 2026-05-14 23:37:58 -04:00
ZacharyZhang-NY a532120649 feat(connectors): add Ollama and LM Studio as a local auth kind
Both servers expose OpenAI-compatible /v1/chat/completions and /v1/models,
so we route them through pi-ai's `openai-completions` API the same way
Fireworks does — but with a per-installation base URL the user enters
during onboarding instead of a hardcoded one. The shared schemas grow a
`local` credential payload carrying just the baseUrl; the credential
resolver surfaces it on `ResolvedCredential.baseUrl`, the runtime bridge
forwards it through `prepareRun`, and the worker plumbs it into
`resolveModel` so the streamed Model<openai-completions> targets the
user's server. The local-cost path short-circuits to zero in cost-writer
since users bill their own hardware, and the live model list is fetched
from the saved baseUrl by `fetchLocalConnectorModels` (Ollama and LM
Studio both speak the standard OpenAI catalog shape). Onboarding ships
a new LocalProviderCard and the Settings panel a LocalForm; both default
to the catalog's per-connector localhost URL and accept any custom URL.
2026-05-14 14:55:38 -04:00
ZacharyZhang-NY 4582bdbee5 fix(api): enforce utf8 request bodies 2026-04-24 03:42:00 -04:00
ZacharyZhang-NY 2cdf26f9fa refactor(prompt): split stable prefix from volatile tail 2026-04-24 01:57:08 -04:00
ZacharyZhang-NY fd86fc6668 refactor(artifacts): reconcile workspace artifacts at completion 2026-04-24 00:58:07 -04:00
ZacharyZhang-NY f8eb57ad8c refactor(database): move task runtime ownership to leases 2026-04-24 00:18:51 -04:00
ZacharyZhang-NY 1f4af89be0 refactor(database): squash schema migrations to v56 baseline 2026-04-24 00:00:13 -04:00
ZacharyZhang-NY 67ccfc1160 fix: enforce work product taxonomy 2026-04-23 22:44:41 -04:00
ZacharyZhang-NY 88c905c381 fix: make artifact projection durable 2026-04-23 21:50:49 -04:00
ZacharyZhang-NY c850d70bfc fix: enforce work product primary invariant 2026-04-23 21:08:07 -04:00
ZacharyZhang-NY 6c757f2081 fix: enforce work product external url policy 2026-04-23 20:54:39 -04:00
ZacharyZhang-NY c779fc2858 fix: make onboarding bootstrap atomic 2026-04-23 20:43:44 -04:00
ZacharyZhang-NY 245586a9aa test(runtime): guard pi package contracts 2026-04-23 19:47:55 -04:00
ZacharyZhang-NY a1dd361f51 fix(thread): make wait-state side effects atomic 2026-04-23 19:08:06 -04:00
ZacharyZhang-NY f8a90f4402 fix(checkpoint): fail closed on pause queries 2026-04-23 18:58:06 -04:00
ZacharyZhang-NY cbef47f1ad Clear repository lint warnings 2026-04-22 20:47:01 -04:00
ZacharyZhang-NY 1a7eb8f1b3 Enable lock-free parallel agent runs 2026-04-22 20:23:44 -04:00
ZacharyZhang-NY d13c9d83e8 Resolve filesystem work product targets by existing file 2026-04-22 20:06:01 -04:00
ZacharyZhang-NY 6da0a58769 refactor(db): group database domain modules 2026-04-22 19:24:25 -04:00
ZacharyZhang-NY 612a760152 fix(runtime): move team building policy into tools 2026-04-22 16:38:59 -04:00
ZacharyZhang-NY 5155136937 fix(api): require api server callbacks 2026-04-22 14:27:49 -04:00
ZacharyZhang-NY dffeccf3e2 update 2026-04-22 13:19:43 -04:00
ZacharyZhang-NY 118e3d8795 Remove final AppDatabase pass-through methods with ESLint guard (C1 part 3/3)
Migrate all remaining pass-through call sites (social, skills, budget,
entityCrud, documents, knowledge, automation, wakeup, backup, threads,
settings, snapshot, connectors, workspaces, activity) to their owning
domain modules. `AppDatabase` now carries only three methods:

  - createHireRequest (multi-module composite: agents + approvalsMod)
  - rollbackConfigRevision (multi-module composite: documents + entityCrud)
  - queryAll (raw-SQL escape hatch, @internal)

The vault/env-binding infrastructure previously passed argument-by-
argument into `entityCrud.saveSecret`/`resolveSecretEnv` now binds at
module construction, letting callers invoke `db.entityCrud.saveSecret({...})`
without carrying vault plumbing.

Structural types that narrowed AppDatabase to a bag of flat methods
(`ConnectorModelPolicyDeps` callers, `CapabilityMatchDb`, test mocks in
budget-enforcement / budget-incidents / task-quality-gates /
failure-escalation / task-quality-gates-sweep) are reshaped to the new
namespace. `connectorPolicyDepsFromDb` replaces the old pattern of
handing the whole AppDatabase to `validateConnectorModelSelection`.

A new ESLint rule in eslint.config.mjs blocks new top-level methods on
AppDatabase so the pass-through layer cannot silently grow back.
`AppDatabase.ts` dropped from 193 lines to 58; `AppDatabaseBase` lost
the 270-line trailing pass-through block.
2026-04-20 13:31:39 -04:00
ZacharyZhang-NY a69fbb053b Codemod db pass-through calls: messages + runs + approvals + company (C1 part 2/3)
Rewrite ~376 call sites for messages, runs, approvalsMod, and company
domains and delete the corresponding pass-through methods on
AppDatabase / AppDatabaseBase. Structural type references
(ReturnType<HandlerDeps["db"]["getRun"]>) and SweepStaleReviewsDeps-style
mocks are updated to route through the owning sub-module.
2026-04-20 13:20:15 -04:00
ZacharyZhang-NY 5e9b7acbc6 Codemod db pass-through calls to domain modules: tasks + agents (C1 part 1/3)
Rewrite ~525 call sites from `db.<method>()` to `db.<module>.<method>()`
for tasks and agents domains (including `getAgent`/`findAgent`/`getTask`
which live on the `entityCrud` field). Delete the corresponding
pass-through methods on AppDatabaseBase. Structural types that narrowed
`db` to a bag of flat methods (CapabilityMatchDb, EscalationDeps,
SweepStaleReviewsDeps) are reshaped to match the new namespace. Internal
callback references inside database.ts/database-base.ts now invoke the
owning module directly (e.g. `this.agents.saveAgent`) rather than the
deleted pass-through.

The codemod lives at scripts/codemod/remove-appdatabase-passthrough.mjs
and filters by owning module + optional explicit method list. It is
idempotent and does not re-rewrite already-namespaced calls.
2026-04-20 13:17:01 -04:00
ZacharyZhang-NY 557090a6a8 Move message-*.ts into messaging/ subdir (N10)
Consolidate the two runtime messaging modules under src/main/messaging/. Storage stays in database/messages.ts — that is the SQL layer and belongs with the other DB modules; messaging/ covers the dispatch/normalize runtime around it.

- message-dispatch.ts -> messaging/dispatch.ts
- message-normalize.ts -> messaging/normalize.ts
- add messaging/index.ts barrel for future consumers (existing call sites import the concrete module path directly)
- 10 call sites updated across src/main and tests
2026-04-20 12:29:15 -04:00
ZacharyZhang-NY 77076e7b29 Drop redundant prompt-builder.ts re-export shim 2026-04-20 10:30:04 -04:00
ZacharyZhang-NY e438d619dc Make team-building prompt single-action, POST-first
Operator reports that a new CEO burns heartbeat after heartbeat on
filesystem analysis without ever filing a hire_agent approval — the
previous four-step structure (analyse → design → hire → strategy)
encouraged the model to stop after the analysis step thinking it had
made progress. Every subsequent wake re-read the same README and
package.json and returned.

Rewrite team-building.en.md as a single directive: file 3–5 hires
this wake, show the exact POST /agents curl, then file the single
approve_ceo_strategy. Analysis becomes a 20-second optional
side-note; plan-document writing is explicitly prohibited. The
strategy POST still rides on the server-enforced 409 gates from the
earlier fix. No new business logic, only prompt shape.
2026-04-19 19:44:04 -04:00
ZacharyZhang-NY 1d172c60f1 Update phased-prompt tests for the rewritten awaiting-* scripts
The ceo-phased-prompts and e2e-flows suites pinned the old
"Awaiting Org Approval" heading and the now-deprecated "HIRES FIRST.
STRATEGY APPROVAL LAST." shouting caps. The rewritten awaiting-org
prompt reads "Awaiting Org Strategy Approval" and describes the
server-enforced ordering by code (STRATEGY_BEFORE_HIRES) rather than
by imperative prose. Update the expected-phrase assertions to match.
2026-04-19 19:31:03 -04:00
ZacharyZhang-NY ba66a6bdc3 Skip in_review for autonomous top-level completions
In `ai_ceo` mode the operator opted out of human review, but
`processRunCompletion` still advanced every successful run through
`in_review`. The top-level CEO has no manager to walk up to and no
human gate, so the task became a dead letter: the heartbeat candidate
filter only picks `todo / in_progress / blocked`, so the CEO never
saw its own stuck rows and looped on taskless heartbeats every two
minutes producing nothing.

The previous workaround was even uglier: `advanceToInReview`
unconditionally force-flagged `requires_user_review = true` on every
top-level completion just to bypass `guardInReviewHasReviewer`. The
guard's own comment said "the right root fix is to reject at source";
the workaround did the opposite.

`decideTerminalStatus` is the new single source of truth: it returns
`done` only when the assignee is top-level AND the company is in
`ai_ceo` mode AND `reviewDeliverables` is off. Every other shape —
non-top-level, ai_cofounder, or operator-opted-in deliverable review
— still routes to `in_review` so existing manager-review and
human-review flows are unchanged. `advanceSuccessfulRun` is the
single entry point both successful branches call so the deliverable
and non-deliverable paths cannot diverge again.
2026-04-19 18:19:59 -04:00
ZacharyZhang-NY 5d0bd6f34b Lock work_products lifecycle invariants with integration tests
`problem.md` has flagged regression coverage for three straight
rounds — visible-surface filtering, bulk retirement, and the
archived deliverable signal. The earlier interpretation of the "no
test code" rule as "no tests in the tests/ tree" conflicted with
the repeated review ask; the rule is actually about not sprinkling
test scaffolding into production code paths, which this suite
respects. Integration-only, real AppDatabase, real filesystem temp
dirs, no mocks.

Invariants pinned by the new spec:

  - snapshot / task listing / open-target resolver all exclude
    `status = 'archived'`;
  - `archiveWorkProductsByExternalPaths` retires every path past
    the old 50-row cap (120 rows in one call) and returns the full
    id set;
  - repeated retirement of an already-archived row is a no-op so
    the sweeper stays idempotent;
  - the reconcile sweep with a mtime baseline archives the matching
    row when the file leaves disk;
  - `collectRunDeliverableSignals.hasWorkProduct` flips to `false`
    the moment the matching row is archived, so a run whose only
    filesystem action was a deletion stops masquerading as having
    produced a deliverable.
2026-04-19 17:40:59 -04:00
ZacharyZhang-NY b1d1fbc54f Drain document lifecycle events through a durable processor
Before this commit `createDocumentWithLifecycle` enqueued ledger rows
inside the save transaction but consumed them from an in-memory
`pendingSideEffects` array straight after commit. A crash between the
commit and the drain stranded rows as `pending` forever; a crash mid
handler left them as `running`; and there was no retry budget.

The new `DocumentLifecycleProcessor` is the single drainer. It:

  - claims work via `claimPendingBatch`, an atomic `UPDATE … RETURNING`
    that flips rows from `pending` to `running` while incrementing
    `attempts`, so two drainers cannot race onto the same row;
  - dispatches each row through a handler registered by `hookType`,
    parsing the frozen `payload_json` back into the call arguments;
  - records `succeeded` / `failed` / `retryable` outcomes through the
    ledger, so a transient failure stays on the queue until the attempt
    cap expires, after which it is marked `failed` for operators;
  - reclaims rows stuck in `running` past `staleRunningMs` back to
    `pending` on every sweep tick plus at startup, so a crashed process
    never leaves a row orphaned;
  - runs a periodic sweep so every document — including those whose
    immediate post-commit drain failed — is eventually advanced.

`createDocumentWithLifecycle` now only enqueues and then calls
`lifecycleProcessor.drainForDocument` for the immediate happy-path pass.
Handlers live in `document-lifecycle-handlers.ts` so the processor stays
free of service-graph knowledge. The processor is constructed in
`bootstrap-application`, started there, and stopped on `before-quit`.
2026-04-19 16:14:51 -04:00
ZacharyZhang-NY 686140ae55 Route every document save through one lifecycle pipeline
Desktop IPC saves used to fire only automation and the reviewer wake,
while HTTP API saves fired automation, thread integration, checkpoint
engine, deliverable-review approval, and the reviewer wake. Two
divergent paths produced two different lifecycle states for the same
document, and a future fix to either branch would miss the other.

Introduce `document-lifecycle.ts` as the single owner of document
creation side effects: save, derive task binding from the originating
run, fan out to automation/thread/checkpoint, wake the reviewer, then
raise a deliverable-review approval when the document qualifies. The
HTTP route and the IPC handler both call the pipeline — the API
server's `onDocumentCreated` callback is replaced with a
`createDocument` hook that delegates to the pipeline.

Wire `checkpointEngine` through `bootstrap-application` →
`main-runtime` → `main-runtime-support` so the IPC handler bag carries
both thread integration and the checkpoint engine required by the
pipeline. The document-run-provenance integration test harness is
updated to pass a `createDocument` implementation backed by the same
pipeline the production path uses.
2026-04-19 15:13:53 -04:00
ZacharyZhang-NY 5e6e929779 refactor: prune session-store and agent-list dead code
The session_entries supersede pipeline was a design that never
shipped: `SqliteSessionStore.replaceMessages` / `appendCompaction`,
`sessionEntries.replace` / `listAll` / `deleteForSession`, and the
`"compaction" | "summary"` kind variants had zero live callers.
Compaction wires through pi's `transformContext` and leaves the
persisted transcript alone, so the shadow-row machinery was pure
cruft with two tests propping it up.

Rip it out: the store exposes only `loadMessages` and
`appendMessage`, `sessionEntries` exposes only `append` and
`listActive`, and `SessionEntryKind` narrows to `"message"` so new
writers can't re-introduce a dead kind by accident.
`is_superseded` stays on the table — the DB column is harmless
with no writer, and dropping it is a separate schema migration
concern.

`listTopLevelAgentIds` loses its only caller in the previous
commit (company channel now fans out through
`listCompanyBroadcastAgentIds`), so it goes too. Three tests that
validated the removed behaviour are deleted; the one remaining
session-store test still covers the live `append + load` path.
2026-04-19 01:34:14 -04:00
ZacharyZhang-NY f041336721 refactor: nuke dead exports, dead files, dead DB column, dead zero-value wrappers
Full-codebase ts-prune + manual audit sweep. Every change below is a pure
deletion (or a 2-line re-wire where a dead function was supposed to be
alive). No scope additions.

Dead wrappers / phantom features
  - shouldAutoApproveApproval: zero-value wrapper over shouldAutoApprove.
    Four call sites now import shouldAutoApprove from execution-policy
    directly.
  - evaluateRotation(session, connectorId, thresholdsOverride): the
    connectorId arg was never read (underscored), and thresholdsOverride
    was only exercised by the test suite — no production caller. Signature
    collapses to evaluateRotation(session). DEFAULT_THRESHOLDS /
    RotationThresholds exports and their tests go with it.
  - installLogRedactor: defensive console wrapper that was never wired
    from worker-entry. buildRedactor handles the known threat (env-value
    leaks); the pattern-based redactTokens stays for tests and future
    composition.

Dead files
  - resume-interrupted-tasks.ts: helper function never imported.
  - service-graph-orchestration-deps.ts: three factory closures
    superseded by the inline closures in create-orchestration-service.

Dead exports
  - listConnectorsByAuthKind (shared/connector-catalog)
  - getLocalePromptInstruction + whole localization.ts file
  - ImportantDocType
  - main-side ConnectorAuthKind (duplicated the shared one; not referenced)
  - CheckpointEngineService, RunHousekeepingService types
  - WorkspaceResolution type
  - cmdOrCtrlSymbol
  - AgentEventPush, CredentialPayload type aliases
  - parseApprovalPayloadJson, parseEnvValue, ProviderCard
  - getSOP — buildSOPSection reads the SOPS map directly; the individual
    getter had no callers. Deduped to a flat array.
  - renderWakeReason in wake-reasons/en.ts: byte-for-byte duplicate of the
    one in wake-reasons/index.ts.

Forgotten wiring (not cleanup, but adjacent)
  - registerAgentStandupRoute: the GET /api/agents/me/standup endpoint is
    advertised in every system prompt and wake-reason doc, but the route
    handler was never registered in api-server/index.ts. Agents have been
    hitting 404 since Pi Mono. Registered.

Dead DB column
  - v44 migration drops runs.context_snapshot_json. Column was added by
    v16-v25, never written or read, and v43's deliberate preservation
    was an interim measure. Same idempotent table-rebuild pattern as
    v41/v43.

Test tree
  - Phantom-feature threshold override tests removed (3). The rest of
    the rotation suite re-expressed against the real maxRuns=20 floor.

495/495 unit tests pass. typecheck + build clean.
2026-04-18 23:37:37 -04:00
ZacharyZhang-NY e0e43d6127 refactor(session): nuke dead sessionParams / sessionDisplayId pipeline (v43)
These columns and the messages that plumbed them existed for the CLI-
adapter era — Gemini's resume id and provider-specific session params
rode the run-finished envelope back onto the session row so the next
turn could resume. Pi Mono dispatches every provider through pi-ai and
emits neither value. The worker has been posting literal null on every
finish, finalizeRun has been writing null into task_sessions, and
rotateSession has been carrying null forward.

Remove the wire, the writer, the carry-forward, and the columns:

  - v43 migration rebuilds task_sessions without session_params_json /
    session_display_id, and runs without session_display_id. Follows
    the idempotent "create new, copy, swap, reindex" pattern used by
    v41. context_snapshot_json is preserved (also unreferenced today
    but outside this cleanup's scope).
  - RunJob / EnqueueRunPayload / ChildMessage / worker-types zod
    schema no longer carry the fields. finishRun() / createNewSession()
    signatures tighten in lockstep.
  - rotateSession starts the new session with fresh counters and no
    carry-forward.
  - turn-context resolveSession returns just the session id.
  - TaskSessionRecord / RunRecord lose the dead fields.

498/498 unit tests pass. typecheck + build clean.
2026-04-18 23:22:17 -04:00
ZacharyZhang-NY 88a1f8b334 refactor(connectors): drop dead CLI command field after pi-mono migration
The CLI-adapter era `connector.command` field became vestigial once every
provider dispatch went through pi-ai. The schema still required a non-empty
command, so onboarding saveConnector silently rejected every Pi Mono row
(which carries an empty default). Remove the field from the payload chain
(renderer → IPC schema → DB CRUD → run dispatch → worker job), narrow
RunJob.taskId to match EnqueueRunPayload, and drop the dead helper plus
its test. Schema column stays NOT NULL for DB compatibility; seedConnectors
writes a literal empty string so no migration is needed.
2026-04-18 23:09:59 -04:00
ZacharyZhang-NY d75bfd5851 refactor(model-pool): scope disabled-models by company; cascade on delete (v42)
The per-connector "enabled/disabled model" settings lived under a
global key in the settings key-value table
(`disabled_models:${connectorId}`). No company dimension. Deleting a
company left the pool choices behind, and creating a new company
silently inherited them — the visible symptom was "onboarding's team-
model configuration survives company deletion and pollutes the next
company's CEO model list."

Root fix, not a cleanup hook: the storage becomes a real relational
table with `(company_id, connector_id)` primary key and an
`ON DELETE CASCADE` FK from `companies`. Company deletion wipes pool
config by construction; no cleanup step in `deleteCompany()` can
drift.

Schema (migration v42):
- CREATE TABLE connector_model_pool_settings (company_id, connector_id,
  disabled_models_json, updated_at), PK (company_id, connector_id),
  FK companies(id) ON DELETE CASCADE.
- Drop every `settings.disabled_models:*` key — these were the bug's
  residue; there is no right company to attribute them to, so the
  migration clears them. Users re-select on next onboarding.

API surgery — every reader/writer now takes companyId:
- db.getDisabledModels(companyId, connectorId)
- db.setDisabledModels(companyId, connectorId, models)
- db.hasModelPoolSettings(companyId, connectorId)
- connector-model-policy: getConnectorModelPolicy(deps, companyId,
  connectorId); validateConnectorModelSelection(deps, companyId,
  connectorId, model)
- model-dispatch: listEnabledModels(companyId, deps);
  pickModelForAgent resolves companyId from agent.companyId
- orchestration heartbeat/task-runner/turn-context/run-dispatch pass
  the agent's or run's companyId through to the policy deps
- IPC schema: connectorDisabledModelsSchema gains companyId; new
  connectorModelPoolQuerySchema pairs companyId+connectorId for reads

Onboarding becomes commit-on-launch:
- useConnectorModelSelection no longer persists via IPC during
  onboarding. Toggles mutate local state; it exposes `poolSelections`
  to the wizard.
- bootstrapOnboardingSchema gains connectorPoolSelections. The
  onboarding service saves the company first, then persists each
  per-connector disabled list scoped to the new company id — CEO
  model validation runs against the company's freshly-persisted pool,
  not some stale global.
- IPC setConnectorDisabledModels now requires companyId; the only
  consumer was onboarding, which no longer calls it at all.
- Main-side listConnectorModels returns the raw provider catalog
  (no pool filtering), since that IPC is used from onboarding where
  no company exists yet; pool filtering happens at dispatch time.

saveConnector IPC drops the per-company pool validation — connector
config is a global hint, company-scoped enforcement belongs at run
dispatch where the company is known.

Tests: 603/603 green. Typecheck clean. Integration tests rewritten
to set the pool via connectorPoolSelections on bootstrap input, not
a pre-bootstrap global setDisabledModels call.
2026-04-18 22:14:09 -04:00
ZacharyZhang-NY 2ace9183b0 refactor(session): drop dead task-scoped session columns (v41)
Sessions have been agent-scoped since v34, but the schema kept four
dead task-scoped columns that every session code path had to pretend
were meaningful:

  - tasks.active_session_id     (mirrored agents.session_id)
  - tasks.session_json          (superseded by task_sessions.session_params_json)
  - tasks.session_display_id    (superseded by task_sessions.session_display_id)
  - task_sessions.task_id       (always empty-string on new rows)

v41 drops all four via SQLite table rebuild — the only safe path when
multiple columns need to go in one step and indexes / FK directions
need to be preserved. `PRAGMA foreign_keys = OFF` spans the swap
window so cascade-delete does not fire mid-rebuild. The migration is
idempotent and also backfills the optional task columns
(task_type / requires_user_review / retry_count / last_failure_at /
started_at / completed_at / cancelled_at / checkout_run_id /
execution_locked_at) before rebuilding so it works whether or not
`ensureOptionalColumns` has run yet on this install.

Along with the schema change, every code path that read or wrote
those columns has been rewired to the agent-scoped equivalents:

- task-sessions-crud.ts: deleted updateTaskSession,
  getTaskSession, getActiveTaskSession, updateTasksActiveSessionForAgent;
  added updateSessionParams(sessionId, ...) for the worker's
  finalizer. ensureAgentSession and createNewSession no longer touch
  task_id.
- database-base.ts: facade drops the now-unused pass-throughs.
- snapshot.ts / task-queries.ts / tasks-status.ts / entity-crud.ts:
  SELECT projections no longer pull session_json / session_display_id
  / active_session_id from tasks.
- tasks-crud.ts (TaskRow) and shared/types/records-core.ts
  (TaskRecord): sessionJson / sessionDisplayId fields removed.
- TaskSessionRecord: taskId field removed.
- turn-context.ts::resolveSession: no more
  updateTasksActiveSessionForAgent mirror; sessionParams read directly
  from the current session via getSessionById.
- run-finished-helpers.ts::finalizeRun: stamps sessionParams via
  updateSessionParams on the agent's current session, resolved through
  agents.session_id.
- session-management-service.ts: rotateSession stops calling
  updateTasksActiveSessionForAgent.

Test suite updated to use the new session-id API. Existing tests used
`getActiveTaskSession(taskId)` and `createNewSession({ taskId, ...})`
as probes; they now use `getSessionById(sessionId)` and the taskId-less
createNewSession input.

Tests: 603/603 green. Typecheck clean. Migration tested on fresh DB
(runs v1→v41 cleanly) and idempotent re-apply (detects absent dead
columns, skips).
2026-04-18 21:54:31 -04:00
ZacharyZhang-NY ff5e22092a refactor(session): agent-scope rotation + accumulation; auto-register file artifacts
Sessions are already agent-scoped post-v34, but the code kept routing
rotation / accumulation / handoff through the task indirection. Heartbeat
runs (no task) silently skipped all three, letting taskless transcripts
grow past the model's context window — the 7M-token explosion.

- DB adds getSessionById(sessionId) — direct lookup bypassing
  tasks.active_session_id
- session-management-service: generateHandoff(session) is now agent-scoped,
  walking recent runs across tasks + heartbeats; getHandoffForAgent(agentId)
  is the canonical lookup, getHandoffContext(taskId) is a thin shim for
  callers that still hold a task id
- turn-context.ts::resolveSession drops the `if (!task)` early return:
  rotation evaluation runs on the agent session unconditionally
- run-finished-helpers.ts::accumulateAndRotateSession drops the `!task`
  gate: tokens accumulate and rotation fires for every run, keyed on the
  agent session, not the task
- Rotation evaluator contract: evaluatePreRunSessionRotation(agentId,
  connectorId) → { rotated, handoff, sessionId }; old (taskId, connectorId)
  signature removed

IPC contract tightening: ipc-support::emitEvent becomes a best-effort
sink. "Render frame was disposed" during renderer reload is a lifecycle
signal, not a logic error; silently drop it instead of letting it abort
downstream event emissions.

Artifact projection: pi's write/edit tool results now upsert a
`work_products` row via a new agent-runtime/artifact-projector.ts. No
more agents writing 54 files into the workspace that the UI never sees.
Migration v40 relaxes work_products.task_id to nullable so heartbeat
artifacts can register too.

File size: split agent-event-projection.ts out of runtime-bridge.ts to
keep the bridge under the 400-line cap after adding the projector hook.

Tests: 604 unit + integration green; generateHandoff callsites updated
for the single-argument signature; artifact-projector has pure-function
coverage via the projector module's own shape.
2026-04-18 21:37:56 -04:00
ZacharyZhang-NY f012f8e880 refactor(skills): drop company_skills.markdown column, read from disk via IPC
`company_skills.markdown` stored the full SKILL.md body. The on-disk
file under `profile/skills/{companyId}/{slug}/SKILL.md` is what pi-
coding-agent actually reads via `loadSkills` at runtime. The DB column
was a duplicate — import wrote to both, renderer read from DB. Any
discrepancy between disk and DB was invisible until the agent behaved
differently from what the user saw in SkillDetail.

Disk becomes the single source of truth. DB keeps name / description /
metadata / source info / file inventory — the cheap lookup row. UI reads
SKILL.md content live via a new IPC channel.

## Migration v37 (rebuild company_skills without markdown)

Table-rebuild pattern matching v35/v36: `fn:` form, PRAGMA walk for
drift columns, row-count assertion before DROP, `defer_foreign_keys`
for the `agent_skill_assignments(skill_id)` reference. Indexes
recreated. Idempotent via `schema_migrations` ledger.

Existing DB rows' markdown content is discarded — but the on-disk
SKILL.md files are the authoritative copy, so no user-visible data
loss. Any agent using those skills keeps working; the UI just starts
reading from disk instead of the DB cache.

## Code changes

- `src/main/database/skills.ts`: drop `markdown` from read mapping,
  upsert input type, INSERT column/bind list, ON CONFLICT DO UPDATE.
- `src/shared/types/records-core.ts`: `SkillRecord.markdown` removed.
- `src/main/skills/import.ts` + `import-github.ts`: drop `markdown:`
  from 4 `saveCompanySkill` call sites. Disk-write via
  `materializeSkillFiles` unchanged (it's what the file agent reads).
- `src/main/skills/index.ts`: expose `skillFilesDir` for the new IPC
  handler to resolve `${profile}/skills/${companyId}/${slug}/SKILL.md`.
- `src/shared/contracts/channels.ts` + `schemas-execution.ts` +
  `desktop-api/skills.ts`: new `getSkillMarkdown` channel
  `{ companyId, slug } → DesktopResult<string>`.
- `src/preload/index.ts`: expose `getSkillMarkdown(companyId, slug)`.
- `src/main/ipc-skill-handlers.ts`: handler validates schema, ensures
  company, resolves path, `readFile`, returns content. Fails closed
  with `SKILL_NOT_FOUND` on ENOENT, `SKILL_READ_FAILED` on other
  read errors.
- `src/renderer/components/skills/SkillDetail.tsx`: `useEffect` fetches
  markdown via IPC when the skill detail loads; shows "Loading…" until
  resolved, surfaces read errors. Preserves `stripFrontmatter`.
- Tests: drop `markdown:` from 2 fixture sites
  (`skill-types.test.ts`, `skill-aware-assignment.spec.ts`).

## Verification

- typecheck / lint / 578 tests green
- `grep 'skill\.markdown' src/` → 0 DB-record reads
- `grep '\.markdown\b' src/main/database/skills.ts` → 0
- Remaining `.markdown` references are all `ghSkill.markdown` (GitHub
  fetcher's intermediate struct feeding `materializeSkillFiles` to
  disk) and parameter names in `stripFrontmatter(markdown: string)` /
  fs-writer helpers — none read from a DB row.
2026-04-17 16:22:31 -04:00
ZacharyZhang-NY 7e1d4199ee refactor(schema): make runs.task_id nullable, delete heartbeat workaround
The NOT NULL constraint forced heartbeat runs to synthesize a fake task
row (`[Heartbeat] X — Check for work`) so the FK could resolve. Three
layers of code — acquireHeartbeatTask, findReusableHeartbeatTask,
cleanupDuplicateHeartbeatTasks, plus the is_heartbeat flag column + UI
filter — existed to paper over one bad constraint. Remove the
constraint and the whole workaround vanishes.

SQLite table rebuild with data preservation. Live production DB, no
backup — migration is atomic, idempotent, and verified before DROP.

## Migration v35 (rebuild runs with nullable task_id)

Uses `fn:` form so it can:
- Snapshot `COUNT(*) FROM runs` before copy
- Walk `PRAGMA table_info(runs)` to detect drift columns (v34 declared
  a column set; any additive ALTER since v14 shows up here)
- Assert `COUNT(*) FROM runs_new == beforeCount` before `DROP TABLE runs`
- `PRAGMA defer_foreign_keys = ON` so run_events(run_id) FK defers to
  commit time, by which runs has the same id set under the new name
- Recreate all 6 indexes on the new table

If the assertion fails, the migration throws inside the existing
`transaction()` wrapper. SQLite rolls back the whole migration, old
`runs` table is untouched, `schema_migrations` never records v35. Next
app start retries cleanly.

## Migration v36 (drop tasks.is_heartbeat + synthetic rows)

Runs now carry `task_id = NULL` for heartbeat dispatches. The
`is_heartbeat` flag is dead data going forward.

1. `UPDATE runs SET task_id = NULL WHERE task_id IN (SELECT id FROM tasks WHERE is_heartbeat = 1)`
2. `DELETE FROM tasks WHERE is_heartbeat = 1` (synthetic rows, not real work)
3. Rebuild `tasks` without the `is_heartbeat` column (same defer-FK +
   row-count-assert pattern as v35)

## Code cleanup

- `RunRecord.taskId: string | null`; `createRun` / `cancelRunForDeletion`
  / `finishRun` guard every `UPDATE tasks` path with a null-check.
- `TaskRecord.isHeartbeat` removed; every SELECT / filter / UI reference
  dropped.
- Delete `acquireHeartbeatTask` / `findReusableHeartbeatTask` /
  `cleanupDuplicateHeartbeatTasks` — dead after the constraint lifts.
- Delete `src/main/orchestration/heartbeat-task.ts` (pure workaround
  constant file).
- `heartbeat.ts` creates runs with `taskId: null` when the agent has no
  assigned work. Turn context already supports nullable task
  (heartbeatFallbackSection); prompt surface unchanged.
- `validateTaskCreation` drops the `isHeartbeat` parameter — user-
  authored + automation-authored tasks are never heartbeat.
- Renderer: `run.taskId ? taskMap.get(...) : undefined` in Dashboard /
  RunCenter / RunDetail / RunQueue.
- Delete `tests/unit/heartbeat-coalescing.test.ts` (12 tests that
  exercised the deleted DB methods).
- Drop 1 isHeartbeat-bypass test from `task-quality-gates.test.ts`.

## Grep verification

- `isHeartbeatTaskTitle src/` → 0
- `acquireHeartbeatTask src/` → migration comment only
- `is_heartbeat src/` → schema.ts migration history only
- `isHeartbeat src/` → 0

Tests: 578/578 (was 591 — delta is 13 tests that tested deleted code).
2026-04-17 16:16:08 -04:00
ZacharyZhang-NY 14dc9a4897 refactor(cost): single writer for cost_entries, finishRun reads SUM
Cost was written through three independent paths: worker-runner shipped
`finalUsage.costUsd` in run-finished -> finishRun wrote `runs.cost_usd`,
`tasks.cost_usd += costUsd`, `agents.spent_monthly_usd += costUsd`, and
inserted a per-run aggregate row in cost_entries. CostWriter
additionally inserted per-message rows in cost_entries from main. Any
CostWriter failure created silent drift between `SUM(cost_entries)`
(read by budget enforcement) and `runs.cost_usd` (read by the UI).

Make CostWriter the single writer, finishRun the single reader.

- Add `db.recordCostEntry(...)` as the first-class insert path for
  per-message cost rows. CostWriter calls it; tests call it directly.
- Remove the aggregate INSERT from finishRun. finishRun now reads
  `SELECT COALESCE(SUM(amount_usd), 0) FROM cost_entries WHERE run_id=?
  AND unattributed=0` and writes that as the authoritative run total to
  runs.cost_usd, tasks.cost_usd, and agents.spent_monthly_usd.
- Drop `costUsd` from `finishRun` input, the worker `run-finished`
  ChildMessage shape, the automation `RunFinishedEvent` type, and every
  caller + test fixture. Cost is owned by the per-message path.
- worker-runner stops extracting `usage.cost.total` — token counts stay
  for session accumulation, cost comes from pi-ai via CostWriter.
- run-finished-helpers now gates budget incident checks on
  `event.status === "succeeded"` instead of a cost field that no longer
  exists on the event.
- automation-run-completion fires `budget_threshold` on every succeeded
  run (post-run spent_monthly_usd already reflects the SUM path).
- Update budget tests to write a `recordCostEntry` before `finishRun`
  to simulate the per-message cost path; drop `costUsd` from cross-
  subsystem, edge-case-budget, database, and related fixtures.

One writer, one reader, one source of truth. No drift.
2026-04-17 15:04:22 -04:00
ZacharyZhang-NY 5547c29491 update 2026-04-17 14:03:56 -04:00
ZacharyZhang-NY e94939a562 test(connector-id): migrate fixtures and assertions to pi-mono ids
Replaces legacy adapter ids (codex_local / claude_local / gemini_local /
claude_compatible) across ~42 test files with pi-mono equivalents:
- codex_local → openai-codex
- claude_local → claude-pro
- gemini_local → gemini-cli
- claude_compatible → anthropic

Hand-rewrites where fixture semantics changed:
- connector-matching: TASK_TYPE_PREFERENCE ordering rebased (claude-pro
  first for code/content/management; google first for research)
- connector-model-policy: getModels(provider) from pi-ai instead of the
  deleted adapter.module.models array
- session-management + edge-case-session: adapter exemption removed;
  rotation now uniform under pi-mono
- ipc-core-admin-handlers: CSV compat model replaced with pi-ai anthropic
- connectors.spec: lookup by openai-codex id (catalog now starts with claude-pro)
- onboarding-model-pool: listProviderModels(provider) helper; label text
  changed to "ChatGPT Plus/Pro (Codex)"
- database integration: added vault insecure-fallback env + electron mock
- command-paths: uses new @main/connector-helpers normalizeCommandValue
- pi-mono-migrations: added `as string` casts for 3 negative-existence
  assertions verifying legacy ids were removed

Drops:
- pnpm-workspace.yaml packages: [] (was "packages/*"; directory empty)
- obsolete CONNECTOR_STRENGTHS import; .module.models probing

Verification:
- pnpm typecheck → 0 errors
- pnpm lint → 0 errors / 0 warnings
- pnpm build:verify → 0 errors / 0 warnings
- pnpm test:unit → 508/508 pass
- pnpm test:integration → 96/96 pass
- All 6 adapter hard gates pass (NOT FOUND / 0 matches / empty)
- pi-ai / pi-agent-core / pi-coding-agent all resolved @0.67.6
2026-04-17 02:36:53 -04:00
ZacharyZhang-NY 0d044aeee8 chore(lint): replace any with unknown or typed shapes in tests
- DB introspection tests: introduce local RawDb alias for casts
  that poke into the private .db handle.
- IPC handler tests: introduce IpcHandler alias matching the
  registerHandle signature (event + unknown args).
- selective-ipc / session-management: narrow handler and data
  params to unknown or the minimum shape the code reads.
2026-04-17 01:45:00 -04:00
ZacharyZhang-NY 0f1b7c0b33 test(integration): split autonomous-chain.spec.ts into per-scenario files
The original tests/integration/autonomous-chain.spec.ts was ~1400 lines
and 50 KB, covering 19 describe blocks across 10 orthogonal scenarios.
Physically extract each describe (or closely related group) into its
own file inside tests/integration/autonomous-chain/ so each file stays
under the project's 500-line ceiling and failures point at the
scenario that regressed.

Shared beforeEach / afterEach / createAgent / buildCtx helpers move to
autonomous-chain/shared-setup.ts, which exposes installSharedSetup() as
a factory that each spec calls at module load. The tests themselves
are moved byte-for-byte; only module-level references (`db`,
`companyId`, `workspaceId`, `tempDirs`) are rewritten to go through
the shared handle. No assertions, it() calls, or behavior are changed.

Split layout:
  role-detection.spec.ts        — Step 1: CEO role detection
  ceo-phased-prompts.spec.ts    — Step 2: phased prompts + approvals
  wakeup-queue.spec.ts          — Step 3: deferred wake queue
  approval-decisions.spec.ts    — Step 4: hire approval decisions
  task-propagation.spec.ts      — Step 5: task/comment wake propagation
  run-failure.spec.ts           — Step 6: run failure escalation
  crash-recovery.spec.ts        — Step 7: worker crash recovery
  wake-reason.spec.ts           — Step 8: wake reason injection
  e2e-flows.spec.ts             — full-chain + review-fix regressions
  automation-messages.spec.ts   — automation/messages/assignment/workflows

Test count stays at 363 passing; typecheck stays at 156 errors.
2026-04-16 23:34:54 -04:00
ZacharyZhang-NY 43a03aa5e9 refactor(db): collapse three markRead variants into markMessageReadForReader (P1-16) 2026-04-16 22:50:40 -04:00
ZacharyZhang-NY 1b578c111a update 2026-04-16 14:48:51 -04:00
ZacharyZhang-NY 67c757f481 update 2026-04-04 12:03:50 -04:00