Commit Graph
41 Commits
Author SHA1 Message Date
ZacharyZhang-NY 54a97f67ac B2 Phase 1 follow-up: plug lease orphan gap in advanceTaskStatus + scope interruptStaleRuns cleanup
Linus review T1 surfaced three real bugs in the B2 Phase 1 shadow
dual-write surface. Fix them in one pass.

1. advanceTaskStatus left task_leases behind on terminal (done /
   cancelled / blocked) and "return to queue" (todo / backlog)
   transitions. The status->phase trigger advanced tasks.phase but the
   lease row lingered as an orphan. dev-mode verifyPhaseConsistency
   could see the divergence; production silently drifted. Wrap the
   status UPDATE and an explicit DELETE FROM task_leases in a single
   transaction so no caller observes the intermediate state.

2. The no-restricted-syntax guard in eslint.config.mjs only matched
   literals shaped "UPDATE tasks SET status" -- advanceTaskStatus's
   dynamic `UPDATE tasks SET ${timestampSets.join(", ")}` slipped past,
   and so did saveTask / recoverStaleLocks (status not first column).
   Tighten the selectors to (a) match any status-write within UPDATE
   tasks regardless of column order, and (b) flag template literals
   whose head ends with `SET ` so dynamic helpers must carry an
   explicit eslint-disable with rationale. Five grandfathered sites
   now carry matching disables.

3. interruptStaleRuns scoped its task+lease cleanup with
   `WHERE finished_at = ?` against the freshly stamped timestamp --
   safe in isolation, vulnerable to ISO-millisecond ties with other
   already-interrupted runs. Capture the stale run ids before
   mutating anything and drive every downstream statement from that
   explicit id list. Original semantics preserved; scoping is exact.

Unit tests added in tests/unit/tasks-phase.test.ts cover the
advanceTaskStatus lease cleanup across done / cancelled / blocked /
todo, and pin the in_review case as "lease stays".
2026-04-20 13:51:53 -04:00
ZacharyZhang-NY 69d03312e6 Cleanup residual dead code and stale annotations (N-scan residual)
- Drop the "pass-through remains for back-compat" note on
  AppDatabaseBase's domain modules; the pass-through layer is gone, so
  the comment was out of date. Replace with a pointer to the ESLint
  guard that blocks regression.

- Flag the two remaining >500-line files (`agent-runtime/runtime-bridge.ts`
  648 LOC, `agent-runtime/artifact-projector.ts` 637 LOC) with an
  inline `TODO: decompose — M-followup` note. No structural change;
  the decomposition is tracked for a later milestone to keep this pass
  scope-safe.
2026-04-20 13:39:56 -04:00
ZacharyZhang-NY 118e3d8795 Remove final AppDatabase pass-through methods with ESLint guard (C1 part 3/3)
Migrate all remaining pass-through call sites (social, skills, budget,
entityCrud, documents, knowledge, automation, wakeup, backup, threads,
settings, snapshot, connectors, workspaces, activity) to their owning
domain modules. `AppDatabase` now carries only three methods:

  - createHireRequest (multi-module composite: agents + approvalsMod)
  - rollbackConfigRevision (multi-module composite: documents + entityCrud)
  - queryAll (raw-SQL escape hatch, @internal)

The vault/env-binding infrastructure previously passed argument-by-
argument into `entityCrud.saveSecret`/`resolveSecretEnv` now binds at
module construction, letting callers invoke `db.entityCrud.saveSecret({...})`
without carrying vault plumbing.

Structural types that narrowed AppDatabase to a bag of flat methods
(`ConnectorModelPolicyDeps` callers, `CapabilityMatchDb`, test mocks in
budget-enforcement / budget-incidents / task-quality-gates /
failure-escalation / task-quality-gates-sweep) are reshaped to the new
namespace. `connectorPolicyDepsFromDb` replaces the old pattern of
handing the whole AppDatabase to `validateConnectorModelSelection`.

A new ESLint rule in eslint.config.mjs blocks new top-level methods on
AppDatabase so the pass-through layer cannot silently grow back.
`AppDatabase.ts` dropped from 193 lines to 58; `AppDatabaseBase` lost
the 270-line trailing pass-through block.
2026-04-20 13:31:39 -04:00
ZacharyZhang-NY a69fbb053b Codemod db pass-through calls: messages + runs + approvals + company (C1 part 2/3)
Rewrite ~376 call sites for messages, runs, approvalsMod, and company
domains and delete the corresponding pass-through methods on
AppDatabase / AppDatabaseBase. Structural type references
(ReturnType<HandlerDeps["db"]["getRun"]>) and SweepStaleReviewsDeps-style
mocks are updated to route through the owning sub-module.
2026-04-20 13:20:15 -04:00
ZacharyZhang-NY 5e9b7acbc6 Codemod db pass-through calls to domain modules: tasks + agents (C1 part 1/3)
Rewrite ~525 call sites from `db.<method>()` to `db.<module>.<method>()`
for tasks and agents domains (including `getAgent`/`findAgent`/`getTask`
which live on the `entityCrud` field). Delete the corresponding
pass-through methods on AppDatabaseBase. Structural types that narrowed
`db` to a bag of flat methods (CapabilityMatchDb, EscalationDeps,
SweepStaleReviewsDeps) are reshaped to match the new namespace. Internal
callback references inside database.ts/database-base.ts now invoke the
owning module directly (e.g. `this.agents.saveAgent`) rather than the
deleted pass-through.

The codemod lives at scripts/codemod/remove-appdatabase-passthrough.mjs
and filters by owning module + optional explicit method list. It is
idempotent and does not re-rewrite already-namespaced calls.
2026-04-20 13:17:01 -04:00
ZacharyZhang-NY 411294afa6 Lint rule: block new UPDATE tasks SET status outside advanceTaskPhase (B2 Phase 1 step 5/5) 2026-04-20 13:00:53 -04:00
ZacharyZhang-NY ab4afba9c7 Add phase/status consistency check observability (B2 Phase 1 step 4/5) 2026-04-20 12:57:24 -04:00
ZacharyZhang-NY 2656684392 Dual-write phase and lease in task status transitions (B2 Phase 1 step 3/5) 2026-04-20 12:55:53 -04:00
ZacharyZhang-NY dade5eb333 Wire pre-migration auto-backup with named API and manifest (B2/backup precursor)
Before running any pending migration, take a synchronous snapshot of the
profile to backups/pre-migration-v<target>-<ts>/ and write a
backup-manifest.json recording reason, schema version, app version, and
git SHA. A failed migration now has a well-labelled recovery point
instead of a half-upgraded profile.

- createAutomaticBackup(reason): synchronous snapshot via VACUUM INTO +
  recursive cpSync of logs/artifacts, plus JSON manifest. No fallback —
  failure throws so the caller refuses to touch the schema.
- runMigrations accepts createAutomaticBackup and invokes it when any
  migration is pending. No optional chaining; callers must supply it.
- AppDatabase constructor takes appVersion so main-runtime stamps the
  manifest with app.getVersion().
2026-04-20 12:16:39 -04:00
ZacharyZhang-NY 2b2a6446cb Wake the CEO when the team stalls, not only when tasks are zero
The bootstrap gate fired on goalCount > 0 && taskCount === 0, which
missed the common failure mode: every task stuck in_review or blocked
so the counter stays positive while no progress happens. The CEO
then waited for the normal timer interval to spot the stall.

Add countAdvancingTasksForCompany (todo + in_progress only) and gate
bootstrap on that. Also pull messages to the top of the timer wake
step list so an agent waking on the normal cadence doesn't skip over
unanswered peer DMs in favour of its own task work.
2026-04-19 19:27:26 -04:00
ZacharyZhang-NY 0443b4a197 Block approve_ceo_strategy until hires exist
The CEO prompt carried a plea — "do not submit strategy before hires,
we have observed a loop" — in place of an actual gate. The model
could (and did) ignore it, land back in team_building mode, and
resubmit. Move the invariant into the /approvals POST handler:
reject approve_ceo_strategy with 409 STRATEGY_BEFORE_HIRES when no
hire_agent approval exists for the company, and 409 STRATEGY_ALREADY_FILED
when one is already pending or approved.

Introduce listCompanyApprovalsByType on the database facade to keep
the route stateless. Trim the prompt to describe the gate instead of
begging the model to obey.
2026-04-19 19:22:34 -04:00
ZacharyZhang-NY edbebe6109 Make document lifecycle fan-out durable and atomic
Pre-v47 `createDocumentWithLifecycle` wrapped each hook in its own
try/catch and always returned success, so a failure in any of the
downstream hooks (thread event, checkpoint approval, deliverable
review, automation dispatch, reviewer wake) left a live document row
behind with a silently incomplete lifecycle.

Two invariants now hold:

- DB-writing hooks (thread integration, checkpoint approval,
  deliverable review) run inside the same `BEGIN IMMEDIATE`
  transaction as `saveDocument`. A throw rolls back the document row
  along with every partial write, so an operator never sees a
  persisted document with a missing thread or skipped approval.

- Cross-process side effects (automation dispatch, reviewer wake) are
  enqueued as `document_lifecycle_events` rows inside the same
  transaction and processed post-commit. Success stamps
  `status = 'succeeded'`; failure stamps `status = 'failed'` with
  `last_error`, so partial lifecycle state is observable via
  `listFailedForCompany` instead of disappearing into the log tail.

Migration v47 installs the ledger; `buildDocumentLifecycleEventMethods`
exposes the enqueue/running/succeeded/failed transitions; the lifecycle
service drives both.
2026-04-19 15:53:55 -04:00
ZacharyZhang-NY cb0ac485da Close inbox projection under one transaction
The old `prepareRun` projection drifted: append-to-shadow, thread-event,
and `markAgentInboxDelivered` ran as three independent writes, so a
crash between the append and the delivered mark replayed the same
inbox row on the next run and double-populated the transcript.

v46 installs `agent_message_projections` keyed by (message_id,
session_id) as the idempotent gate. The projection closure now wraps
the ledger claim, the shadow append, the thread-event write, and the
delivered-mark inside one `BEGIN IMMEDIATE` transaction (exposed via
`AppDatabase.withTransaction`). `claimProjection` returns false on a
replay, so the whole closure short-circuits — no double transcript,
no double thread event.
2026-04-19 15:46:12 -04:00
ZacharyZhang-NY 1a0fdc6f31 Collapse the GET /api/agents/me/standup duplicate to one module
Two registrations of the same route lived in the codebase: one inline
in routes-agents.ts that filtered three full company-wide SELECTs in
memory, and the dedicated routes-agents-standup.ts that uses targeted
standup-shaped DB queries. The router keeps the first match so the
inline copy was serving every request and the focused module was dead
code — any future fix to standup would have silently missed
production.

Remove the inline standup handler and its three companion helpers
(`listAgentsForStandup`, `listTasksForStandup`, `listApprovalsForStandup`)
plus the unused `AgentStandupRow` type. The dedicated standup module is
now the single source of truth.
2026-04-19 15:01:41 -04:00
ZacharyZhang-NY 5e6e929779 refactor: prune session-store and agent-list dead code
The session_entries supersede pipeline was a design that never
shipped: `SqliteSessionStore.replaceMessages` / `appendCompaction`,
`sessionEntries.replace` / `listAll` / `deleteForSession`, and the
`"compaction" | "summary"` kind variants had zero live callers.
Compaction wires through pi's `transformContext` and leaves the
persisted transcript alone, so the shadow-row machinery was pure
cruft with two tests propping it up.

Rip it out: the store exposes only `loadMessages` and
`appendMessage`, `sessionEntries` exposes only `append` and
`listActive`, and `SessionEntryKind` narrows to `"message"` so new
writers can't re-introduce a dead kind by accident.
`is_superseded` stays on the table — the DB column is harmless
with no writer, and dropping it is a separate schema migration
concern.

`listTopLevelAgentIds` loses its only caller in the previous
commit (company channel now fans out through
`listCompanyBroadcastAgentIds`), so it goes too. Three tests that
validated the removed behaviour are deleted; the one remaining
session-store test still covers the live `append + load` path.
2026-04-19 01:34:14 -04:00
ZacharyZhang-NY d47e637a45 fix(messages): company channel fans out to every active agent
"Company" channel fell through to `listTopLevelAgentIds`, so a
broadcast on the most ambient-sounding channel in the product only
reached agents with `reports_to IS NULL`. The label said company;
the semantics said board-only. The rest of the org never knew a
message existed.

Add `listCompanyBroadcastAgentIds` — returns every non-terminated,
non-paused, non-pending agent for the company — and wire it through
the `company` case in `resolveMessageWakeTargets`. Board-only
fan-outs are still reachable: the `incident` channel keeps the
broader pingable set (paused-by-budget included) and direct DMs to
top-level agents work as before.

The prompt doc catches up to the runtime: company is explicitly
framed as all-hands, with board-only guidance redirected to DMs or
incidents. The message-dispatch unit test asserts the new fan-out.
2026-04-19 01:21:26 -04:00
ZacharyZhang-NY 7f23f40e87 fix(db): strip implicit data mutations from startup
`init()` ran three implicit data-repair queries on every boot:
`reconcileOrphanedTasks`, `reconcileUnreviewableTasks`, and a
secrets-table migration whose fallback glued legacy rows onto a
synthetic zero UUID when no current company existed. The first
two papered over state-machine leaks at init time — hiding the
real bug and making startup observably mutate production data;
the third silently connected ciphertext to a non-existent company
with no UI visibility and no deletion path.

Root fix: the startup path is now allowed exactly one mutation —
`interruptStaleRuns`, which is crash recovery (running-at-boot =
worker died mid-turn) and cannot be expressed anywhere else.

The dead-letter `in_review` case moves to the transition point:
`tasks-status.guardInReviewHasReviewer` refuses to advance a task
into `in_review` when its assignee is a top-level agent
(`reports_to IS NULL`) and the task is not flagged for user
review. Callers either pre-flag the task (the CEO-completion path
in `automation-run-completion.advanceToInReview` now does this)
or the transition is rejected. No more silent resurrection at
boot; the failure is visible and actionable at source.

The orphan-task reconcile disappears entirely — the `finishRun`
guard already enforces the invariant at the one place a run can
leave the in-progress state, and `recoverStaleLocks` handles
stuck executions at runtime.

The secrets migration now throws when neither
`settings.current_company_id` nor any company row exists, rather
than binding legacy ciphertext to `00000000-0000-0000-0000-000000000000`.
Legacy rows stay on the old table until the operator creates a
real company, at which point the upgrade succeeds deterministically.

Tests rewritten: the reconciliation regression suite now asserts
the transition-time guard and the lone `interruptStaleRuns`
behavior. `resolve-workspace.test.ts` catches up with the
exclusive-by-default lock policy committed earlier.
2026-04-19 01:12:38 -04:00
ZacharyZhang-NY e0e43d6127 refactor(session): nuke dead sessionParams / sessionDisplayId pipeline (v43)
These columns and the messages that plumbed them existed for the CLI-
adapter era — Gemini's resume id and provider-specific session params
rode the run-finished envelope back onto the session row so the next
turn could resume. Pi Mono dispatches every provider through pi-ai and
emits neither value. The worker has been posting literal null on every
finish, finalizeRun has been writing null into task_sessions, and
rotateSession has been carrying null forward.

Remove the wire, the writer, the carry-forward, and the columns:

  - v43 migration rebuilds task_sessions without session_params_json /
    session_display_id, and runs without session_display_id. Follows
    the idempotent "create new, copy, swap, reindex" pattern used by
    v41. context_snapshot_json is preserved (also unreferenced today
    but outside this cleanup's scope).
  - RunJob / EnqueueRunPayload / ChildMessage / worker-types zod
    schema no longer carry the fields. finishRun() / createNewSession()
    signatures tighten in lockstep.
  - rotateSession starts the new session with fresh counters and no
    carry-forward.
  - turn-context resolveSession returns just the session id.
  - TaskSessionRecord / RunRecord lose the dead fields.

498/498 unit tests pass. typecheck + build clean.
2026-04-18 23:22:17 -04:00
ZacharyZhang-NY 88a1f8b334 refactor(connectors): drop dead CLI command field after pi-mono migration
The CLI-adapter era `connector.command` field became vestigial once every
provider dispatch went through pi-ai. The schema still required a non-empty
command, so onboarding saveConnector silently rejected every Pi Mono row
(which carries an empty default). Remove the field from the payload chain
(renderer → IPC schema → DB CRUD → run dispatch → worker job), narrow
RunJob.taskId to match EnqueueRunPayload, and drop the dead helper plus
its test. Schema column stays NOT NULL for DB compatibility; seedConnectors
writes a literal empty string so no migration is needed.
2026-04-18 23:09:59 -04:00
ZacharyZhang-NY d75bfd5851 refactor(model-pool): scope disabled-models by company; cascade on delete (v42)
The per-connector "enabled/disabled model" settings lived under a
global key in the settings key-value table
(`disabled_models:${connectorId}`). No company dimension. Deleting a
company left the pool choices behind, and creating a new company
silently inherited them — the visible symptom was "onboarding's team-
model configuration survives company deletion and pollutes the next
company's CEO model list."

Root fix, not a cleanup hook: the storage becomes a real relational
table with `(company_id, connector_id)` primary key and an
`ON DELETE CASCADE` FK from `companies`. Company deletion wipes pool
config by construction; no cleanup step in `deleteCompany()` can
drift.

Schema (migration v42):
- CREATE TABLE connector_model_pool_settings (company_id, connector_id,
  disabled_models_json, updated_at), PK (company_id, connector_id),
  FK companies(id) ON DELETE CASCADE.
- Drop every `settings.disabled_models:*` key — these were the bug's
  residue; there is no right company to attribute them to, so the
  migration clears them. Users re-select on next onboarding.

API surgery — every reader/writer now takes companyId:
- db.getDisabledModels(companyId, connectorId)
- db.setDisabledModels(companyId, connectorId, models)
- db.hasModelPoolSettings(companyId, connectorId)
- connector-model-policy: getConnectorModelPolicy(deps, companyId,
  connectorId); validateConnectorModelSelection(deps, companyId,
  connectorId, model)
- model-dispatch: listEnabledModels(companyId, deps);
  pickModelForAgent resolves companyId from agent.companyId
- orchestration heartbeat/task-runner/turn-context/run-dispatch pass
  the agent's or run's companyId through to the policy deps
- IPC schema: connectorDisabledModelsSchema gains companyId; new
  connectorModelPoolQuerySchema pairs companyId+connectorId for reads

Onboarding becomes commit-on-launch:
- useConnectorModelSelection no longer persists via IPC during
  onboarding. Toggles mutate local state; it exposes `poolSelections`
  to the wizard.
- bootstrapOnboardingSchema gains connectorPoolSelections. The
  onboarding service saves the company first, then persists each
  per-connector disabled list scoped to the new company id — CEO
  model validation runs against the company's freshly-persisted pool,
  not some stale global.
- IPC setConnectorDisabledModels now requires companyId; the only
  consumer was onboarding, which no longer calls it at all.
- Main-side listConnectorModels returns the raw provider catalog
  (no pool filtering), since that IPC is used from onboarding where
  no company exists yet; pool filtering happens at dispatch time.

saveConnector IPC drops the per-company pool validation — connector
config is a global hint, company-scoped enforcement belongs at run
dispatch where the company is known.

Tests: 603/603 green. Typecheck clean. Integration tests rewritten
to set the pool via connectorPoolSelections on bootstrap input, not
a pre-bootstrap global setDisabledModels call.
2026-04-18 22:14:09 -04:00
ZacharyZhang-NY 2ace9183b0 refactor(session): drop dead task-scoped session columns (v41)
Sessions have been agent-scoped since v34, but the schema kept four
dead task-scoped columns that every session code path had to pretend
were meaningful:

  - tasks.active_session_id     (mirrored agents.session_id)
  - tasks.session_json          (superseded by task_sessions.session_params_json)
  - tasks.session_display_id    (superseded by task_sessions.session_display_id)
  - task_sessions.task_id       (always empty-string on new rows)

v41 drops all four via SQLite table rebuild — the only safe path when
multiple columns need to go in one step and indexes / FK directions
need to be preserved. `PRAGMA foreign_keys = OFF` spans the swap
window so cascade-delete does not fire mid-rebuild. The migration is
idempotent and also backfills the optional task columns
(task_type / requires_user_review / retry_count / last_failure_at /
started_at / completed_at / cancelled_at / checkout_run_id /
execution_locked_at) before rebuilding so it works whether or not
`ensureOptionalColumns` has run yet on this install.

Along with the schema change, every code path that read or wrote
those columns has been rewired to the agent-scoped equivalents:

- task-sessions-crud.ts: deleted updateTaskSession,
  getTaskSession, getActiveTaskSession, updateTasksActiveSessionForAgent;
  added updateSessionParams(sessionId, ...) for the worker's
  finalizer. ensureAgentSession and createNewSession no longer touch
  task_id.
- database-base.ts: facade drops the now-unused pass-throughs.
- snapshot.ts / task-queries.ts / tasks-status.ts / entity-crud.ts:
  SELECT projections no longer pull session_json / session_display_id
  / active_session_id from tasks.
- tasks-crud.ts (TaskRow) and shared/types/records-core.ts
  (TaskRecord): sessionJson / sessionDisplayId fields removed.
- TaskSessionRecord: taskId field removed.
- turn-context.ts::resolveSession: no more
  updateTasksActiveSessionForAgent mirror; sessionParams read directly
  from the current session via getSessionById.
- run-finished-helpers.ts::finalizeRun: stamps sessionParams via
  updateSessionParams on the agent's current session, resolved through
  agents.session_id.
- session-management-service.ts: rotateSession stops calling
  updateTasksActiveSessionForAgent.

Test suite updated to use the new session-id API. Existing tests used
`getActiveTaskSession(taskId)` and `createNewSession({ taskId, ...})`
as probes; they now use `getSessionById(sessionId)` and the taskId-less
createNewSession input.

Tests: 603/603 green. Typecheck clean. Migration tested on fresh DB
(runs v1→v41 cleanly) and idempotent re-apply (detects absent dead
columns, skips).
2026-04-18 21:54:31 -04:00
ZacharyZhang-NY 6abc1a823f refactor(inbox): derive from snapshot; delete getInbox IPC chain
Sibling refactor to the sidebar-badges cleanup. The inbox aggregates
five item categories (pending approvals, failed runs, budget warnings,
blocked/review tasks, unread company/incident/urgent messages) — every
row is already in `ProfileSnapshot`. The server-side `getInbox`
aggregation (180 lines of SQL + projection) was a redundant second
path.

New: `src/shared/inbox-items.ts::deriveInboxItems(snapshot, companyId)`
produces the same InboxItem[] shape with the same severity ordering
and 24h failed-run window. Pure function, no IPC, no state.

Deleted:
- database/company-crud.ts::getInbox (180-line aggregation)
- database-base.ts::getInbox pass-through
- ipc-core-read-handlers.ts: handler
- preload/index.ts: bridge binding
- shared/contracts/channels.ts: channel name
- shared/contracts/desktop-api/companies.ts: API signature
- state/slices/company.ts: `inboxItems` field, `setInboxItems` action
- useDataLoader.ts: both getInbox call sites (initial snapshot load
  and section-change effect)
- useCompanyController.ts: inbox reset on company switch

OperationsSections now computes `inboxItems` via `useMemo` off the
current snapshot + company id. The snapshot is already company-scoped
at the main/database/snapshot.ts layer when `currentCompanyId` is
non-null; the derivation re-filters defensively so the function works
correctly even for unscoped snapshots.

Tests: 603/603 green. No new tests added — the derivation is a pure
function whose shape is exercised end-to-end by the existing Inbox
component integration.
2026-04-18 21:42:58 -04:00
ZacharyZhang-NY ff5e22092a refactor(session): agent-scope rotation + accumulation; auto-register file artifacts
Sessions are already agent-scoped post-v34, but the code kept routing
rotation / accumulation / handoff through the task indirection. Heartbeat
runs (no task) silently skipped all three, letting taskless transcripts
grow past the model's context window — the 7M-token explosion.

- DB adds getSessionById(sessionId) — direct lookup bypassing
  tasks.active_session_id
- session-management-service: generateHandoff(session) is now agent-scoped,
  walking recent runs across tasks + heartbeats; getHandoffForAgent(agentId)
  is the canonical lookup, getHandoffContext(taskId) is a thin shim for
  callers that still hold a task id
- turn-context.ts::resolveSession drops the `if (!task)` early return:
  rotation evaluation runs on the agent session unconditionally
- run-finished-helpers.ts::accumulateAndRotateSession drops the `!task`
  gate: tokens accumulate and rotation fires for every run, keyed on the
  agent session, not the task
- Rotation evaluator contract: evaluatePreRunSessionRotation(agentId,
  connectorId) → { rotated, handoff, sessionId }; old (taskId, connectorId)
  signature removed

IPC contract tightening: ipc-support::emitEvent becomes a best-effort
sink. "Render frame was disposed" during renderer reload is a lifecycle
signal, not a logic error; silently drop it instead of letting it abort
downstream event emissions.

Artifact projection: pi's write/edit tool results now upsert a
`work_products` row via a new agent-runtime/artifact-projector.ts. No
more agents writing 54 files into the workspace that the UI never sees.
Migration v40 relaxes work_products.task_id to nullable so heartbeat
artifacts can register too.

File size: split agent-event-projection.ts out of runtime-bridge.ts to
keep the bridge under the 400-line cap after adding the projector hook.

Tests: 604 unit + integration green; generateHandoff callsites updated
for the single-argument signature; artifact-projector has pure-function
coverage via the projector module's own shape.
2026-04-18 21:37:56 -04:00
ZacharyZhang-NY 7e1d4199ee refactor(schema): make runs.task_id nullable, delete heartbeat workaround
The NOT NULL constraint forced heartbeat runs to synthesize a fake task
row (`[Heartbeat] X — Check for work`) so the FK could resolve. Three
layers of code — acquireHeartbeatTask, findReusableHeartbeatTask,
cleanupDuplicateHeartbeatTasks, plus the is_heartbeat flag column + UI
filter — existed to paper over one bad constraint. Remove the
constraint and the whole workaround vanishes.

SQLite table rebuild with data preservation. Live production DB, no
backup — migration is atomic, idempotent, and verified before DROP.

## Migration v35 (rebuild runs with nullable task_id)

Uses `fn:` form so it can:
- Snapshot `COUNT(*) FROM runs` before copy
- Walk `PRAGMA table_info(runs)` to detect drift columns (v34 declared
  a column set; any additive ALTER since v14 shows up here)
- Assert `COUNT(*) FROM runs_new == beforeCount` before `DROP TABLE runs`
- `PRAGMA defer_foreign_keys = ON` so run_events(run_id) FK defers to
  commit time, by which runs has the same id set under the new name
- Recreate all 6 indexes on the new table

If the assertion fails, the migration throws inside the existing
`transaction()` wrapper. SQLite rolls back the whole migration, old
`runs` table is untouched, `schema_migrations` never records v35. Next
app start retries cleanly.

## Migration v36 (drop tasks.is_heartbeat + synthetic rows)

Runs now carry `task_id = NULL` for heartbeat dispatches. The
`is_heartbeat` flag is dead data going forward.

1. `UPDATE runs SET task_id = NULL WHERE task_id IN (SELECT id FROM tasks WHERE is_heartbeat = 1)`
2. `DELETE FROM tasks WHERE is_heartbeat = 1` (synthetic rows, not real work)
3. Rebuild `tasks` without the `is_heartbeat` column (same defer-FK +
   row-count-assert pattern as v35)

## Code cleanup

- `RunRecord.taskId: string | null`; `createRun` / `cancelRunForDeletion`
  / `finishRun` guard every `UPDATE tasks` path with a null-check.
- `TaskRecord.isHeartbeat` removed; every SELECT / filter / UI reference
  dropped.
- Delete `acquireHeartbeatTask` / `findReusableHeartbeatTask` /
  `cleanupDuplicateHeartbeatTasks` — dead after the constraint lifts.
- Delete `src/main/orchestration/heartbeat-task.ts` (pure workaround
  constant file).
- `heartbeat.ts` creates runs with `taskId: null` when the agent has no
  assigned work. Turn context already supports nullable task
  (heartbeatFallbackSection); prompt surface unchanged.
- `validateTaskCreation` drops the `isHeartbeat` parameter — user-
  authored + automation-authored tasks are never heartbeat.
- Renderer: `run.taskId ? taskMap.get(...) : undefined` in Dashboard /
  RunCenter / RunDetail / RunQueue.
- Delete `tests/unit/heartbeat-coalescing.test.ts` (12 tests that
  exercised the deleted DB methods).
- Drop 1 isHeartbeat-bypass test from `task-quality-gates.test.ts`.

## Grep verification

- `isHeartbeatTaskTitle src/` → 0
- `acquireHeartbeatTask src/` → migration comment only
- `is_heartbeat src/` → schema.ts migration history only
- `isHeartbeat src/` → 0

Tests: 578/578 (was 591 — delta is 13 tests that tested deleted code).
2026-04-17 16:16:08 -04:00
ZacharyZhang-NY 92795f7bfb refactor(session): scope pi sessions to agents instead of tasks
Sessions were keyed by (task, agent). Every task switch started a fresh
transcript — the agent lost everything it had learned on the previous
task. That breaks the product premise: a CEO should remember its past
decisions, an engineer should carry context between bugs.

Move pi's AgentMessage transcript to be agent-scoped. One session per
agent (per company). Tasks enter the ongoing conversation as user turns;
rotation/compaction boundaries follow the agent's long-running session.

Additive schema only — no row rewrites on the live production DB.

- Migration v34: `ALTER TABLE agents ADD COLUMN session_id TEXT` +
  `idx_agents_session_id`. `ensureOptionalColumns` mirror for fresh DBs.
- `AgentRecord.sessionId?: string | null` in the read model; included
  in every full-agent SELECT (entity-crud, snapshot, agents-hierarchy
  chain-of-command / direct-reports / heartbeat-eligible).
- `db.ensureAgentSession({ agentId, companyId, adapterType })` —
  idempotent: returns existing `agents.session_id` if set, else inserts
  a new `task_sessions` row with an empty `task_id` sentinel (agent-
  scoped, task-id is now metadata) and stores the id on the agent.
- `db.setAgentSessionId(agentId, sessionId)` — used by rotation.
- `db.updateTasksActiveSessionForAgent(agentId, sessionId)` — mirrors
  the agent's current session onto every task the agent owns, so
  existing per-task readers (`tasks.active_session_id`) still work.
- `turn-context.resolveSession`: pulls the agent's session via
  `ensureAgentSession` instead of `getActiveTaskSession(task.id)`,
  then points the current task at it so UI continues to resolve.
- `session-management-service.rotateSession`: new session carries
  `taskId:""`, then `setAgentSessionId` + `updateTasksActiveSessionForAgent`
  so every task the agent owns follows the rotation boundary.
- `session-entries.listActive`: `ORDER BY created_at, sequence` — the
  created_at timestamp is authoritative; sequence stays as same-ms
  tiebreaker. Forward-safety for future merge scenarios.

Existing agents' first post-upgrade run creates a fresh agent-scoped
session. Their pre-migration per-task transcripts remain in the DB as
historical records but are not loaded for new runs. This is a one-time
transcript reset, accepted over a risky lossy merge on the live DB.

Documented in a code comment on `ensureAgentSession`.
2026-04-17 15:42:13 -04:00
ZacharyZhang-NY 14dc9a4897 refactor(cost): single writer for cost_entries, finishRun reads SUM
Cost was written through three independent paths: worker-runner shipped
`finalUsage.costUsd` in run-finished -> finishRun wrote `runs.cost_usd`,
`tasks.cost_usd += costUsd`, `agents.spent_monthly_usd += costUsd`, and
inserted a per-run aggregate row in cost_entries. CostWriter
additionally inserted per-message rows in cost_entries from main. Any
CostWriter failure created silent drift between `SUM(cost_entries)`
(read by budget enforcement) and `runs.cost_usd` (read by the UI).

Make CostWriter the single writer, finishRun the single reader.

- Add `db.recordCostEntry(...)` as the first-class insert path for
  per-message cost rows. CostWriter calls it; tests call it directly.
- Remove the aggregate INSERT from finishRun. finishRun now reads
  `SELECT COALESCE(SUM(amount_usd), 0) FROM cost_entries WHERE run_id=?
  AND unattributed=0` and writes that as the authoritative run total to
  runs.cost_usd, tasks.cost_usd, and agents.spent_monthly_usd.
- Drop `costUsd` from `finishRun` input, the worker `run-finished`
  ChildMessage shape, the automation `RunFinishedEvent` type, and every
  caller + test fixture. Cost is owned by the per-message path.
- worker-runner stops extracting `usage.cost.total` — token counts stay
  for session accumulation, cost comes from pi-ai via CostWriter.
- run-finished-helpers now gates budget incident checks on
  `event.status === "succeeded"` instead of a cost field that no longer
  exists on the event.
- automation-run-completion fires `budget_threshold` on every succeeded
  run (post-run spent_monthly_usd already reflects the SUM path).
- Update budget tests to write a `recordCostEntry` before `finishRun`
  to simulate the per-message cost path; drop `costUsd` from cross-
  subsystem, edge-case-budget, database, and related fixtures.

One writer, one reader, one source of truth. No drift.
2026-04-17 15:04:22 -04:00
ZacharyZhang-NY b50716d4e8 refactor(messaging): project inter-agent messages into pi session transcript
Inter-agent messages previously entered the recipient's context as a
separate prompt block (recentMessages), rebuilt every turn. This double-
charged tokens and kept messages outside pi's native AgentMessage stream.

Now at prepareRun: pull the recipient's unread inbox, project each
AgentMessageRecord into a pi user-role AgentMessage, persist to
SqliteSessionStore, and mark delivered. The recipient reads messages
as natural conversation turns in the transcript.

- db: listAgentInboxUnread + markAgentInboxDelivered (transactional)
- agent-runtime/inbox-projection.ts: AgentMessageRecord -> pi AgentMessage
- runtime-bridge.prepareRun: inject + deliver before loadMessages
- prompt-builder: delete recentMessages context block + buildMessagesSection
- message-normalize: drop unused normalizeMessagesForPrompt
- delete prompt-builder/attachment-resolver.ts (orphan after section removal)

Send path (api-server/routes-messages, ipc, UI) unchanged — inbox table
remains the UI source. Transactional delivery marking prevents duplicate
injection across runs. Failed append -> no mark -> retry on next run.
2026-04-17 14:48:22 -04:00
ZacharyZhang-NY 5547c29491 update 2026-04-17 14:03:56 -04:00
ZacharyZhang-NY 2b63588604 feat(db): wire sessionEntries + credentialsTable and add migration 32
- Expose buildSessionEntryMethods and buildCredentialMethods on
  AppDatabaseBase as public readonly members so agent-runtime and
  credential-store can access them via db.sessionEntries /
  db.credentialsTable
- Add idempotent schema migration v32 creating session_entries and
  credentials tables plus supporting indexes; uses CREATE TABLE IF NOT
  EXISTS so existing profile DBs upgrade safely without destructive ops
2026-04-17 01:28:20 -04:00
ZacharyZhang-NY 279af3b034 refactor(database): expose domain modules as public readonly fields
Changes the AppDatabaseBase module slots (tasks, agents, runs, ...)
from protected to public readonly so new call sites can reach them
directly as db.tasks.x / db.agents.x. The 156 pass-through methods on
AppDatabase remain untouched, so all existing callers keep working;
this is the first step of a two-part migration to eventually drop
the pass-through layer.
2026-04-16 23:29:07 -04:00
ZacharyZhang-NY 821855931d refactor(db): add findAgent/findConnector; wrap linkManyTasksToApproval in tx; collapse saveAgent IIFEs (P1-28, P1-29) 2026-04-16 23:02:13 -04:00
ZacharyZhang-NY 9874d825b6 refactor(api-server): move agents-standup SQL into db domain methods
Replace the six inline queryAll calls in routes-agents-standup.ts with
typed domain methods: db.agents gains listDirectReportsForStandup and
listReportStatusesForStandup (LEFT JOIN aggregation), db.tasks gains
listAssigneeTasksForStandup and listTasksForAssignees (safe on empty
assignee list), db.approvals gains listAgentPendingApprovals and
listAgentResolvedApprovals. Shared listRecentTaskComments is reused.
2026-04-16 22:39:43 -04:00
ZacharyZhang-NY 4342cf3e2f refactor(api-server): move approvals SQL into db.approvals domain methods
Replace the ad-hoc getApproval helper and the filterable list query
in routes-approvals.ts with db.approvals methods. Add findApprovalById
(non-throwing lookup returning null) and listApprovalsForApi (accepts
requestedByAgentId / relatedTaskId / statuses filters and builds the
prepared statement inside the domain layer).
2026-04-16 22:38:05 -04:00
ZacharyZhang-NY 28296b3d5f refactor(api-server): move agents SQL into db.agents/db.activity/db.approvals methods
Replace seven inline queryAll calls in routes-agents.ts with typed
domain methods. db.agents gains listAgentsForApi, listAgentsForStandup,
listExistingAgentsForHireCheck. db.tasks gains listTasksForStandup.
db.approvals gains listApprovalsForStandup and
listPendingHireApprovalPayloads. db.activity gains listRecentTaskComments
(safe on empty taskIds). All HTTP response projections stay identical.
2026-04-16 22:37:10 -04:00
ZacharyZhang-NY efb9fdb8cc refactor(api-server): move work-graph SQL into db domain methods
Replace six inline queryAll calls in routes-work-graph.ts with domain
methods. db.entityCrud gains listGoals, getGoalForUpdate, listProjects,
getProjectForUpdate. db.knowledge gains listSprints. db.company gains
getDashboardSnapshot, which returns the raw agent/task/activity rows
the dashboard handler aggregates. The route-level row shapes are
preserved in the new GoalListRow / ProjectListRow / SprintListRow
types so HTTP responses remain identical.
2026-04-16 22:34:46 -04:00
ZacharyZhang-NY 7f097e3319 refactor(api-server): move tasks SQL into db.tasks/db.documents domain methods
Replace inline queryAll calls in routes-tasks.ts with two new domain
methods: tasks.listTasksByStatuses (used by GET /api/companies/:id/issues)
and documents.listImportantDocsForReview (used by the deliverable-review
gate). Routes now hold no SQL or column-mapping logic; the
TaskListByStatusesRow shape preserves the existing HTTP response fields
including requiresUserReview as a boolean.
2026-04-16 22:29:36 -04:00
ZacharyZhang-NY 8f7054630b fix(heartbeat): atomic find-or-create + checkout to eliminate duplicate task race
Replace the racy three-step heartbeat path (listAgentTasks ->
findReusableHeartbeatTask -> saveTask -> checkoutTask) with a single
transactional acquireHeartbeatTask method. The prior flow allowed two
concurrent scheduler ticks to both observe "no reusable heartbeat" and
both INSERT a fresh [Heartbeat] task before either CAS-checked out,
producing duplicates that cleanupDuplicateHeartbeatTasks had to mop up.

acquireHeartbeatTask runs inside BEGIN IMMEDIATE so concurrent ticks
serialize on the writer lock. Inside the transaction it first attempts
to claim any existing claimable [Heartbeat] task (todo/backlog/done/
in_review with checkout_run_id IS NULL), reviving done/in_review by
resetting status. If nothing is claimable but an in-flight heartbeat
exists (status NOT IN done/cancelled), it returns null instead of
creating a duplicate. Only when no heartbeat task exists at all does it
INSERT a new one, already in 'in_progress' with checkout_run_id set.

cleanupDuplicateHeartbeatTasks remains as deletion-side defense.
2026-04-16 22:19:42 -04:00
ZacharyZhang-NY c81d72fb9c fix(api-server): SSE stream pulls incremental events via seq cursor
Replace per-poll 5000-row scan + JS filter with indexed WHERE seq > ?
query (uses idx_run_events_run). Batch up to 500 per call, drain loop
on reconnect. Poll interval raised 500ms -> 1000ms.
2026-04-16 21:57:35 -04:00
ZacharyZhang-NY 5201b6623b fix(database): enable WAL mode and busy_timeout on every DB connection
Applies WAL journal, 5s busy_timeout, synchronous=NORMAL, and
foreign_keys=ON pragmas immediately after opening each connection so
concurrent main+worker writers share the same locking configuration
and avoid SQLITE_BUSY under contention.
2026-04-16 21:52:25 -04:00
ZacharyZhang-NY 9779c4d1ca update 2026-04-07 20:18:47 -04:00
ZacharyZhang-NY 67c757f481 update 2026-04-04 12:03:50 -04:00