Commit Graph
12 Commits
Author SHA1 Message Date
ZacharyZhang-NY 6da0a58769 refactor(db): group database domain modules 2026-04-22 19:24:25 -04:00
ZacharyZhang-NY 118e3d8795 Remove final AppDatabase pass-through methods with ESLint guard (C1 part 3/3)
Migrate all remaining pass-through call sites (social, skills, budget,
entityCrud, documents, knowledge, automation, wakeup, backup, threads,
settings, snapshot, connectors, workspaces, activity) to their owning
domain modules. `AppDatabase` now carries only three methods:

  - createHireRequest (multi-module composite: agents + approvalsMod)
  - rollbackConfigRevision (multi-module composite: documents + entityCrud)
  - queryAll (raw-SQL escape hatch, @internal)

The vault/env-binding infrastructure previously passed argument-by-
argument into `entityCrud.saveSecret`/`resolveSecretEnv` now binds at
module construction, letting callers invoke `db.entityCrud.saveSecret({...})`
without carrying vault plumbing.

Structural types that narrowed AppDatabase to a bag of flat methods
(`ConnectorModelPolicyDeps` callers, `CapabilityMatchDb`, test mocks in
budget-enforcement / budget-incidents / task-quality-gates /
failure-escalation / task-quality-gates-sweep) are reshaped to the new
namespace. `connectorPolicyDepsFromDb` replaces the old pattern of
handing the whole AppDatabase to `validateConnectorModelSelection`.

A new ESLint rule in eslint.config.mjs blocks new top-level methods on
AppDatabase so the pass-through layer cannot silently grow back.
`AppDatabase.ts` dropped from 193 lines to 58; `AppDatabaseBase` lost
the 270-line trailing pass-through block.
2026-04-20 13:31:39 -04:00
ZacharyZhang-NY a69fbb053b Codemod db pass-through calls: messages + runs + approvals + company (C1 part 2/3)
Rewrite ~376 call sites for messages, runs, approvalsMod, and company
domains and delete the corresponding pass-through methods on
AppDatabase / AppDatabaseBase. Structural type references
(ReturnType<HandlerDeps["db"]["getRun"]>) and SweepStaleReviewsDeps-style
mocks are updated to route through the owning sub-module.
2026-04-20 13:20:15 -04:00
ZacharyZhang-NY 5e9b7acbc6 Codemod db pass-through calls to domain modules: tasks + agents (C1 part 1/3)
Rewrite ~525 call sites from `db.<method>()` to `db.<module>.<method>()`
for tasks and agents domains (including `getAgent`/`findAgent`/`getTask`
which live on the `entityCrud` field). Delete the corresponding
pass-through methods on AppDatabaseBase. Structural types that narrowed
`db` to a bag of flat methods (CapabilityMatchDb, EscalationDeps,
SweepStaleReviewsDeps) are reshaped to match the new namespace. Internal
callback references inside database.ts/database-base.ts now invoke the
owning module directly (e.g. `this.agents.saveAgent`) rather than the
deleted pass-through.

The codemod lives at scripts/codemod/remove-appdatabase-passthrough.mjs
and filters by owning module + optional explicit method list. It is
idempotent and does not re-rewrite already-namespaced calls.
2026-04-20 13:17:01 -04:00
ZacharyZhang-NY e0e43d6127 refactor(session): nuke dead sessionParams / sessionDisplayId pipeline (v43)
These columns and the messages that plumbed them existed for the CLI-
adapter era — Gemini's resume id and provider-specific session params
rode the run-finished envelope back onto the session row so the next
turn could resume. Pi Mono dispatches every provider through pi-ai and
emits neither value. The worker has been posting literal null on every
finish, finalizeRun has been writing null into task_sessions, and
rotateSession has been carrying null forward.

Remove the wire, the writer, the carry-forward, and the columns:

  - v43 migration rebuilds task_sessions without session_params_json /
    session_display_id, and runs without session_display_id. Follows
    the idempotent "create new, copy, swap, reindex" pattern used by
    v41. context_snapshot_json is preserved (also unreferenced today
    but outside this cleanup's scope).
  - RunJob / EnqueueRunPayload / ChildMessage / worker-types zod
    schema no longer carry the fields. finishRun() / createNewSession()
    signatures tighten in lockstep.
  - rotateSession starts the new session with fresh counters and no
    carry-forward.
  - turn-context resolveSession returns just the session id.
  - TaskSessionRecord / RunRecord lose the dead fields.

498/498 unit tests pass. typecheck + build clean.
2026-04-18 23:22:17 -04:00
ZacharyZhang-NY 88a1f8b334 refactor(connectors): drop dead CLI command field after pi-mono migration
The CLI-adapter era `connector.command` field became vestigial once every
provider dispatch went through pi-ai. The schema still required a non-empty
command, so onboarding saveConnector silently rejected every Pi Mono row
(which carries an empty default). Remove the field from the payload chain
(renderer → IPC schema → DB CRUD → run dispatch → worker job), narrow
RunJob.taskId to match EnqueueRunPayload, and drop the dead helper plus
its test. Schema column stays NOT NULL for DB compatibility; seedConnectors
writes a literal empty string so no migration is needed.
2026-04-18 23:09:59 -04:00
ZacharyZhang-NY 2ace9183b0 refactor(session): drop dead task-scoped session columns (v41)
Sessions have been agent-scoped since v34, but the schema kept four
dead task-scoped columns that every session code path had to pretend
were meaningful:

  - tasks.active_session_id     (mirrored agents.session_id)
  - tasks.session_json          (superseded by task_sessions.session_params_json)
  - tasks.session_display_id    (superseded by task_sessions.session_display_id)
  - task_sessions.task_id       (always empty-string on new rows)

v41 drops all four via SQLite table rebuild — the only safe path when
multiple columns need to go in one step and indexes / FK directions
need to be preserved. `PRAGMA foreign_keys = OFF` spans the swap
window so cascade-delete does not fire mid-rebuild. The migration is
idempotent and also backfills the optional task columns
(task_type / requires_user_review / retry_count / last_failure_at /
started_at / completed_at / cancelled_at / checkout_run_id /
execution_locked_at) before rebuilding so it works whether or not
`ensureOptionalColumns` has run yet on this install.

Along with the schema change, every code path that read or wrote
those columns has been rewired to the agent-scoped equivalents:

- task-sessions-crud.ts: deleted updateTaskSession,
  getTaskSession, getActiveTaskSession, updateTasksActiveSessionForAgent;
  added updateSessionParams(sessionId, ...) for the worker's
  finalizer. ensureAgentSession and createNewSession no longer touch
  task_id.
- database-base.ts: facade drops the now-unused pass-throughs.
- snapshot.ts / task-queries.ts / tasks-status.ts / entity-crud.ts:
  SELECT projections no longer pull session_json / session_display_id
  / active_session_id from tasks.
- tasks-crud.ts (TaskRow) and shared/types/records-core.ts
  (TaskRecord): sessionJson / sessionDisplayId fields removed.
- TaskSessionRecord: taskId field removed.
- turn-context.ts::resolveSession: no more
  updateTasksActiveSessionForAgent mirror; sessionParams read directly
  from the current session via getSessionById.
- run-finished-helpers.ts::finalizeRun: stamps sessionParams via
  updateSessionParams on the agent's current session, resolved through
  agents.session_id.
- session-management-service.ts: rotateSession stops calling
  updateTasksActiveSessionForAgent.

Test suite updated to use the new session-id API. Existing tests used
`getActiveTaskSession(taskId)` and `createNewSession({ taskId, ...})`
as probes; they now use `getSessionById(sessionId)` and the taskId-less
createNewSession input.

Tests: 603/603 green. Typecheck clean. Migration tested on fresh DB
(runs v1→v41 cleanly) and idempotent re-apply (detects absent dead
columns, skips).
2026-04-18 21:54:31 -04:00
ZacharyZhang-NY 7e1d4199ee refactor(schema): make runs.task_id nullable, delete heartbeat workaround
The NOT NULL constraint forced heartbeat runs to synthesize a fake task
row (`[Heartbeat] X — Check for work`) so the FK could resolve. Three
layers of code — acquireHeartbeatTask, findReusableHeartbeatTask,
cleanupDuplicateHeartbeatTasks, plus the is_heartbeat flag column + UI
filter — existed to paper over one bad constraint. Remove the
constraint and the whole workaround vanishes.

SQLite table rebuild with data preservation. Live production DB, no
backup — migration is atomic, idempotent, and verified before DROP.

## Migration v35 (rebuild runs with nullable task_id)

Uses `fn:` form so it can:
- Snapshot `COUNT(*) FROM runs` before copy
- Walk `PRAGMA table_info(runs)` to detect drift columns (v34 declared
  a column set; any additive ALTER since v14 shows up here)
- Assert `COUNT(*) FROM runs_new == beforeCount` before `DROP TABLE runs`
- `PRAGMA defer_foreign_keys = ON` so run_events(run_id) FK defers to
  commit time, by which runs has the same id set under the new name
- Recreate all 6 indexes on the new table

If the assertion fails, the migration throws inside the existing
`transaction()` wrapper. SQLite rolls back the whole migration, old
`runs` table is untouched, `schema_migrations` never records v35. Next
app start retries cleanly.

## Migration v36 (drop tasks.is_heartbeat + synthetic rows)

Runs now carry `task_id = NULL` for heartbeat dispatches. The
`is_heartbeat` flag is dead data going forward.

1. `UPDATE runs SET task_id = NULL WHERE task_id IN (SELECT id FROM tasks WHERE is_heartbeat = 1)`
2. `DELETE FROM tasks WHERE is_heartbeat = 1` (synthetic rows, not real work)
3. Rebuild `tasks` without the `is_heartbeat` column (same defer-FK +
   row-count-assert pattern as v35)

## Code cleanup

- `RunRecord.taskId: string | null`; `createRun` / `cancelRunForDeletion`
  / `finishRun` guard every `UPDATE tasks` path with a null-check.
- `TaskRecord.isHeartbeat` removed; every SELECT / filter / UI reference
  dropped.
- Delete `acquireHeartbeatTask` / `findReusableHeartbeatTask` /
  `cleanupDuplicateHeartbeatTasks` — dead after the constraint lifts.
- Delete `src/main/orchestration/heartbeat-task.ts` (pure workaround
  constant file).
- `heartbeat.ts` creates runs with `taskId: null` when the agent has no
  assigned work. Turn context already supports nullable task
  (heartbeatFallbackSection); prompt surface unchanged.
- `validateTaskCreation` drops the `isHeartbeat` parameter — user-
  authored + automation-authored tasks are never heartbeat.
- Renderer: `run.taskId ? taskMap.get(...) : undefined` in Dashboard /
  RunCenter / RunDetail / RunQueue.
- Delete `tests/unit/heartbeat-coalescing.test.ts` (12 tests that
  exercised the deleted DB methods).
- Drop 1 isHeartbeat-bypass test from `task-quality-gates.test.ts`.

## Grep verification

- `isHeartbeatTaskTitle src/` → 0
- `acquireHeartbeatTask src/` → migration comment only
- `is_heartbeat src/` → schema.ts migration history only
- `isHeartbeat src/` → 0

Tests: 578/578 (was 591 — delta is 13 tests that tested deleted code).
2026-04-17 16:16:08 -04:00
ZacharyZhang-NY 14dc9a4897 refactor(cost): single writer for cost_entries, finishRun reads SUM
Cost was written through three independent paths: worker-runner shipped
`finalUsage.costUsd` in run-finished -> finishRun wrote `runs.cost_usd`,
`tasks.cost_usd += costUsd`, `agents.spent_monthly_usd += costUsd`, and
inserted a per-run aggregate row in cost_entries. CostWriter
additionally inserted per-message rows in cost_entries from main. Any
CostWriter failure created silent drift between `SUM(cost_entries)`
(read by budget enforcement) and `runs.cost_usd` (read by the UI).

Make CostWriter the single writer, finishRun the single reader.

- Add `db.recordCostEntry(...)` as the first-class insert path for
  per-message cost rows. CostWriter calls it; tests call it directly.
- Remove the aggregate INSERT from finishRun. finishRun now reads
  `SELECT COALESCE(SUM(amount_usd), 0) FROM cost_entries WHERE run_id=?
  AND unattributed=0` and writes that as the authoritative run total to
  runs.cost_usd, tasks.cost_usd, and agents.spent_monthly_usd.
- Drop `costUsd` from `finishRun` input, the worker `run-finished`
  ChildMessage shape, the automation `RunFinishedEvent` type, and every
  caller + test fixture. Cost is owned by the per-message path.
- worker-runner stops extracting `usage.cost.total` — token counts stay
  for session accumulation, cost comes from pi-ai via CostWriter.
- run-finished-helpers now gates budget incident checks on
  `event.status === "succeeded"` instead of a cost field that no longer
  exists on the event.
- automation-run-completion fires `budget_threshold` on every succeeded
  run (post-run spent_monthly_usd already reflects the SUM path).
- Update budget tests to write a `recordCostEntry` before `finishRun`
  to simulate the per-message cost path; drop `costUsd` from cross-
  subsystem, edge-case-budget, database, and related fixtures.

One writer, one reader, one source of truth. No drift.
2026-04-17 15:04:22 -04:00
ZacharyZhang-NY e94939a562 test(connector-id): migrate fixtures and assertions to pi-mono ids
Replaces legacy adapter ids (codex_local / claude_local / gemini_local /
claude_compatible) across ~42 test files with pi-mono equivalents:
- codex_local → openai-codex
- claude_local → claude-pro
- gemini_local → gemini-cli
- claude_compatible → anthropic

Hand-rewrites where fixture semantics changed:
- connector-matching: TASK_TYPE_PREFERENCE ordering rebased (claude-pro
  first for code/content/management; google first for research)
- connector-model-policy: getModels(provider) from pi-ai instead of the
  deleted adapter.module.models array
- session-management + edge-case-session: adapter exemption removed;
  rotation now uniform under pi-mono
- ipc-core-admin-handlers: CSV compat model replaced with pi-ai anthropic
- connectors.spec: lookup by openai-codex id (catalog now starts with claude-pro)
- onboarding-model-pool: listProviderModels(provider) helper; label text
  changed to "ChatGPT Plus/Pro (Codex)"
- database integration: added vault insecure-fallback env + electron mock
- command-paths: uses new @main/connector-helpers normalizeCommandValue
- pi-mono-migrations: added `as string` casts for 3 negative-existence
  assertions verifying legacy ids were removed

Drops:
- pnpm-workspace.yaml packages: [] (was "packages/*"; directory empty)
- obsolete CONNECTOR_STRENGTHS import; .module.models probing

Verification:
- pnpm typecheck → 0 errors
- pnpm lint → 0 errors / 0 warnings
- pnpm build:verify → 0 errors / 0 warnings
- pnpm test:unit → 508/508 pass
- pnpm test:integration → 96/96 pass
- All 6 adapter hard gates pass (NOT FOUND / 0 matches / empty)
- pi-ai / pi-agent-core / pi-coding-agent all resolved @0.67.6
2026-04-17 02:36:53 -04:00
ZacharyZhang-NY 67c757f481 update 2026-04-04 12:03:50 -04:00
ZacharyZhang-NY 047b09eae4 test(10-01): add cross-subsystem pipeline integration tests
- 6 tests exercising full 5-subsystem pipeline (D-04)
- Quality gate -> CAS checkout -> budget gate -> session -> completion
- Budget exhaustion blocks subsequent runs correctly
- Session token accumulation across multiple runs verified
- CAS double-checkout prevention in pipeline context
- Uses real AppDatabase, budget enforcement, session management services
2026-03-27 20:32:54 -04:00