schema.ts had grown to 852 LOC — migration history is append-only, so
the file gets bigger every release. Organize by era without fragmenting
the history.
| File | LOC | Contents |
|-----------------------------------|-----|---------------------------------------|
| schema.ts | 195 | helpers, runMigrations, |
| | | ensureOptionalColumns, seed, init |
| migrations/types.ts | 47 | Migration/Deps types, column/table |
| | | helpers |
| migrations/v1-v15.ts | 50 | createMigrationsV1V15 — v1..v15 |
| migrations/v16-v25.ts | 157 | createMigrationsV16V25 — v16..v25 |
| migrations/v26-v34.ts | 158 | createMigrationsV26V34 — v26..v34 |
| migrations/v35-v37.ts | 326 | createMigrationsV35V37 — three heavy |
| | | table-rebuilds (runs/tasks/skills) |
Each era exports a factory `createMigrationsVxxVyy(deps)` that takes
`{ db, readAll }` — migrations' `fn` bodies captured those names via
closure in the original; factory pattern preserves the closure shape
byte-for-byte while making the dependency flow explicit.
Master `schema.ts` imports all four and concatenates them into the
final ascending list. Order preserved including the quirky v31 -> v33
-> v32 -> v34 sequence that predated my changes.
All 37 migrations' `version`, `name`, and `sql`/`fn` bodies are
byte-for-byte identical. No behavior change; 578 tests green.
`database-base.ts` (449 LOC) kept as-is. It's a thin linear pass-
through facade — every method is 1–3 lines. Splitting a facade of
this kind by forcing mixins or abstract classes costs more than it
saves. Not a god component in the Linus sense (one responsibility:
delegate); flagged here so future readers know it's deliberate.
`company_skills.markdown` stored the full SKILL.md body. The on-disk
file under `profile/skills/{companyId}/{slug}/SKILL.md` is what pi-
coding-agent actually reads via `loadSkills` at runtime. The DB column
was a duplicate — import wrote to both, renderer read from DB. Any
discrepancy between disk and DB was invisible until the agent behaved
differently from what the user saw in SkillDetail.
Disk becomes the single source of truth. DB keeps name / description /
metadata / source info / file inventory — the cheap lookup row. UI reads
SKILL.md content live via a new IPC channel.
## Migration v37 (rebuild company_skills without markdown)
Table-rebuild pattern matching v35/v36: `fn:` form, PRAGMA walk for
drift columns, row-count assertion before DROP, `defer_foreign_keys`
for the `agent_skill_assignments(skill_id)` reference. Indexes
recreated. Idempotent via `schema_migrations` ledger.
Existing DB rows' markdown content is discarded — but the on-disk
SKILL.md files are the authoritative copy, so no user-visible data
loss. Any agent using those skills keeps working; the UI just starts
reading from disk instead of the DB cache.
## Code changes
- `src/main/database/skills.ts`: drop `markdown` from read mapping,
upsert input type, INSERT column/bind list, ON CONFLICT DO UPDATE.
- `src/shared/types/records-core.ts`: `SkillRecord.markdown` removed.
- `src/main/skills/import.ts` + `import-github.ts`: drop `markdown:`
from 4 `saveCompanySkill` call sites. Disk-write via
`materializeSkillFiles` unchanged (it's what the file agent reads).
- `src/main/skills/index.ts`: expose `skillFilesDir` for the new IPC
handler to resolve `${profile}/skills/${companyId}/${slug}/SKILL.md`.
- `src/shared/contracts/channels.ts` + `schemas-execution.ts` +
`desktop-api/skills.ts`: new `getSkillMarkdown` channel
`{ companyId, slug } → DesktopResult<string>`.
- `src/preload/index.ts`: expose `getSkillMarkdown(companyId, slug)`.
- `src/main/ipc-skill-handlers.ts`: handler validates schema, ensures
company, resolves path, `readFile`, returns content. Fails closed
with `SKILL_NOT_FOUND` on ENOENT, `SKILL_READ_FAILED` on other
read errors.
- `src/renderer/components/skills/SkillDetail.tsx`: `useEffect` fetches
markdown via IPC when the skill detail loads; shows "Loading…" until
resolved, surfaces read errors. Preserves `stripFrontmatter`.
- Tests: drop `markdown:` from 2 fixture sites
(`skill-types.test.ts`, `skill-aware-assignment.spec.ts`).
## Verification
- typecheck / lint / 578 tests green
- `grep 'skill\.markdown' src/` → 0 DB-record reads
- `grep '\.markdown\b' src/main/database/skills.ts` → 0
- Remaining `.markdown` references are all `ghSkill.markdown` (GitHub
fetcher's intermediate struct feeding `materializeSkillFiles` to
disk) and parameter names in `stripFrontmatter(markdown: string)` /
fs-writer helpers — none read from a DB row.
The NOT NULL constraint forced heartbeat runs to synthesize a fake task
row (`[Heartbeat] X — Check for work`) so the FK could resolve. Three
layers of code — acquireHeartbeatTask, findReusableHeartbeatTask,
cleanupDuplicateHeartbeatTasks, plus the is_heartbeat flag column + UI
filter — existed to paper over one bad constraint. Remove the
constraint and the whole workaround vanishes.
SQLite table rebuild with data preservation. Live production DB, no
backup — migration is atomic, idempotent, and verified before DROP.
## Migration v35 (rebuild runs with nullable task_id)
Uses `fn:` form so it can:
- Snapshot `COUNT(*) FROM runs` before copy
- Walk `PRAGMA table_info(runs)` to detect drift columns (v34 declared
a column set; any additive ALTER since v14 shows up here)
- Assert `COUNT(*) FROM runs_new == beforeCount` before `DROP TABLE runs`
- `PRAGMA defer_foreign_keys = ON` so run_events(run_id) FK defers to
commit time, by which runs has the same id set under the new name
- Recreate all 6 indexes on the new table
If the assertion fails, the migration throws inside the existing
`transaction()` wrapper. SQLite rolls back the whole migration, old
`runs` table is untouched, `schema_migrations` never records v35. Next
app start retries cleanly.
## Migration v36 (drop tasks.is_heartbeat + synthetic rows)
Runs now carry `task_id = NULL` for heartbeat dispatches. The
`is_heartbeat` flag is dead data going forward.
1. `UPDATE runs SET task_id = NULL WHERE task_id IN (SELECT id FROM tasks WHERE is_heartbeat = 1)`
2. `DELETE FROM tasks WHERE is_heartbeat = 1` (synthetic rows, not real work)
3. Rebuild `tasks` without the `is_heartbeat` column (same defer-FK +
row-count-assert pattern as v35)
## Code cleanup
- `RunRecord.taskId: string | null`; `createRun` / `cancelRunForDeletion`
/ `finishRun` guard every `UPDATE tasks` path with a null-check.
- `TaskRecord.isHeartbeat` removed; every SELECT / filter / UI reference
dropped.
- Delete `acquireHeartbeatTask` / `findReusableHeartbeatTask` /
`cleanupDuplicateHeartbeatTasks` — dead after the constraint lifts.
- Delete `src/main/orchestration/heartbeat-task.ts` (pure workaround
constant file).
- `heartbeat.ts` creates runs with `taskId: null` when the agent has no
assigned work. Turn context already supports nullable task
(heartbeatFallbackSection); prompt surface unchanged.
- `validateTaskCreation` drops the `isHeartbeat` parameter — user-
authored + automation-authored tasks are never heartbeat.
- Renderer: `run.taskId ? taskMap.get(...) : undefined` in Dashboard /
RunCenter / RunDetail / RunQueue.
- Delete `tests/unit/heartbeat-coalescing.test.ts` (12 tests that
exercised the deleted DB methods).
- Drop 1 isHeartbeat-bypass test from `task-quality-gates.test.ts`.
## Grep verification
- `isHeartbeatTaskTitle src/` → 0
- `acquireHeartbeatTask src/` → migration comment only
- `is_heartbeat src/` → schema.ts migration history only
- `isHeartbeat src/` → 0
Tests: 578/578 (was 591 — delta is 13 tests that tested deleted code).
Sessions were keyed by (task, agent). Every task switch started a fresh
transcript — the agent lost everything it had learned on the previous
task. That breaks the product premise: a CEO should remember its past
decisions, an engineer should carry context between bugs.
Move pi's AgentMessage transcript to be agent-scoped. One session per
agent (per company). Tasks enter the ongoing conversation as user turns;
rotation/compaction boundaries follow the agent's long-running session.
Additive schema only — no row rewrites on the live production DB.
- Migration v34: `ALTER TABLE agents ADD COLUMN session_id TEXT` +
`idx_agents_session_id`. `ensureOptionalColumns` mirror for fresh DBs.
- `AgentRecord.sessionId?: string | null` in the read model; included
in every full-agent SELECT (entity-crud, snapshot, agents-hierarchy
chain-of-command / direct-reports / heartbeat-eligible).
- `db.ensureAgentSession({ agentId, companyId, adapterType })` —
idempotent: returns existing `agents.session_id` if set, else inserts
a new `task_sessions` row with an empty `task_id` sentinel (agent-
scoped, task-id is now metadata) and stores the id on the agent.
- `db.setAgentSessionId(agentId, sessionId)` — used by rotation.
- `db.updateTasksActiveSessionForAgent(agentId, sessionId)` — mirrors
the agent's current session onto every task the agent owns, so
existing per-task readers (`tasks.active_session_id`) still work.
- `turn-context.resolveSession`: pulls the agent's session via
`ensureAgentSession` instead of `getActiveTaskSession(task.id)`,
then points the current task at it so UI continues to resolve.
- `session-management-service.rotateSession`: new session carries
`taskId:""`, then `setAgentSessionId` + `updateTasksActiveSessionForAgent`
so every task the agent owns follows the rotation boundary.
- `session-entries.listActive`: `ORDER BY created_at, sequence` — the
created_at timestamp is authoritative; sequence stays as same-ms
tiebreaker. Forward-safety for future merge scenarios.
Existing agents' first post-upgrade run creates a fresh agent-scoped
session. Their pre-migration per-task transcripts remain in the DB as
historical records but are not loaded for new runs. This is a one-time
transcript reset, accepted over a risky lossy merge on the live DB.
Documented in a code comment on `ensureAgentSession`.
Migration 28 (add_snapshot_hot_path_indexes) referenced task_comments,
documents, meetings, sprints, agent_messages, activity — all tables
created by ensureOptionalColumns() which runs AFTER runMigrations() on
a fresh DB. Converted from sql-string migration to fn-based so each
index is gated by tableExists(); safe on both fresh and migrated DBs.
Also updated tests/unit/selective-ipc.test.ts exhaustive switch to
include the 7 new DesktopEvent variants added in ef90a58.
- Expose buildSessionEntryMethods and buildCredentialMethods on
AppDatabaseBase as public readonly members so agent-runtime and
credential-store can access them via db.sessionEntries /
db.credentialsTable
- Add idempotent schema migration v32 creating session_entries and
credentials tables plus supporting indexes; uses CREATE TABLE IF NOT
EXISTS so existing profile DBs upgrade safely without destructive ops
Add explicit LIMIT to all hot-path tables in listSnapshot regardless of
companyId, raise cross-company caps from 500/250 to 2000/1000 so dashboard
no longer truncates silently at 500.
Add migration 28 with composite indexes covering snapshot WHERE/ORDER BY
columns: tasks(company_id, created_at|updated_at), approvals(company_id,
created_at), task_comments(company_id, created_at), documents(company_id,
updated_at), meetings(company_id, scheduled_at), sprints(company_id,
start_date), runs(agent_id, created_at), cost_entries(agent_id,
created_at), activity(company_id, actor, action), tasks(company_id,
assignee_agent_id, status). Indexes match exact ORDER BY direction so
SQLite can satisfy reads via index scan without temp sort.
Stop writing new session data back to the legacy tasks.session_json
column; updateTaskSession now upserts the active task_sessions row
(updating the session pointed to by tasks.active_session_id, creating
one when absent), and getTaskSession reads from the same row so
session data has a single source of truth. Adds migration 27 to
backfill pre-existing tasks.session_json values into task_sessions
idempotently via INSERT OR IGNORE guarded on task_id. The legacy
column is kept for read-only history and is not dropped.
Stop silently swallowing duplicate-column errors in the migration
runner; any real migration failure now throws. Replace try/catch-based
column additions in ensureOptionalColumns and migrations 19-22 with
pragma table_info checks so repeated startup is idempotent without
tolerating error signals. Reorder initializeDatabase so runMigrations
runs before ensureOptionalColumns to avoid migration 13 adding columns
that ensureOptionalColumns already created on fresh databases.
ZacharyZhang-NY