refactor(schema): make runs.task_id nullable, delete heartbeat workaround
The NOT NULL constraint forced heartbeat runs to synthesize a fake task row (`[Heartbeat] X — Check for work`) so the FK could resolve. Three layers of code — acquireHeartbeatTask, findReusableHeartbeatTask, cleanupDuplicateHeartbeatTasks, plus the is_heartbeat flag column + UI filter — existed to paper over one bad constraint. Remove the constraint and the whole workaround vanishes. SQLite table rebuild with data preservation. Live production DB, no backup — migration is atomic, idempotent, and verified before DROP. ## Migration v35 (rebuild runs with nullable task_id) Uses `fn:` form so it can: - Snapshot `COUNT(*) FROM runs` before copy - Walk `PRAGMA table_info(runs)` to detect drift columns (v34 declared a column set; any additive ALTER since v14 shows up here) - Assert `COUNT(*) FROM runs_new == beforeCount` before `DROP TABLE runs` - `PRAGMA defer_foreign_keys = ON` so run_events(run_id) FK defers to commit time, by which runs has the same id set under the new name - Recreate all 6 indexes on the new table If the assertion fails, the migration throws inside the existing `transaction()` wrapper. SQLite rolls back the whole migration, old `runs` table is untouched, `schema_migrations` never records v35. Next app start retries cleanly. ## Migration v36 (drop tasks.is_heartbeat + synthetic rows) Runs now carry `task_id = NULL` for heartbeat dispatches. The `is_heartbeat` flag is dead data going forward. 1. `UPDATE runs SET task_id = NULL WHERE task_id IN (SELECT id FROM tasks WHERE is_heartbeat = 1)` 2. `DELETE FROM tasks WHERE is_heartbeat = 1` (synthetic rows, not real work) 3. Rebuild `tasks` without the `is_heartbeat` column (same defer-FK + row-count-assert pattern as v35) ## Code cleanup - `RunRecord.taskId: string | null`; `createRun` / `cancelRunForDeletion` / `finishRun` guard every `UPDATE tasks` path with a null-check. - `TaskRecord.isHeartbeat` removed; every SELECT / filter / UI reference dropped. - Delete `acquireHeartbeatTask` / `findReusableHeartbeatTask` / `cleanupDuplicateHeartbeatTasks` — dead after the constraint lifts. - Delete `src/main/orchestration/heartbeat-task.ts` (pure workaround constant file). - `heartbeat.ts` creates runs with `taskId: null` when the agent has no assigned work. Turn context already supports nullable task (heartbeatFallbackSection); prompt surface unchanged. - `validateTaskCreation` drops the `isHeartbeat` parameter — user- authored + automation-authored tasks are never heartbeat. - Renderer: `run.taskId ? taskMap.get(...) : undefined` in Dashboard / RunCenter / RunDetail / RunQueue. - Delete `tests/unit/heartbeat-coalescing.test.ts` (12 tests that exercised the deleted DB methods). - Drop 1 isHeartbeat-bypass test from `task-quality-gates.test.ts`. ## Grep verification - `isHeartbeatTaskTitle src/` → 0 - `acquireHeartbeatTask src/` → migration comment only - `is_heartbeat src/` → schema.ts migration history only - `isHeartbeat src/` → 0 Tests: 578/578 (was 591 — delta is 13 tests that tested deleted code).
This commit is contained in:
ZacharyZhang-NY committed
1 parent
92795f7bfb
commit
7e1d4199ee
44 files changed
+385
-674
No files matched your search
+227
-1
@@ -42,7 +42,6 @@ export function ensureOptionalColumns(db: DatabaseSync) {
|
||||
addColumnIfMissing("tasks", "checkout_run_id", "alter table tasks add column checkout_run_id text");
|
||||
addColumnIfMissing("tasks", "execution_locked_at", "alter table tasks add column execution_locked_at text");
|
||||
addColumnIfMissing("tasks", "active_session_id", "alter table tasks add column active_session_id text");
|
||||
addColumnIfMissing("tasks", "is_heartbeat", "alter table tasks add column is_heartbeat integer not null default 0");
|
||||
addColumnIfMissing("cost_entries", "project_id", "alter table cost_entries add column project_id text");
|
||||
db.exec(`
|
||||
create table if not exists task_comments (id text primary key, company_id text not null, task_id text not null, author_agent_id text, author_name text not null, body text not null, created_at text not null);
|
||||
@@ -513,6 +512,233 @@ export function runMigrations(
|
||||
sql: `ALTER TABLE agents ADD COLUMN session_id TEXT;
|
||||
CREATE INDEX IF NOT EXISTS idx_agents_session_id ON agents(session_id)`,
|
||||
},
|
||||
{
|
||||
// Rebuild runs table with task_id NULLABLE. Heartbeat runs (agent
|
||||
// wake-ups with no real task) no longer need to synthesize a fake
|
||||
// task row; they simply insert a run with task_id = NULL. The old
|
||||
// workaround (is_heartbeat flag + acquireHeartbeatTask + cleanup
|
||||
// sweeps) becomes dead code in v36.
|
||||
//
|
||||
// SQLite can't ALTER a column's NOT NULL constraint, so we rebuild:
|
||||
// 1. Snapshot row count
|
||||
// 2. Discover actual column set via PRAGMA (drift safe)
|
||||
// 3. CREATE runs_new with the same columns + task_id nullable
|
||||
// 4. Copy every row
|
||||
// 5. Assert copied count == snapshot count (abort before DROP
|
||||
// if any row went missing)
|
||||
// 6. DROP old, RENAME new
|
||||
// 7. Recreate every index on runs
|
||||
//
|
||||
// FK from run_events(run_id) references runs(id); PRAGMA foreign_keys
|
||||
// OFF during the rebuild (we're inside a transaction already, so we
|
||||
// toggle the defer_foreign_keys flag instead — SQLite forbids
|
||||
// changing foreign_keys mid-transaction).
|
||||
version: 35, name: "make_runs_task_id_nullable",
|
||||
fn: () => {
|
||||
const beforeCount = (db
|
||||
.prepare("SELECT COUNT(*) as cnt FROM runs")
|
||||
.get() as { cnt: number }).cnt;
|
||||
|
||||
const existingCols = (db
|
||||
.prepare("PRAGMA table_info(runs)")
|
||||
.all() as Array<{ name: string; type: string; notnull: number; dflt_value: unknown; pk: number }>);
|
||||
if (existingCols.length === 0) {
|
||||
throw new Error("runs table is missing; refusing to migrate");
|
||||
}
|
||||
const colNames = existingCols.map((c) => c.name);
|
||||
|
||||
// Declare the known v34 column set. If the live DB has drift (extra
|
||||
// columns we did not declare), preserve those verbatim using the
|
||||
// column metadata from PRAGMA.
|
||||
const knownCols = new Set([
|
||||
"id", "company_id", "task_id", "agent_id", "workspace_id",
|
||||
"connector_id", "status", "summary", "error_message", "exit_code",
|
||||
"signal", "model", "session_display_id", "cost_usd",
|
||||
"unattributed_cost", "started_at", "finished_at", "log_path",
|
||||
"created_at", "updated_at", "context_snapshot_json",
|
||||
"actor_type", "actor_id",
|
||||
]);
|
||||
const driftCols = existingCols.filter((c) => !knownCols.has(c.name));
|
||||
|
||||
// Defer FK checks within this transaction.
|
||||
db.exec("PRAGMA defer_foreign_keys = ON");
|
||||
|
||||
// Build CREATE TABLE runs_new. task_id is nullable. Other columns
|
||||
// reuse the declared types / constraints. For drift columns we
|
||||
// emit a best-effort column definition (type, NULL, default).
|
||||
const declaredDdl = [
|
||||
"id TEXT PRIMARY KEY",
|
||||
"company_id TEXT NOT NULL",
|
||||
"task_id TEXT", // NULLABLE (was NOT NULL)
|
||||
"agent_id TEXT NOT NULL",
|
||||
"workspace_id TEXT",
|
||||
"connector_id TEXT NOT NULL",
|
||||
"status TEXT NOT NULL",
|
||||
"summary TEXT NOT NULL",
|
||||
"error_message TEXT",
|
||||
"exit_code INTEGER",
|
||||
"signal TEXT",
|
||||
"model TEXT",
|
||||
"session_display_id TEXT",
|
||||
"cost_usd REAL",
|
||||
"unattributed_cost INTEGER NOT NULL DEFAULT 0",
|
||||
"started_at TEXT",
|
||||
"finished_at TEXT",
|
||||
"log_path TEXT",
|
||||
"created_at TEXT NOT NULL",
|
||||
"updated_at TEXT NOT NULL",
|
||||
"context_snapshot_json TEXT",
|
||||
"actor_type TEXT",
|
||||
"actor_id TEXT",
|
||||
];
|
||||
for (const drift of driftCols) {
|
||||
const type = drift.type || "TEXT";
|
||||
declaredDdl.push(`${drift.name} ${type}`);
|
||||
}
|
||||
|
||||
db.exec(`CREATE TABLE runs_new (${declaredDdl.join(", ")})`);
|
||||
|
||||
// Copy preserving every column present in the old table.
|
||||
const copyCols = colNames.join(", ");
|
||||
db.exec(`INSERT INTO runs_new (${copyCols}) SELECT ${copyCols} FROM runs`);
|
||||
|
||||
const afterCount = (db
|
||||
.prepare("SELECT COUNT(*) as cnt FROM runs_new")
|
||||
.get() as { cnt: number }).cnt;
|
||||
if (afterCount !== beforeCount) {
|
||||
throw new Error(
|
||||
`runs copy row count mismatch: before=${beforeCount} after=${afterCount}; aborting before DROP`,
|
||||
);
|
||||
}
|
||||
|
||||
db.exec("DROP TABLE runs");
|
||||
db.exec("ALTER TABLE runs_new RENAME TO runs");
|
||||
|
||||
// Recreate every index on runs.
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_runs_company ON runs(company_id)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_runs_task ON runs(task_id)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_runs_agent ON runs(agent_id)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_runs_company_status ON runs(company_id, status)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_runs_company_created_at ON runs(company_id, created_at DESC)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_runs_agent_created_at ON runs(agent_id, created_at DESC)");
|
||||
|
||||
log.info(`[migration v35] runs table rebuilt with nullable task_id (${afterCount} rows preserved)`);
|
||||
},
|
||||
},
|
||||
{
|
||||
// Drop the `tasks.is_heartbeat` column and all synthetic heartbeat
|
||||
// task rows. Prerequisite: v35 must have made runs.task_id nullable
|
||||
// so we can null-out runs that pointed at heartbeat tasks before
|
||||
// deleting those task rows.
|
||||
version: 36, name: "drop_tasks_is_heartbeat_and_synthetic_rows",
|
||||
fn: () => {
|
||||
const hasColumn = columnExists(db, "tasks", "is_heartbeat");
|
||||
if (!hasColumn) {
|
||||
log.info("[migration v36] tasks.is_heartbeat already absent; skipping");
|
||||
return;
|
||||
}
|
||||
|
||||
// 1. NULL-out runs that pointed at heartbeat tasks.
|
||||
const updated = db
|
||||
.prepare(
|
||||
"UPDATE runs SET task_id = NULL WHERE task_id IN (SELECT id FROM tasks WHERE is_heartbeat = 1)",
|
||||
)
|
||||
.run();
|
||||
const runsNulledOut = Number(updated.changes);
|
||||
|
||||
// 2. Delete synthetic heartbeat tasks. Cost entries referring to
|
||||
// these tasks remain with their task_id pointing at now-deleted
|
||||
// ids — acceptable, cost_entries.task_id has no FK constraint.
|
||||
// task_comments and approvals attached to these tasks are
|
||||
// already cascaded by the task row delete or would remain
|
||||
// orphaned; heartbeat tasks carry neither in practice.
|
||||
const beforeHeartbeatCount = (db
|
||||
.prepare("SELECT COUNT(*) as cnt FROM tasks WHERE is_heartbeat = 1")
|
||||
.get() as { cnt: number }).cnt;
|
||||
db.exec("DELETE FROM tasks WHERE is_heartbeat = 1");
|
||||
|
||||
// 3. Rebuild `tasks` without the is_heartbeat column.
|
||||
const beforeTasksCount = (db
|
||||
.prepare("SELECT COUNT(*) as cnt FROM tasks")
|
||||
.get() as { cnt: number }).cnt;
|
||||
|
||||
const existingCols = (db
|
||||
.prepare("PRAGMA table_info(tasks)")
|
||||
.all() as Array<{ name: string; type: string; notnull: number; dflt_value: unknown; pk: number }>);
|
||||
const keepCols = existingCols.filter((c) => c.name !== "is_heartbeat");
|
||||
|
||||
db.exec("PRAGMA defer_foreign_keys = ON");
|
||||
|
||||
// Declared v20 + v13 + v38+v39/v20 column set for `tasks`. Drift
|
||||
// columns (any column we haven't declared here) are preserved
|
||||
// verbatim using PRAGMA metadata — same drift-safe shape as v35.
|
||||
const declaredCols = new Map<string, string>([
|
||||
["id", "TEXT PRIMARY KEY"],
|
||||
["company_id", "TEXT NOT NULL"],
|
||||
["project_id", "TEXT"],
|
||||
["goal_id", "TEXT"],
|
||||
["parent_id", "TEXT"],
|
||||
["title", "TEXT NOT NULL"],
|
||||
["description", "TEXT NOT NULL"],
|
||||
["assignee_agent_id", "TEXT"],
|
||||
["workspace_id", "TEXT"],
|
||||
["status", "TEXT NOT NULL"],
|
||||
["priority", "TEXT NOT NULL"],
|
||||
["active_run_id", "TEXT"],
|
||||
["session_json", "TEXT"],
|
||||
["session_display_id", "TEXT"],
|
||||
["cost_usd", "REAL NOT NULL DEFAULT 0"],
|
||||
["created_at", "TEXT NOT NULL"],
|
||||
["updated_at", "TEXT NOT NULL"],
|
||||
["task_type", "TEXT NOT NULL DEFAULT 'general'"],
|
||||
["requires_user_review", "INTEGER NOT NULL DEFAULT 0"],
|
||||
["retry_count", "INTEGER NOT NULL DEFAULT 0"],
|
||||
["last_failure_at", "TEXT"],
|
||||
["checkout_run_id", "TEXT"],
|
||||
["execution_locked_at", "TEXT"],
|
||||
["active_session_id", "TEXT"],
|
||||
["started_at", "TEXT"],
|
||||
["completed_at", "TEXT"],
|
||||
["cancelled_at", "TEXT"],
|
||||
]);
|
||||
|
||||
const ddl = keepCols.map((c) => {
|
||||
const declared = declaredCols.get(c.name);
|
||||
if (declared) return `${c.name} ${declared}`;
|
||||
return `${c.name} ${c.type || "TEXT"}`;
|
||||
});
|
||||
|
||||
db.exec(`CREATE TABLE tasks_new (${ddl.join(", ")})`);
|
||||
|
||||
const copyCols = keepCols.map((c) => c.name).join(", ");
|
||||
db.exec(`INSERT INTO tasks_new (${copyCols}) SELECT ${copyCols} FROM tasks`);
|
||||
|
||||
const afterTasksCount = (db
|
||||
.prepare("SELECT COUNT(*) as cnt FROM tasks_new")
|
||||
.get() as { cnt: number }).cnt;
|
||||
if (afterTasksCount !== beforeTasksCount) {
|
||||
throw new Error(
|
||||
`tasks copy row count mismatch: before=${beforeTasksCount} after=${afterTasksCount}; aborting before DROP`,
|
||||
);
|
||||
}
|
||||
|
||||
db.exec("DROP TABLE tasks");
|
||||
db.exec("ALTER TABLE tasks_new RENAME TO tasks");
|
||||
|
||||
// Recreate every index on tasks.
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_tasks_company ON tasks(company_id)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_tasks_assignee ON tasks(assignee_agent_id)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_tasks_project ON tasks(project_id)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_tasks_company_status ON tasks(company_id, status)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_tasks_company_created_at ON tasks(company_id, created_at DESC)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_tasks_company_updated_at ON tasks(company_id, updated_at DESC)");
|
||||
db.exec("CREATE INDEX IF NOT EXISTS idx_tasks_company_assignee_status ON tasks(company_id, assignee_agent_id, status)");
|
||||
|
||||
log.info(
|
||||
`[migration v36] removed tasks.is_heartbeat; deleted ${beforeHeartbeatCount} synthetic heartbeat task row(s); nulled task_id on ${runsNulledOut} run(s); preserved ${afterTasksCount} task row(s)`,
|
||||
);
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
for (const migration of migrations) {
|
||||
|
||||
Reference in new issue
Block a user