T

25c2b3aed2 perf(agents): cut codex usage collector memory with SQL filter and cache (#6780 )

* perf(agents): cut codex usage collector memory with SQL filter and cache

The codex collector scanned every row of opencode.db (1.7 GB, 55k+ rows)
with Python-side json.loads, peaking around 716 MB of RSS on every run
-- including the panel's refreshLimits() call, which passed --limits-only
that the collector silently ignored.

Filter rows in SQL (LIKE gates + json_valid + json_extract authority,
mirroring the old Python filter semantics) so giant blobs are never
parsed, and cache the local stats scan in XDG_CACHE_HOME following the
claude collector's pattern (atomic writes, flock, schemaVersion).
--force rescans, --limits-only and normal mode reuse a fresh cache and
fall back to a full scan when it is missing, stale, or corrupt.

Measured: cold scan 716 MB -> 158 MB peak; warm --limits-only ~85 MB
and ~1.4 s. Output record schema and values are unchanged for the same
data (parity verified against the old filter, including malformed rows).

* Scope the codex scan cache's 15-minute reuse to --limits-only

A no-flag run is the widget's periodic refresh, and refreshIntervalSec is
configurable down to 30 seconds; holding every mode to a 15-minute cache
meant stats could lag far behind the interval the user asked for. Mirror
the claude collector: normal runs reuse a scan for ~20 seconds purely to
dedup concurrent collectors, and only --limits-only, which promises just
fresh limits, may reuse a scan for up to 15 minutes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Invalidate the codex scan cache across day boundaries

The cached stats embed date-dependent fields (todayPrompts,
todayTotalTokens, recentDays), but only the file's age was checked, so a
cache written at 23:58 served yesterday's numbers as "today" for up to
15 minutes past midnight. Stamp the envelope with the scan's local date
and treat any other date as a miss.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Reject codex scan caches with a future mtime

A cache whose mtime is ahead of the clock has a negative age, which the
freshness check accepted forever: setting the clock backwards froze the
stats until real time caught up with the file. Require a non-negative
age before trusting the cache.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Never cache an interrupted opencode scan

A transient lock, schema migration, or corrupted database aborts the
opencode scan mid-flight; the partial numbers still serve the current
run, but persisting them let a single bad read suppress opencode usage
for every cache reader until expiry. The claude collector already skips
its opencode cache write on a database error; do the same here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Make the json_valid guard order explicit in the opencode query

The query relied on json_valid(data) evaluating before json_extract(),
but SQLite does not promise that AND terms run left to right; a
reordered plan would let json_extract raise on a malformed row and
silently truncate the scan. Wrap each json_extract in a CASE so the
guard is structural rather than positional.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Drop a claude-collector comment that is false for codex

"These caches were world-readable before" was copied from the claude
collector; codex had no caches before this one existed. Explain the
chmod on its own terms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: markbusking <marcosbustos.dev@gmail.com>
Co-authored-by: David Heinemeier Hansson <david@hey.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

2026-08-13 13:47:44 +02:00