Run the timers every 5 minutes, with overlap and failure guards

A push reaching the mirror should take minutes, not up to six hours. All four
units now fire every 5 minutes, staggered a minute apart. Three guards make
that cadence safe:

- Scheduled runs take the release lock NON-BLOCKING (try_release_lock) and
  skip the tick when a build is running. Blocking would stack one stalled
  process per tick behind a long build and stampede when it finished. Manual
  commands still wait, as an operator expects.
- check-versions takes the lock too, and now owns its git pull (--pull, passed
  by the unit) instead of an ExecStartPre: at this cadence an unlocked pull
  would swap PKGBUILDs out from under a running build.
- A failed release records .build-failed-<channel> and backs off
  exponentially (10m, 20m, 40m … capped at 6h) rather than rebuilding the same
  broken tree every 5 minutes. Any new commit clears the backoff, since a push
  is the most likely fix.

Idle ticks exit without output so the journal keeps showing the runs that
matter, and bin/repo timers reports backoff state — a paused channel is
otherwise indistinguishable from an idle one.
This commit is contained in:
Ryan Hughes
2026-08-27 01:10:33 -04:00
parent 9d86123264
commit 71e72e581b
10 changed files with 243 additions and 57 deletions
+3 -3
View File
@@ -1,10 +1,10 @@
[Unit]
Description=Auto-release stable mirror every 6 hours
Description=Release queued stable builds every 5 minutes
[Timer]
# Run every 6 hours at :00 (after version check at :30)
OnCalendar=*-*-* 01,07,13,19:00:00 America/New_York
OnCalendar=*:3/5
Persistent=true
RandomizedDelaySec=20
[Install]
WantedBy=timers.target