When edge already holds the version, a PKGBUILD whose recipe changed
(comments and blank lines aside) builds nothing and publishes nothing.
Arch bumps pkgrel only when the built package changes, which is the
reviewer's call, so this is a warning annotation on the PKGBUILD and a
job summary line, not a failure. Comment-only and hook-only changes
stay silent.
A PR can change a package's directory without bumping its version: an
upstream hook, its tests, a README. bin/build then plans nothing, and
the pack step failed on the empty build output, turning the required
result check red (seen on #769). Make the same dry-run check
publish.yml's rebuild job already makes, and skip packing and uploading
when nothing was built. publish.yml finds no artifact for such a merge
and records the package as already published.
The planner took 'git diff base.sha head.sha', a two-dot diff between
the current master tip and the PR head. For a PR that branched before
later merges, that counts every package master has changed since, so
the PR plans those too and builds its own stale copies of them: wasted
builds, and 'already up to date' Pack failures that turn the PR red for
packages it never touched. #397 (lazyjournal) planned 22 packages for a
one-package change. The checkout is shallow, so ask GitHub for the PR's
files, which are listed against the merge base.
'git status --short | head' under set -o pipefail: once the PR's
pkgbuilds/ differs from base in more than ten files, head exits, git
takes SIGPIPE and the step fails with 141 before anything builds. Every
PR branched far enough behind master hit it (omadev #423, llmman-bin
#428 and others on 2026-09-28). sed -n '1,10p' prints the same preview
and drains the stream.
The droplet pool is x86, so aarch64 builds ran under QEMU about 30x slower;
omarchy-mac-boot's check() took 2h47m of the 180-minute job limit. Heavy
packages stay on the droplets until a native build of each is shown to fit.
makepkg runs check() inside the build container and meson writes each
test's output to the build tree, not to stdout. A failing test therefore
leaves only a summary line in the job log. Diagnosing it means reading
the package source and inferring the cause.
bin/build bind-mounts $SRC_DIR at /src, so the logs outlive the
container at src/**/meson-logs/ on the runner. Upload them when the
build step fails.
owe 0.2.7 showed the cost: owe:transition failed on aarch64 and passed
on x86_64, and CI carried no record of which assertion tripped.
Co-authored-by: Bjarne Oeverli <1419214+bjarneo@users.noreply.github.com>
Three things kept the upstream sync PR (#589) from ever finishing a build:
Scoped dispatches wiped the shared PR. A workflow_dispatch with `packages`
regenerates only those packages from master, and pushing that to
auto/sync-upstream replaced 38 pending updates with one. Scoped runs now
push to their own auto/sync-{upstream,rebuilds}-<packages> branch and PR;
scheduled runs keep the shared branch.
build-approved stopped working after the first bot push. A GITHUB_TOKEN
push creates pull_request runs held for approval but no pull_request_target
run, so approve-pr.yml never saw it: its last run on the branch was the
label itself (2026-09-25T19:26), and each of the next four syncs sat at
action_required. The sync workflows now release the held runs for the
commit they just pushed, from a separate job holding actions: write, and
only for their own bot-authored, same-repo PR while build-approved is on
it.
Each approved push cancelled the in-flight build. Approving the 21:43
sync's build cancelled the label-triggered one still queued on strata and
schist-bin. On auto/sync-* branches a new build now waits for the running
one instead, then reuses its artifacts. The approval script no longer
waits for a lone approved build to start before releasing tests, which a
queued build would have turned into a timeout.
Every push to a PR rebuilt every package the PR touches, on every
architecture, even when only one of them changed. The daily sync PR
carries around thirty package/arch pairs; fixing one package meant
rebuilding all of them, and an aarch64 build under QEMU takes up to an
hour. Nine runs of that PR cost about 36 droplet-hours in two days.
The planner now asks the artifact store for <pkg>-<arch>-<tree hash>
before adding an entry to the matrix and drops entries that already
have one. That is the same lookup publish.yml makes on merge, so a
reused entry publishes exactly the file it would have anyway. Dry run
against the current sync PR: 27 of 31 entries reused, 4 built.
Pack and Upload no longer run with always(): only a successful build
uploads, so an artifact's existence means that tree built.
workflow_dispatch always builds; it is an explicit request.
github.event.pull_request.base.sha is a snapshot taken when the PR was
last pushed, not the current tip of the base branch. A reopened or rerun
PR therefore builds with whatever master looked like at its last push,
and a tooling fix that landed on master since then never reaches it:
the daily sync PR reopened after #553 merged checked out a base without
helpers/artifact-helpers.sh and failed at "Pack artifact".
Check out base.ref instead. The plan's diff and the empty-PR check still
compare base.sha to head.sha, so the list of changed packages is
unaffected; only the tooling that runs on the droplet moves to the tip.
actions/upload-artifact rejects any path containing ':', and makepkg names
a package with an epoch `name-1:ver-rel-arch.pkg.tar.zst`. Every PR that
built such a package (cursor-cli in the sync PRs, omasnap once it gained an
epoch) failed at "Upload artifact" after a successful build, and publish
then rebuilt from scratch on merge.
The files now ride inside packages.tar for the artifact hop and come back
out with makepkg's names untouched: pacman clients and bin/publish-artifact
both require the filename to match PKGINFO, and the channels already carry
these names. publish.yml still accepts bare pre-packing artifacts until the
7-day retention drains them.
helpers/artifact-helpers.sh holds both halves; tests/artifact-helpers.sh
covers the round trip and runs with the other self-tests.
A PR whose diff against its base is empty has already landed some other
way, typically a sync PR carrying the same bump or a merge from master
that swallowed it. Merging it records a change that isn't one and could
mask a real mistake. result now fails with a message saying to close it.
Every pull request now builds the package directories it touches on
ephemeral DigitalOcean droplets, and every merge to master publishes the
resulting artifacts into the channels each package belongs to. The
repository host's timers become the fallback rather than the pipeline.
Build (.github/workflows/build-pr.yml)
One job per package per architecture, always against edge. The artifact
is labelled with the package directory's git tree hash. Tooling (bin/,
helpers/, build/) is checked out from the base branch; the PR supplies
only pkgbuilds/, so a PR can change what is built, never how. Builds
run only for trusted authors: collaborators, .github/VOUCHED.td, or a
PR carrying the build-approved label. A single required check, result,
aggregates the matrix.
Publish (.github/workflows/publish.yml, bin/publish-artifact)
One job per merge. It collects the PR artifacts for the merged tree,
builds anything that has none, then walks each channel/architecture
slot once: pull that database, repo-add every package that belongs in
it, upload packages, signatures, then the database. A published
filename is immutable; identical bytes under an existing name only
gain a database entry, different bytes are refused. Fast-ring packages
reach edge, rc and stable in the same run from the same file.
Matrix (bin/build-matrix)
Package x architecture, with the channels the artifact ships to,
decided by package_builds_for_mirror so CI and the host agree.
arch=any packages build once and land in every architecture database.
Builder (build/build.sh, bin/build, build/Dockerfile)
With no local published tree, plan against and resolve from the public
channel over HTTPS. PACKAGER is set to Omarchy. rclone is in the image.
Runners (ci/)
A controller droplet polls GitHub with curl and creates one g5 droplet
per queued job from cloud-init, deleting them when off or over-age.
Builders carry QEMU with credential support for aarch64. Operator SSH
keys come from GitHub. tests/controller.sh and tests/publish-artifact.sh
cover the decisions against fixtures and real makepkg output.
Tests run on pull requests only; branch protection requires result,
self-tests and build-isolation with up-to-date branches.
Emir Beganovic