Both upstreams set fat LTO in their release profile (strata also
codegen-units = 1), which compiles the final crate on one thread. That
is a reasonable trade for the binary users run, and build() keeps it.
check() then compiled the same crate a second time under the same
profile for the test harness, and threw it away. On the aarch64 builds,
which run under QEMU, those two serial compiles were 106 of strata's
118 minutes.
check() now builds the harness with thin LTO and 16 codegen units in
its own target directory. The shipped binary is unchanged. Measured on
the x86_64 builder for strata's test binary: 708 s serial, 223 s with
this profile. Local run: shipped strata still 7m30s from target/, tests
3m43s from target-check/, 1998 passed.
ttfx has the same upstream profile but no check(), so nothing to do.
flea 0.3.0 never published: its shelfundo suite (new in 0.3.0) failed check() on the 2026-09-18 publish build, and 0.3.1 fails the same way on the sync PR. undo_refuses_to_walk_a_stranger_back deletes a file, creates another under the same name, and expects undo to tell them apart by (dev, ino, kind). ext4 hands the freed inode number straight back to the next create, so on the droplet's /tmp the stranger is identical and undo walks it back into a directory that does not exist. tmpfs allocates inode numbers from a counter and never reuses one. The suite joins the filesystem tests that already run with TMPDIR on /dev/shm.
On aarch64 three more tests fail: two in backend::child and one in menu_registry expect spawning a missing program to be reported as not started. Under QEMU user-mode emulation glibc's posix_spawn cannot observe the child's failed execve; the spawn succeeds with exit 127. Skipped on aarch64 only.
0.19.0 adds two restart_waiter tests that run `sh -c 'while kill -0 "$1"; do sleep; done'` with the pid u32::MAX. Arch's sh is bash 5.3, whose kill builtin in POSIX mode prints "not a pid or valid job spec" for that number but exits 0, so the loop never ends; every droplet build of 0.19.0 sat there until the 180-minute job timeout. Upstream CI runs on Ubuntu, where sh is dash and rejects the number.
It also adds ownership_probe_errors_disable_in_place_updates, which points the pacman path at a directory and expects Command::output() to fail. aarch64 builds run under QEMU user-mode emulation, where glibc's posix_spawn cannot observe the child's failed execve; the spawn succeeds with exit 127 and the assertion fails. Skipped on aarch64 only; it passes natively.
Both skips carry comments; the sync bot rewrites only version fields, so they survive future syncs.
Every push to a PR rebuilt every package the PR touches, on every
architecture, even when only one of them changed. The daily sync PR
carries around thirty package/arch pairs; fixing one package meant
rebuilding all of them, and an aarch64 build under QEMU takes up to an
hour. Nine runs of that PR cost about 36 droplet-hours in two days.
The planner now asks the artifact store for <pkg>-<arch>-<tree hash>
before adding an entry to the matrix and drops entries that already
have one. That is the same lookup publish.yml makes on merge, so a
reused entry publishes exactly the file it would have anyway. Dry run
against the current sync PR: 27 of 31 entries reused, 4 built.
Pack and Upload no longer run with always(): only a successful build
uploads, so an artifact's existence means that tree built.
workflow_dispatch always builds; it is an explicit request.
Upstream moved to omacom/omasnap and cut 1.21.0 from the same line as 1.20.1, so the version simply increases and no epoch is needed.
check() flushes dirty pages before the smoke suite. The suite fsyncs its working documents under /tmp; on the CI droplets the build leaves about a gigabyte of dirty pages, and the flush that starts a few seconds into the suite made those fsyncs stall past the suite's 5-second settle windows on the first run after every build.
`restore=$(shopt -p nullglob)` exits 1 when nullglob is off, which it is
in the workflow shell. Under the `bash -e` that GitHub runs steps with,
that ended "Pack artifact" before tar ran: every job on #512 and #550
built fine and then failed with nothing in the log but the command.
The self-test never saw it because `pack_packages ... || fail` suppresses
errexit. Enumerate package files with a loop instead of toggling shell
options, and add a test that calls both helpers under `bash -e` exactly
as the workflows do; it fails against the old helper.
github.event.pull_request.base.sha is a snapshot taken when the PR was
last pushed, not the current tip of the base branch. A reopened or rerun
PR therefore builds with whatever master looked like at its last push,
and a tooling fix that landed on master since then never reaches it:
the daily sync PR reopened after #553 merged checked out a base without
helpers/artifact-helpers.sh and failed at "Pack artifact".
Check out base.ref instead. The plan's diff and the empty-PR check still
compare base.sha to head.sha, so the list of changed packages is
unaffected; only the tooling that runs on the droplet moves to the tip.
actions/upload-artifact rejects any path containing ':', and makepkg names
a package with an epoch `name-1:ver-rel-arch.pkg.tar.zst`. Every PR that
built such a package (cursor-cli in the sync PRs, omasnap once it gained an
epoch) failed at "Upload artifact" after a successful build, and publish
then rebuilt from scratch on merge.
The files now ride inside packages.tar for the artifact hop and come back
out with makepkg's names untouched: pacman clients and bin/publish-artifact
both require the filename to match PKGINFO, and the channels already carry
these names. publish.yml still accepts bare pre-packing artifacts until the
7-day retention drains them.
helpers/artifact-helpers.sh holds both halves; tests/artifact-helpers.sh
covers the round trip and runs with the other self-tests.
The package was added pointing at v0.1.0 with a placeholder checksum,
but that tag was never cut; upstream went straight to v1.0.0. The
package has never built anywhere. Point at the real tag and fill the
digest. The upstream watch keeps it current from here.
A workflow_dispatch runs from master's head. That commit's PR merged
something unrelated, so looking the PR up by commit attached a failed
elsewhen report to #515, whose merge had nothing to do with elsewhen.
Dispatch runs now go to the log only. The log append also moves ahead
of the PR comment so the record exists by the time anyone follows the
comment to it.
multiarch/qemu-user-static stopped at QEMU 7.2 (January 2023). Under it,
qmake's compiler probe (`g++ -E -v` in toolchain.prf) returns nothing on the
current gcc 16 toolchain, so every qmake package fails on aarch64 with
"failed to parse default include paths from compiler output" (hype, PR #517).
The same PKGBUILD builds under QEMU 10.2.3 and 11.1.
Register through tonistiigi/binfmt at a pinned tag, on both the ephemeral
builder droplets and the rootful-Docker path of setup_qemu, uninstalling any
existing entry first because the tool keeps an older registration in place.
The tag is now the one place that decides what every emulated build runs
under. Flags stay F and C, which rootless sudo inside the builder needs.
Re-running publish for a package that is already live (a dispatch for
something that turned out fine, or a retry after a partial failure) made
bin/build report nothing to build and exit 2, which the publish step
treated as an error. The collect step now dry-runs first: if the channel
already holds master's version the package is recorded as
already-published and skipped, and a run where every package is in that
state exits cleanly with a record saying so.
When a package had no PR artifact and its build failed, the publish
step never ran, no record was written, and the report job failed
looking for it. The collect step now records each package's source
(PR artifact, built here, or build-failed) and writes the record itself
when a build fails, so the report can say plainly that nothing was
published and why.
A PR whose diff against its base is empty has already landed some other
way, typically a sync PR carrying the same bump or a merge from master
that swallowed it. Merging it records a change that isn't one and could
mask a real mistake. result now fails with a message saying to close it.
The publish job now writes publish-record.json describing every
channel/architecture slot it touched: the packages, whether the slot was
published or failed, the target (live or a proof prefix), the commit and
the run. A report job renders that as a comment on the PR the merge
commit came from (looked up by commit, so squash and rebase merges work)
and appends the record as one line to publish-log.jsonl in the bucket,
served next to the packages at https://pkgs.omarchy.org/publish-log.jsonl.
Failures are reported too, with the slots that landed before the failure,
which is when a human most needs to know.
Every pull request now builds the package directories it touches on
ephemeral DigitalOcean droplets, and every merge to master publishes the
resulting artifacts into the channels each package belongs to. The
repository host's timers become the fallback rather than the pipeline.
Build (.github/workflows/build-pr.yml)
One job per package per architecture, always against edge. The artifact
is labelled with the package directory's git tree hash. Tooling (bin/,
helpers/, build/) is checked out from the base branch; the PR supplies
only pkgbuilds/, so a PR can change what is built, never how. Builds
run only for trusted authors: collaborators, .github/VOUCHED.td, or a
PR carrying the build-approved label. A single required check, result,
aggregates the matrix.
Publish (.github/workflows/publish.yml, bin/publish-artifact)
One job per merge. It collects the PR artifacts for the merged tree,
builds anything that has none, then walks each channel/architecture
slot once: pull that database, repo-add every package that belongs in
it, upload packages, signatures, then the database. A published
filename is immutable; identical bytes under an existing name only
gain a database entry, different bytes are refused. Fast-ring packages
reach edge, rc and stable in the same run from the same file.
Matrix (bin/build-matrix)
Package x architecture, with the channels the artifact ships to,
decided by package_builds_for_mirror so CI and the host agree.
arch=any packages build once and land in every architecture database.
Builder (build/build.sh, bin/build, build/Dockerfile)
With no local published tree, plan against and resolve from the public
channel over HTTPS. PACKAGER is set to Omarchy. rclone is in the image.
Runners (ci/)
A controller droplet polls GitHub with curl and creates one g5 droplet
per queued job from cloud-init, deleting them when off or over-age.
Builders carry QEMU with credential support for aarch64. Operator SSH
keys come from GitHub. tests/controller.sh and tests/publish-artifact.sh
cover the decisions against fixtures and real makepkg output.
Tests run on pull requests only; branch protection requires result,
self-tests and build-isolation with up-to-date branches.
Ryan Hughes