Harden multi-architecture release pipeline

This commit is contained in:
Ryan Hughes committed 2026-09-04 23:40:49 -04:00
1 parent b677f50358
commit 76687fcc82
23 files changed
+557 -271

No files matched your search

+1
View File
@@ -27,6 +27,7 @@ jobs:
set -euo pipefail set -euo pipefail
pacman -Syu --noconfirm jq pacman -Syu --noconfirm jq
./bin/sync-upstream self-test ./bin/sync-upstream self-test
./bin/sync-rebuilds --self-test
./bin/omarchy-pkgs self-test ./bin/omarchy-pkgs self-test
./bin/omarchy-release self-test ./bin/omarchy-release self-test
' '
+46 -20
View File
@@ -57,14 +57,25 @@ file per channel and architecture (`.sync-needed-<channel>-<arch>`);
`auto-release <channel>` works through the queues one architecture at a `auto-release <channel>` works through the queues one architecture at a
time, each with its own backoff (`.build-failed-<channel>-<arch>`), so a time, each with its own backoff (`.build-failed-<channel>-<arch>`), so a
failing build on one architecture never holds up the other; and the release failing build on one architecture never holds up the other; and the release
train advances channels with `--arch all`, so a channel never moves for one train advances channels with `--arch all`: it takes one host-wide lock and
architecture and not another. The first entry is the reference architecture verifies every architecture's source database before moving any of them. The
the release train observes channels through. first entry is the reference architecture the release train observes channels
through. A remote sync failure can still leave a promotion temporarily partial;
rerunning the same advance completes it safely.
Adding an architecture is therefore one change to that list (or Adding an architecture to the scheduled pipeline is therefore one checked-in
`OMARCHY_ARCHES` in the host's build credentials): the next `check-versions` change to that list: the next `check-versions` tick queues everything the new
tick queues everything the new architecture lacks, and the next architecture lacks, and the next `auto-release` tick starts building it. A
`auto-release` tick starts building it. The builder image bootstraps checked-in list also means the rebuild workflow and release host cannot drift
onto different architecture sets. For a one-off run, override it directly:
```bash
OMARCHY_ARCHES=x86_64 bin/check-versions
OMARCHY_ARCHES=aarch64 bin/check-versions
OMARCHY_ARCHES="x86_64 aarch64" bin/check-versions
```
The builder image bootstraps
`omarchy-keyring` from the x86_64 tree for every architecture, so the first `omarchy-keyring` from the x86_64 tree for every architecture, so the first
build of a new architecture does not depend on a repository that only it can build of a new architecture does not depend on a repository that only it can
create. create.
@@ -450,7 +461,12 @@ A package names those dependencies in `.omarchy/package.json`:
{ "source": "aur", "sync": false, "rebuild_on": ["qt6-base", "qt6-declarative", "qt6-wayland"] } { "source": "aur", "sync": false, "rebuild_on": ["qt6-base", "qt6-declarative", "qt6-wayland"] }
``` ```
`bin/sync-rebuilds` reads each named package's version from the official repositories and compares it to `rebuilt_against`, the record of what the checked-in pkgrel was last bumped for. pkgrel is bumped unless every name in `rebuild_on` is recorded and still matches, so a name the record does not carry reads as changed rather than going unexamined forever. Opting a package in therefore buys one rebuild: what its published build actually linked against is not knowable from here, and a record written without a rebuild would certify a build nobody checked. `bin/sync-rebuilds` reads each named package's version from the official
repositories for every published architecture the package supports and compares
it to `rebuilt_against`. Records are kept per architecture because Arch and
Arch Linux ARM can carry different dependency versions. pkgrel is bumped once
when any recorded version moves; that one source revision is then rebuilt by
each architecture's normal queue.
The bump is the point of the command, and it has to land in git rather than in the builder. A rebuild that reuses the published version string produces a package pacman will never offer anyone, so merely unlocking the build gate would ship nothing. Bumping pkgrel needs no other change: `bin/check-versions` and the builder both already rebuild when pkgrel moves. The bump is the point of the command, and it has to land in git rather than in the builder. A rebuild that reuses the published version string produces a package pacman will never offer anyone, so merely unlocking the build gate would ship nothing. Bumping pkgrel needs no other change: `bin/check-versions` and the builder both already rebuild when pkgrel moves.
@@ -458,9 +474,12 @@ For an AUR-synced package the bump is expressed as the dotted Omarchy pkgrel suf
The bumped version is checked against the published one as well as the checked-in one, and refused when pacman would not order it higher. The checked-in version is not the floor; what a user already has is, and a checkout that has fallen behind the repository can otherwise be bumped to something that loses to the package it means to replace. That check is skipped with a warning when the published database cannot be read. The bumped version is checked against the published one as well as the checked-in one, and refused when pacman would not order it higher. The checked-in version is not the floor; what a user already has is, and a checkout that has fallen behind the repository can otherwise be bumped to something that loses to the package it means to replace. That check is skipped with a warning when the published database cannot be read.
Versions are read from the local pacman database, so this runs on Arch or in an Arch container against a synced database. Only `core`, `extra` and `multilib` count: a Qt release sitting in testing or kde-unstable is not what the builder will link against, and rebuilding for it would ship a package built against the wrong ABI. The workflow points that database at `mirror.omarchy.org`, the mirror the x86_64 builder itself uses, because a mirror running ahead of the builder would record a version the build never linked against and nothing re-fires once the record matches. x86_64 versions are read from the local pacman database, so the workflow runs
in an Arch container pointed at `mirror.omarchy.org`, the same mirror as the
aarch64 is not covered. Those builds resolve Qt from Arch Linux ARM, which can lag Arch, so one record cannot describe both architectures. Only x86_64 is published today, so nothing currently ships from the untracked side; if ARM publishing starts, `rebuilt_against` has to become per-architecture before this can be trusted there. x86_64 builder. aarch64 versions are read directly from the live Arch Linux ARM
repository database, which is also what the ARM builder uses. Testing and
staging repositories do not count. A legacy flat `rebuilt_against` record is
read as x86_64 and is migrated naturally the next time a rebuild is needed.
### Other ### Other
@@ -668,7 +687,7 @@ Fields:
- `skip_build`: optional boolean; defaults to `false`. Set `true` to exclude a package from scheduled version checks and unscoped builds. The package can still be built explicitly with `bin/repo release --package <name>`. - `skip_build`: optional boolean; defaults to `false`. Set `true` to exclude a package from scheduled version checks and unscoped builds. The package can still be built explicitly with `bin/repo release --package <name>`.
- `pkgrel`: optional Omarchy pkgrel suffix for a version-pinned rebuild bump. This emits `<aur pkgrel>.<suffix>` instead of replacing AUR's pkgrel. `offset` can be used only when preserving monotonic upgrades from old absolute pkgrel bumps. The metadata is removed automatically when AUR sync changes `pkgver`; the current package version is read from the checked-in PKGBUILD, so the version is not duplicated in JSON. - `pkgrel`: optional Omarchy pkgrel suffix for a version-pinned rebuild bump. This emits `<aur pkgrel>.<suffix>` instead of replacing AUR's pkgrel. `offset` can be used only when preserving monotonic upgrades from old absolute pkgrel bumps. The metadata is removed automatically when AUR sync changes `pkgver`; the current package version is read from the checked-in PKGBUILD, so the version is not duplicated in JSON.
- `rebuild_on`: optional array of package names this package links against closely enough that it must be rebuilt when they change, independent of its own source. Read by `bin/sync-rebuilds`. - `rebuild_on`: optional array of package names this package links against closely enough that it must be rebuilt when they change, independent of its own source. Read by `bin/sync-rebuilds`.
- `rebuilt_against`: written by `bin/sync-rebuilds`. Records the version of each `rebuild_on` package that the current pkgrel was bumped for. - `rebuilt_against`: written by `bin/sync-rebuilds`. Maps each published architecture to the versions of its `rebuild_on` packages that the current pkgrel was bumped for.
- `upstream_commit`: set by `bin/sync-aur` for AUR packages. Used by `bin/package-worktree` to recreate the exact raw AUR package that Omarchy last synced. - `upstream_commit`: set by `bin/sync-aur` for AUR packages. Used by `bin/package-worktree` to recreate the exact raw AUR package that Omarchy last synced.
### Build Matrix ### Build Matrix
@@ -764,8 +783,10 @@ bin/repo release --package my-package
### aarch64 ### aarch64
- Built on the repository host like x86_64; under QEMU when the host is x86_64 - Built on the repository host like x86_64; under QEMU when the host is x86_64
- Uses Arch Linux ARM repositories (one mirrorlist for every channel — Arch - On an ARM host, package builds and the signing/database utility containers
Linux ARM publishes no dated snapshots to pin a channel's base to) run natively; only an explicitly requested x86_64 package build is emulated
- Uses Arch Linux ARM repositories through the same HTTPS mirror for every
channel (Arch Linux ARM publishes no dated snapshots to pin a channel's base)
- Additional repos: `[alarm]`, `[aur]` - Additional repos: `[alarm]`, `[aur]`
- Same workflow, just add `--arch aarch64`; the scheduled pipeline runs it - Same workflow, just add `--arch aarch64`; the scheduled pipeline runs it
automatically once `aarch64` is in `PUBLISHED_ARCHES` automatically once `aarch64` is in `PUBLISHED_ARCHES`
@@ -835,10 +856,11 @@ That cadence is only safe because of three guards:
an operator expects. `check-versions` takes it too — its `git pull` would an operator expects. `check-versions` takes it too — its `git pull` would
otherwise swap PKGBUILDs out from under a running build. otherwise swap PKGBUILDs out from under a running build.
- **Backoff on failure.** A failed release records the attempt in - **Backoff on failure.** A failed release records the attempt in
`.build-failed-<channel>` and backs off exponentially — 10m, 20m, 40m, up to `.build-failed-<channel>-<arch>` and backs off exponentially — 10m, 20m, 40m, up to
a 6h ceiling — instead of rebuilding the same broken tree every 5 minutes. a 6h ceiling — instead of rebuilding the same broken tree every 5 minutes.
**Any new commit clears the backoff immediately**, since a push is the most **Any new commit clears the backoff immediately**, since a push is the most
likely fix. Clear it by hand with `rm /root/.state/.build-failed-<channel>`. likely fix. Clear it by hand with
`rm /root/.state/.build-failed-<channel>-<arch>`.
- **Quiet when idle.** With nothing queued a tick exits without output, so the - **Quiet when idle.** With nothing queued a tick exits without output, so the
journal shows the runs that mattered rather than 288 no-ops a day. journal shows the runs that mattered rather than 288 no-ops a day.
@@ -891,10 +913,14 @@ bin/repo timers --local # inspect this machine instead
``` ```
State files are stored in `/root/.state/`: State files are stored in `/root/.state/`:
- `.sync-needed-<channel>` — the packages queued for that channel, one per - `.sync-needed-<channel>-<arch>` — the packages queued for that channel and
line; the release run reads them to name what it is building architecture, one per line; the release run reads them to name what it is
- `.build-failed-<channel>` — consecutive failure count, timestamp, and the building
commit it failed on (drives the backoff; removing it forces a retry) - `.build-failed-<channel>-<arch>` — consecutive failure count, timestamp, and
the commit it failed on (drives the backoff; removing it forces a retry)
Legacy files without the architecture suffix are consumed once as x86_64
state, so upgrading the host does not lose an in-flight build.
### Schedule (America/New_York) ### Schedule (America/New_York)
+40 -33
View File
@@ -25,31 +25,21 @@ SKIP_PROD_CHECK=false
FAST_RING_ONLY=false FAST_RING_ONLY=false
BOOTSTRAP=false BOOTSTRAP=false
PACKAGES="" PACKAGES=""
ALL_ARCHES=false
# Kept for --arch all, which re-invokes this script per architecture with the # Kept for --arch all, which re-invokes this script per architecture with the
# other arguments unchanged (minus the --arch all pair itself). # other arguments unchanged (minus the --arch all pair itself).
ORIGINAL_ARGS=() ORIGINAL_ARGS=()
skip_next=false arch_option=false
prev=""
for arg in "$@"; do for arg in "$@"; do
if [[ "$skip_next" == true ]]; then if [[ "$arch_option" == true ]]; then
skip_next=false [[ "$arg" != "all" ]] && ORIGINAL_ARGS+=("--arch" "$arg")
prev="" arch_option=false
continue elif [[ "$arg" == "--arch" ]]; then
arch_option=true
else
ORIGINAL_ARGS+=("$arg")
fi fi
if [[ "$arg" == "--arch" ]]; then
prev="--arch"
continue
fi
if [[ "$prev" == "--arch" ]]; then
prev=""
if [[ "$arg" == "all" ]]; then
continue
fi
ORIGINAL_ARGS+=("--arch" "$arg")
continue
fi
ORIGINAL_ARGS+=("$arg")
done done
usage() { usage() {
@@ -93,21 +83,13 @@ while [[ $# -gt 0 ]]; do
;; ;;
--arch) --arch)
if [[ "$2" == "all" ]]; then if [[ "$2" == "all" ]]; then
# A release train advances every published architecture together; a ALL_ARCHES=true
# channel that moved for one and not another would serve mismatched ARCH=all
# versions to users on the two. Re-run this script once per else
# architecture with the same arguments, stopping at the first failure require_valid_arch "$2"
# so the operator sees which architecture is stuck (the advance is ARCH="$2"
# idempotent, so re-running with --arch all resumes where it stopped). update_arch_paths
shift 2
for arch in $(published_arches); do
"$0" --arch "$arch" "${ORIGINAL_ARGS[@]}" || exit $?
done
exit 0
fi fi
require_valid_arch "$2"
ARCH="$2"
update_arch_paths
shift 2 shift 2
;; ;;
--package) --package)
@@ -175,6 +157,31 @@ case "$FROM->$TO" in
;; ;;
esac esac
if [[ "$ALL_ARCHES" == true ]]; then
# Hold one lock across the whole operation so a timer cannot mutate a
# channel between architectures. Check every source database before moving
# the first one; a failed sync can still require an idempotent retry, but a
# missing architecture never creates a knowingly partial advance.
if [[ "$DRY_RUN" != true ]]; then
acquire_release_lock || exit 1
fi
ARCHES=$(published_arches)
for arch in $ARCHES; do
source_db="$REPO_ROOT/$FROM/$arch/omarchy.db.tar.zst"
if [[ ! -f "$source_db" ]]; then
print_error "Source database not found: $source_db"
echo "Nothing has been published to the $FROM channel for $arch."
exit 1
fi
done
for arch in $ARCHES; do
"$0" --arch "$arch" "${ORIGINAL_ARGS[@]}" || exit $?
done
exit 0
fi
SOURCE_DIR="$REPO_ROOT/$FROM/$ARCH" SOURCE_DIR="$REPO_ROOT/$FROM/$ARCH"
TARGET_DIR="$REPO_ROOT/$TO/$ARCH" TARGET_DIR="$REPO_ROOT/$TO/$ARCH"
SOURCE_DB="$SOURCE_DIR/omarchy.db.tar.zst" SOURCE_DB="$SOURCE_DIR/omarchy.db.tar.zst"
+13 -2
View File
@@ -131,17 +131,28 @@ release_arch() {
if "$BUILD_ROOT/bin/repo" release --mirror "$MIRROR" --arch "$arch" --skip-prod-check; then if "$BUILD_ROOT/bin/repo" release --mirror "$MIRROR" --arch "$arch" --skip-prod-check; then
print_success "Release completed successfully for $MIRROR ($arch)" print_success "Release completed successfully for $MIRROR ($arch)"
rm -f "$state_file" "$fail_file" local completed_state=("$state_file" "$fail_file")
if [[ "$arch" == "x86_64" ]]; then
completed_state+=("$(legacy_sync_queue_file "$MIRROR")" "$(legacy_sync_fail_file "$MIRROR")")
fi
if ! rm -f "${completed_state[@]}"; then
print_error "Release succeeded, but its queue state could not be cleared"
return 1
fi
print_success "State file removed: $state_file" print_success "State file removed: $state_file"
return 0 return 0
fi fi
fail_count=$((fail_count + 1)) fail_count=$((fail_count + 1))
cat >"$fail_file" <<EOF if ! cat >"$fail_file" <<EOF
FAILURE_COUNT=$fail_count FAILURE_COUNT=$fail_count
FAILURE_AT=$(date +%s) FAILURE_AT=$(date +%s)
FAILURE_FINGERPRINT=$(current_fingerprint) FAILURE_FINGERPRINT=$(current_fingerprint)
EOF EOF
then
print_error "Could not record failure state: $fail_file"
return 1
fi
local next local next
next=$(backoff_seconds "$fail_count") next=$(backoff_seconds "$fail_count")
print_error "Release failed for $MIRROR ($arch) (attempt $fail_count)" print_error "Release failed for $MIRROR ($arch) (attempt $fail_count)"
+5 -4
View File
@@ -102,6 +102,8 @@ while [[ $# -gt 0 ]]; do
esac esac
done done
require_valid_arch "$ARCH"
if [[ $DEFER_RUNTIME_DEPS != "false" && $DEFER_RUNTIME_DEPS != "true" ]]; then if [[ $DEFER_RUNTIME_DEPS != "false" && $DEFER_RUNTIME_DEPS != "true" ]]; then
print_error "OMARCHY_DEFER_RUNTIME_DEPS must be true or false" print_error "OMARCHY_DEFER_RUNTIME_DEPS must be true or false"
exit 1 exit 1
@@ -170,8 +172,7 @@ check_docker
HOST_ARCH=$(uname -m) HOST_ARCH=$(uname -m)
[[ "$HOST_ARCH" == "arm64" ]] && HOST_ARCH=aarch64 [[ "$HOST_ARCH" == "arm64" ]] && HOST_ARCH=aarch64
if [[ "$HOST_ARCH" != "$ARCH" ]]; then if [[ "$HOST_ARCH" != "$ARCH" ]]; then
# shellcheck disable=SC2086 if ! docker run --rm "$(get_platform_arg "$ARCH")" alpine:3.21 /bin/true >/dev/null 2>&1; then
if ! docker run --rm $(get_platform_arg "$ARCH") alpine:3.21 /bin/true >/dev/null 2>&1; then
print_info "Setting up QEMU for $ARCH emulation on this $HOST_ARCH host..." print_info "Setting up QEMU for $ARCH emulation on this $HOST_ARCH host..."
setup_qemu setup_qemu
fi fi
@@ -183,7 +184,7 @@ if [[ $KEEP_BUILD_WORKSPACE == "1" ]]; then
print_info "Keeping existing build workspace..." print_info "Keeping existing build workspace..."
else else
print_info "Cleaning build workspace..." print_info "Cleaning build workspace..."
rm -rf "$BUILD_OUTPUT_DIR"/* rm -rf "${BUILD_OUTPUT_DIR:?}"/*
fi fi
mkdir -p "$BUILD_OUTPUT_DIR" mkdir -p "$BUILD_OUTPUT_DIR"
@@ -237,7 +238,7 @@ DOCKER_ARGS=(
# Run the builder with assembled args # Run the builder with assembled args
PLATFORM_ARG=$(get_platform_arg "$ARCH") PLATFORM_ARG=$(get_platform_arg "$ARCH")
docker run $PLATFORM_ARG "${DOCKER_ARGS[@]}" "$IMAGE_TAG" /build/build.sh docker run "$PLATFORM_ARG" "${DOCKER_ARGS[@]}" "$IMAGE_TAG" /build/build.sh
BUILD_RESULT=$? BUILD_RESULT=$?
+1 -1
View File
@@ -180,7 +180,7 @@ check_mirror() {
needs_build=true needs_build=true
packages+=("$pkg") packages+=("$pkg")
fi fi
done < <(packages_for_unscoped_build "$mirror") done < <(packages_for_unscoped_build "$mirror" "$ARCH")
echo "" echo ""
+7 -6
View File
@@ -31,12 +31,13 @@ clean_packages() {
# Skip signature files # Skip signature files
[[ "$pkg" == *.sig ]] && continue [[ "$pkg" == *.sig ]] && continue
# Extract package name (remove version and architecture) # Format: name-version-release-arch.pkg.tar.*. Work from the right because
# Format: name-version-release-arch.pkg.tar.* # package names can themselves contain version-like pieces (qt6-5compat,
# The version starts at the first "-<digit>" and neither pkgver nor pkgrel # nvidia-580xx-utils), while pkgver and pkgrel cannot contain hyphens.
# can contain a hyphen, so the architecture is whatever sits between the local stem="${pkg%%.pkg.tar.*}"
# last hyphen and .pkg.tar — any, x86_64, aarch64, and whatever comes next. stem="${stem%-*}" # architecture
local pkgname=$(echo "$pkg" | sed -E 's/-[0-9]+.*-[^-]+\.pkg\.tar\..*//') stem="${stem%-*}" # pkgrel
local pkgname="${stem%-*}" # pkgver
# Add to array # Add to array
if [[ -n "${packages[$pkgname]}" ]]; then if [[ -n "${packages[$pkgname]}" ]]; then
+12 -6
View File
@@ -23,7 +23,7 @@ source "$BUILD_ROOT/helpers/paths.sh"
source "$BUILD_ROOT/helpers/host-helpers.sh" source "$BUILD_ROOT/helpers/host-helpers.sh"
UPSTREAM_URL="${OMARCHY_UPSTREAM_URL:-https://github.com/basecamp/omarchy.git}" UPSTREAM_URL="${OMARCHY_UPSTREAM_URL:-https://github.com/basecamp/omarchy.git}"
EDGE_DB_URL="${OMARCHY_EDGE_DB_URL:-https://pkgs.omarchy.org/edge/x86_64/omarchy.db.tar.zst}" EDGE_DB_URL="${OMARCHY_EDGE_DB_URL:-https://pkgs.omarchy.org/edge/$(reference_arch)/omarchy.db.tar.zst}"
RELEASE_PACKAGES=(omarchy omarchy-settings) RELEASE_PACKAGES=(omarchy omarchy-settings)
DEFAULT_RC_REF="quattro" DEFAULT_RC_REF="quattro"
SRCDEST_DIR="$BUILD_ROOT/.srcdest" SRCDEST_DIR="$BUILD_ROOT/.srcdest"
@@ -327,13 +327,19 @@ regenerate_checksums() {
trigger_build_host() { trigger_build_host() {
local host local host
queue_edge_builds() {
mkdir -p "$STATE_DIR" || return 1
local arch
for arch in $(published_arches); do
touch "$(sync_queue_file edge "$arch")" || return 1
done
}
# Explicit host configuration outranks the local-host inference (a # Explicit host configuration outranks the local-host inference (a
# workstation that ran a full local release carries the db marker too). # workstation that ran a full local release carries the db marker too).
if ! resolve_repo_host "${REPO_HOST_OVERRIDE:-}" >/dev/null && on_repo_host; then if ! resolve_repo_host "${REPO_HOST_OVERRIDE:-}" >/dev/null && on_repo_host; then
print_info "Triggering edge build locally (this is the build host)..." print_info "Triggering edge build locally (this is the build host)..."
if mkdir -p "${OMARCHY_STATE_DIR:-/root/.state}" && if queue_edge_builds && systemctl start --no-block omarchy-auto-release-edge.service; then
touch "${OMARCHY_STATE_DIR:-/root/.state}/.sync-needed-edge" &&
systemctl start --no-block omarchy-auto-release-edge.service; then
print_success "Edge build triggered" print_success "Edge build triggered"
else else
print_warning "Could not start the edge release service — the 6-hourly timer will pick it up" print_warning "Could not start the edge release service — the 6-hourly timer will pick it up"
@@ -343,11 +349,11 @@ trigger_build_host() {
if ! host=$(resolve_repo_host "${REPO_HOST_OVERRIDE:-}"); then if ! host=$(resolve_repo_host "${REPO_HOST_OVERRIDE:-}"); then
print_info "No repository host configured (set OMARCHY_REPO_HOST or $BUILD_ROOT/.repo-host — any ssh destination, e.g. root@<host> or an ssh-config alias)." print_info "No repository host configured (set OMARCHY_REPO_HOST or $BUILD_ROOT/.repo-host — any ssh destination, e.g. root@<host> or an ssh-config alias)."
print_info "The 6-hourly auto-release timer will pick this up, or trigger manually:" print_info "The 6-hourly auto-release timer will pick this up, or trigger manually:"
echo " ssh <build-host> 'git -C /root/omarchy-pkgs pull --ff-only && touch /root/.state/.sync-needed-edge && systemctl start omarchy-auto-release-edge.service'" echo " ssh <build-host> 'git -C /root/omarchy-pkgs pull --ff-only && systemctl start omarchy-check-versions.service omarchy-auto-release-edge.service'"
return 0 return 0
fi fi
print_info "Triggering edge build on $host..." print_info "Triggering edge build on $host..."
if ssh "$host" 'git -C /root/omarchy-pkgs pull --ff-only && mkdir -p /root/.state && touch /root/.state/.sync-needed-edge && systemctl start --no-block omarchy-auto-release-edge.service'; then if ssh "$host" 'git -C /root/omarchy-pkgs pull --ff-only && cd /root/omarchy-pkgs && export BUILD_ROOT=/root/omarchy-pkgs && source helpers/paths.sh && mkdir -p "$STATE_DIR" && for arch in $(published_arches); do touch "$(sync_queue_file edge "$arch")"; done && systemctl start --no-block omarchy-auto-release-edge.service'; then
print_success "Edge build triggered on $host" print_success "Edge build triggered on $host"
else else
print_warning "Could not trigger $host — the 6-hourly timer will pick it up" print_warning "Could not trigger $host — the 6-hourly timer will pick it up"
+75 -50
View File
@@ -28,11 +28,9 @@ ISO_REPO="${OMARCHY_ISO_REPO:-omacom-io/omarchy-iso}"
DEV_BRANCH="${OMARCHY_DEV_BRANCH:-quattro}" DEV_BRANCH="${OMARCHY_DEV_BRANCH:-quattro}"
PKGS_DB_BASE="${OMARCHY_PKGS_DB_BASE:-https://pkgs.omarchy.org}" PKGS_DB_BASE="${OMARCHY_PKGS_DB_BASE:-https://pkgs.omarchy.org}"
# The train observes channels through the reference architecture (the first # The first published architecture supplies the version-ordering floor when
# published one). The pair is pinned to one version for every architecture and # cutting pins. Readiness checks below still verify every published
# built for all of them in one rc trigger, so one architecture's database is # architecture before an RC or final release can move forward.
# the state of the release; per-architecture drift is a build failure the
# scheduled pipeline reports, not something the train has to poll for.
OBSERVED_ARCH=$(reference_arch) OBSERVED_ARCH=$(reference_arch)
RC_DB_URL="$PKGS_DB_BASE/rc/$OBSERVED_ARCH/omarchy.db.tar.zst" RC_DB_URL="$PKGS_DB_BASE/rc/$OBSERVED_ARCH/omarchy.db.tar.zst"
@@ -187,12 +185,12 @@ ensure_work_clone() {
# Prints omarchy's published version in a channel; empty when absent, rc 2 when # Prints omarchy's published version in a channel; empty when absent, rc 2 when
# the database cannot be read (callers must not mistake an outage for absence). # the database cannot be read (callers must not mistake an outage for absence).
published_version() { published_version() {
local channel="$1" tmp descs local channel="$1" arch="${2:-$OBSERVED_ARCH}" tmp descs
tmp=$(mktemp) || return 2 tmp=$(mktemp) || return 2
# A unique query string busts the CDN cache: right after a sync the plain # A unique query string busts the CDN cache: right after a sync the plain
# URL can keep serving the previous db for a while, which reads as "not # URL can keep serving the previous db for a while, which reads as "not
# published yet" to status, the wait loop, and ship's pre-checks. # published yet" to status, the wait loop, and ship's pre-checks.
if ! curl -sf "$PKGS_DB_BASE/$channel/$OBSERVED_ARCH/omarchy.db.tar.zst?$(date +%s%N)" -o "$tmp"; then if ! curl -sf "$PKGS_DB_BASE/$channel/$arch/omarchy.db.tar.zst?$(date +%s%N)" -o "$tmp"; then
rm -f "$tmp" rm -f "$tmp"
return 2 return 2
fi fi
@@ -213,6 +211,14 @@ published_version() {
' <<<"$descs" ' <<<"$descs"
} }
all_arches_at_version() { # all_arches_at_version <channel> <pkgver>
local channel="$1" want="$2" arch got
for arch in $(published_arches); do
got=$(published_version "$channel" "$arch" 2>/dev/null) || return 1
[[ "${got%-*}" == "$want" ]] || return 1
done
}
# The rc branch of THIS repo carries the current pins. Read them without # The rc branch of THIS repo carries the current pins. Read them without
# touching the working tree. # touching the working tree.
rc_branch_pin() { # prints "pkgver commit", empty when no rc branch rc_branch_pin() { # prints "pkgver commit", empty when no rc branch
@@ -297,12 +303,16 @@ host_advance() { # host_advance <from> <to> [extra args...]
} }
wait_for_published() { # wait_for_published <channel> <version> [timeout-seconds] wait_for_published() { # wait_for_published <channel> <version> [timeout-seconds]
local channel="$1" want="$2" timeout="${3:-3600}" waited=0 got local channel="$1" want="$2" timeout="${3:-3600}" waited=0 arch got pending
print_info "Waiting for $want to appear in the $channel channel (up to $((timeout / 60))m)..." print_info "Waiting for $want in $channel for: $(published_arches | tr '\n' ' ')"
while ((waited < timeout)); do while ((waited < timeout)); do
got=$(published_version "$channel" 2>/dev/null) || got="" pending=""
if [[ "${got%-*}" == "$want" ]]; then for arch in $(published_arches); do
print_success "$channel now serves omarchy $got" got=$(published_version "$channel" "$arch" 2>/dev/null) || got=""
[[ "${got%-*}" == "$want" ]] || pending+=" $arch=${got:-unreachable}"
done
if [[ -z "$pending" ]]; then
print_success "$channel now serves omarchy $want on every published architecture"
return 0 return 0
fi fi
sleep 60 sleep 60
@@ -310,7 +320,7 @@ wait_for_published() { # wait_for_published <channel> <version> [timeout-seconds
printf '.' >&2 printf '.' >&2
done done
echo "" >&2 echo "" >&2
print_warning "Timed out waiting for $want in $channel (currently: ${got:-unknown})" print_warning "Timed out waiting for $want in $channel (pending:$pending)"
print_info "The build may still be running — re-run this command to resume." print_info "The build may still be running — re-run this command to resume."
return 1 return 1
} }
@@ -674,15 +684,13 @@ cmd_rc() {
if [[ -n "$pin" ]]; then if [[ -n "$pin" ]]; then
local pin_ver="${pin%% *}" pin_commit="${pin##* }" local pin_ver="${pin%% *}" pin_commit="${pin##* }"
if [[ "$pin_commit" == "$head" && "$pin_ver" =~ ^${version//./\\.}rc[0-9]+$ ]]; then if [[ "$pin_commit" == "$head" && "$pin_ver" =~ ^${version//./\\.}rc[0-9]+$ ]]; then
local pub if all_arches_at_version rc "$pin_ver"; then
pub=$(published_version rc 2>/dev/null) || pub="" print_success "$pin_ver is already cut from this head and published to rc on every architecture"
if [[ "${pub%-*}" == "$pin_ver" ]]; then
print_success "$pin_ver is already cut from this head and published to rc"
maybe_iso "$pin_ver" rc "$iso_mode" maybe_iso "$pin_ver" rc "$iso_mode"
return 0 return 0
fi fi
print_info "$pin_ver is pinned from this head but not published yet — re-triggering the build" print_info "$pin_ver is pinned from this head but not published yet — re-triggering the build"
trigger_rc_build || true trigger_rc_build || return 1
[[ "$wait" == true ]] && wait_for_published rc "$pin_ver" [[ "$wait" == true ]] && wait_for_published rc "$pin_ver"
maybe_iso "$pin_ver" rc "$iso_mode" maybe_iso "$pin_ver" rc "$iso_mode"
return 0 return 0
@@ -698,7 +706,7 @@ cmd_rc() {
new_ver="${new_pin%% *}" new_ver="${new_pin%% *}"
print_success "Pinned $new_ver (rc branch pushed)" print_success "Pinned $new_ver (rc branch pushed)"
trigger_rc_build || true trigger_rc_build || return 1
if [[ "$wait" == true ]]; then if [[ "$wait" == true ]]; then
wait_for_published rc "$new_ver" || return 1 wait_for_published rc "$new_ver" || return 1
fi fi
@@ -720,9 +728,7 @@ cmd_ship() {
exit 1 exit 1
fi fi
version=$(branch_to_version "$branch") version=$(branch_to_version "$branch")
local stable_now if all_arches_at_version stable "$version" &&
stable_now=$(published_version stable 2>/dev/null) || stable_now=""
if [[ "${stable_now%-*}" == "$version" ]] &&
gh release view "v$version" --repo "$UPSTREAM_REPO" >/dev/null 2>&1; then gh release view "v$version" --repo "$UPSTREAM_REPO" >/dev/null 2>&1; then
print_success "Nothing to ship — $version is tagged, released, and live on stable" print_success "Nothing to ship — $version is tagged, released, and live on stable"
exit 0 exit 0
@@ -764,10 +770,8 @@ cmd_ship() {
echo " omarchy-release rc" echo " omarchy-release rc"
exit 1 exit 1
fi fi
local pub if ! all_arches_at_version rc "$pin_ver"; then
pub=$(published_version rc 2>/dev/null) || pub="" print_error "$pin_ver is pinned but is not published for every architecture"
if [[ "${pub%-*}" != "$pin_ver" ]]; then
print_error "$pin_ver is pinned but rc serves '${pub:-nothing}' — the candidate build hasn't published"
echo "Wait for it (or re-run: omarchy-release rc), then ship." echo "Wait for it (or re-run: omarchy-release rc), then ship."
exit 1 exit 1
fi fi
@@ -811,10 +815,9 @@ cmd_ship() {
# 2. Final pins into rc. Resolve the tag we just established so the final # 2. Final pins into rc. Resolve the tag we just established so the final
# PKGBUILDs record both its provenance and its exact commit. # PKGBUILDs record both its provenance and its exact commit.
local rc_pub stable_pub local stable_pub
rc_pub=$(published_version rc 2>/dev/null) || rc_pub="" if all_arches_at_version rc "$version"; then
if [[ "${rc_pub%-*}" == "$version" ]]; then print_success "2/7 Final $version already published to rc on every architecture"
print_success "2/7 Final $version already published to rc"
else else
if [[ "$pin_ver" != "$version" ]]; then if [[ "$pin_ver" != "$version" ]]; then
print_info "2/7 Pinning final $version from tag v$version..." print_info "2/7 Pinning final $version from tag v$version..."
@@ -822,22 +825,21 @@ cmd_ship() {
else else
print_info "2/7 Final $version pinned — re-triggering build" print_info "2/7 Final $version pinned — re-triggering build"
fi fi
trigger_rc_build || true trigger_rc_build || exit 1
wait_for_published rc "$version" || exit 1 wait_for_published rc "$version" || exit 1
fi fi
# 3. Promote rc -> stable # 3. Promote rc -> stable
stable_pub=$(published_version stable 2>/dev/null) || stable_pub="" if all_arches_at_version stable "$version"; then
if [[ "${stable_pub%-*}" == "$version" ]]; then print_success "3/7 Stable already serves $version on every architecture"
print_success "3/7 Stable already serves $version"
else else
host_advance rc stable || exit 1 host_advance rc stable || exit 1
stable_pub=$(published_version stable 2>/dev/null) || stable_pub="" if ! all_arches_at_version stable "$version"; then
if [[ "${stable_pub%-*}" != "$version" ]]; then print_error "Promotion ran but stable does not serve $version on every architecture"
print_error "Promotion ran but stable serves '${stable_pub:-nothing}' — investigate before continuing"
exit 1 exit 1
fi fi
print_success "3/7 Promoted to stable: omarchy $stable_pub" stable_pub=$(published_version stable 2>/dev/null) || stable_pub="$version"
print_success "3/7 Promoted to stable: omarchy $stable_pub on every architecture"
fi fi
# 4. Final pins onto master (keeps edge overlap publishing and the repo record) # 4. Final pins onto master (keeps edge overlap publishing and the repo record)
@@ -920,7 +922,7 @@ next_step() { # prints "<command>|<description>"
last=$(newest_release_branch 2>/dev/null) || last="" last=$(newest_release_branch 2>/dev/null) || last=""
if [[ -n "$last" && "$STABLE_VER" != "<unreachable>" ]]; then if [[ -n "$last" && "$STABLE_VER" != "<unreachable>" ]]; then
last_ver=$(branch_to_version "$last") last_ver=$(branch_to_version "$last")
if [[ "${STABLE_VER%-*}" != "$last_ver" ]] || if ! all_arches_at_version stable "$last_ver" ||
{ command -v gh >/dev/null && ! gh release view "v$last_ver" --repo "$UPSTREAM_REPO" >/dev/null 2>&1; }; then { command -v gh >/dev/null && ! gh release view "v$last_ver" --repo "$UPSTREAM_REPO" >/dev/null 2>&1; }; then
echo "ship|$last_ver is tagged but not fully shipped — resume ship" echo "ship|$last_ver is tagged but not fully shipped — resume ship"
return return
@@ -936,7 +938,7 @@ next_step() { # prints "<command>|<description>"
echo "ship|Final $TRAIN_VER is pinned — finish shipping (re-runs are safe)" echo "ship|Final $TRAIN_VER is pinned — finish shipping (re-runs are safe)"
elif [[ "$pin_commit" != "$TRAIN_HEAD" ]]; then elif [[ "$pin_commit" != "$TRAIN_HEAD" ]]; then
echo "rc|$TRAIN has commits newer than $pin_ver — cut the next candidate" echo "rc|$TRAIN has commits newer than $pin_ver — cut the next candidate"
elif [[ "${RC_VER%-*}" != "$pin_ver" ]]; then elif ! all_arches_at_version rc "$pin_ver"; then
echo "rc|$pin_ver is pinned but not published — re-run rc to re-trigger/wait" echo "rc|$pin_ver is pinned but not published — re-run rc to re-trigger/wait"
else else
echo "ship|$pin_ver is published to rc — test it, then ship" echo "ship|$pin_ver is published to rc — test it, then ship"
@@ -1006,20 +1008,24 @@ cmd_doctor() {
check "makepkg available (checksums)" command -v makepkg check "makepkg available (checksums)" command -v makepkg
check "curl available" command -v curl check "curl available" command -v curl
check "upstream reachable ($UPSTREAM_URL)" git ls-remote "$UPSTREAM_URL" HEAD check "upstream reachable ($UPSTREAM_URL)" git ls-remote "$UPSTREAM_URL" HEAD
local ch local ch arch
for ch in edge stable; do for ch in edge stable; do
if published_version "$ch" >/dev/null 2>&1; then for arch in $(published_arches); do
print_success "$ch channel db readable" if published_version "$ch" "$arch" >/dev/null 2>&1; then
print_success "$ch/$arch channel db readable"
else
print_error "$ch/$arch channel db readable"
failures=$((failures + 1))
fi
done
done
for arch in $(published_arches); do
if published_version rc "$arch" >/dev/null 2>&1; then
print_success "rc/$arch channel db readable"
else else
print_error "$ch channel db readable" print_warning "rc/$arch channel db not readable — bootstrap it first: bin/repo bootstrap-rc (on the build host)"
failures=$((failures + 1))
fi fi
done done
if published_version rc >/dev/null 2>&1; then
print_success "rc channel db readable"
else
print_warning "rc channel db not readable — bootstrap it first: bin/repo bootstrap-rc (on the build host)"
fi
local host local host
if host=$(repo_host); then if host=$(repo_host); then
check "build host ssh ($host)" ssh -o ConnectTimeout=10 "$host" true check "build host ssh ($host)" ssh -o ConnectTimeout=10 "$host" true
@@ -1066,6 +1072,25 @@ cmd_self_test() {
expect "version_is_patch 5.0.0" "$(version_is_patch 5.0.0 && echo yes || echo no)" "no" expect "version_is_patch 5.0.0" "$(version_is_patch 5.0.0 && echo yes || echo no)" "no"
expect "previous_patch_tag 4.0.2" "$(previous_patch_tag 4.0.2)" "v4.0.1" expect "previous_patch_tag 4.0.2" "$(previous_patch_tag 4.0.2)" "v4.0.1"
expect "previous_patch_tag 4.0.10" "$(previous_patch_tag 4.0.10)" "v4.0.9" expect "previous_patch_tag 4.0.10" "$(previous_patch_tag 4.0.10)" "v4.0.9"
# Keep release readiness fail-closed when only one architecture has reached
# the requested version. This replaces the network reader for this process;
# self-test exits immediately afterwards.
published_version() {
case "$2" in
x86_64) echo "${TEST_X86_VERSION:-4.0.2-1}" ;;
aarch64) echo "${TEST_ARM_VERSION:-4.0.2-1}" ;;
esac
}
PUBLISHED_ARCHES=x86_64
expect "x86-only readiness" "$(all_arches_at_version rc 4.0.2 && echo ready || echo blocked)" "ready"
PUBLISHED_ARCHES=aarch64
expect "ARM-only readiness" "$(all_arches_at_version rc 4.0.2 && echo ready || echo blocked)" "ready"
PUBLISHED_ARCHES="x86_64 aarch64"
TEST_ARM_VERSION=4.0.1-1
expect "mixed versions block release" "$(all_arches_at_version rc 4.0.2 && echo ready || echo blocked)" "blocked"
TEST_ARM_VERSION=4.0.2-1
expect "both architectures ready" "$(all_arches_at_version rc 4.0.2 && echo ready || echo blocked)" "ready"
echo "" echo ""
if ((failures == 0)); then if ((failures == 0)); then
print_success "Self-test passed" print_success "Self-test passed"
+9 -6
View File
@@ -88,9 +88,12 @@ if [[ ! $REPLY =~ ^[Yy]$ ]]; then
exit 0 exit 0
fi fi
# Build/update the Docker image (always use x86_64 for removal - it's architecture independent) # repo-remove is architecture-independent, so use a host-native edge image.
# repo-remove is mirror-independent — always use the edge x86_64 image TOOL_ARCH=$(docker_native_arch) || {
build_docker_image "$BUILD_DIR" "x86_64" "edge" print_error "Unsupported host architecture: $(uname -m)"
exit 1
}
build_docker_image "$BUILD_DIR" "$TOOL_ARCH" "edge"
acquire_release_lock || exit 1 acquire_release_lock || exit 1
@@ -99,13 +102,13 @@ print_info "Removing package..."
# Ensure directory is writable by container user # Ensure directory is writable by container user
make_dir_writable "$REPO_DIR" make_dir_writable "$REPO_DIR"
# Run the removal script in Docker (always use x86_64 image) # Run the removal script in the host-native image.
docker run --rm --platform linux/amd64 \ docker run --rm "$(get_platform_arg "$TOOL_ARCH")" \
-e ARCH="$ARCH" \ -e ARCH="$ARCH" \
-e MIRROR="$MIRROR" \ -e MIRROR="$MIRROR" \
-v "$REPO_ROOT:/pkgs.omarchy.org" \ -v "$REPO_ROOT:/pkgs.omarchy.org" \
-v "$BUILD_DIR:/build:ro" \ -v "$BUILD_DIR:/build:ro" \
omarchy-pkg-builder:latest-x86_64-edge /build/remove-package.sh "$PACKAGE_NAME" "omarchy-pkg-builder:latest-$TOOL_ARCH-edge" /build/remove-package.sh "$PACKAGE_NAME"
RESULT=$? RESULT=$?
+2 -1
View File
@@ -15,6 +15,7 @@ set -e
BUILD_ROOT=$(realpath "${BASH_SOURCE[0]%/*}/..") BUILD_ROOT=$(realpath "${BASH_SOURCE[0]%/*}/..")
source "$BUILD_ROOT/helpers/message-helpers.sh" source "$BUILD_ROOT/helpers/message-helpers.sh"
source "$BUILD_ROOT/helpers/paths.sh"
CHECK_ONLY=false CHECK_ONLY=false
SKIP_TIMERS=false SKIP_TIMERS=false
@@ -238,7 +239,7 @@ echo ""
# --- release timers ---------------------------------------------------------- # --- release timers ----------------------------------------------------------
print_info "Published architectures: $(published_arches | tr '\n' ' ')" print_info "Published architectures: $(published_arches | tr '\n' ' ')"
echo " (PUBLISHED_ARCHES in helpers/paths.sh, or OMARCHY_ARCHES in $CREDENTIALS)" echo " (PUBLISHED_ARCHES in helpers/paths.sh; OMARCHY_ARCHES overrides a one-off command)"
echo "" echo ""
TIMERS=(omarchy-check-versions omarchy-auto-release-edge omarchy-auto-release-rc omarchy-auto-release-stable) TIMERS=(omarchy-check-versions omarchy-auto-release-edge omarchy-auto-release-rc omarchy-auto-release-stable)
+10 -5
View File
@@ -66,23 +66,28 @@ if [[ -z "$GPG_PASSPHRASE" ]]; then
exit 1 exit 1
fi fi
# Build/update the Docker image (always use x86_64 for signing - it's architecture independent) # Signing is architecture-independent, so run its utility container natively
build_docker_image "$BUILD_DIR" "x86_64" "$MIRROR" # on either an x86_64 or ARM host.
TOOL_ARCH=$(docker_native_arch) || {
print_error "Unsupported host architecture: $(uname -m)"
exit 1
}
build_docker_image "$BUILD_DIR" "$TOOL_ARCH" "$MIRROR"
print_info "Running package signing..." print_info "Running package signing..."
# Ensure output directory is writable by container user # Ensure output directory is writable by container user
make_dir_writable "$BUILD_OUTPUT_DIR" make_dir_writable "$BUILD_OUTPUT_DIR"
# Run the signing script in Docker (always use x86_64 image) # Run the signing script in the host-native image.
docker run --rm --platform linux/amd64 \ docker run --rm "$(get_platform_arg "$TOOL_ARCH")" \
-e ARCH="$ARCH" \ -e ARCH="$ARCH" \
-e MIRROR="$MIRROR" \ -e MIRROR="$MIRROR" \
-e GPG_PRIVATE_KEY="$GPG_PRIVATE_KEY" \ -e GPG_PRIVATE_KEY="$GPG_PRIVATE_KEY" \
-e GPG_PASSPHRASE="$GPG_PASSPHRASE" \ -e GPG_PASSPHRASE="$GPG_PASSPHRASE" \
-v "$BUILD_ROOT/build-output:/build-output" \ -v "$BUILD_ROOT/build-output:/build-output" \
-v "$BUILD_DIR:/build:ro" \ -v "$BUILD_DIR:/build:ro" \
omarchy-pkg-builder:latest-x86_64-$MIRROR /build/sign.sh "omarchy-pkg-builder:latest-$TOOL_ARCH-$MIRROR" /build/sign.sh
SIGN_RESULT=$? SIGN_RESULT=$?
+228 -59
View File
@@ -17,11 +17,14 @@ SELF_TEST=false
# rebuilding for it would ship a package built against the wrong ABI. # rebuilding for it would ship a package built against the wrong ABI.
OFFICIAL_REPOS=" core extra multilib core-debug extra-debug " OFFICIAL_REPOS=" core extra multilib core-debug extra-debug "
# The published repository, used as the floor a bumped pkgrel has to clear. # The published repositories are the floor a bumped pkgrel has to clear.
# Only x86_64 is published today; aarch64 has no repository to compare against.
PUBLISHED_BASE_URL="${OMARCHY_PUBLISHED_BASE_URL:-https://pkgs.omarchy.org}" PUBLISHED_BASE_URL="${OMARCHY_PUBLISHED_BASE_URL:-https://pkgs.omarchy.org}"
PUBLISHED_MIRRORS=(edge stable) PUBLISHED_MIRRORS=(edge stable)
PUBLISHED_ARCH=x86_64
# Arch Linux ARM has no dated snapshots. This is the same live repository the
# aarch64 builder resolves; override it only when the builder mirror changes.
ALARM_BASE_URL="${OMARCHY_ALARM_BASE_URL:-https://fl.us.mirror.archlinuxarm.org/aarch64}"
ALARM_REPOS=(core extra alarm aur)
usage() { usage() {
cat <<EOF cat <<EOF
@@ -34,10 +37,13 @@ A package opts in by naming those dependencies in .omarchy/package.json:
{ "source": "aur", "sync": false, "rebuild_on": ["qt6-base"] } { "source": "aur", "sync": false, "rebuild_on": ["qt6-base"] }
The versions the current pkgrel was bumped for are recorded alongside, in The versions the current pkgrel was bumped for are recorded per published
rebuilt_against, and written by this command: architecture in rebuilt_against, and written by this command:
{ "rebuild_on": ["qt6-base"], "rebuilt_against": { "qt6-base": "6.11.2-2" } } { "rebuild_on": ["qt6-base"], "rebuilt_against": {
"x86_64": { "qt6-base": "6.11.2-3" },
"aarch64": { "qt6-base": "6.11.2-2" }
} }
pkgrel is bumped unless every package named in rebuild_on is recorded and still pkgrel is bumped unless every package named in rebuild_on is recorded and still
matches. A name that is missing from the record counts as changed, so opting a matches. A name that is missing from the record counts as changed, so opting a
@@ -60,8 +66,9 @@ Examples:
$0 # Update every package that declares rebuild_on $0 # Update every package that declares rebuild_on
$0 quickshell-git # Update specific packages $0 quickshell-git # Update specific packages
Trigger versions are read from the local pacman database, so sync it first x86_64 trigger versions come from the local pacman database; aarch64 versions
(pacman -Sy) or this reports whatever that database last saw. come from the live Arch Linux ARM repository database. Only architectures in
PUBLISHED_ARCHES that the package supports are considered.
EOF EOF
} }
@@ -94,7 +101,7 @@ SPECIFIC_MODE=false
# The version of a trigger package as the build container would resolve it. # The version of a trigger package as the build container would resolve it.
# A name pacman does not know reports nothing rather than failing, so the caller # A name pacman does not know reports nothing rather than failing, so the caller
# gets to say which package was left alone instead of the run dying here. # gets to say which package was left alone instead of the run dying here.
repo_version() { native_repo_version() {
local package="$1" local package="$1"
local info local info
@@ -108,9 +115,59 @@ repo_version() {
' <<<"$info" ' <<<"$info"
} }
load_alarm_repo() {
local repo="$1" db index
index="$TEMP_DIR/alarm-$repo.index"
[[ -f "$index" ]] && return 0
db="$TEMP_DIR/alarm-$repo.db"
if ! curl -fsSL --max-time 120 -o "$db" "$ALARM_BASE_URL/$repo/$repo.db" 2>/dev/null; then
print_error "Could not read the aarch64 $repo repository database"
return 1
fi
if ! tar -tf "$db" >/dev/null 2>&1; then
print_error "Unreadable aarch64 $repo repository database"
return 1
fi
tar -xOf "$db" --wildcards '*/desc' 2>/dev/null | awk '
function emit() {
if (name != "" && version != "") {
print name "\t" version
if (base != "" && base != name) print base "\t" version
}
name=""; base=""; version=""
}
$0 == "%FILENAME%" { emit(); next }
$0 == "%NAME%" { if (name != "" && version != "") emit(); getline; name=$0; next }
$0 == "%BASE%" { getline; base=$0; next }
$0 == "%VERSION%" { getline; version=$0; next }
END { emit() }
' >"$index"
}
alarm_repo_version() {
local package="$1" repo version
for repo in "${ALARM_REPOS[@]}"; do
load_alarm_repo "$repo" || return 1
version=$(awk -F '\t' -v package="$package" '$1 == package { print $2; exit }' "$TEMP_DIR/alarm-$repo.index")
if [[ -n "$version" ]]; then
echo "$version"
return 0
fi
done
}
repo_version() { # repo_version <arch> <package>
case "$1" in
x86_64) native_repo_version "$2" ;;
aarch64) alarm_repo_version "$2" ;;
*) return 1 ;;
esac
}
declare -A PUBLISHED_VERSION=() declare -A PUBLISHED_VERSION=()
PUBLISHED_LOADED=false PUBLISHED_LOADED=false
PUBLISHED_AVAILABLE=true
remember_published() { remember_published() {
local name="$1" local name="$1"
@@ -131,34 +188,35 @@ load_published_versions() {
[[ "$PUBLISHED_LOADED" == true ]] && return 0 [[ "$PUBLISHED_LOADED" == true ]] && return 0
PUBLISHED_LOADED=true PUBLISHED_LOADED=true
local mirror db name base version local arch mirror db name base version
for mirror in "${PUBLISHED_MIRRORS[@]}"; do for arch in $(published_arches); do
db="$TEMP_DIR/published-$mirror.db.tar.zst" for mirror in "${PUBLISHED_MIRRORS[@]}"; do
db="$TEMP_DIR/published-$mirror-$arch.db.tar.zst"
if ! curl -fsSL --max-time 120 -o "$db" \ if ! curl -fsSL --max-time 120 -o "$db" \
"$PUBLISHED_BASE_URL/$mirror/$PUBLISHED_ARCH/omarchy.db.tar.zst" 2>/dev/null; then "$PUBLISHED_BASE_URL/$mirror/$arch/omarchy.db.tar.zst" 2>/dev/null; then
print_warning "Could not read the published $mirror database; bumps are not checked against it this run" print_warning "Could not read the published $mirror/$arch database; bumps are not checked against it this run"
PUBLISHED_AVAILABLE=false continue
continue fi
fi
while IFS=$'\t' read -r name base version; do while IFS=$'\t' read -r name base version; do
[[ -n "$name" && -n "$version" ]] && remember_published "$name" "$version" [[ -n "$name" && -n "$version" ]] && remember_published "$name" "$version"
[[ -n "$base" && -n "$version" ]] && remember_published "$base" "$version" [[ -n "$base" && -n "$version" ]] && remember_published "$base" "$version"
done < <( done < <(
tar -xOf "$db" --wildcards '*/desc' 2>/dev/null | awk ' tar -xOf "$db" --wildcards '*/desc' 2>/dev/null | awk '
function emit() { function emit() {
if (name != "" && version != "") print name "\t" base "\t" version if (name != "" && version != "") print name "\t" base "\t" version
name=""; base=""; version="" name=""; base=""; version=""
} }
$0 == "%FILENAME%" { emit(); next } $0 == "%FILENAME%" { emit(); next }
$0 == "%NAME%" { if (name != "" && version != "") emit(); getline; name=$0; next } $0 == "%NAME%" { if (name != "" && version != "") emit(); getline; name=$0; next }
$0 == "%BASE%" { getline; base=$0; next } $0 == "%BASE%" { getline; base=$0; next }
$0 == "%VERSION%" { getline; version=$0; next } $0 == "%VERSION%" { getline; version=$0; next }
END { emit() } END { emit() }
' '
) )
done
done done
} }
@@ -252,7 +310,17 @@ record_triggers() {
local package_dir="$1" local package_dir="$1"
local current="$2" local current="$2"
write_metadata "$package_dir" '.rebuilt_against = $current' --argjson current "$current" # A flat record is the legacy x86_64 shape. Preserve records for
# architectures outside this run, then replace the ones just rebuilt.
write_metadata "$package_dir" '
(.rebuilt_against // {}) as $old |
(if ($old | length) == 0 then {}
elif ($old | to_entries | all(.value | type == "string"))
then {x86_64: $old}
else $old
end) as $by_arch |
.rebuilt_against = ($by_arch * $current)
' --argjson current "$current"
} }
# Metadata that does not parse would otherwise drop its package out of the run # Metadata that does not parse would otherwise drop its package out of the run
@@ -300,21 +368,41 @@ sync_package() {
print_info "Checking $package against ${triggers[*]}..." print_info "Checking $package against ${triggers[*]}..."
local current="{}" trigger version local current="{}" arch arch_current trigger version considered=0
for trigger in "${triggers[@]}"; do for arch in $(published_arches); do
version=$(repo_version "$trigger") package_supports_arch "$package_dir" "$arch" || continue
if [[ -z "$version" ]]; then considered=$((considered + 1))
print_error " $trigger is in no official repository; leaving $package alone" arch_current="{}"
((++FAILED)) for trigger in "${triggers[@]}"; do
return 0 if ! version=$(repo_version "$arch" "$trigger"); then
fi print_error " Could not read $arch repository versions; leaving $package alone"
if ! current=$(jq -c --arg name "$trigger" --arg version "$version" '.[$name] = $version' <<<"$current"); then ((++FAILED))
print_error " Could not record $trigger $version for $package" return 0
fi
if [[ -z "$version" ]]; then
print_error " $trigger is in no $arch repository; leaving $package alone"
((++FAILED))
return 0
fi
if ! arch_current=$(jq -c --arg name "$trigger" --arg version "$version" '.[$name] = $version' <<<"$arch_current"); then
print_error " Could not record $arch/$trigger $version for $package"
((++FAILED))
return 0
fi
done
if ! current=$(jq -c --arg arch "$arch" --argjson versions "$arch_current" '.[$arch] = $versions' <<<"$current"); then
print_error " Could not record $arch trigger versions for $package"
((++FAILED)) ((++FAILED))
return 0 return 0
fi fi
done done
if ((considered == 0)); then
print_info " Skipping: not built for any published architecture"
((++SKIPPED))
return 0
fi
local recorded local recorded
if ! recorded=$(package_metadata_value "$package_dir" '.rebuilt_against' ""); then if ! recorded=$(package_metadata_value "$package_dir" '.rebuilt_against' ""); then
print_error " Could not read .omarchy/package.json for $package" print_error " Could not read .omarchy/package.json for $package"
@@ -323,13 +411,20 @@ sync_package() {
fi fi
[[ -n "$recorded" && "$recorded" != "null" ]] || recorded="{}" [[ -n "$recorded" && "$recorded" != "null" ]] || recorded="{}"
# Before architecture-specific records existed, rebuilt_against described
# x86_64. Read it that way without forcing a metadata-only migration.
if jq -e 'to_entries | all(.value | type == "string")' >/dev/null <<<"$recorded"; then
recorded=$(jq -c '{x86_64: .}' <<<"$recorded")
fi
# Walk the declared triggers rather than the record, so a name the record does # Walk the declared triggers rather than the record, so a name the record does
# not carry reads as changed instead of going unexamined forever. # not carry reads as changed instead of going unexamined forever.
local moved local moved
if ! moved=$(jq -r --argjson recorded "$recorded" ' if ! moved=$(jq -r --argjson recorded "$recorded" '
to_entries [to_entries[] as $arch
| map(select($recorded[.key] != .value) | $arch.value | to_entries[] as $trigger
| "\(.key) \($recorded[.key] // "unrecorded") -> \(.value)") | select($recorded[$arch.key][$trigger.key] != $trigger.value)
| "\($arch.key)/\($trigger.key) \($recorded[$arch.key][$trigger.key] // "unrecorded") -> \($trigger.value)"]
| join(", ") | join(", ")
' <<<"$current"); then ' <<<"$current"); then
print_error " Could not compare recorded trigger versions for $package" print_error " Could not compare recorded trigger versions for $package"
@@ -338,7 +433,7 @@ sync_package() {
fi fi
if [[ -z "$moved" ]]; then if [[ -z "$moved" ]]; then
print_info " Already rebuilt against $(jq -r 'to_entries | map("\(.key) \(.value)") | join(", ")' <<<"$current")" print_info " Already rebuilt against every published architecture"
((++SKIPPED)) ((++SKIPPED))
return 0 return 0
fi fi
@@ -466,11 +561,11 @@ selftest_root() {
} }
selftest_package() { selftest_package() {
local root="$1" name="$2" pkgrel="$3" metadata="$4" pkgver="${5:-1.0}" local root="$1" name="$2" pkgrel="$3" metadata="$4" pkgver="${5:-1.0}" arches="${6:-x86_64}"
local dir="$root/pkgbuilds/$name" local dir="$root/pkgbuilds/$name"
mkdir -p "$dir/.omarchy" mkdir -p "$dir/.omarchy"
printf 'pkgname=%s\npkgver=%s\npkgrel=%s\narch=(x86_64)\n' "$name" "$pkgver" "$pkgrel" > "$dir/PKGBUILD" printf 'pkgname=%s\npkgver=%s\npkgrel=%s\narch=(%s)\n' "$name" "$pkgver" "$pkgrel" "$arches" > "$dir/PKGBUILD"
printf '%s\n' "$metadata" > "$dir/.omarchy/package.json" printf '%s\n' "$metadata" > "$dir/.omarchy/package.json"
} }
@@ -514,19 +609,51 @@ selftest_published() {
cat > "$root/stub/curl" <<'STUB' cat > "$root/stub/curl" <<'STUB'
#!/bin/bash #!/bin/bash
out="" out=""
url=""
while [[ $# -gt 0 ]]; do while [[ $# -gt 0 ]]; do
case "$1" in case "$1" in
-o) out="$2"; shift 2 ;; -o) out="$2"; shift 2 ;;
*) shift ;; *) url="$1"; shift ;;
esac esac
done done
db="$(dirname "$0")/omarchy.db.tar.zst" if [[ "$url" == */aarch64/* ]]; then
repo="${url%/*}"
repo="${repo##*/}"
db="$(dirname "$0")/alarm-$repo.db"
else
db="$(dirname "$0")/omarchy.db.tar.zst"
fi
[[ -f "$db" && -n "$out" ]] || exit 22 [[ -f "$db" && -n "$out" ]] || exit 22
cp "$db" "$out" cp "$db" "$out"
STUB STUB
chmod +x "$root/stub/curl" chmod +x "$root/stub/curl"
} }
selftest_alarm() {
local root="$1"
shift
local repo staging="$root/stub/alarm-db" entry name version
for repo in core extra; do
rm -rf "$staging"
mkdir -p "$staging"
if [[ "$repo" == "core" ]]; then
mkdir -p "$staging/unrelated-1-1"
printf '%%FILENAME%%\nunrelated-1-1-aarch64.pkg.tar.zst\n\n%%NAME%%\nunrelated\n\n%%BASE%%\nunrelated\n\n%%VERSION%%\n1-1\n' \
> "$staging/unrelated-1-1/desc"
else
for entry in "$@"; do
name="${entry%=*}"
version="${entry#*=}"
mkdir -p "$staging/$name-$version"
printf '%%FILENAME%%\n%s-%s-aarch64.pkg.tar.zst\n\n%%NAME%%\n%s\n\n%%BASE%%\n%s\n\n%%VERSION%%\n%s\n' \
"$name" "$version" "$name" "$name" "$version" > "$staging/$name-$version/desc"
done
fi
tar -czf "$root/stub/alarm-$repo.db" -C "$staging" .
done
}
cmd_self_test() { cmd_self_test() {
local failures=0 local failures=0
local root local root
@@ -545,7 +672,8 @@ cmd_self_test() {
local root="$1" local root="$1"
shift shift
local status=0 local status=0
PATH="$root/stub:$PATH" "$root/bin/sync-rebuilds" "$@" > "$root/output" 2>&1 || status=$? OMARCHY_ARCHES="${SELFTEST_ARCHES:-x86_64}" \
PATH="$root/stub:$PATH" "$root/bin/sync-rebuilds" "$@" > "$root/output" 2>&1 || status=$?
echo "$status" echo "$status"
} }
@@ -563,7 +691,7 @@ cmd_self_test() {
check "run succeeds" 0 "$(run_case "$root")" check "run succeeds" 0 "$(run_case "$root")"
check "pkgrel bumped" 2 "$(pkgrel_of "$root/pkgbuilds/t-partial")" check "pkgrel bumped" 2 "$(pkgrel_of "$root/pkgbuilds/t-partial")"
check "unrecorded trigger now recorded" "2-2" \ check "unrecorded trigger now recorded" "2-2" \
"$(jq -r '.rebuilt_against["dep-b"]' "$root/pkgbuilds/t-partial/.omarchy/package.json")" "$(jq -r '.rebuilt_against.x86_64["dep-b"]' "$root/pkgbuilds/t-partial/.omarchy/package.json")"
echo "Opting a package in buys a rebuild rather than a bare record:" echo "Opting a package in buys a rebuild rather than a bare record:"
root=$(selftest_root fresh) root=$(selftest_root fresh)
@@ -573,7 +701,7 @@ cmd_self_test() {
check "run succeeds" 0 "$(run_case "$root")" check "run succeeds" 0 "$(run_case "$root")"
check "pkgrel bumped" 2 "$(pkgrel_of "$root/pkgbuilds/t-fresh")" check "pkgrel bumped" 2 "$(pkgrel_of "$root/pkgbuilds/t-fresh")"
check "trigger recorded" "1-1" \ check "trigger recorded" "1-1" \
"$(jq -r '.rebuilt_against["dep-a"]' "$root/pkgbuilds/t-fresh/.omarchy/package.json")" "$(jq -r '.rebuilt_against.x86_64["dep-a"]' "$root/pkgbuilds/t-fresh/.omarchy/package.json")"
echo "An unchanged package is left alone:" echo "An unchanged package is left alone:"
root=$(selftest_root current) root=$(selftest_root current)
@@ -612,6 +740,47 @@ cmd_self_test() {
check "suffix recorded for the next AUR sync" 1 \ check "suffix recorded for the next AUR sync" 1 \
"$(jq -r '.pkgrel.suffix' "$root/pkgbuilds/t-aur/.omarchy/package.json")" "$(jq -r '.pkgrel.suffix' "$root/pkgbuilds/t-aur/.omarchy/package.json")"
echo "A dependency is tracked independently for both published architectures:"
root=$(selftest_root multiarch)
selftest_package "$root" t-multi 1 '{"source":"local","rebuild_on":["dep-a"],"rebuilt_against":{"dep-a":"1-1"}}' 1.0 'x86_64 aarch64'
selftest_pacman "$root" dep-a=1-1
selftest_published "$root"
selftest_alarm "$root" dep-a=2-1
SELFTEST_ARCHES="x86_64 aarch64"
check "run succeeds" 0 "$(run_case "$root")"
unset SELFTEST_ARCHES
check "pkgrel bumped once" 2 "$(pkgrel_of "$root/pkgbuilds/t-multi")"
check "x86_64 trigger recorded" "1-1" \
"$(jq -r '.rebuilt_against.x86_64["dep-a"]' "$root/pkgbuilds/t-multi/.omarchy/package.json")"
check "aarch64 trigger recorded" "2-1" \
"$(jq -r '.rebuilt_against.aarch64["dep-a"]' "$root/pkgbuilds/t-multi/.omarchy/package.json")"
echo "An ARM-only run records only the ARM dependency state:"
root=$(selftest_root arm-only)
selftest_package "$root" t-arm 1 '{"source":"local","rebuild_on":["dep-a"]}' 1.0 'x86_64 aarch64'
selftest_pacman "$root"
selftest_published "$root"
selftest_alarm "$root" dep-a=2-1
SELFTEST_ARCHES=aarch64
check "run succeeds" 0 "$(run_case "$root")"
unset SELFTEST_ARCHES
check "pkgrel bumped" 2 "$(pkgrel_of "$root/pkgbuilds/t-arm")"
check "ARM trigger recorded" "2-1" \
"$(jq -r '.rebuilt_against.aarch64["dep-a"]' "$root/pkgbuilds/t-arm/.omarchy/package.json")"
check "x86_64 was not consulted" "false" \
"$(jq -r '.rebuilt_against | has("x86_64")' "$root/pkgbuilds/t-arm/.omarchy/package.json")"
echo "An x86-only package ignores ARM during a dual-architecture run:"
root=$(selftest_root x86-package)
selftest_package "$root" t-x86 1 '{"source":"local","rebuild_on":["dep-a"],"rebuilt_against":{"dep-a":"1-1"}}'
selftest_pacman "$root" dep-a=1-1
selftest_published "$root"
selftest_alarm "$root" dep-a=2-1
SELFTEST_ARCHES="x86_64 aarch64"
check "run succeeds" 0 "$(run_case "$root")"
unset SELFTEST_ARCHES
check "pkgrel untouched" 1 "$(pkgrel_of "$root/pkgbuilds/t-x86")"
echo "" echo ""
if [[ "$failures" -eq 0 ]]; then if [[ "$failures" -eq 0 ]]; then
print_success "Self-test passed" print_success "Self-test passed"
@@ -626,9 +795,9 @@ if [[ "$SELF_TEST" == true ]]; then
exit $? exit $?
fi fi
for tool in pacman vercmp jq curl; do for tool in pacman vercmp jq curl tar; do
if ! command -v "$tool" >/dev/null 2>&1; then if ! command -v "$tool" >/dev/null 2>&1; then
print_error "$tool not found: reading trigger versions and ordering pkgrels both need pacman" print_error "$tool not found: rebuild trigger sync cannot run"
exit 1 exit 1
fi fi
done done
+26 -26
View File
@@ -105,32 +105,32 @@ echo ""
paused=false paused=false
for channel in edge rc stable; do for channel in edge rc stable; do
for arch in $(published_arches); do for arch in $(published_arches); do
fail_file=$(sync_fail_file "$channel" "$arch") fail_file=$(sync_fail_file "$channel" "$arch")
[[ "$arch" == "x86_64" && ! -f "$fail_file" ]] && fail_file=$(legacy_sync_fail_file "$channel") [[ "$arch" == "x86_64" && ! -f "$fail_file" ]] && fail_file=$(legacy_sync_fail_file "$channel")
[[ -f "$fail_file" ]] || continue [[ -f "$fail_file" ]] || continue
if [[ "$paused" == false ]]; then if [[ "$paused" == false ]]; then
print_error "Failing builds (backoff active)" print_error "Failing builds (backoff active)"
paused=true paused=true
fi fi
FAILURE_COUNT=0 FAILURE_AT=0 FAILURE_FINGERPRINT="" FAILURE_COUNT=0 FAILURE_AT=0 FAILURE_FINGERPRINT=""
# shellcheck disable=SC1090 # shellcheck disable=SC1090
source "$fail_file" 2>/dev/null || true source "$fail_file" 2>/dev/null || true
delay=600 delay=600
for ((i = 1; i < FAILURE_COUNT; i++)); do for ((i = 1; i < FAILURE_COUNT; i++)); do
delay=$((delay * 2)) delay=$((delay * 2))
((delay >= 21600)) && { delay=21600; break; } ((delay >= 21600)) && { delay=21600; break; }
done done
retry_at=$((FAILURE_AT + delay)) retry_at=$((FAILURE_AT + delay))
now=$(date +%s) now=$(date +%s)
if ((now < retry_at)); then if ((now < retry_at)); then
when="retries at $(date -d "@$retry_at" '+%H:%M:%S' 2>/dev/null || echo "+$((retry_at - now))s")" when="retries at $(date -d "@$retry_at" '+%H:%M:%S' 2>/dev/null || echo "+$((retry_at - now))s")"
else else
when="retries on the next tick" when="retries on the next tick"
fi fi
printf ' ✗ %-7s %-8s %s consecutive failure(s), %s\n' "$channel" "$arch" "$FAILURE_COUNT" "$when" printf ' ✗ %-7s %-8s %s consecutive failure(s), %s\n' "$channel" "$arch" "$FAILURE_COUNT" "$when"
printf ' last attempt %s on commit %s\n' \ printf ' last attempt %s on commit %s\n' \
"$(date -d "@$FAILURE_AT" '+%Y-%m-%d %H:%M:%S' 2>/dev/null || echo "$FAILURE_AT")" \ "$(date -d "@$FAILURE_AT" '+%Y-%m-%d %H:%M:%S' 2>/dev/null || echo "$FAILURE_AT")" \
"${FAILURE_FINGERPRINT:0:12}" "${FAILURE_FINGERPRINT:0:12}"
done done
done done
if [[ "$paused" == true ]]; then if [[ "$paused" == true ]]; then
+10 -6
View File
@@ -28,17 +28,21 @@ update_database() {
# Make output directory writable for container # Make output directory writable for container
make_dir_writable "$REPO_DIR" make_dir_writable "$REPO_DIR"
# repo-add is architecture- and mirror-independent, so always use the edge # repo-add is architecture- and mirror-independent. Use the host-native edge
# x86_64 image. This also lets bootstrap-rc build the rc database before the # image, which also lets bootstrap-rc run before that channel exists remotely.
# rc channel exists remotely (an rc image can only build after it does). local tool_arch
build_docker_image "$BUILD_DIR" "x86_64" "edge" tool_arch=$(docker_native_arch) || {
print_error "Unsupported host architecture: $(uname -m)"
exit 1
}
build_docker_image "$BUILD_DIR" "$tool_arch" "edge"
docker run --rm --platform linux/amd64 \ docker run --rm "$(get_platform_arg "$tool_arch")" \
-e ARCH="$ARCH" \ -e ARCH="$ARCH" \
-e MIRROR="$MIRROR" \ -e MIRROR="$MIRROR" \
-v "$REPO_ROOT:/output" \ -v "$REPO_ROOT:/output" \
-v "$BUILD_DIR:/build:ro" \ -v "$BUILD_DIR:/build:ro" \
omarchy-pkg-builder:latest-x86_64-edge /build/update-repo.sh "omarchy-pkg-builder:latest-$tool_arch-edge" /build/update-repo.sh
} }
# Main execution # Main execution
+1 -3
View File
@@ -45,9 +45,7 @@ RUN if [ "${TARGETARCH}" = "amd64" ]; then \
printf 'Server = https://mirror.omarchy.org/$repo/os/$arch\n' > /etc/pacman.d/mirrorlist; \ printf 'Server = https://mirror.omarchy.org/$repo/os/$arch\n' > /etc/pacman.d/mirrorlist; \
fi; \ fi; \
else \ else \
curl -L "https://raw.githubusercontent.com/archlinuxarm/PKGBUILDs/master/core/pacman-mirrorlist/mirrorlist" 2>/dev/null | \ printf 'Server = https://fl.us.mirror.archlinuxarm.org/aarch64/$repo\n' > /etc/pacman.d/mirrorlist; \
sed -E 's/^\s*#\s*Server\s*=/Server =/g' > /etc/pacman.d/mirrorlist && \
sed -i 's/\$arch/aarch64/g' /etc/pacman.d/mirrorlist; \
fi fi
# Bootstrap keyrings (required before pacstrap can verify packages) # Bootstrap keyrings (required before pacstrap can verify packages)
+4 -28
View File
@@ -186,26 +186,9 @@ get_local_version() {
# Returns 0 (success) if should build, 1 if should skip # Returns 0 (success) if should build, 1 if should skip
should_build_for_arch() { should_build_for_arch() {
local pkg="$1" local pkg="$1"
local current_arch="$ARCH" local pkgdir
local pkgdir=$(find_package_dir "$pkg") pkgdir=$(find_package_dir "$pkg")
local pkgbuild="$pkgdir/PKGBUILD" [[ -n "$pkgdir" ]] && package_supports_arch "$pkgdir" "$ARCH"
[[ ! -f "$pkgbuild" ]] && return 1
# Check PKGBUILD arch=() array
local pkgbuild_archs=$(cd "$pkgdir" && bash -c 'source PKGBUILD 2>/dev/null; echo "${arch[@]}"')
# If arch=('any'), build for all architectures
if [[ "$pkgbuild_archs" == "any" ]]; then
return 0
fi
# Check if current arch is in PKGBUILD arch=()
if echo "$pkgbuild_archs" | grep -qw "$current_arch"; then
return 0 # Build
else
return 1 # Skip
fi
} }
# For VCS packages, makepkg recalculates pkgver() before the build. If the # For VCS packages, makepkg recalculates pkgver() before the build. If the
@@ -488,7 +471,7 @@ check_needs_build() {
# Collect packages that should be built for the selected mirror # Collect packages that should be built for the selected mirror
collect_packages() { collect_packages() {
packages_for_unscoped_build "$MIRROR" packages_for_unscoped_build "$MIRROR" "$ARCH"
} }
# Main execution # Main execution
@@ -540,13 +523,6 @@ if [[ -n "$PACKAGES" ]]; then
else else
# Build all packages that need updates from the relevant directories # Build all packages that need updates from the relevant directories
while IFS= read -r pkg; do while IFS= read -r pkg; do
# Check if package should be built for this architecture
if ! should_build_for_arch "$pkg"; then
echo " - $pkg - not built for $ARCH"
SKIPPED_PACKAGES="$SKIPPED_PACKAGES $pkg"
continue
fi
if check_needs_build "$pkg"; then if check_needs_build "$pkg"; then
PACKAGES_TO_BUILD+=("$pkg") PACKAGES_TO_BUILD+=("$pkg")
else else
+14 -6
View File
@@ -13,13 +13,21 @@ check_docker() {
fi fi
} }
docker_native_arch() {
case "$(uname -m)" in
x86_64) echo x86_64 ;;
aarch64 | arm64) echo aarch64 ;;
*) return 1 ;;
esac
}
setup_qemu() { setup_qemu() {
# Setup QEMU for building ARM64 packages on x86_64 hosts # Register emulators for builds whose target differs from the host.
if ! docker run --rm --privileged multiarch/qemu-user-static --reset -p yes --credential yes >/dev/null 2>&1; then if ! docker run --rm --privileged multiarch/qemu-user-static --reset -p yes --credential yes >/dev/null 2>&1; then
print_error "Failed to setup QEMU for ARM64 emulation" print_error "Failed to set up QEMU emulation"
exit 1 exit 1
fi fi
print_success "QEMU ARM64 emulation enabled" print_success "QEMU emulation enabled"
} }
build_docker_image() { build_docker_image() {
@@ -53,9 +61,9 @@ build_docker_image() {
get_platform_arg() { get_platform_arg() {
local arch="$1" local arch="$1"
case "$arch" in case "$arch" in
x86_64) echo "--platform linux/amd64" ;; x86_64) echo "--platform=linux/amd64" ;;
aarch64) echo "--platform linux/arm64" ;; aarch64) echo "--platform=linux/arm64" ;;
*) echo "" ;; *) return 1 ;;
esac esac
} }
+48 -4
View File
@@ -143,6 +143,31 @@ package_has_pkgbuild() {
[[ -f "$pkgdir/PKGBUILD" ]] [[ -f "$pkgdir/PKGBUILD" ]]
} }
# The architectures declared by a PKGBUILD. Set CARCH while reading it so a
# conditional arch=() assignment is evaluated for the architecture we are
# actually checking, even when the repository host is a different one.
package_arches() {
local pkgdir="$1"
local arch="${2:-${ARCH:-x86_64}}"
(cd "$pkgdir" && env -u OMARCHY_SRC CARCH="$arch" bash -c '
source PKGBUILD >/dev/null 2>&1
printf "%s\n" "${arch[*]}"
')
}
package_supports_arch() {
local pkgdir="$1"
local target="${2:-${ARCH:-x86_64}}"
local arches
arches=$(package_arches "$pkgdir" "$target") || return 1
case " $arches " in
*" any "* | *" $target "*) return 0 ;;
*) return 1 ;;
esac
}
# Channel membership: where a package may be published. Packages without a # Channel membership: where a package may be published. Packages without a
# `channels` key are members of every channel (they flow edge -> rc -> stable). # `channels` key are members of every channel (they flow edge -> rc -> stable).
package_has_channels() { package_has_channels() {
@@ -301,9 +326,12 @@ packages_for_mirror() {
packages_for_unscoped_build() { packages_for_unscoped_build() {
local mirror="$1" local mirror="$1"
local arch="${2:-${ARCH:-x86_64}}"
package_dirs | while IFS= read -r pkgdir; do package_dirs | while IFS= read -r pkgdir; do
if package_builds_for_mirror "$pkgdir" "$mirror" && ! package_build_skipped "$pkgdir"; then if package_builds_for_mirror "$pkgdir" "$mirror" &&
! package_build_skipped "$pkgdir" &&
package_supports_arch "$pkgdir" "$arch"; then
basename "$pkgdir" basename "$pkgdir"
fi fi
done done
@@ -502,12 +530,28 @@ validate_package_metadata() {
return 1 return 1
fi fi
if ! jq -e '(.rebuilt_against // {}) | type == "object" and (to_entries | all(.value | type == "string" and length > 0))' "$metadata" >/dev/null; then if ! jq -e '
echo "invalid rebuilt_against for $(basename "$pkgdir"): must be an object mapping package names to versions" def version_map:
type == "object" and (to_entries | all(.value | type == "string" and length > 0));
(.rebuilt_against // {}) as $record |
($record | version_map) or
(($record | type) == "object"
and ((($record | keys) - ["x86_64", "aarch64"]) | length == 0)
and ($record | to_entries | all(.value | version_map)))
' "$metadata" >/dev/null; then
echo "invalid rebuilt_against for $(basename "$pkgdir"): must map architectures to package-version maps"
return 1 return 1
fi fi
if ! jq -e '((.rebuilt_against // {}) | keys) - (.rebuild_on // []) | length == 0' "$metadata" >/dev/null; then if ! jq -e '
(.rebuild_on // []) as $triggers |
(.rebuilt_against // {}) as $record |
if ($record | to_entries | all(.value | type == "string")) then
((($record | keys) - $triggers) | length == 0)
else
($record | to_entries | all((((.value | keys) - $triggers) | length) == 0))
end
' "$metadata" >/dev/null; then
echo "invalid rebuilt_against for $(basename "$pkgdir"): records a package that rebuild_on does not name" echo "invalid rebuilt_against for $(basename "$pkgdir"): records a package that rebuild_on does not name"
return 1 return 1
fi fi
+2 -2
View File
@@ -13,8 +13,8 @@ VALID_ARCHES="x86_64 aarch64"
# in this order; the first entry is the reference architecture that the # in this order; the first entry is the reference architecture that the
# release train observes channels through. Adding an architecture here is the # release train observes channels through. Adding an architecture here is the
# enablement step: the next check-versions tick queues its packages and the # enablement step: the next check-versions tick queues its packages and the
# next auto-release tick builds them. OMARCHY_ARCHES overrides it for a host # next auto-release tick builds them. OMARCHY_ARCHES overrides it for a
# or a one-off run. # one-off run.
PUBLISHED_ARCHES="${OMARCHY_ARCHES:-x86_64}" PUBLISHED_ARCHES="${OMARCHY_ARCHES:-x86_64}"
validate_arch() { validate_arch() {
+1 -1
View File
@@ -7,7 +7,7 @@ Wants=network-online.target
Type=oneshot Type=oneshot
ExecStart=/bin/bash -c 'source /root/.omarchy/build-credentials && /root/omarchy-pkgs/bin/auto-release edge' ExecStart=/bin/bash -c 'source /root/.omarchy/build-credentials && /root/omarchy-pkgs/bin/auto-release edge'
Environment=OMARCHY_STATE_DIR=/root/.state Environment=OMARCHY_STATE_DIR=/root/.state
TimeoutStartSec=7200 TimeoutStartSec=43200
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target
+1 -1
View File
@@ -12,7 +12,7 @@ Type=oneshot
# branch's worktree with OMARCHY_RC_PINS=1. # branch's worktree with OMARCHY_RC_PINS=1.
ExecStart=/bin/bash -c 'source /root/.omarchy/build-credentials && /root/omarchy-pkgs/bin/auto-release rc' ExecStart=/bin/bash -c 'source /root/.omarchy/build-credentials && /root/omarchy-pkgs/bin/auto-release rc'
Environment=OMARCHY_STATE_DIR=/root/.state Environment=OMARCHY_STATE_DIR=/root/.state
TimeoutStartSec=7200 TimeoutStartSec=43200
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target
+1 -1
View File
@@ -7,7 +7,7 @@ Wants=network-online.target
Type=oneshot Type=oneshot
ExecStart=/bin/bash -c 'source /root/.omarchy/build-credentials && /root/omarchy-pkgs/bin/auto-release stable' ExecStart=/bin/bash -c 'source /root/.omarchy/build-credentials && /root/omarchy-pkgs/bin/auto-release stable'
Environment=OMARCHY_STATE_DIR=/root/.state Environment=OMARCHY_STATE_DIR=/root/.state
TimeoutStartSec=7200 TimeoutStartSec=43200
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target