Make aarch64 a first-class architecture in the scheduled pipeline
One list, PUBLISHED_ARCHES in helpers/paths.sh (default x86_64, overridable with OMARCHY_ARCHES), now drives everything the repository host schedules. check-versions compares PKGBUILDs against each architecture's channel databases and writes one queue per channel and architecture; auto-release works through the queues one architecture at a time, each with its own backoff, so a failing build on one never blocks the other; advance-channel --arch all re-runs an advance for every published architecture and omarchy-release uses it for start and ship, building the pinned pair once per architecture in its rc trigger; the train observes channels through the reference (first) architecture instead of a hard-coded x86_64. Queue and backoff files written under the old per-channel names are treated as x86_64 until consumed. Two things made an aarch64 builder image impossible to create: the keyring bootstrap fetched omarchy-keyring from the target architecture's own channel tree, which does not exist before that architecture has published anything, and the QEMU probe only knew the x86_64-host, aarch64-target case. The keyring (arch=any) now always comes from the x86_64 tree, and the probe compares host and target architectures and runs a container for the target platform. clean-repo grouped versions with a regex that only knew any, x86_64 and i686, so aarch64 packages would never have been pruned.
This commit is contained in:
1 parent
069ffc8c3a
commit
91843ab099
12 files changed
+330
-104
No files matched your search
@@ -30,18 +30,44 @@ The filesystem no longer encodes release policy. Instead:
|
|||||||
## Prerequisites
|
## Prerequisites
|
||||||
### aarch64 Builds (Optional)
|
### aarch64 Builds (Optional)
|
||||||
|
|
||||||
To build ARM64 packages on x86_64, enable QEMU emulation:
|
The repository host builds every architecture it publishes on the same
|
||||||
|
machine. A foreign architecture runs under QEMU user emulation, which
|
||||||
|
`bin/build` sets up on first use (and re-checks after each reboot by
|
||||||
|
actually running a container for the target platform):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Run after each reboot
|
|
||||||
docker run --privileged --rm tonistiigi/binfmt --install arm64
|
|
||||||
|
|
||||||
# Verify
|
# Verify
|
||||||
docker run --rm --platform linux/arm64 alpine:latest uname -m
|
docker run --rm --platform linux/arm64 alpine:latest uname -m
|
||||||
# Should output: aarch64
|
# Should output: aarch64
|
||||||
```
|
```
|
||||||
|
|
||||||
**Note**: aarch64 builds use QEMU and slower than native x86_64 builds.
|
**Note**: emulated builds are much slower than native ones.
|
||||||
|
|
||||||
|
### Published architectures
|
||||||
|
|
||||||
|
`helpers/paths.sh` names the architectures this repository publishes:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PUBLISHED_ARCHES="${OMARCHY_ARCHES:-x86_64}"
|
||||||
|
```
|
||||||
|
|
||||||
|
That list drives the whole scheduled pipeline. `check-versions` compares
|
||||||
|
PKGBUILDs against each architecture's channel databases and writes one queue
|
||||||
|
file per channel and architecture (`.sync-needed-<channel>-<arch>`);
|
||||||
|
`auto-release <channel>` works through the queues one architecture at a
|
||||||
|
time, each with its own backoff (`.build-failed-<channel>-<arch>`), so a
|
||||||
|
failing build on one architecture never holds up the other; and the release
|
||||||
|
train advances channels with `--arch all`, so a channel never moves for one
|
||||||
|
architecture and not another. The first entry is the reference architecture
|
||||||
|
the release train observes channels through.
|
||||||
|
|
||||||
|
Adding an architecture is therefore one change to that list (or
|
||||||
|
`OMARCHY_ARCHES` in the host's build credentials): the next `check-versions`
|
||||||
|
tick queues everything the new architecture lacks, and the next
|
||||||
|
`auto-release` tick starts building it. The builder image bootstraps
|
||||||
|
`omarchy-keyring` from the x86_64 tree for every architecture, so the first
|
||||||
|
build of a new architecture does not depend on a repository that only it can
|
||||||
|
create.
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
@@ -737,10 +763,13 @@ bin/repo release --package my-package
|
|||||||
- Mirrors: mirror.omarchy.org, rackspace, pkgbuild.com
|
- Mirrors: mirror.omarchy.org, rackspace, pkgbuild.com
|
||||||
|
|
||||||
### aarch64
|
### aarch64
|
||||||
- QEMU emulation required on x86_64 hosts (slower)
|
- Built on the repository host like x86_64; under QEMU when the host is x86_64
|
||||||
- Uses Arch Linux ARM repositories
|
- Uses Arch Linux ARM repositories (one mirrorlist for every channel — Arch
|
||||||
|
Linux ARM publishes no dated snapshots to pin a channel's base to)
|
||||||
- Additional repos: `[alarm]`, `[aur]`
|
- Additional repos: `[alarm]`, `[aur]`
|
||||||
- Same workflow, just add `--arch aarch64`
|
- Same workflow, just add `--arch aarch64`; the scheduled pipeline runs it
|
||||||
|
automatically once `aarch64` is in `PUBLISHED_ARCHES`
|
||||||
|
- Packages whose `arch=()` lacks `aarch64` are skipped, not failed
|
||||||
|
|
||||||
### Building for Both Architectures
|
### Building for Both Architectures
|
||||||
|
|
||||||
@@ -791,8 +820,8 @@ The repository includes GitHub workflows and systemd services for automated rele
|
|||||||
All four units run **every 5 minutes**, staggered by a minute each, so a push
|
All four units run **every 5 minutes**, staggered by a minute each, so a push
|
||||||
reaches the mirror in minutes rather than hours:
|
reaches the mirror in minutes rather than hours:
|
||||||
|
|
||||||
1. **check-versions** (`*:0/5`): Pulls latest from git, compares PKGBUILD versions to published versions, creates state files if builds are needed
|
1. **check-versions** (`*:0/5`): Pulls latest from git, compares PKGBUILD versions to published versions for every published architecture, creates one state file per channel and architecture if builds are needed
|
||||||
2. **auto-release-edge** (`*:1/5`): If a state file exists, builds all edge packages that need updates
|
2. **auto-release-edge** (`*:1/5`): For each published architecture with a state file, builds all edge packages that need updates
|
||||||
3. **auto-release-rc** (`*:2/5`): Builds fast-ring packages for rc, from the main checkout like the other two — natively in the rc image, not copied from another channel. The pinned release pair is built separately by `omarchy-release rc` in the `rc` branch worktree
|
3. **auto-release-rc** (`*:2/5`): Builds fast-ring packages for rc, from the main checkout like the other two — natively in the rc image, not copied from another channel. The pinned release pair is built separately by `omarchy-release rc` in the `rc` branch worktree
|
||||||
4. **auto-release-stable** (`*:3/5`): If a state file exists, builds `release_ring=fast` packages for stable and replicates them to rc
|
4. **auto-release-stable** (`*:3/5`): If a state file exists, builds `release_ring=fast` packages for stable and replicates them to rc
|
||||||
|
|
||||||
|
|||||||
+42
-1
@@ -26,6 +26,32 @@ FAST_RING_ONLY=false
|
|||||||
BOOTSTRAP=false
|
BOOTSTRAP=false
|
||||||
PACKAGES=""
|
PACKAGES=""
|
||||||
|
|
||||||
|
# Kept for --arch all, which re-invokes this script per architecture with the
|
||||||
|
# other arguments unchanged (minus the --arch all pair itself).
|
||||||
|
ORIGINAL_ARGS=()
|
||||||
|
skip_next=false
|
||||||
|
prev=""
|
||||||
|
for arg in "$@"; do
|
||||||
|
if [[ "$skip_next" == true ]]; then
|
||||||
|
skip_next=false
|
||||||
|
prev=""
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
if [[ "$arg" == "--arch" ]]; then
|
||||||
|
prev="--arch"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
if [[ "$prev" == "--arch" ]]; then
|
||||||
|
prev=""
|
||||||
|
if [[ "$arg" == "all" ]]; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
ORIGINAL_ARGS+=("--arch" "$arg")
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
ORIGINAL_ARGS+=("$arg")
|
||||||
|
done
|
||||||
|
|
||||||
usage() {
|
usage() {
|
||||||
echo "Usage: $0 --from <channel> --to <channel> [OPTIONS]"
|
echo "Usage: $0 --from <channel> --to <channel> [OPTIONS]"
|
||||||
echo ""
|
echo ""
|
||||||
@@ -36,7 +62,8 @@ usage() {
|
|||||||
echo " or --bootstrap (one-time initial seed)"
|
echo " or --bootstrap (one-time initial seed)"
|
||||||
echo ""
|
echo ""
|
||||||
echo "Options:"
|
echo "Options:"
|
||||||
echo " --arch <arch> Target architecture (x86_64 or aarch64, default: x86_64)"
|
echo " --arch <arch>|all Target architecture (x86_64 or aarch64, default: x86_64);"
|
||||||
|
echo " all = every published architecture, one after another"
|
||||||
echo " --package <names...> Advance only the named package(s)"
|
echo " --package <names...> Advance only the named package(s)"
|
||||||
echo " --fast-ring Restrict to fast-ring packages (stable -> rc parity copy)"
|
echo " --fast-ring Restrict to fast-ring packages (stable -> rc parity copy)"
|
||||||
echo " --bootstrap One-time stable -> rc seed before forward-only enforcement"
|
echo " --bootstrap One-time stable -> rc seed before forward-only enforcement"
|
||||||
@@ -65,6 +92,20 @@ while [[ $# -gt 0 ]]; do
|
|||||||
shift 2
|
shift 2
|
||||||
;;
|
;;
|
||||||
--arch)
|
--arch)
|
||||||
|
if [[ "$2" == "all" ]]; then
|
||||||
|
# A release train advances every published architecture together; a
|
||||||
|
# channel that moved for one and not another would serve mismatched
|
||||||
|
# versions to users on the two. Re-run this script once per
|
||||||
|
# architecture with the same arguments, stopping at the first failure
|
||||||
|
# so the operator sees which architecture is stuck (the advance is
|
||||||
|
# idempotent, so re-running with --arch all resumes where it stopped).
|
||||||
|
shift 2
|
||||||
|
for arch in $(published_arches); do
|
||||||
|
"$0" --arch "$arch" "${ORIGINAL_ARGS[@]}" || exit $?
|
||||||
|
done
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
require_valid_arch "$2"
|
||||||
ARCH="$2"
|
ARCH="$2"
|
||||||
update_arch_paths
|
update_arch_paths
|
||||||
shift 2
|
shift 2
|
||||||
|
|||||||
+93
-57
@@ -1,6 +1,10 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# Run the release workflow for a channel when work is queued.
|
# Run the release workflow for a channel when work is queued.
|
||||||
# Usage: auto-release <edge|rc|stable>
|
# Usage: auto-release <edge|rc|stable> [<arch>|all]
|
||||||
|
#
|
||||||
|
# With no architecture (what the timers pass), every published architecture
|
||||||
|
# is processed in turn, each against its own queue and its own backoff, so a
|
||||||
|
# failing aarch64 build never holds up x86_64 or the other way round.
|
||||||
#
|
#
|
||||||
# Safe to run on a tight schedule. Three guards make that true:
|
# Safe to run on a tight schedule. Three guards make that true:
|
||||||
#
|
#
|
||||||
@@ -21,7 +25,7 @@ source "$BUILD_ROOT/helpers/paths.sh"
|
|||||||
source "$BUILD_ROOT/helpers/lock-helpers.sh"
|
source "$BUILD_ROOT/helpers/lock-helpers.sh"
|
||||||
|
|
||||||
MIRROR="${1:-}"
|
MIRROR="${1:-}"
|
||||||
STATE_DIR="${OMARCHY_STATE_DIR:-/root/.state}"
|
ARCH_ARG="${2:-all}"
|
||||||
|
|
||||||
# Backoff schedule: 10m, 20m, 40m, 80m, 160m, 320m, then hourly-ish forever
|
# Backoff schedule: 10m, 20m, 40m, 80m, 160m, 320m, then hourly-ish forever
|
||||||
# (capped at 6h, the cadence this system ran at before frequent timers).
|
# (capped at 6h, the cadence this system ran at before frequent timers).
|
||||||
@@ -29,8 +33,9 @@ BACKOFF_BASE_SECONDS="${OMARCHY_BACKOFF_BASE:-600}"
|
|||||||
BACKOFF_MAX_SECONDS="${OMARCHY_BACKOFF_MAX:-21600}"
|
BACKOFF_MAX_SECONDS="${OMARCHY_BACKOFF_MAX:-21600}"
|
||||||
|
|
||||||
if [[ -z "$MIRROR" ]]; then
|
if [[ -z "$MIRROR" ]]; then
|
||||||
print_error "Usage: $0 <mirror>"
|
print_error "Usage: $0 <mirror> [<arch>|all]"
|
||||||
echo " mirror: edge, rc, or stable"
|
echo " mirror: edge, rc, or stable"
|
||||||
|
echo " arch: one of $VALID_ARCHES, or all (default) for every published architecture"
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -39,17 +44,13 @@ if [[ "$MIRROR" != "edge" && "$MIRROR" != "rc" && "$MIRROR" != "stable" ]]; then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
STATE_FILE="$STATE_DIR/.sync-needed-$MIRROR"
|
if [[ "$ARCH_ARG" == "all" ]]; then
|
||||||
FAIL_FILE="$STATE_DIR/.build-failed-$MIRROR"
|
ARCHES=$(published_arches)
|
||||||
|
else
|
||||||
# Nothing queued: stay quiet. At a 5-minute cadence this is most invocations,
|
require_valid_arch "$ARCH_ARG"
|
||||||
# and a header for each would bury the runs that matter in the journal.
|
ARCHES="$ARCH_ARG"
|
||||||
if [[ ! -f "$STATE_FILE" ]]; then
|
|
||||||
exit 0
|
|
||||||
fi
|
fi
|
||||||
|
|
||||||
print_header "Processing Sync for $MIRROR"
|
|
||||||
|
|
||||||
# The build inputs are this checkout's contents; its HEAD identifies them.
|
# The build inputs are this checkout's contents; its HEAD identifies them.
|
||||||
current_fingerprint() {
|
current_fingerprint() {
|
||||||
git -C "$BUILD_ROOT" rev-parse HEAD 2>/dev/null || echo "unknown"
|
git -C "$BUILD_ROOT" rev-parse HEAD 2>/dev/null || echo "unknown"
|
||||||
@@ -65,57 +66,92 @@ backoff_seconds() {
|
|||||||
echo "$delay"
|
echo "$delay"
|
||||||
}
|
}
|
||||||
|
|
||||||
FAIL_COUNT=0
|
# Returns 0 when this architecture's queue was processed (or was empty), 1 when
|
||||||
if [[ -f "$FAIL_FILE" ]]; then
|
# the release failed. Backoff and "someone else holds the lock" are not
|
||||||
# shellcheck disable=SC1090
|
# failures: they are this tick deciding to do nothing.
|
||||||
source "$FAIL_FILE" 2>/dev/null || true
|
release_arch() {
|
||||||
FAIL_COUNT="${FAILURE_COUNT:-0}"
|
local arch="$1"
|
||||||
failed_at="${FAILURE_AT:-0}"
|
local state_file fail_file
|
||||||
failed_fingerprint="${FAILURE_FINGERPRINT:-}"
|
state_file=$(sync_queue_file "$MIRROR" "$arch")
|
||||||
|
fail_file=$(sync_fail_file "$MIRROR" "$arch")
|
||||||
|
|
||||||
if [[ "$failed_fingerprint" != "$(current_fingerprint)" ]]; then
|
# A queue written under the pre-architecture name belongs to x86_64.
|
||||||
print_info "Repository changed since the last failure — clearing backoff and retrying"
|
if [[ "$arch" == "x86_64" && ! -f "$state_file" && -f "$(legacy_sync_queue_file "$MIRROR")" ]]; then
|
||||||
rm -f "$FAIL_FILE"
|
state_file=$(legacy_sync_queue_file "$MIRROR")
|
||||||
FAIL_COUNT=0
|
fi
|
||||||
else
|
if [[ "$arch" == "x86_64" && ! -f "$fail_file" && -f "$(legacy_sync_fail_file "$MIRROR")" ]]; then
|
||||||
delay=$(backoff_seconds "$FAIL_COUNT")
|
fail_file=$(legacy_sync_fail_file "$MIRROR")
|
||||||
now=$(date +%s)
|
|
||||||
retry_at=$((failed_at + delay))
|
|
||||||
if ((now < retry_at)); then
|
|
||||||
print_warning "$MIRROR has failed $FAIL_COUNT time(s) on this tree — not retrying until $(date -d "@$retry_at" '+%H:%M:%S' 2>/dev/null || echo "+$((retry_at - now))s")"
|
|
||||||
echo " Push a fix (any new commit clears this), or: rm $FAIL_FILE"
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
print_info "Backoff elapsed — retrying $MIRROR (failure #$((FAIL_COUNT + 1)) if this fails)"
|
|
||||||
fi
|
fi
|
||||||
fi
|
|
||||||
|
|
||||||
# Non-blocking: a build in progress means this tick has nothing to do.
|
# Nothing queued: stay quiet. At a 5-minute cadence this is most
|
||||||
if ! try_release_lock; then
|
# invocations, and a header for each would bury the runs that matter in
|
||||||
holder=$(release_lock_holder)
|
# the journal.
|
||||||
print_info "A release is already running (${holder:-holder unknown}) — skipping this tick"
|
[[ -f "$state_file" ]] || return 0
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
print_info "State file found: $STATE_FILE"
|
print_header "Processing Sync for $MIRROR ($arch)"
|
||||||
print_info "Starting release workflow for $MIRROR..."
|
|
||||||
|
|
||||||
if "$BUILD_ROOT/bin/repo" release --mirror "$MIRROR" --skip-prod-check; then
|
local fail_count=0 failed_at failed_fingerprint delay now retry_at
|
||||||
print_success "Release completed successfully for $MIRROR"
|
if [[ -f "$fail_file" ]]; then
|
||||||
rm -f "$STATE_FILE"
|
FAILURE_COUNT=0 FAILURE_AT=0 FAILURE_FINGERPRINT=""
|
||||||
rm -f "$FAIL_FILE"
|
# shellcheck disable=SC1090
|
||||||
print_success "State file removed: $STATE_FILE"
|
source "$fail_file" 2>/dev/null || true
|
||||||
else
|
fail_count="${FAILURE_COUNT:-0}"
|
||||||
status=$?
|
failed_at="${FAILURE_AT:-0}"
|
||||||
FAIL_COUNT=$((FAIL_COUNT + 1))
|
failed_fingerprint="${FAILURE_FINGERPRINT:-}"
|
||||||
cat >"$FAIL_FILE" <<EOF
|
|
||||||
FAILURE_COUNT=$FAIL_COUNT
|
if [[ "$failed_fingerprint" != "$(current_fingerprint)" ]]; then
|
||||||
|
print_info "Repository changed since the last failure — clearing backoff and retrying"
|
||||||
|
rm -f "$fail_file"
|
||||||
|
fail_count=0
|
||||||
|
else
|
||||||
|
delay=$(backoff_seconds "$fail_count")
|
||||||
|
now=$(date +%s)
|
||||||
|
retry_at=$((failed_at + delay))
|
||||||
|
if ((now < retry_at)); then
|
||||||
|
print_warning "$MIRROR ($arch) has failed $fail_count time(s) on this tree — not retrying until $(date -d "@$retry_at" '+%H:%M:%S' 2>/dev/null || echo "+$((retry_at - now))s")"
|
||||||
|
echo " Push a fix (any new commit clears this), or: rm $fail_file"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
print_info "Backoff elapsed — retrying $MIRROR ($arch) (failure #$((fail_count + 1)) if this fails)"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Non-blocking: a build in progress means this tick has nothing to do. The
|
||||||
|
# lock is reentrant, so once this run holds it the remaining architectures
|
||||||
|
# run under the same acquisition.
|
||||||
|
if ! try_release_lock; then
|
||||||
|
local holder
|
||||||
|
holder=$(release_lock_holder)
|
||||||
|
print_info "A release is already running (${holder:-holder unknown}) — skipping this tick"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
print_info "State file found: $state_file"
|
||||||
|
print_info "Starting release workflow for $MIRROR ($arch)..."
|
||||||
|
|
||||||
|
if "$BUILD_ROOT/bin/repo" release --mirror "$MIRROR" --arch "$arch" --skip-prod-check; then
|
||||||
|
print_success "Release completed successfully for $MIRROR ($arch)"
|
||||||
|
rm -f "$state_file" "$fail_file"
|
||||||
|
print_success "State file removed: $state_file"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
fail_count=$((fail_count + 1))
|
||||||
|
cat >"$fail_file" <<EOF
|
||||||
|
FAILURE_COUNT=$fail_count
|
||||||
FAILURE_AT=$(date +%s)
|
FAILURE_AT=$(date +%s)
|
||||||
FAILURE_FINGERPRINT=$(current_fingerprint)
|
FAILURE_FINGERPRINT=$(current_fingerprint)
|
||||||
EOF
|
EOF
|
||||||
next=$(backoff_seconds "$FAIL_COUNT")
|
local next
|
||||||
print_error "Release failed for $MIRROR (attempt $FAIL_COUNT)"
|
next=$(backoff_seconds "$fail_count")
|
||||||
print_warning "State file retained for retry: $STATE_FILE"
|
print_error "Release failed for $MIRROR ($arch) (attempt $fail_count)"
|
||||||
|
print_warning "State file retained for retry: $state_file"
|
||||||
print_warning "Backing off $((next / 60))m before the next attempt; a new commit retries sooner"
|
print_warning "Backing off $((next / 60))m before the next attempt; a new commit retries sooner"
|
||||||
exit "$status"
|
return 1
|
||||||
fi
|
}
|
||||||
|
|
||||||
|
status=0
|
||||||
|
for arch in $ARCHES; do
|
||||||
|
release_arch "$arch" || status=1
|
||||||
|
done
|
||||||
|
exit "$status"
|
||||||
@@ -164,11 +164,15 @@ mkdir -p "$BUILD_OUTPUT_DIR" "$REPO_DIR" "$SRC_DIR"
|
|||||||
# Check Docker is available
|
# Check Docker is available
|
||||||
check_docker
|
check_docker
|
||||||
|
|
||||||
# Setup QEMU for aarch64 builds on x86_64 hosts
|
# A foreign target architecture runs under QEMU user emulation. Probe by
|
||||||
if [[ "$(uname -m)" == "x86_64" && "$ARCH" == "aarch64" ]]; then
|
# actually running a container for the target platform: that is the only
|
||||||
# Check if QEMU is already working
|
# test that covers both "binfmt not registered" and "registered but broken".
|
||||||
if ! docker run --rm --platform linux/arm64 alpine:3.21 /bin/true >/dev/null 2>&1; then
|
HOST_ARCH=$(uname -m)
|
||||||
print_info "Setting up QEMU for ARM64 emulation..."
|
[[ "$HOST_ARCH" == "arm64" ]] && HOST_ARCH=aarch64
|
||||||
|
if [[ "$HOST_ARCH" != "$ARCH" ]]; then
|
||||||
|
# shellcheck disable=SC2086
|
||||||
|
if ! docker run --rm $(get_platform_arg "$ARCH") alpine:3.21 /bin/true >/dev/null 2>&1; then
|
||||||
|
print_info "Setting up QEMU for $ARCH emulation on this $HOST_ARCH host..."
|
||||||
setup_qemu
|
setup_qemu
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|||||||
+37
-11
@@ -1,6 +1,10 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# Check PKGBUILD versions against published repo versions
|
# Check PKGBUILD versions against published repo versions
|
||||||
# Creates state files for edge and/or stable if any packages need building
|
# Creates a state file per channel and architecture when packages need building
|
||||||
|
#
|
||||||
|
# Usage: check-versions [--pull] [--arch <arch>]
|
||||||
|
# --pull pull the repository first (the scheduled run does this)
|
||||||
|
# --arch <arch> check one architecture; default: every published one
|
||||||
|
|
||||||
set -e
|
set -e
|
||||||
|
|
||||||
@@ -10,11 +14,27 @@ source "$BUILD_ROOT/helpers/paths.sh"
|
|||||||
source "$BUILD_ROOT/helpers/package-metadata.sh"
|
source "$BUILD_ROOT/helpers/package-metadata.sh"
|
||||||
source "$BUILD_ROOT/helpers/lock-helpers.sh"
|
source "$BUILD_ROOT/helpers/lock-helpers.sh"
|
||||||
|
|
||||||
STATE_DIR="${OMARCHY_STATE_DIR:-/root/.state}"
|
|
||||||
ARCH="${ARCH:-x86_64}"
|
|
||||||
PULL=false
|
PULL=false
|
||||||
|
ARCHES=""
|
||||||
|
|
||||||
[[ "${1:-}" == "--pull" ]] && PULL=true
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case $1 in
|
||||||
|
--pull)
|
||||||
|
PULL=true
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--arch)
|
||||||
|
require_valid_arch "$2"
|
||||||
|
ARCHES="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
print_error "Unknown option: $1"
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
[[ -n "$ARCHES" ]] || ARCHES=$(published_arches)
|
||||||
|
|
||||||
mkdir -p "$STATE_DIR"
|
mkdir -p "$STATE_DIR"
|
||||||
|
|
||||||
@@ -147,11 +167,12 @@ check_package() {
|
|||||||
|
|
||||||
check_mirror() {
|
check_mirror() {
|
||||||
local mirror="$1"
|
local mirror="$1"
|
||||||
local state_file="$STATE_DIR/.sync-needed-$mirror"
|
local state_file
|
||||||
|
state_file=$(sync_queue_file "$mirror" "$ARCH")
|
||||||
local needs_build=false
|
local needs_build=false
|
||||||
local packages=()
|
local packages=()
|
||||||
|
|
||||||
print_info "Checking $mirror packages..."
|
print_info "Checking $mirror packages for $ARCH..."
|
||||||
|
|
||||||
while IFS= read -r pkg; do
|
while IFS= read -r pkg; do
|
||||||
local pkgdir="$PKGBUILDS_DIR/$pkg"
|
local pkgdir="$PKGBUILDS_DIR/$pkg"
|
||||||
@@ -168,18 +189,23 @@ check_mirror() {
|
|||||||
# this to name the packages in its start report, which is the difference
|
# this to name the packages in its start report, which is the difference
|
||||||
# between "a build is running" and "your package is in this build".
|
# between "a build is running" and "your package is in this build".
|
||||||
printf '%s\n' "${packages[@]}" >"$state_file"
|
printf '%s\n' "${packages[@]}" >"$state_file"
|
||||||
print_success "${mirror^} needs building (${#packages[@]} packages)"
|
print_success "${mirror^} ($ARCH) needs building (${#packages[@]} packages)"
|
||||||
print_info "Packages: ${packages[*]}"
|
print_info "Packages: ${packages[*]}"
|
||||||
print_info "State file created: $state_file"
|
print_info "State file created: $state_file"
|
||||||
else
|
else
|
||||||
print_info "${mirror^} is up to date"
|
print_info "${mirror^} ($ARCH) is up to date"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo ""
|
echo ""
|
||||||
}
|
}
|
||||||
|
|
||||||
check_mirror edge
|
# One pass per published architecture: the version comparison reads that
|
||||||
check_mirror rc
|
# architecture's channel databases, and each queue is its own file.
|
||||||
check_mirror stable
|
for ARCH in $ARCHES; do
|
||||||
|
update_arch_paths
|
||||||
|
check_mirror edge
|
||||||
|
check_mirror rc
|
||||||
|
check_mirror stable
|
||||||
|
done
|
||||||
|
|
||||||
print_success "Version check complete!"
|
print_success "Version check complete!"
|
||||||
+4
-1
@@ -33,7 +33,10 @@ clean_packages() {
|
|||||||
|
|
||||||
# Extract package name (remove version and architecture)
|
# Extract package name (remove version and architecture)
|
||||||
# Format: name-version-release-arch.pkg.tar.*
|
# Format: name-version-release-arch.pkg.tar.*
|
||||||
local pkgname=$(echo "$pkg" | sed -E 's/-[0-9]+.*-(any|x86_64|i686)\.pkg\.tar\..*//')
|
# The version starts at the first "-<digit>" and neither pkgver nor pkgrel
|
||||||
|
# can contain a hyphen, so the architecture is whatever sits between the
|
||||||
|
# last hyphen and .pkg.tar — any, x86_64, aarch64, and whatever comes next.
|
||||||
|
local pkgname=$(echo "$pkg" | sed -E 's/-[0-9]+.*-[^-]+\.pkg\.tar\..*//')
|
||||||
|
|
||||||
# Add to array
|
# Add to array
|
||||||
if [[ -n "${packages[$pkgname]}" ]]; then
|
if [[ -n "${packages[$pkgname]}" ]]; then
|
||||||
|
|||||||
+18
-6
@@ -28,7 +28,13 @@ ISO_REPO="${OMARCHY_ISO_REPO:-omacom-io/omarchy-iso}"
|
|||||||
DEV_BRANCH="${OMARCHY_DEV_BRANCH:-quattro}"
|
DEV_BRANCH="${OMARCHY_DEV_BRANCH:-quattro}"
|
||||||
|
|
||||||
PKGS_DB_BASE="${OMARCHY_PKGS_DB_BASE:-https://pkgs.omarchy.org}"
|
PKGS_DB_BASE="${OMARCHY_PKGS_DB_BASE:-https://pkgs.omarchy.org}"
|
||||||
RC_DB_URL="$PKGS_DB_BASE/rc/x86_64/omarchy.db.tar.zst"
|
# The train observes channels through the reference architecture (the first
|
||||||
|
# published one). The pair is pinned to one version for every architecture and
|
||||||
|
# built for all of them in one rc trigger, so one architecture's database is
|
||||||
|
# the state of the release; per-architecture drift is a build failure the
|
||||||
|
# scheduled pipeline reports, not something the train has to poll for.
|
||||||
|
OBSERVED_ARCH=$(reference_arch)
|
||||||
|
RC_DB_URL="$PKGS_DB_BASE/rc/$OBSERVED_ARCH/omarchy.db.tar.zst"
|
||||||
|
|
||||||
SRCDEST_DIR="$BUILD_ROOT/.srcdest"
|
SRCDEST_DIR="$BUILD_ROOT/.srcdest"
|
||||||
MIRROR_CLONE="$SRCDEST_DIR/omarchy" # bare mirror (shared with bin/omarchy-pkgs)
|
MIRROR_CLONE="$SRCDEST_DIR/omarchy" # bare mirror (shared with bin/omarchy-pkgs)
|
||||||
@@ -186,7 +192,7 @@ published_version() {
|
|||||||
# A unique query string busts the CDN cache: right after a sync the plain
|
# A unique query string busts the CDN cache: right after a sync the plain
|
||||||
# URL can keep serving the previous db for a while, which reads as "not
|
# URL can keep serving the previous db for a while, which reads as "not
|
||||||
# published yet" to status, the wait loop, and ship's pre-checks.
|
# published yet" to status, the wait loop, and ship's pre-checks.
|
||||||
if ! curl -sf "$PKGS_DB_BASE/$channel/x86_64/omarchy.db.tar.zst?$(date +%s%N)" -o "$tmp"; then
|
if ! curl -sf "$PKGS_DB_BASE/$channel/$OBSERVED_ARCH/omarchy.db.tar.zst?$(date +%s%N)" -o "$tmp"; then
|
||||||
rm -f "$tmp"
|
rm -f "$tmp"
|
||||||
return 2
|
return 2
|
||||||
fi
|
fi
|
||||||
@@ -251,8 +257,12 @@ fi
|
|||||||
git -C /root/omarchy-pkgs-rc fetch origin rc
|
git -C /root/omarchy-pkgs-rc fetch origin rc
|
||||||
git -C /root/omarchy-pkgs-rc reset --hard origin/rc
|
git -C /root/omarchy-pkgs-rc reset --hard origin/rc
|
||||||
cd /root/omarchy-pkgs-rc
|
cd /root/omarchy-pkgs-rc
|
||||||
OMARCHY_RC_PINS=1 OMARCHY_REPO_ROOT=/root/omarchy-pkgs/pkgs.omarchy.org \
|
# The pair is built once per published architecture (helpers/paths.sh on the
|
||||||
bin/repo release --mirror rc --package omarchy omarchy-settings --skip-prod-check
|
# host decides which), so every architecture'"'"'s rc channel carries the pins.
|
||||||
|
for arch in $(BUILD_ROOT=/root/omarchy-pkgs-rc bash -c "source helpers/paths.sh; published_arches"); do
|
||||||
|
OMARCHY_RC_PINS=1 OMARCHY_REPO_ROOT=/root/omarchy-pkgs/pkgs.omarchy.org \
|
||||||
|
bin/repo release --mirror rc --arch "$arch" --package omarchy omarchy-settings --skip-prod-check
|
||||||
|
done
|
||||||
'
|
'
|
||||||
|
|
||||||
trigger_rc_build() {
|
trigger_rc_build() {
|
||||||
@@ -278,10 +288,12 @@ host_advance() { # host_advance <from> <to> [extra args...]
|
|||||||
# against this machine's (likely stale) local tree would be wrong.
|
# against this machine's (likely stale) local tree would be wrong.
|
||||||
if ! resolve_repo_host "$REPO_HOST_OVERRIDE" >/dev/null && ! on_repo_host; then
|
if ! resolve_repo_host "$REPO_HOST_OVERRIDE" >/dev/null && ! on_repo_host; then
|
||||||
print_no_host_help "advance $from -> $to" \
|
print_no_host_help "advance $from -> $to" \
|
||||||
" cd /root/omarchy-pkgs && bin/repo advance --from $from --to $to --skip-prod-check $*"
|
" cd /root/omarchy-pkgs && bin/repo advance --from $from --to $to --arch all --skip-prod-check $*"
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
"$BUILD_ROOT/bin/repo" advance --from "$from" --to "$to" --skip-prod-check "$@"
|
# --arch all: the host advances every architecture it publishes, so a train
|
||||||
|
# never moves a channel for one architecture and not another.
|
||||||
|
"$BUILD_ROOT/bin/repo" advance --from "$from" --to "$to" --arch all --skip-prod-check "$@"
|
||||||
}
|
}
|
||||||
|
|
||||||
wait_for_published() { # wait_for_published <channel> <version> [timeout-seconds]
|
wait_for_published() { # wait_for_published <channel> <version> [timeout-seconds]
|
||||||
|
|||||||
+5
-1
@@ -114,7 +114,11 @@ BUILT_FILES=""
|
|||||||
# What the scheduled version check queued, when it was the one that asked for
|
# What the scheduled version check queued, when it was the one that asked for
|
||||||
# this run. Absent for a manual run, which is fine — the report just omits it.
|
# this run. Absent for a manual run, which is fine — the report just omits it.
|
||||||
QUEUED_PACKAGES=""
|
QUEUED_PACKAGES=""
|
||||||
QUEUE_FILE="${OMARCHY_STATE_DIR:-/root/.state}/.sync-needed-$MIRROR"
|
QUEUE_FILE=$(sync_queue_file "$MIRROR" "$ARCH")
|
||||||
|
# A queue written under the pre-architecture name belongs to x86_64.
|
||||||
|
if [[ "$ARCH" == "x86_64" && ! -s "$QUEUE_FILE" && -s "$(legacy_sync_queue_file "$MIRROR")" ]]; then
|
||||||
|
QUEUE_FILE=$(legacy_sync_queue_file "$MIRROR")
|
||||||
|
fi
|
||||||
[[ -s "$QUEUE_FILE" ]] && QUEUED_PACKAGES=$(grep -c '' "$QUEUE_FILE")
|
[[ -s "$QUEUE_FILE" ]] && QUEUED_PACKAGES=$(grep -c '' "$QUEUE_FILE")
|
||||||
|
|
||||||
if [[ "$DRY_RUN" != true ]]; then
|
if [[ "$DRY_RUN" != true ]]; then
|
||||||
|
|||||||
@@ -237,6 +237,10 @@ echo ""
|
|||||||
|
|
||||||
# --- release timers ----------------------------------------------------------
|
# --- release timers ----------------------------------------------------------
|
||||||
|
|
||||||
|
print_info "Published architectures: $(published_arches | tr '\n' ' ')"
|
||||||
|
echo " (PUBLISHED_ARCHES in helpers/paths.sh, or OMARCHY_ARCHES in $CREDENTIALS)"
|
||||||
|
echo ""
|
||||||
|
|
||||||
TIMERS=(omarchy-check-versions omarchy-auto-release-edge omarchy-auto-release-rc omarchy-auto-release-stable)
|
TIMERS=(omarchy-check-versions omarchy-auto-release-edge omarchy-auto-release-rc omarchy-auto-release-stable)
|
||||||
|
|
||||||
if [[ "$SKIP_TIMERS" == true ]]; then
|
if [[ "$SKIP_TIMERS" == true ]]; then
|
||||||
|
|||||||
+16
-10
@@ -84,15 +84,18 @@ echo ""
|
|||||||
|
|
||||||
# --- queued work -------------------------------------------------------------
|
# --- queued work -------------------------------------------------------------
|
||||||
|
|
||||||
print_info "Queued builds (state files in $STATE_DIR)"
|
print_info "Queued builds (state files in $STATE_DIR; architectures: $(published_arches | tr '\n' ' '))"
|
||||||
queued=false
|
queued=false
|
||||||
for channel in edge rc stable; do
|
for channel in edge rc stable; do
|
||||||
state_file="$STATE_DIR/.sync-needed-$channel"
|
for arch in $(published_arches); do
|
||||||
if [[ -f "$state_file" ]]; then
|
state_file=$(sync_queue_file "$channel" "$arch")
|
||||||
queued=true
|
[[ "$arch" == "x86_64" && ! -f "$state_file" ]] && state_file=$(legacy_sync_queue_file "$channel")
|
||||||
since=$(date -r "$state_file" '+%Y-%m-%d %H:%M:%S' 2>/dev/null || echo "unknown")
|
if [[ -f "$state_file" ]]; then
|
||||||
printf ' • %-7s queued since %s\n' "$channel" "$since"
|
queued=true
|
||||||
fi
|
since=$(date -r "$state_file" '+%Y-%m-%d %H:%M:%S' 2>/dev/null || echo "unknown")
|
||||||
|
printf ' • %-7s %-8s queued since %s\n' "$channel" "$arch" "$since"
|
||||||
|
fi
|
||||||
|
done
|
||||||
done
|
done
|
||||||
[[ "$queued" == false ]] && echo " (nothing queued — all channels up to date)"
|
[[ "$queued" == false ]] && echo " (nothing queued — all channels up to date)"
|
||||||
echo ""
|
echo ""
|
||||||
@@ -101,7 +104,9 @@ echo ""
|
|||||||
# say so plainly: it is queued but deliberately not being retried yet.
|
# say so plainly: it is queued but deliberately not being retried yet.
|
||||||
paused=false
|
paused=false
|
||||||
for channel in edge rc stable; do
|
for channel in edge rc stable; do
|
||||||
fail_file="$STATE_DIR/.build-failed-$channel"
|
for arch in $(published_arches); do
|
||||||
|
fail_file=$(sync_fail_file "$channel" "$arch")
|
||||||
|
[[ "$arch" == "x86_64" && ! -f "$fail_file" ]] && fail_file=$(legacy_sync_fail_file "$channel")
|
||||||
[[ -f "$fail_file" ]] || continue
|
[[ -f "$fail_file" ]] || continue
|
||||||
if [[ "$paused" == false ]]; then
|
if [[ "$paused" == false ]]; then
|
||||||
print_error "Failing builds (backoff active)"
|
print_error "Failing builds (backoff active)"
|
||||||
@@ -122,13 +127,14 @@ for channel in edge rc stable; do
|
|||||||
else
|
else
|
||||||
when="retries on the next tick"
|
when="retries on the next tick"
|
||||||
fi
|
fi
|
||||||
printf ' ✗ %-7s %s consecutive failure(s), %s\n' "$channel" "$FAILURE_COUNT" "$when"
|
printf ' ✗ %-7s %-8s %s consecutive failure(s), %s\n' "$channel" "$arch" "$FAILURE_COUNT" "$when"
|
||||||
printf ' last attempt %s on commit %s\n' \
|
printf ' last attempt %s on commit %s\n' \
|
||||||
"$(date -d "@$FAILURE_AT" '+%Y-%m-%d %H:%M:%S' 2>/dev/null || echo "$FAILURE_AT")" \
|
"$(date -d "@$FAILURE_AT" '+%Y-%m-%d %H:%M:%S' 2>/dev/null || echo "$FAILURE_AT")" \
|
||||||
"${FAILURE_FINGERPRINT:0:12}"
|
"${FAILURE_FINGERPRINT:0:12}"
|
||||||
|
done
|
||||||
done
|
done
|
||||||
if [[ "$paused" == true ]]; then
|
if [[ "$paused" == true ]]; then
|
||||||
echo " Any new commit clears the backoff; or: rm $STATE_DIR/.build-failed-<channel>"
|
echo " Any new commit clears the backoff; or: rm $STATE_DIR/.build-failed-<channel>-<arch>"
|
||||||
echo ""
|
echo ""
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
|||||||
+7
-2
@@ -108,11 +108,16 @@ RUN ln -sf /usr/lib/os-release /etc/os-release && \
|
|||||||
# Setup Omarchy keyring manually before adding repo (avoids keyserver trust issues)
|
# Setup Omarchy keyring manually before adding repo (avoids keyserver trust issues)
|
||||||
# Note: Repository is removed at the end since build scripts add it dynamically.
|
# Note: Repository is removed at the end since build scripts add it dynamically.
|
||||||
# The keyring comes from this image's own channel (the bare /$arch path is a
|
# The keyring comes from this image's own channel (the bare /$arch path is a
|
||||||
# stale legacy layout); %s keeps pacman's $arch literal while MIRROR expands.
|
# stale legacy layout). It is always taken from the x86_64 tree, whatever the
|
||||||
|
# image's own architecture: omarchy-keyring is an arch=any package, and the
|
||||||
|
# x86_64 tree is the one that exists before a new architecture has published
|
||||||
|
# anything. Bootstrapping from the target's own tree would make the first
|
||||||
|
# aarch64 build depend on an aarch64 repository that only that build can
|
||||||
|
# create.
|
||||||
ARG MIRROR=edge
|
ARG MIRROR=edge
|
||||||
RUN pacman-key --recv-keys 40DFB630FF42BCFFB047046CF0134EE680CAC571 --keyserver keys.openpgp.org && \
|
RUN pacman-key --recv-keys 40DFB630FF42BCFFB047046CF0134EE680CAC571 --keyserver keys.openpgp.org && \
|
||||||
pacman-key --lsign-key 40DFB630FF42BCFFB047046CF0134EE680CAC571 && \
|
pacman-key --lsign-key 40DFB630FF42BCFFB047046CF0134EE680CAC571 && \
|
||||||
printf '\n[omarchy]\nSigLevel = Optional\nServer = https://pkgs.omarchy.org/%s/$arch\n' "${MIRROR}" >> /etc/pacman.conf && \
|
printf '\n[omarchy]\nSigLevel = Optional\nServer = https://pkgs.omarchy.org/%s/x86_64\n' "${MIRROR}" >> /etc/pacman.conf && \
|
||||||
pacman -Sy --noconfirm && \
|
pacman -Sy --noconfirm && \
|
||||||
pacman -S --noconfirm omarchy-keyring && \
|
pacman -S --noconfirm omarchy-keyring && \
|
||||||
pacman-key --populate omarchy && \
|
pacman-key --populate omarchy && \
|
||||||
|
|||||||
@@ -5,6 +5,62 @@
|
|||||||
ARCH=${ARCH:-x86_64}
|
ARCH=${ARCH:-x86_64}
|
||||||
MIRROR=${MIRROR:-edge}
|
MIRROR=${MIRROR:-edge}
|
||||||
|
|
||||||
|
# Architectures the tooling knows how to build.
|
||||||
|
VALID_ARCHES="x86_64 aarch64"
|
||||||
|
|
||||||
|
# Architectures this repository PUBLISHES. The scheduled pipeline (version
|
||||||
|
# check, auto-release, channel advance with --arch all) runs once per entry,
|
||||||
|
# in this order; the first entry is the reference architecture that the
|
||||||
|
# release train observes channels through. Adding an architecture here is the
|
||||||
|
# enablement step: the next check-versions tick queues its packages and the
|
||||||
|
# next auto-release tick builds them. OMARCHY_ARCHES overrides it for a host
|
||||||
|
# or a one-off run.
|
||||||
|
PUBLISHED_ARCHES="${OMARCHY_ARCHES:-x86_64}"
|
||||||
|
|
||||||
|
validate_arch() {
|
||||||
|
case " $VALID_ARCHES " in
|
||||||
|
*" $1 "*) return 0 ;;
|
||||||
|
*) return 1 ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
require_valid_arch() {
|
||||||
|
if ! validate_arch "$1"; then
|
||||||
|
echo "Invalid architecture: $1 (must be one of: $VALID_ARCHES)" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
published_arches() {
|
||||||
|
local arch
|
||||||
|
for arch in $PUBLISHED_ARCHES; do
|
||||||
|
require_valid_arch "$arch"
|
||||||
|
echo "$arch"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
reference_arch() {
|
||||||
|
published_arches | head -1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Scheduled-pipeline state, one file per channel and architecture, so one
|
||||||
|
# architecture's queue or backoff never gates another's.
|
||||||
|
STATE_DIR="${OMARCHY_STATE_DIR:-/root/.state}"
|
||||||
|
|
||||||
|
sync_queue_file() { # sync_queue_file <mirror> <arch>
|
||||||
|
echo "$STATE_DIR/.sync-needed-$1-$2"
|
||||||
|
}
|
||||||
|
|
||||||
|
sync_fail_file() { # sync_fail_file <mirror> <arch>
|
||||||
|
echo "$STATE_DIR/.build-failed-$1-$2"
|
||||||
|
}
|
||||||
|
|
||||||
|
# The pre-architecture names, .sync-needed-<mirror> and .build-failed-<mirror>,
|
||||||
|
# meant x86_64. A host upgraded mid-cycle may still hold one; readers treat
|
||||||
|
# it as the x86_64 file until it is consumed.
|
||||||
|
legacy_sync_queue_file() { echo "$STATE_DIR/.sync-needed-$1"; }
|
||||||
|
legacy_sync_fail_file() { echo "$STATE_DIR/.build-failed-$1"; }
|
||||||
|
|
||||||
# Valid package channels, in pipeline order: packages move edge -> rc -> stable
|
# Valid package channels, in pipeline order: packages move edge -> rc -> stable
|
||||||
VALID_MIRRORS="edge rc stable"
|
VALID_MIRRORS="edge rc stable"
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user