Ryan Hughes
537c377fa5
Every pull request now builds the package directories it touches on ephemeral DigitalOcean droplets, and every merge to master publishes the resulting artifacts into the channels each package belongs to. The repository host's timers become the fallback rather than the pipeline. Build (.github/workflows/build-pr.yml) One job per package per architecture, always against edge. The artifact is labelled with the package directory's git tree hash. Tooling (bin/, helpers/, build/) is checked out from the base branch; the PR supplies only pkgbuilds/, so a PR can change what is built, never how. Builds run only for trusted authors: collaborators, .github/VOUCHED.td, or a PR carrying the build-approved label. A single required check, result, aggregates the matrix. Publish (.github/workflows/publish.yml, bin/publish-artifact) One job per merge. It collects the PR artifacts for the merged tree, builds anything that has none, then walks each channel/architecture slot once: pull that database, repo-add every package that belongs in it, upload packages, signatures, then the database. A published filename is immutable; identical bytes under an existing name only gain a database entry, different bytes are refused. Fast-ring packages reach edge, rc and stable in the same run from the same file. Matrix (bin/build-matrix) Package x architecture, with the channels the artifact ships to, decided by package_builds_for_mirror so CI and the host agree. arch=any packages build once and land in every architecture database. Builder (build/build.sh, bin/build, build/Dockerfile) With no local published tree, plan against and resolve from the public channel over HTTPS. PACKAGER is set to Omarchy. rclone is in the image. Runners (ci/) A controller droplet polls GitHub with curl and creates one g5 droplet per queued job from cloud-init, deleting them when off or over-age. Builders carry QEMU with credential support for aarch64. Operator SSH keys come from GitHub. tests/controller.sh and tests/publish-artifact.sh cover the decisions against fixtures and real makepkg output. Tests run on pull requests only; branch protection requires result, self-tests and build-isolation with up-to-date branches.
75 lines
4.7 KiB
Bash
Executable File
75 lines
4.7 KiB
Bash
Executable File
#!/bin/bash
|
|
# Self-test for bin/publish-artifact against a local directory as the remote.
|
|
# Needs repo-add, gpg, rclone, bsdtar (run in the Arch builder/test container).
|
|
set -euo pipefail
|
|
ROOT=$(realpath "${BASH_SOURCE[0]%/*}/..")
|
|
T=$(mktemp -d); chmod 755 "$T"; trap 'rm -rf "$T"' EXIT
|
|
REMOTE="$T/r2"; mkdir -p "$REMOTE"
|
|
|
|
# throwaway signing key
|
|
export GNUPGHOME="$T/g"; mkdir -m700 "$GNUPGHOME"
|
|
gpg --batch --quiet --passphrase '' --quick-gen-key 'Test <t@t>' ed25519 sign 0 2>/dev/null
|
|
export GPG_PRIVATE_KEY=$(gpg --batch --armor --export-secret-keys 'Test <t@t>') GPG_PASSPHRASE=''
|
|
unset GNUPGHOME
|
|
|
|
# minimal real packages via makepkg
|
|
mkpkg() { # mkpkg <name> <pkgrel> <arch> [payload]
|
|
local d="$T/src/$1-$2${4:+-$4}"; mkdir -p "$d"; cd "$d"
|
|
printf 'pkgname=%s\npkgver=1.0\npkgrel=%s\narch=(%s)\npackage(){ install -Dm644 /dev/null "$pkgdir/usr/share/%s-%s"; echo "%s" > "$pkgdir/usr/share/%s-%s"; }\n' "$1" "$2" "$3" "$1" "$2" "${4:-payload}" "$1" "$2" > PKGBUILD
|
|
# CARCH so the PKGINFO records the requested arch (--ignorearch would
|
|
# stamp the host's).
|
|
# makepkg refuses to run as root (the CI test container does); build the
|
|
# fixture as an unprivileged user in that case.
|
|
if (( EUID == 0 )); then
|
|
id -u fixture >/dev/null 2>&1 || useradd -m fixture
|
|
chmod 755 "$T/src"; chown -R fixture "$d"
|
|
runuser -u fixture -- env CARCH=$3 makepkg -f --nodeps --ignorearch >/dev/null 2>&1
|
|
else
|
|
CARCH=$3 makepkg -f --nodeps --ignorearch >/dev/null 2>&1
|
|
fi
|
|
ls "$d"/*.pkg.tar.zst
|
|
}
|
|
A1=$(mkpkg alpha 1 any); A2=$(mkpkg alpha 2 any); B1=$(mkpkg beta 1 x86_64); C1=$(mkpkg gamma 1 aarch64)
|
|
|
|
pub() { "$ROOT/bin/publish-artifact" --remote "$REMOTE" --mirror edge --arch x86_64 "$@" >"$T/out" 2>&1; }
|
|
entries() { tar -tf "$REMOTE/edge/x86_64/omarchy.db.tar.zst" | grep '/$' | sort | tr '\n' ' '; }
|
|
pass() { echo "PASS: $1"; }
|
|
fail() { echo "FAIL: $1"; cat "$T/out"; exit 1; }
|
|
|
|
pub "$A1" && [[ "$(entries)" == "alpha-1.0-1/ " ]] && [[ -f "$REMOTE/edge/x86_64/$(basename "$A1").sig" ]] \
|
|
&& pass "first publish creates db with one entry and a signature" || fail "first publish"
|
|
|
|
sum_before=$(sha256sum "$REMOTE/edge/x86_64/$(basename "$A1")")
|
|
pub "$B1" && [[ "$(entries)" == "alpha-1.0-1/ beta-1.0-1/ " ]] && [[ "$(sha256sum "$REMOTE/edge/x86_64/$(basename "$A1")")" == "$sum_before" ]] \
|
|
&& pass "second package added incrementally; first file untouched" || fail "incremental add"
|
|
|
|
pub "$A2" && [[ "$(entries)" == "alpha-1.0-2/ beta-1.0-1/ " ]] && [[ -f "$REMOTE/edge/x86_64/$(basename "$A1")" ]] \
|
|
&& pass "new pkgrel replaces the db entry, old file remains on remote" || fail "replace entry"
|
|
|
|
# Same bytes again: allowed, idempotent (this is how a fast-ring artifact
|
|
# reaches rc and stable after edge, and how a re-run recovers).
|
|
pub "$A2" && grep -q 'identical bytes' "$T/out" && [[ "$(entries)" == "alpha-1.0-2/ beta-1.0-1/ " ]] \
|
|
&& pass "identical bytes under an existing name: accepted, db unchanged" || fail "identical republish"
|
|
|
|
# Orphan repair: a file that reached the remote but whose db entry was lost
|
|
# (a concurrent publish overwrote the db) is fixed by publishing it again.
|
|
( cd "$REMOTE/edge/x86_64" && repo-remove --quiet omarchy.db.tar.zst alpha >/dev/null 2>&1 )
|
|
[[ "$(entries)" == "beta-1.0-1/ " ]] || fail "fixture: could not drop alpha from the db"
|
|
pub "$A2" && [[ "$(entries)" == "alpha-1.0-2/ beta-1.0-1/ " ]] \
|
|
&& pass "orphaned file regains its db entry on republish" || fail "orphan repair"
|
|
|
|
# Different bytes under an existing name: refused. Build alpha-2 again with
|
|
# a different payload (makepkg is reproducible, so the content must change).
|
|
A2b=$(mkpkg alpha 2 any different-payload)
|
|
[[ "$(md5sum < "$A2")" != "$(md5sum < "$A2b")" ]] || { echo "fixture: rebuilt package is byte-identical, cannot test"; exit 1; }
|
|
if pub "$A2b"; then fail "different bytes under same filename should refuse"; else grep -q 'DIFFERENT bytes' "$T/out" && pass "different bytes under an existing name refused" || fail "wrong refusal reason"; fi
|
|
|
|
if pub "$C1"; then fail "aarch64 package into x86_64 should refuse"; else grep -q 'publishing to x86_64' "$T/out" && pass "wrong-arch package refused" || fail "wrong-arch reason"; fi
|
|
|
|
cp "$B1" "$T/renamed-1.0-1-x86_64.pkg.tar.zst"
|
|
if pub "$T/renamed-1.0-1-x86_64.pkg.tar.zst"; then fail "filename/PKGINFO mismatch should refuse"; else grep -q 'does not match PKGINFO' "$T/out" && pass "filename must match PKGINFO" || fail "mismatch reason"; fi
|
|
|
|
# db must verify: pacman can read it and each package's signature checks
|
|
gpg --batch --quiet --import <<<"$GPG_PRIVATE_KEY" 2>/dev/null || true
|
|
( cd "$REMOTE/edge/x86_64" && for f in *.pkg.tar.zst; do gpg --batch --quiet --verify "$f.sig" "$f" 2>/dev/null || { echo "FAIL: signature $f"; exit 1; }; done ) && pass "all signatures verify"
|