Emir Beganovic
f9d184f2bc
The image was a folder, vm/, holding base.qcow2 and what boots it. It is now one file next to the ISO, with the ISO's name and its own checksum file, the way Ubuntu, Debian, Arch, Fedora and CentOS publish theirs: edge/<version>/omarchy-edge-<version>.qcow2 stable/<version>/<run>/omarchy-<version>.qcow2 The image boots by itself through UEFI. What a direct boot needs, the kernel, initramfs, kernel arguments and vm-boot, is in an unpacked/ folder beside it, Ubuntu's name for the same thing. The firmware variables are no longer published: a boot with the firmware's defaults finds the disk. latest.json follows: vm.url, vm.sha256 and vm.size are the image, as url, sha256 and size are the ISO; vm.unpacked and vm.unpacked_files are the folder. vm-boot takes the image file. The publish job compares every file it uploaded, whatever the layout. README, tests and the workflow's header say the same.
281 lines
12 KiB
YAML
281 lines
12 KiB
YAML
name: Nightly ISO Build
|
|
|
|
# Every night, for two channels: installs an ISO unattended and boots what it
|
|
# installed (the install test), turns that installed disk into a VM image, and
|
|
# publishes it with a latest.json saying what is in it.
|
|
#
|
|
# edge the ISO is built here, from the edge channel: its packages, its
|
|
# mirrors and the edge builds of Omarchy (omarchy-dev)
|
|
# stable the ISO is the last release: omacom/omarchy's latest release names
|
|
# the version, iso.omarchy.org has the ISO, its signature is checked.
|
|
# Done once for each release, skipped on the nights after.
|
|
#
|
|
# https://nightly.omarchy.org/<channel>/latest.json
|
|
# https://nightly.omarchy.org/edge/<version>/omarchy-edge-<version>.iso
|
|
# https://nightly.omarchy.org/edge/<version>/omarchy-edge-<version>.qcow2
|
|
# https://nightly.omarchy.org/edge/<version>/unpacked/ kernel, initramfs, vm-boot
|
|
# https://nightly.omarchy.org/stable/<version>/<run>/omarchy-<version>.qcow2
|
|
# https://nightly.omarchy.org/stable/<version>/<run>/unpacked/
|
|
#
|
|
# Nothing is published unless the ISO installed and the image passed its
|
|
# checks. A version's files are written once and never replaced: the CDN keeps
|
|
# them for a year. Only latest.json changes, and it is written last.
|
|
#
|
|
# No pull_request trigger: this workflow holds the key to the bucket, so it
|
|
# never runs for a fork. To try a change, run it by hand on a branch; that
|
|
# builds and checks everything and publishes nothing.
|
|
|
|
on:
|
|
schedule:
|
|
- cron: "1 0 * * *" # Daily at 00:01 UTC
|
|
workflow_dispatch:
|
|
inputs:
|
|
publish:
|
|
description: "Publish to nightly.omarchy.org (only from quattro)"
|
|
type: boolean
|
|
default: false
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: nightly-${{ github.ref }}
|
|
cancel-in-progress: false
|
|
|
|
env:
|
|
SITE: https://nightly.omarchy.org
|
|
|
|
jobs:
|
|
# Which channels this run makes, under which version, and where each goes:
|
|
# test/vm-image/nightly-plan decides, from what is fetched here.
|
|
plan:
|
|
name: Plan
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 5
|
|
outputs:
|
|
channels: ${{ steps.plan.outputs.channels }}
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
|
|
|
|
- id: plan
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
run: |
|
|
set -euo pipefail
|
|
release=$(gh api repos/omacom/omarchy/releases/latest --jq .tag_name)
|
|
published=$(curl -fsL "$SITE/stable/latest.json?run=$GITHUB_RUN_ID" | jq -r .version || true)
|
|
channels=$(./test/vm-image/nightly-plan "$(date -u +%Y.%m.%d)" "${{ github.run_number }}" "${{ github.run_attempt }}" \
|
|
"$GITHUB_EVENT_NAME" "$release" "$published")
|
|
echo "channels=$channels" | tee -a "$GITHUB_OUTPUT"
|
|
|
|
build:
|
|
name: ${{ matrix.channel }} ${{ matrix.version }}
|
|
needs: plan
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include: ${{ fromJSON(needs.plan.outputs.channels) }}
|
|
runs-on: namespace-profile-linux-amd64-16x32
|
|
timeout-minutes: 90
|
|
env:
|
|
CHANNEL: ${{ matrix.channel }}
|
|
VERSION: ${{ matrix.version }}
|
|
RELEASE: ${{ matrix.release }}
|
|
steps:
|
|
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
|
|
|
|
- name: Runner
|
|
run: |
|
|
uname -srm
|
|
echo "$(nproc) CPUs, $(free -h | awk 'NR==2 {print $2}') memory"
|
|
grep -m1 'model name' /proc/cpuinfo || true
|
|
echo "LSMs: $(cat /sys/kernel/security/lsm 2>/dev/null || echo unreadable)"
|
|
ls -l /dev/kvm
|
|
|
|
- name: Check disk space
|
|
run: |
|
|
df -h /
|
|
avail_gb=$(df -BG --output=avail / | tail -1 | tr -dc 0-9)
|
|
[ "$avail_gb" -ge 50 ] || { echo "::error::only ${avail_gb} GB free, the build, the install test and the image need ~50 GB"; exit 1; }
|
|
|
|
- uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
|
|
if: matrix.channel == 'edge'
|
|
|
|
# No package cache, on purpose: every package the ISO is made of is
|
|
# fetched from the mirrors again each night, so one that has gone
|
|
# missing there fails this build instead of being served from a cache.
|
|
- name: Build the edge ISO
|
|
if: matrix.channel == 'edge'
|
|
run: NO_BOOT_OFFER=1 ./bin/omarchy-iso-make --edge
|
|
|
|
# Good only if the signature is the release key's, the one in this repo.
|
|
- name: Download the released ISO and check its signature
|
|
if: matrix.channel == 'stable'
|
|
run: |
|
|
set -euo pipefail
|
|
mkdir -p release
|
|
iso=omarchy-$VERSION.iso
|
|
curl -fSL --retry 3 -o "release/$iso" "https://iso.omarchy.org/$iso"
|
|
curl -fsSL --retry 3 -o "release/$iso.sig" "https://iso.omarchy.org/$iso.sig"
|
|
home=$(mktemp -d)
|
|
gpg --homedir "$home" --quiet --batch --import builder/omarchy.gpg
|
|
key=$(gpg --homedir "$home" --batch --with-colons --list-keys | awk -F: '$1 == "fpr" { print $10; exit }')
|
|
gpg --homedir "$home" --batch --status-fd 1 --verify "release/$iso.sig" "release/$iso" 2>/dev/null |
|
|
grep -q "^\[GNUPG:\] VALIDSIG $key " || { echo "::error::$iso is not signed by $key"; exit 1; }
|
|
echo "$iso is signed by $key"
|
|
rm -rf "$home" "release/$iso.sig"
|
|
|
|
# The same install test a pull request gets (install-test.yml), then the
|
|
# image: the installed disk that test leaves behind, sealed and checked.
|
|
- name: Install unattended, boot the installed system, seal it
|
|
run: |
|
|
sudo chmod 0666 /dev/kvm
|
|
docker run --rm --device /dev/kvm -v "$PWD:/w" -w /w \
|
|
-e OMARCHY_INTEGRATION_DISK_CACHE=none \
|
|
archlinux:latest bash -euo pipefail -c '
|
|
# pacman 7 fails every sync on a kernel without Landlock, which
|
|
# these runners have; builder/build-iso.sh does the same.
|
|
if ! grep -qw landlock /sys/kernel/security/lsm 2>/dev/null; then
|
|
printf "#!/bin/sh\nexec /usr/bin/pacman --disable-sandbox \"\$@\"\n" >/usr/local/bin/pacman
|
|
chmod 0755 /usr/local/bin/pacman
|
|
fi
|
|
pacman -Syu --noconfirm --needed qemu-system-x86 qemu-hw-display-virtio-gpu qemu-hw-display-virtio-vga qemu-img edk2-ovmf socat \
|
|
imagemagick tesseract tesseract-data-eng mtools dosfstools jq openssh openssl python curl binutils >/dev/null
|
|
# test/integration installs its tools through Omarchy'"'"'s helper; they are installed above.
|
|
printf "#!/bin/sh\nexit 0\n" >/usr/local/bin/omarchy-pkg-add
|
|
chmod 0755 /usr/local/bin/omarchy-pkg-add
|
|
qemu-system-x86_64 --version | head -1
|
|
./test/integration release/*.iso boot --no-preview --memory 8192 --timeout 1800
|
|
|
|
base=$(ls -d test-runs/*-integration)
|
|
./test/vm-image/vm-seal "$base" sealed
|
|
./test/vm-image/vm-verify sealed | tee vm-verify.txt
|
|
./test/vm-image/vm-boot sealed "pacman -Qq omarchy omarchy-dev 2>/dev/null | xargs pacman -Q; pacman -Qo /usr/lib/modules/\$(uname -r)/pkgbase; uname -r" >vm-packages.txt
|
|
cat vm-packages.txt
|
|
'
|
|
|
|
# The container ran as root; what it wrote belongs to root until here.
|
|
- name: Hand the results to the runner's user
|
|
if: always()
|
|
run: |
|
|
mkdir -p test-runs sealed
|
|
sudo chown -R "$(id -u):$(id -g)" test-runs sealed release vm-*.txt 2>/dev/null || true
|
|
|
|
- name: Lay out the release and describe it
|
|
env:
|
|
ISO_COMMIT: ${{ github.sha }}
|
|
run: |
|
|
set -euo pipefail
|
|
./test/vm-image/nightly-layout "$CHANNEL" "$VERSION" "$RELEASE" release/*.iso sealed vm-packages.txt out
|
|
dir=out/$RELEASE
|
|
|
|
{
|
|
echo "## Nightly $CHANNEL $VERSION"
|
|
echo
|
|
echo "| File | Size |"
|
|
echo "| --- | ---: |"
|
|
(cd "$dir" && find . -type f -printf '| `%P` | %s |\n' | sort) | awk -F'|' '{ printf "|%s| %.0f MiB |\n", $2, $3 / 1048576 }'
|
|
echo
|
|
echo '```'
|
|
cat vm-verify.txt
|
|
echo '```'
|
|
} >>"$GITHUB_STEP_SUMMARY"
|
|
|
|
- name: Keep the release for the publish job
|
|
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
|
|
with:
|
|
name: nightly-${{ matrix.channel }}-${{ github.run_id }}
|
|
path: out/
|
|
compression-level: 0
|
|
retention-days: 2
|
|
if-no-files-found: error
|
|
|
|
- name: Keep the install test's logs
|
|
if: always()
|
|
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v6
|
|
with:
|
|
name: nightly-install-test-${{ matrix.channel }}-${{ github.run_id }}-${{ github.run_attempt }}
|
|
path: |
|
|
test-runs/*/omarchy-install-timing.json
|
|
test-runs/*/omarchy-install.log
|
|
test-runs/*/first-boot-systemd-analyze.txt
|
|
test-runs/*/runs/**/*.png
|
|
test-runs/*/runs/**/*.txt
|
|
test-runs/*/runs/**/*.json
|
|
test-runs/*/runs/**/*.log
|
|
vm-verify.txt
|
|
vm-packages.txt
|
|
if-no-files-found: ignore
|
|
retention-days: 14
|
|
|
|
# The only job that holds the bucket's key, and it builds nothing: it copies
|
|
# what the job above made, checks what the CDN then serves, and only then
|
|
# points latest.json at it.
|
|
# One channel failing does not hold back the other: each is published if
|
|
# its own build left a release behind.
|
|
publish:
|
|
name: Publish ${{ matrix.channel }}
|
|
needs: [plan, build]
|
|
if: ${{ !cancelled() && github.ref == 'refs/heads/quattro' && (github.event_name == 'schedule' || inputs.publish) }}
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include: ${{ fromJSON(needs.plan.outputs.channels) }}
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
environment: nightly
|
|
env:
|
|
CHANNEL: ${{ matrix.channel }}
|
|
VERSION: ${{ matrix.version }}
|
|
RELEASE: ${{ matrix.release }}
|
|
RCLONE_CONFIG_R2_TYPE: s3
|
|
RCLONE_CONFIG_R2_PROVIDER: Cloudflare
|
|
RCLONE_CONFIG_R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }}
|
|
RCLONE_CONFIG_R2_SECRET_ACCESS_KEY: ${{ secrets.R2_SECRET_ACCESS_KEY }}
|
|
RCLONE_CONFIG_R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
|
|
BUCKET: omarchy-nightly
|
|
steps:
|
|
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
|
with:
|
|
name: nightly-${{ matrix.channel }}-${{ github.run_id }}
|
|
path: out
|
|
|
|
- name: rclone
|
|
run: |
|
|
[ -n "$RCLONE_CONFIG_R2_ACCESS_KEY_ID" ] || { echo "::error::the nightly environment has no bucket key"; exit 1; }
|
|
sudo apt-get update -qq && sudo apt-get install -y -qq rclone >/dev/null
|
|
rclone version | head -1
|
|
|
|
# --immutable: a name that exists with other bytes is an error, not an
|
|
# overwrite.
|
|
- name: Upload this version
|
|
run: |
|
|
rclone copy "out/$RELEASE" "r2:$BUCKET/$RELEASE" \
|
|
--immutable --s3-no-check-bucket --transfers 8 --s3-upload-concurrency 8 --s3-chunk-size 64M --stats 30s --stats-one-line
|
|
|
|
# Fetched the way a user fetches them. It proves what is served, and it
|
|
# is the first request the CDN sees for each file.
|
|
- name: Check what the CDN serves
|
|
run: |
|
|
set -euo pipefail
|
|
dir=out/$RELEASE
|
|
check() { # url, expected sha256
|
|
got=$(curl -fsSL --retry 3 "$1" | sha256sum | cut -d" " -f1)
|
|
[ "$got" = "$2" ] || { echo "::error::$1 serves $got, expected $2"; exit 1; }
|
|
echo "ok $1"
|
|
}
|
|
# Every file of the version, the checksum files too.
|
|
(cd "$dir" && find . -type f -printf '%P\n' | sort) >files.txt
|
|
[ -s files.txt ] || { echo "::error::nothing to check under $dir"; exit 1; }
|
|
while read -r file; do
|
|
check "$SITE/$RELEASE/$file" "$(sha256sum "$dir/$file" | cut -d" " -f1)"
|
|
done <files.txt
|
|
echo "$(wc -l <files.txt) files published and checked"
|
|
|
|
- name: Point latest.json at it
|
|
run: |
|
|
rclone copyto "out/$CHANNEL/latest.json" "r2:$BUCKET/$CHANNEL/latest.json" --s3-no-check-bucket
|
|
served=$(curl -fsSL "$SITE/$CHANNEL/latest.json?run=$GITHUB_RUN_ID" | jq -r .version)
|
|
[ "$served" = "$VERSION" ] || { echo "::error::latest.json serves version $served, expected $VERSION"; exit 1; }
|
|
echo "Published [$VERSION]($SITE/$CHANNEL/latest.json)" >>"$GITHUB_STEP_SUMMARY"
|