Nightly: fetch the release over 16 connections
The first publish spent 9 min 42 s in the download action for the 4.6 GB stable release; the 11.5 GB edge release would not have fitted in the job's 45 minutes. GitHub's artifact storage limits each connection to a few megabytes a second, and the action uses few. An artifact is one zip at a signed address. test/vm-image/nightly-fetch asks the API for that address and takes it with aria2c over 16 connections, checks the size and unpacks it. The same 4.6 GB artifact came down in 123 s that way. The publish job gets actions: read for it, and checks out test/vm-image for the script. test/unit/nightly-test.sh plays the network with stand-ins: the newest artifact of a name is the one fetched, a failed download is tried again with a new address, a wrong size is refused and not unpacked.
This commit is contained in:
1 parent
bb1ad28887
commit
b01a396e62
4 files changed
+117
-4
No files matched your search
@@ -232,6 +232,10 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 45
|
||||
environment: nightly
|
||||
# actions: read is for finding and fetching the build job's artifact.
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
env:
|
||||
CHANNEL: ${{ matrix.channel }}
|
||||
VERSION: ${{ matrix.version }}
|
||||
@@ -243,10 +247,20 @@ jobs:
|
||||
RCLONE_CONFIG_R2_ENDPOINT: ${{ secrets.R2_ENDPOINT }}
|
||||
BUCKET: omarchy-nightly
|
||||
steps:
|
||||
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
|
||||
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
name: nightly-${{ matrix.channel }}-${{ github.run_id }}
|
||||
path: out
|
||||
sparse-checkout: test/vm-image
|
||||
persist-credentials: false
|
||||
|
||||
# Not the download action: GitHub's storage limits each connection to a
|
||||
# few megabytes a second, and the action took 10 minutes over 4.6 GB.
|
||||
# nightly-fetch takes the same artifact over 16 connections.
|
||||
- name: Fetch the release from the build job
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
sudo apt-get update -qq && sudo apt-get install -y -qq aria2 unzip >/dev/null
|
||||
./test/vm-image/nightly-fetch "$GITHUB_REPOSITORY" "$GITHUB_RUN_ID" "nightly-$CHANNEL-$GITHUB_RUN_ID" out
|
||||
|
||||
- name: rclone
|
||||
run: |
|
||||
|
||||
@@ -50,7 +50,7 @@ jobs:
|
||||
# mirror that drops one file must not fail the tests.
|
||||
sed -i "1i Server = https://mirror.omarchy.org/\$repo/os/\$arch" /etc/pacman.d/mirrorlist
|
||||
for attempt in 1 2 3; do
|
||||
pacman -Syu --noconfirm --needed python parted diffutils jq >/dev/null && break
|
||||
pacman -Syu --noconfirm --needed python parted diffutils jq zip unzip >/dev/null && break
|
||||
[ "$attempt" -lt 3 ] || { echo "::error::the test container could not install its tools"; exit 1; }
|
||||
sleep 15
|
||||
done
|
||||
|
||||
Reference in new issue
Block a user