Nightly: the retry warning covers latest.json and counts files
The warning was written for the bulk upload only, and counted lines: latest.json is uploaded by a second rclone call, which could still pass on a retry unseen, and a file that failed on two attempts was counted as two uploads. Both calls go through test/vm-image/nightly-rclone, which passes rclone's output and exit status through and names each file once. A stand-in rclone tests it.
This commit is contained in:
1 parent
22ef6e12c5
commit
ed0dfc5052
3 files changed
+52
-10
No files matched your search
@@ -285,17 +285,13 @@ jobs:
|
||||
|
||||
# --immutable: a name that exists with other bytes is an error, not an
|
||||
# overwrite.
|
||||
# An upload that only passes on a retry is said so on the run's page:
|
||||
# the files are compared afterwards either way, but it should not go
|
||||
# unnoticed again.
|
||||
# nightly-rclone is rclone, with a warning on the run's page when an
|
||||
# upload only passed on a retry: the files are compared afterwards either
|
||||
# way, but it should not go unnoticed again.
|
||||
- name: Upload this version
|
||||
run: |
|
||||
set -euo pipefail
|
||||
rclone copy "out/$RELEASE" "r2:$BUCKET/$RELEASE" \
|
||||
--immutable --s3-no-check-bucket --transfers 8 --s3-upload-concurrency 8 --s3-chunk-size 64M --stats 30s --stats-one-line 2>&1 | tee upload.log
|
||||
if grep -q "Failed to copy" upload.log; then
|
||||
echo "::warning::rclone needed a retry: $(grep -c "Failed to copy" upload.log) uploads failed on an earlier attempt"
|
||||
fi
|
||||
./test/vm-image/nightly-rclone copy "out/$RELEASE" "r2:$BUCKET/$RELEASE" \
|
||||
--immutable --s3-no-check-bucket --transfers 8 --s3-upload-concurrency 8 --s3-chunk-size 64M --stats 30s --stats-one-line
|
||||
|
||||
# Fetched the way a user fetches them. It proves what is served, and it
|
||||
# is the first request the CDN sees for each file.
|
||||
@@ -318,7 +314,7 @@ jobs:
|
||||
|
||||
- name: Point latest.json at it
|
||||
run: |
|
||||
rclone copyto "out/$CHANNEL/latest.json" "r2:$BUCKET/$CHANNEL/latest.json" --s3-no-check-bucket
|
||||
./test/vm-image/nightly-rclone copyto "out/$CHANNEL/latest.json" "r2:$BUCKET/$CHANNEL/latest.json" --s3-no-check-bucket
|
||||
served=$(curl -fsSL "$SITE/$CHANNEL/latest.json?run=$GITHUB_RUN_ID" | jq -r .version)
|
||||
[ "$served" = "$VERSION" ] || { echo "::error::latest.json serves version $served, expected $VERSION"; exit 1; }
|
||||
echo "Published [$VERSION]($SITE/$CHANNEL/latest.json)" >>"$GITHUB_STEP_SUMMARY"
|
||||
Reference in new issue
Block a user