Nightly: the retry warning covers latest.json and counts files

The warning was written for the bulk upload only, and counted lines:
latest.json is uploaded by a second rclone call, which could still pass
on a retry unseen, and a file that failed on two attempts was counted as
two uploads.

Both calls go through test/vm-image/nightly-rclone, which passes
rclone's output and exit status through and names each file once. A
stand-in rclone tests it.
This commit is contained in:
Emir Beganovic committed 2026-10-09 22:36:31 +02:00
1 parent 22ef6e12c5
commit ed0dfc5052
3 files changed
+52 -10

No files matched your search

+19
View File
@@ -0,0 +1,19 @@
#!/bin/bash
# Runs rclone for the publish job and says when it only got there on a retry.
#
# nightly-rclone <rclone arguments...>
#
# rclone tries a failed transfer again by itself, and a run that passes on the
# second attempt looks like any other. This passes rclone's output and exit
# status through, and adds a warning for the run's page naming the files that
# failed on an earlier attempt.
set -uo pipefail
log=$(mktemp)
trap 'rm -f "$log"' EXIT
rclone "$@" 2>&1 | tee "$log"
status=${PIPESTATUS[0]}
files=$(sed -nE 's/^.*ERROR : (.*): Failed to copy: .*$/\1/p' "$log" | sort -u)
if [[ $status -eq 0 && -n $files ]]; then
echo "::warning::rclone needed a retry for $(wc -l <<<"$files") file(s): $(paste -sd' ' <<<"$files")"
fi
exit "$status"