Skip to content

Release workflows should retry svn checkout in case of failure #3393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pingtimeout merged 3 commits into from
Jan 12, 2026
Merged

Release workflows should retry svn checkout in case of failure #3393

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e8aa117
Retry svn checkout in case of ASF rate limiting
pingtimeout Jan 8, 2026
4bd9ef0
Delete svn folder if exists after failure
pingtimeout Jan 8, 2026
aca99e8
Move retry function to _exec.sh
pingtimeout Jan 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/workflows/release-3-build-and-publish-artifacts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,9 @@ jobs:
source "${LIBS_DIR}/_exec.sh"

dist_dev_dir=${RELEASEY_DIR}/polaris-dist-dev
exec_process svn checkout --username "$SVN_USERNAME" --password "$SVN_PASSWORD" --non-interactive "${APACHE_DIST_URL}${APACHE_DIST_PATH}" "${dist_dev_dir}"

# Retry logic for SVN checkout (Apache SVN can have transient connectivity issues)
Copy link
Member

@snazy snazy Jan 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind moving the retry-loop to a new function like exec_process_with_retries or so?

Copy link
Contributor Author

@pingtimeout pingtimeout Jan 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Implementation part is done, I just need to re-test.

Copy link
Contributor Author

@pingtimeout pingtimeout Jan 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmation that it still works : https://github.com/pingtimeout/polaris/actions/runs/20915713316/job/60088667452

Note that the workflow failed because of an unrelated issue: I am using a temporary GPG key that is not published to ubuntu keyserver. So the gradle task that closes the staging repository fails and that in turn fails the release process.

exec_process_with_retries 5 60 "${dist_dev_dir}" svn checkout --username "$SVN_USERNAME" --password "$SVN_PASSWORD" --non-interactive "${APACHE_DIST_URL}${APACHE_DIST_PATH}" "${dist_dev_dir}"

version_dir="${dist_dev_dir}/${version_without_rc}"
exec_process mkdir -p "${version_dir}"
Expand Down Expand Up @@ -393,7 +395,9 @@ jobs:
source "${LIBS_DIR}/_exec.sh"

dist_dev_dir=${RELEASEY_DIR}/polaris-dist-dev
exec_process svn checkout --username "$SVN_USERNAME" --password "$SVN_PASSWORD" --non-interactive "${APACHE_DIST_URL}${APACHE_DIST_PATH}" "${dist_dev_dir}"

# Retry logic for SVN checkout (Apache SVN can have transient connectivity issues)
exec_process_with_retries 5 60 "${dist_dev_dir}" svn checkout --username "$SVN_USERNAME" --password "$SVN_PASSWORD" --non-interactive "${APACHE_DIST_URL}${APACHE_DIST_PATH}" "${dist_dev_dir}"

exec_process mkdir -p "${dist_dev_dir}/helm-chart/${version_without_rc}"
exec_process cp helm/polaris-${version_without_rc}.tgz* "${dist_dev_dir}/helm-chart/${version_without_rc}/"
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/release-4-publish-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,8 @@ jobs:
release_helm_dir="${RELEASEY_DIR}/polaris-dist-release-helm-chart"
release_helm_url="${APACHE_DIST_URL}/release/incubator/polaris/helm-chart"

exec_process svn checkout --username "$SVN_USERNAME" --password "$SVN_PASSWORD" --non-interactive "${release_helm_url}" "${release_helm_dir}"
# Retry logic for SVN checkout (Apache SVN can have transient connectivity issues)
exec_process_with_retries 5 60 "${release_helm_dir}" svn checkout --username "$SVN_USERNAME" --password "$SVN_PASSWORD" --non-interactive "${release_helm_url}" "${release_helm_dir}"

exec_process cd "${release_helm_dir}"
exec_process helm repo index .
Expand Down
36 changes: 36 additions & 0 deletions releasey/libs/_exec.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,42 @@ function exec_process {
fi
}

# Executes a command with retry logic
# Args:
# $1: max_attempts - Maximum number of retry attempts
# $2: sleep_duration - Seconds to wait between retries
# $3: cleanup_path - Path to clean up before retrying (can be empty)
# $@: Command and arguments to execute
function exec_process_with_retries {
if [[ $# -lt 4 ]]; then
echo "ERROR: exec_process_with_retries requires: max_attempts sleep_duration cleanup_path command [args...]"
exit 1
fi

local max_attempts="${1}"
local sleep_duration="${2}"
local cleanup_path="${3}"
shift 3

local attempt=1
while true; do
if exec_process "$@"; then
break
fi
if [[ $attempt -ge $max_attempts ]]; then
echo "ERROR: Command failed after ${max_attempts} attempts: ${*}"
exit 1
fi
echo "WARNING: Command failed (attempt ${attempt}/${max_attempts}), retrying in ${sleep_duration} seconds..."
# Clean up any partial state before retrying
if [[ -n "${cleanup_path}" && -e "${cleanup_path}" ]]; then
rm -rf "${cleanup_path}"
fi
sleep "${sleep_duration}"
((attempt++))
done
}

function calculate_sha512 {
local source_file="$1"
local target_file="${source_file}.sha512"
Expand Down