Add configure-aws-credentials composite workflow with release support

[ScottTodd]

Motivation

For multi-arch release workflows (#3334), I want to reuse the existing workflows like .github/workflows/multi_arch_build_portable_linux_artifacts.yml but have them upload to therock-dev-artifacts, therock-nightly-artifacts, etc. instead of therock-ci-artifacts. The _retrieve_bucket_info() function in build_tools/_therock_utils/workflow_outputs.py already selects the right bucket based on the RELEASE_TYPE environment variable, but this also requires switching the role from

-role-to-assume: arn:aws:iam::692859939525:role/therock-ci
+role-to-assume: arn:aws:iam::692859939525:role/therock-dev

I decided to take the chance to translate our s3_buckets.md into a s3_buckets.py file with two users:

• The existing _retrieve_bucket_info() function in build_tools/_therock_utils/workflow_outputs.py
• A new build_tools/github_actions/write_artifacts_bucket_info.py script used by a .github/actions/configure_aws_artifacts_credentials/action.yml composite workflow that handles looking up the role and calling aws-actions/configure-aws-credentials (with special-characters-workaround set on Windows)

Technical Details

This is in contrast to what was recently done for native linux packages (which we can migrate in a follow-up):

TheRock/.github/workflows/multi_arch_build_native_linux_packages.yml

Lines 100 to 131 in aee2385

	- name: Determine IAM role
	id: iam_role
	run: |
	# ================================================================
	# IAM Role Selection Logic
	# ================================================================
	# Determines which AWS IAM role to assume based on job_type from s3_config step.
	#
	# Role Mapping:
	# ├─ IF job_type == "ci"
	# │ └─ Use: therock-ci role
	# │ (For all CI buckets: therock-ci-artifacts, therock-ci-artifacts-external, therock-artifacts-internal)
	# │
	# └─ ELSE (job_type == dev/nightly/prerelease/release)
	# └─ Use: therock-${job_type} role
	# (For package buckets: therock-dev-packages, therock-nightly-packages, etc.)
	#
	# ================================================================
	JOB_TYPE="${{ steps.s3_config.outputs.job_type }}"
	if [[ "${JOB_TYPE}" == "ci" ]]; then
	# CI builds use the shared CI role (for all artifact buckets)
	IAM_ROLE="arn:aws:iam::692859939525:role/therock-ci"
	echo "✓ Using CI role: ${IAM_ROLE}"
	else
	# Release builds use release-type-specific roles (for package buckets)
	IAM_ROLE="arn:aws:iam::692859939525:role/therock-${JOB_TYPE}"
	echo "✓ Using release-type role: ${IAM_ROLE}"
	fi
	echo "iam_role=${IAM_ROLE}" >> $GITHUB_OUTPUT

TheRock/.github/workflows/multi_arch_build_native_linux_packages.yml

Lines 169 to 173 in aee2385

	- name: Configure AWS Credentials
	uses: aws-actions/configure-aws-credentials@8df5847569e6427dd6c4fb1cf565c83acfa8afa7 # v6.0.0
	with:
	aws-region: us-east-2
	role-to-assume: ${{ steps.iam_role.outputs.iam_role }}

We could put similar code directly in the composite workflow instead of adding the python scripts, but this is significantly easier to test and then match the behavior between different workflows and scripts that want to interface with the buckets.

Test Plan

• Unit tests
• CI on this PR (which should authenticate with therock-ci and upload to therock-ci-artifacts)
• Watch CI on PRs from forks (which should skip authentication and upload to therock-ci-artifacts-external as before)

Test Result

Composite action worked: https://github.com/ROCm/TheRock/actions/runs/24109830592/job/70341723332?pr=4386#step:15:1

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[ScottTodd]

We could also update

TheRock/.github/workflows/release_portable_linux_packages.yml

Lines 274 to 279 in aee2385

	- name: Configure AWS Credentials
	if: ${{ github.repository_owner == 'ROCm' && !cancelled() }}
	uses: aws-actions/configure-aws-credentials@8df5847569e6427dd6c4fb1cf565c83acfa8afa7 # v6.0.0
	with:
	aws-region: us-east-2
	role-to-assume: arn:aws:iam::692859939525:role/therock-${{ env.RELEASE_TYPE }}

but that workflow uses more than just the therock-{release_type}-artifacts bucket, it also uses

TheRock/.github/workflows/release_portable_linux_packages.yml

Line 185 in aee2385

S3_BUCKET_PY: "therock-${{ needs.setup_metadata.outputs.release_type }}-python"

For multi-arch release workflows, I plan on having the existing build workflows upload artifacts,logs,etc. to therock-{release_type}-artifacts and then have new workflow code copy from those buckets to e.g. therock-{release_type}-tarball

---

By the way, I decided to NOT update

TheRock/.github/workflows/manifest-diff.yml

Lines 66 to 71 in aee2385

	- name: Configure AWS Credentials
	if: ${{ github.repository == 'ROCm/TheRock' }}
	uses: aws-actions/configure-aws-credentials@8df5847569e6427dd6c4fb1cf565c83acfa8afa7 # v6.0.0
	with:
	aws-region: us-east-2
	role-to-assume: arn:aws:iam::692859939525:role/therock-ci

[marbre]

Updating release_portable_linux_packages.yml should be fine as the role allows to access the other buckets as well. Can be in a follow up however.

[marbre]

From my agent's review:

The event-payload fork detection tests use tempfile and finally: os.unlink(...) - fine, but could use tempfile.NamedTemporaryFile(delete=True) with a context manager instead of manual cleanup.

[ScottTodd]

tempfile.NamedTemporaryFile(delete=True) won't work on Windows, as the file can't be reopened for read by another command while already opened by the context manager:

     >       with open(event_path) as f:
                  ^^^^^^^^^^^^^^^^
     E       PermissionError: [Errno 13] Permission denied:
     'C:\\Users\\NOD-SH~1\\AppData\\Local\\Temp\\tmpjr8p04dq.json'

We could use tempfile.TemporaryDirectory() though:

with tempfile.TemporaryDirectory() as tmpdir:
    event_path = Path(tmpdir) / "event.json"
    event_path.write_text(
        json.dumps({"pull_request": {"head": {"repo": {"fork": True}}}})
    )

How about we keep the NamedTemporaryFile and manual unlink() but add more comments? That matches recently added test code in https://github.com/ROCm/TheRock/blob/main/build_tools/github_actions/tests/configure_multi_arch_ci_test.py more closely too.

[marbre]

Oh we can also keep as is for consistency :)

[marbre]

Updating release_portable_linux_packages.yml should be fine as the role allows to access the other buckets as well. Can be in a follow up however.

[marbre]

arn:aws:iam actually is the prefix, partition, service, which will always be the same here. 692859939525 is the account ID. We could also just store stat one. And should role be deduced from iam_role?

[ScottTodd]

How's this now?

[marbre]

LGTM, thanks for iterating.