Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/multi_arch_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,7 @@ jobs:
with:
build_config: ${{ needs.setup.outputs.windows_build_config }}
rocm_package_version: ${{ needs.setup.outputs.rocm_package_version }}
rocm_msi_package_version: ${{ needs.setup.outputs.rocm_msi_package_version }}
test_type: ${{ needs.setup.outputs.test_type }}
release_type: ${{ inputs.release_type }}
repository: ${{ inputs.repository || github.repository }}
Expand Down
46 changes: 44 additions & 2 deletions .github/workflows/multi_arch_release_windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ on:
description: JSON object with build configuration for this platform.
rocm_package_version:
type: string
rocm_msi_package_version:
type: string
test_type:
type: string
release_type:
Expand Down Expand Up @@ -124,8 +126,27 @@ jobs:
contents: read
id-token: write

build_native_msi_packages:
needs: [build_artifacts]
name: Build Native MSI Packages
uses: ./.github/workflows/multi_arch_build_native_windows_packages.yml
with:
artifact_run_id: ${{ github.run_id }}
rocm_version: ${{ inputs.rocm_package_version }}
msi_version: ${{ inputs.rocm_msi_package_version }}
# Only the runtime package is published in artifacts today; core's Windows
# artifacts (core-amdsmi, core-runtime) are not yet available.
packages: runtime
release_type: ${{ inputs.release_type }}
repository: ${{ inputs.repository }}
ref: ${{ inputs.ref }}
quartz_tracking_id: ${{ inputs.quartz_tracking_id }}
permissions:
contents: read
id-token: write

publish_to_release_buckets:
needs: [build_tarballs, build_python_packages]
needs: [build_tarballs, build_python_packages, build_native_msi_packages]
name: Publish to Release Buckets
runs-on: ubuntu-26.04
permissions:
Expand Down Expand Up @@ -197,6 +218,27 @@ jobs:
"quartz_tracking_id": "${{ inputs.quartz_tracking_id }}"
}

trigger_test_native_windows_install:
needs: [build_native_msi_packages]
name: Trigger Test Native Windows Packages Install
runs-on: ubuntu-26.04
permissions:
actions: write
steps:
- name: Trigger test_native_windows_packages_install
uses: benc-uk/workflow-dispatch@d004979be141727c7a17c28f603e2facc40d0855 # v1.3.3
with:
workflow: test_native_windows_packages_install.yml
Comment on lines +229 to +231

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All use of benc-uk/workflow-dispatch in release workflows need stub workflows in the rockrel repository like https://github.com/ROCm/rockrel/blob/main/.github/workflows/test_native_linux_packages_install.yml. See ROCm/rockrel#49

Please land that before this change so we don't break nightly releases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stub workflow added in ROCm/rockrel#130, following the same pattern as test_native_linux_packages_install.yml.

inputs: |
{ "artifact_run_id": "${{ github.run_id }}",
"packages": "runtime",
"rocm_version": "${{ inputs.rocm_package_version }}",
"release_type": "${{ inputs.release_type }}",
"run_uninstall": "true",
"repository": "${{ inputs.repository }}",
"ref": "${{ inputs.ref }}",
"quartz_tracking_id": "${{ inputs.quartz_tracking_id }}" }

# The PyTorch release workflow publishes its wheels to the release index, then
# tests from that index. Wait for the corresponding ROCm release to be published
# before triggering the downstream release.
Expand Down Expand Up @@ -227,7 +269,7 @@ jobs:

notify_quartz_completed:
if: ${{ always() && vars.NOTIFY_QUARTZ_ENABLED == 'true' }}
needs: [build_artifacts, build_tarballs, build_python_packages, publish_to_release_buckets, trigger_test_artifacts_per_family, trigger_release_pytorch_wheels]
needs: [build_artifacts, build_tarballs, build_python_packages, build_native_msi_packages, publish_to_release_buckets, trigger_test_artifacts_per_family, trigger_release_pytorch_wheels, trigger_test_native_windows_install]
name: "Quartz - completed - release Windows"
runs-on: ubuntu-26.04
timeout-minutes: 10
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/test_native_windows_packages_install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,13 @@ on:
description: "Branch, tag, or SHA to checkout. Defaults to the triggering ref."
type: string
default: ""
quartz_tracking_id:
description: >-
Leave empty. Internal Quartz tracking id, set automatically by the
top-level orchestrator to link this run to its release lineage. Do not
set it manually.
type: string
default: ""
workflow_call:
inputs:
artifact_run_id:
Expand Down Expand Up @@ -74,6 +81,9 @@ on:
ref:
type: string
default: ""
quartz_tracking_id:
type: string
default: ""

permissions:
contents: read
Expand Down
52 changes: 51 additions & 1 deletion build_tools/github_actions/publish_rocm_to_release_buckets.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
- [x] tarballs
- [x] python packages
- [x] native linux packages
- [ ] native windows packages
- [x] native windows packages
Comment on lines 11 to +12

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI we also have some ooooold docs that predate the native linux packaging which we could update to mention native linux and native windows now: https://github.com/ROCm/TheRock/blob/main/docs/development/development_guide.md#overall-build-architecture

See the files in https://github.com/ROCm/TheRock/tree/main/docs/development/assets which can be loaded into https://excalidraw.com/. I think agents are also pretty good at editing the excalidraw svgs now too...


Example with ``--run-id 12345 --platform linux --release-type dev``:

Expand Down Expand Up @@ -43,6 +43,11 @@
s3://therock-prerelease-artifacts/12345-linux/packages/rpm/
-> s3://therock-repo-amd-rc-core/v5/rocm/core/packages/rpm/

native windows packages (dev/nightly), with ``--platform windows``:

s3://therock-dev-artifacts/12345-windows/packages/msi/
-> s3://therock-repo-amd-dev-core/v5/rocm/core/windows-installers/20250101-12345/

ASAN build variant:

For ASAN builds (--build-variant asan), python packages are skipped and
Expand Down Expand Up @@ -278,6 +283,49 @@ def publish_native_linux_packages(
raise FileNotFoundError(f"No {pkg_type} packages found at {source.s3_uri}")


def publish_native_windows_packages(
artifacts_root: WorkflowOutputRoot,
release_type: str,
backend: StorageBackend,
) -> None:
"""Copy native Windows MSI packages from the artifacts bucket to the release bucket.

The source MSIs were uploaded by upload_package_repo.py (called from
multi_arch_build_native_windows_packages.yml) as loose .msi files.

The destination follows RFC0012: Windows installer artifacts live under
``core/windows-installers/`` (not ``core/packages/``). The current MSI is a
single OS-agnostic installer, so it is published to a flat
``windows-installers/`` prefix rather than the per-OS subfolders RFC0012
reserves for OS-specific installers.

dev/nightly example:
s3://therock-dev-artifacts/12345-windows/packages/msi/
-> s3://therock-repo-amd-dev-core/v5/rocm/core/windows-installers/20250101-12345/

prerelease example:
s3://therock-prerelease-artifacts/12345-windows/packages/msi/
-> s3://therock-repo-amd-rc-core/v5/rocm/core/windows-installers/
"""
dest_bucket = get_product_release_bucket_config(release_type, "core")
today = datetime.date.today().strftime("%Y%m%d")

source = artifacts_root.native_windows_packages("msi")

base_path = "v5/rocm/core/windows-installers"
if release_type == "prerelease":
dest_prefix = base_path
else:
dest_prefix = f"{base_path}/{today}-{artifacts_root.run_id}"

dest = StorageLocation(dest_bucket.name, dest_prefix)
logger.info("Native msi packages: %s -> %s", source.s3_uri, dest.s3_uri)
count = backend.copy_directory(source, dest)
logger.info("Copied %d files for msi packages", count)
if count == 0:
raise FileNotFoundError(f"No msi packages found at {source.s3_uri}")


def main(argv: list[str]) -> None:
parser = argparse.ArgumentParser(
description="Publish ROCm release files to release buckets"
Expand Down Expand Up @@ -360,6 +408,8 @@ def main(argv: list[str]) -> None:
publish_native_linux_packages(
artifacts_root, args.release_type, backend, args.build_variant
)
if artifacts_root.platform == "windows" and not args.skip_native_packages:
publish_native_windows_packages(artifacts_root, args.release_type, backend)
Comment on lines +411 to +412

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once we've done enough initial validation on the nightly windows packages we can update the release docs too:

TheRock/RELEASES.md

Lines 38 to 42 in 4236235

- [Installing multi-arch native packages](#installing-multi-arch-native-packages)
- [Installing multi-arch native Linux packages](#installing-multi-arch-native-linux-packages)
<!-- - [Using ROCm from WSL](#using-rocm-from-wsl) -->
<!-- TODO: native Windows packages -->
- [Installing multi-arch tarballs](#installing-multi-arch-tarballs)



if __name__ == "__main__":
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ def test_dev_linux_copies_native_packages(self, mock_copy):
)

@mock.patch("_therock_utils.storage_backend.S3StorageBackend.copy_directory")
def test_windows_skips_native_packages(self, mock_copy):
def test_windows_copies_native_msi_packages(self, mock_copy):
mock_copy.return_value = 1
main(
[
Expand All @@ -146,8 +146,16 @@ def test_windows_skips_native_packages(self, mock_copy):
"--dry-run",
]
)
# Only tarballs + python x2 (3 calls) — native packages skipped for windows
self.assertEqual(mock_copy.call_count, 3)
# Calls: tarballs, python x2, msi (4 calls)
self.assertEqual(mock_copy.call_count, 4)
# msi packages are the last call
msi_source, msi_dest = mock_copy.call_args_list[3].args
self.assertEqual(msi_source.bucket, "therock-nightly-artifacts")
self.assertEqual(msi_source.relative_path, "99-windows/packages/msi")
self.assertEqual(msi_dest.bucket, "therock-repo-amd-nightly-core")
self.assertRegex(
msi_dest.relative_path, r"^v5/rocm/core/windows-installers/\d{8}-99$"
)

@mock.patch("_therock_utils.storage_backend.S3StorageBackend.copy_directory")
def test_raises_when_no_tarballs_found(self, mock_copy):
Expand Down
Loading