Repository navigation
ci: publish Windows MSIs to release buckets + dispatch install test (2/2) #9371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -9,7 +9,7 @@ | |||||||||||
| - [x] tarballs | ||||||||||||
| - [x] python packages | ||||||||||||
| - [x] native linux packages | ||||||||||||
| - [ ] native windows packages | ||||||||||||
| - [x] native windows packages | ||||||||||||
|
Comment on lines
11
to
+12
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. FYI we also have some ooooold docs that predate the native linux packaging which we could update to mention native linux and native windows now: https://github.com/ROCm/TheRock/blob/main/docs/development/development_guide.md#overall-build-architecture See the files in https://github.com/ROCm/TheRock/tree/main/docs/development/assets which can be loaded into https://excalidraw.com/. I think agents are also pretty good at editing the excalidraw svgs now too... |
||||||||||||
|
|
||||||||||||
| Example with ``--run-id 12345 --platform linux --release-type dev``: | ||||||||||||
|
|
||||||||||||
|
|
@@ -43,6 +43,11 @@ | |||||||||||
| s3://therock-prerelease-artifacts/12345-linux/packages/rpm/ | ||||||||||||
| -> s3://therock-repo-amd-rc-core/v5/rocm/core/packages/rpm/ | ||||||||||||
|
|
||||||||||||
| native windows packages (dev/nightly), with ``--platform windows``: | ||||||||||||
|
|
||||||||||||
| s3://therock-dev-artifacts/12345-windows/packages/msi/ | ||||||||||||
| -> s3://therock-repo-amd-dev-core/v5/rocm/core/windows-installers/20250101-12345/ | ||||||||||||
|
|
||||||||||||
| ASAN build variant: | ||||||||||||
|
|
||||||||||||
| For ASAN builds (--build-variant asan), python packages are skipped and | ||||||||||||
|
|
@@ -278,6 +283,49 @@ def publish_native_linux_packages( | |||||||||||
| raise FileNotFoundError(f"No {pkg_type} packages found at {source.s3_uri}") | ||||||||||||
|
|
||||||||||||
|
|
||||||||||||
| def publish_native_windows_packages( | ||||||||||||
| artifacts_root: WorkflowOutputRoot, | ||||||||||||
| release_type: str, | ||||||||||||
| backend: StorageBackend, | ||||||||||||
| ) -> None: | ||||||||||||
| """Copy native Windows MSI packages from the artifacts bucket to the release bucket. | ||||||||||||
|
|
||||||||||||
| The source MSIs were uploaded by upload_package_repo.py (called from | ||||||||||||
| multi_arch_build_native_windows_packages.yml) as loose .msi files. | ||||||||||||
|
|
||||||||||||
| The destination follows RFC0012: Windows installer artifacts live under | ||||||||||||
| ``core/windows-installers/`` (not ``core/packages/``). The current MSI is a | ||||||||||||
| single OS-agnostic installer, so it is published to a flat | ||||||||||||
| ``windows-installers/`` prefix rather than the per-OS subfolders RFC0012 | ||||||||||||
| reserves for OS-specific installers. | ||||||||||||
|
|
||||||||||||
| dev/nightly example: | ||||||||||||
| s3://therock-dev-artifacts/12345-windows/packages/msi/ | ||||||||||||
| -> s3://therock-repo-amd-dev-core/v5/rocm/core/windows-installers/20250101-12345/ | ||||||||||||
|
|
||||||||||||
| prerelease example: | ||||||||||||
| s3://therock-prerelease-artifacts/12345-windows/packages/msi/ | ||||||||||||
| -> s3://therock-repo-amd-rc-core/v5/rocm/core/windows-installers/ | ||||||||||||
| """ | ||||||||||||
| dest_bucket = get_product_release_bucket_config(release_type, "core") | ||||||||||||
| today = datetime.date.today().strftime("%Y%m%d") | ||||||||||||
|
|
||||||||||||
| source = artifacts_root.native_windows_packages("msi") | ||||||||||||
|
|
||||||||||||
| base_path = "v5/rocm/core/windows-installers" | ||||||||||||
| if release_type == "prerelease": | ||||||||||||
| dest_prefix = base_path | ||||||||||||
| else: | ||||||||||||
| dest_prefix = f"{base_path}/{today}-{artifacts_root.run_id}" | ||||||||||||
|
|
||||||||||||
| dest = StorageLocation(dest_bucket.name, dest_prefix) | ||||||||||||
| logger.info("Native msi packages: %s -> %s", source.s3_uri, dest.s3_uri) | ||||||||||||
| count = backend.copy_directory(source, dest) | ||||||||||||
| logger.info("Copied %d files for msi packages", count) | ||||||||||||
| if count == 0: | ||||||||||||
| raise FileNotFoundError(f"No msi packages found at {source.s3_uri}") | ||||||||||||
|
|
||||||||||||
|
|
||||||||||||
| def main(argv: list[str]) -> None: | ||||||||||||
| parser = argparse.ArgumentParser( | ||||||||||||
| description="Publish ROCm release files to release buckets" | ||||||||||||
|
|
@@ -360,6 +408,8 @@ def main(argv: list[str]) -> None: | |||||||||||
| publish_native_linux_packages( | ||||||||||||
| artifacts_root, args.release_type, backend, args.build_variant | ||||||||||||
| ) | ||||||||||||
| if artifacts_root.platform == "windows" and not args.skip_native_packages: | ||||||||||||
| publish_native_windows_packages(artifacts_root, args.release_type, backend) | ||||||||||||
|
Comment on lines
+411
to
+412
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Once we've done enough initial validation on the nightly windows packages we can update the release docs too: Lines 38 to 42 in 4236235
|
||||||||||||
|
|
||||||||||||
|
|
||||||||||||
| if __name__ == "__main__": | ||||||||||||
|
|
||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All use of
benc-uk/workflow-dispatchin release workflows need stub workflows in the rockrel repository like https://github.com/ROCm/rockrel/blob/main/.github/workflows/test_native_linux_packages_install.yml. See ROCm/rockrel#49Please land that before this change so we don't break nightly releases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
stub workflow added in ROCm/rockrel#130, following the same pattern as test_native_linux_packages_install.yml.