Repository navigation
ci: add Windows MSI install test (Milestone 5, #1987) - #8669
Conversation
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
ScottTodd
left a comment
There was a problem hiding this comment.
Quick surface level review, dumping state before end of week.
I also reviewed with an agent using this prompt:
› Please review #8669. In particular:
- github actions style / security, e.g.
set -euo pipefailshouldn't be needed since github applies similar modes to bashrun:steps by default- why do the tests themselves have tests? did we do something similar for native linux packaging...? that seems very weird
- workflow design and location iteration: the build_tools/packaging/windows/native_windows_package_install_test.py script downloads the packages. What about running the test locally? I would prefer to download once and then be able to run and re-run each test using what I've downloaded
- are the tests themselves meaningful? what potential issues are they guarding against? do they miss important cases?
which produced https://github.com/ScottTodd/rocm-workspace/blob/main/reviews/pr_TheRock_8669.md
it found a few notable gaps in there which I haven't verified:
- Uninstall selects a product without validating the requested version
- The payload test does not establish that each selected package installed its payload
- Registry verification accepts a wrong installation location
the native linux packaging does indeed unit test its tests for install and uninstall so this was modelled after that for consistency sake.. However, I do see your point on how this is not the cleanest approach and indeed is a product of excess logic in tests. I will refactor this following the "keep logic out of tests" paradigm |
Addresses ScottTodd's review of the MSI install test (#8669): Tests no longer touch the network. A new download_msi_packages.py downloads a run's MSIs once into a directory; the install/uninstall tests consume them via --msi-dir / MSI_DIR, so CI and developers can re-run the tests against the same files. The workflow gained a separate "Download MSI packages" step. Logic moved out of the tests into generate_msi_wxs.py (with its own unit tests), so the test scripts are thin enough not to need tests-of-tests: parse_major_minor, expected_msi_filename, install_subdir_for (and discovery_registry_key, added in the shared-registry-key change). The two native_windows_package_*_ut_test.py files are deleted; their reusable-logic coverage lives in generate_msi_wxs_test.py and the real behavior is exercised by the live install test on a Windows runner. Fixes the three correctness gaps raised in review: - Payload is verified per-product: the install test resolves each package's ProductCode and asks Windows Installer (MsiEnumComponents / MsiGetComponentPath via ctypes) which files it installed, then requires those files to exist under the install dir. One package's files can no longer satisfy another's check. - Uninstall validates the installed version before removal: the product's recorded InstallLocation (or its component files) must belong to the requested version's directory, so a different version is never removed. - The discovery-key check compares InstallDir against the expected install directory and requires it to exist, instead of only checking it is non-empty. Also: treat msiexec exit 3010 (reboot required) as success; drop the redundant set -euo pipefail from the workflow run: steps; align the test workflow's release_type default with the MSI build workflow (ci). NOTE: depends on the shared-registry-key change (discovery_registry_key) and the System32 opt-in change; keep this PR in draft until both merge, then rebase. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
|
Restructured per the review. Marked draft: this now depends on #8723 (shared registry discovery key) and #8724 (System32 opt-in) — the tests import How the review points were addressed:
Verified end-to-end on Windows against a real MSI built from the combined (#8723 + #8724) generator: install (7 payload files found via the Installer API, System32 scrubbed, discovery key InstallDir/Version match), uninstall + teardown, and the negative wrong-version case all behave as expected. |
Adds a standalone, dispatchable install test for the Windows MSI packages, mirroring the Linux native-package install-test pattern (issue #1987 comment): it consumes an arbitrary BUILD run's MSIs, is callable by ci/release, and is runnable locally for iteration. - native_windows_package_install_test.py: downloads each package's MSI from the public {run_id}-windows/packages/msi/ prefix (filenames derived from the generator's PACKAGES table, so no S3 listing and no credentials -- fork friendly), installs via `msiexec /i /qn`, and verifies the install dir, payload DLLs, System32 legacy DLLs, and the HKLM registry key. Dual CLI + pytest + env-var entry like the Linux script; --msi-dir skips the download for local iteration against a freshly built MSI. - native_windows_package_uninstall_test.py: looks up the ProductCode by DisplayName, runs `msiexec /x /qn`, and verifies the install dir + registry key are gone (shared System32 DLLs are ref-counted, so not asserted). - test_native_windows_packages_install.yml: workflow_dispatch + workflow_call on windows-2022, public download (no creds/guard -> forks run it), conditional uninstall step gated on run_uninstall. zizmor- and actionlint-clean. - Unit tests for both scripts (mock msiexec/winreg/downloads/filesystem) run on any platform; the live modules import winreg lazily and the pytest entries skip on non-Windows, so Linux whole-tree collection stays green. Style: argparse + type hints + pathlib, specific exceptions (no bare sys.exit), SHA-pinned actions, inputs via env (no template injection), safe release_type dispatch default (ci), set -euo pipefail. Test filenames are globally unique. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Addresses ScottTodd's review of the MSI install test (#8669): Tests no longer touch the network. A new download_msi_packages.py downloads a run's MSIs once into a directory; the install/uninstall tests consume them via --msi-dir / MSI_DIR, so CI and developers can re-run the tests against the same files. The workflow gained a separate "Download MSI packages" step. Logic moved out of the tests into generate_msi_wxs.py (with its own unit tests), so the test scripts are thin enough not to need tests-of-tests: parse_major_minor, expected_msi_filename, install_subdir_for (and discovery_registry_key, added in the shared-registry-key change). The two native_windows_package_*_ut_test.py files are deleted; their reusable-logic coverage lives in generate_msi_wxs_test.py and the real behavior is exercised by the live install test on a Windows runner. Fixes the three correctness gaps raised in review: - Payload is verified per-product: the install test resolves each package's ProductCode and asks Windows Installer (MsiEnumComponents / MsiGetComponentPath via ctypes) which files it installed, then requires those files to exist under the install dir. One package's files can no longer satisfy another's check. - Uninstall validates the installed version before removal: the product's recorded InstallLocation (or its component files) must belong to the requested version's directory, so a different version is never removed. - The discovery-key check compares InstallDir against the expected install directory and requires it to exist, instead of only checking it is non-empty. Also: treat msiexec exit 3010 (reboot required) as success; drop the redundant set -euo pipefail from the workflow run: steps; align the test workflow's release_type default with the MSI build workflow (ci). NOTE: depends on the shared-registry-key change (discovery_registry_key) and the System32 opt-in change; keep this PR in draft until both merge, then rebase. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
65e3d89 to
7cb1b3a
Compare
|
Rebased onto
89 packaging tests pass; pre-commit / pr-bot green. |
Two review comments (ScottTodd): - download_msi_packages.py: resolve the bandit B310 finding. Validate the URL scheme is https (and has a netloc) before urlretrieve, raising ValueError otherwise, so the downloader can't be pointed at a file:// or other-scheme path (CWE-22). Mark the call `# nosec B310` now that the scheme is checked, mirroring the pattern in #8877. - install/uninstall tests: drop the unused `raise_instead_of_exit` parameter and branch from parse_cli_arguments. Both call sites now use argparse's default exit-on-error behavior. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
…stone 4) The MSI build workflow (multi_arch_build_native_windows_packages.yml) existed but had no callers — the gap behind the #8617 regression. Wire it into both Windows workflows per Milestone 4 (#1987), with the build/publish distinction each needs: - multi_arch_ci_windows.yml (nightly): add build_native_msi_packages, which builds the runtime MSI from this run's artifacts and uploads it to the run's artifacts bucket (a build artifact). No release-bucket publish in CI. - multi_arch_release_windows.yml (release cut): add build_native_msi_packages, make publish_to_release_buckets depend on it (so the MSIs exist before publish copies them to the release bucket — the dependency edge that prevents #8617), and add trigger_test_native_windows_install, which dispatches the MSI install test against this run's artifacts prefix. Both jobs added to notify_quartz_completed's needs. Only the runtime package is wired (packages: runtime); core's Windows artifacts are not yet published in nightly runs. DRAFT: depends on #8704 (publish_native_windows_packages) and #8669 (test_native_windows_packages_install.yml), which are not yet on main. The workflow_dispatch_inputs test correctly flags the install-test dispatch until #8669 merges. Rebase and validate once both land. Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Summary
Adds a standalone, dispatchable install test for the Windows MSI packages (Milestone 5 of #1987), mirroring the Linux native-package install-test pattern per ScottTodd's guidance: it consumes an arbitrary build run's MSIs, is callable by ci/release, and is runnable locally for iteration.
native_windows_package_install_test.py— downloads each package's MSI from the public{run_id}-windows/packages/msi/prefix (filenames derived from the generator'sPACKAGEStable, so no S3 listing and no credentials — fork-friendly), installs viamsiexec /i /qn, and verifies the install directory, payload DLLs, System32 legacy DLLs, and the HKLM registry key. Dual CLI + pytest + env-var entry like the Linux script;--msi-dirskips the download for local iteration against a freshly built MSI.native_windows_package_uninstall_test.py— looks up the ProductCode by DisplayName, runsmsiexec /x /qn, and verifies the install dir + registry key are gone (shared, ref-counted System32 DLLs are intentionally not asserted).test_native_windows_packages_install.yml—workflow_dispatch+workflow_callonwindows-2022, public download (no creds/guard → forks run it), conditional uninstall step gated onrun_uninstall. actionlint- and zizmor-clean.winreglazily and the pytest entries skip on non-Windows, so Linux whole-tree collection stays green.Style compliance
Audited against
docs/development/style_guides/: argparse + type hints + pathlib, specific exceptions (no baresys.exit), SHA-pinned actions, inputs passed viaenv:(no template injection), saferelease_typedispatch default (choice [ci, dev], matching the Linux sibling),set -euo pipefail. Test filenames are globally unique (whole-tree pytest).Test plan
pytest build_tools/packaging/windows/tests/— install + uninstall UTs pass on Linux (33 tests).zizmor --offlineclean on the new workflow.36895974522,release_type=ci) to produceamdrocm-runtime.msiin the public ci bucket, then dispatched this install test against it:https://github.com/idass1990/TheRock/actions/runs/36896581592 — ✅ green
install directory present: C:\Program Files\AMD\ROCm\core-10.2,payload DLLs present,System32 DLLs present,registry key present: HKLM\Software\AMD\ROCm\runtime\10.2.install directory removed,registry key removed.Notes
if: owner == 'ROCm'), so live-testing requires a real ROCm build run'sartifact_run_id(as done above). Local--msi-dirneeds no S3 at all.Follow-ups (out of scope):
--test-typemodes, smoke-running a bundled exe (needs core pkg/driver/GPU), and wiring build + test into ci/release (Milestone 4).