Skip to content

ci: add Windows MSI install test (Milestone 5, #1987) - #8669

Merged
idass1990 merged 3 commits into
mainfrom
users/idass1990/windows_msi_install_test
Oct 15, 2026
Merged

idass1990 merged 3 commits into
mainfrom
users/idass1990/windows_msi_install_test

Conversation

@idass1990

Copy link
Copy Markdown
Contributor

Summary

Adds a standalone, dispatchable install test for the Windows MSI packages (Milestone 5 of #1987), mirroring the Linux native-package install-test pattern per ScottTodd's guidance: it consumes an arbitrary build run's MSIs, is callable by ci/release, and is runnable locally for iteration.

  • native_windows_package_install_test.py — downloads each package's MSI from the public {run_id}-windows/packages/msi/ prefix (filenames derived from the generator's PACKAGES table, so no S3 listing and no credentials — fork-friendly), installs via msiexec /i /qn, and verifies the install directory, payload DLLs, System32 legacy DLLs, and the HKLM registry key. Dual CLI + pytest + env-var entry like the Linux script; --msi-dir skips the download for local iteration against a freshly built MSI.
  • native_windows_package_uninstall_test.py — looks up the ProductCode by DisplayName, runs msiexec /x /qn, and verifies the install dir + registry key are gone (shared, ref-counted System32 DLLs are intentionally not asserted).
  • test_native_windows_packages_install.yml — workflow_dispatch + workflow_call on windows-2022, public download (no creds/guard → forks run it), conditional uninstall step gated on run_uninstall. actionlint- and zizmor-clean.
  • Unit tests for both scripts (mock msiexec/winreg/downloads/filesystem) run on any platform; the live modules import winreg lazily and the pytest entries skip on non-Windows, so Linux whole-tree collection stays green.

Style compliance

Audited against docs/development/style_guides/: argparse + type hints + pathlib, specific exceptions (no bare sys.exit), SHA-pinned actions, inputs passed via env: (no template injection), safe release_type dispatch default (choice [ci, dev], matching the Linux sibling), set -euo pipefail. Test filenames are globally unique (whole-tree pytest).

Test plan

  • Unit tests: pytest build_tools/packaging/windows/tests/ — install + uninstall UTs pass on Linux (33 tests).
  • actionlint + zizmor --offline clean on the new workflow.
  • End-to-end verified on a real MSI. Dispatched the build workflow on ROCm (run 36895974522, release_type=ci) to produce amdrocm-runtime.msi in the public ci bucket, then dispatched this install test against it:
    https://github.com/idass1990/TheRock/actions/runs/36896581592 — ✅ green
    • Install + verify: install directory present: C:\Program Files\AMD\ROCm\core-10.2, payload DLLs present, System32 DLLs present, registry key present: HKLM\Software\AMD\ROCm\runtime\10.2.
    • Uninstall + teardown: ProductCode looked up by DisplayName, install directory removed, registry key removed.

Notes

  • The install test consumes the MSIs the build workflow uploads to the artifacts bucket (still present). It is independent of the release-bucket publish that was reverted in Revert MSI publishing from the release publish script (#8617) #8629.
  • Fork caveat: a fork build run does not upload MSIs (if: owner == 'ROCm'), so live-testing requires a real ROCm build run's artifact_run_id (as done above). Local --msi-dir needs no S3 at all.

Follow-ups (out of scope): --test-type modes, smoke-running a bundled exe (needs core pkg/driver/GPU), and wiring build + test into ci/release (Milestone 4).

@therock-pr-bot

therock-pr-bot Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass —
🔎 pre-commit ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —
🤖 therock-pr-bot ✅ Pass —

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

therock-pr-bot Bot commented Oct 1, 2026

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

@idass1990
idass1990 requested a review from ScottTodd October 1, 2026 17:25

@ScottTodd ScottTodd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick surface level review, dumping state before end of week.

I also reviewed with an agent using this prompt:

› Please review #8669. In particular:

  • github actions style / security, e.g. set -euo pipefail shouldn't be needed since github applies similar modes to bash run: steps by default
  • why do the tests themselves have tests? did we do something similar for native linux packaging...? that seems very weird
  • workflow design and location iteration: the build_tools/packaging/windows/native_windows_package_install_test.py script downloads the packages. What about running the test locally? I would prefer to download once and then be able to run and re-run each test using what I've downloaded
  • are the tests themselves meaningful? what potential issues are they guarding against? do they miss important cases?

which produced https://github.com/ScottTodd/rocm-workspace/blob/main/reviews/pr_TheRock_8669.md

it found a few notable gaps in there which I haven't verified:

  • Uninstall selects a product without validating the requested version
  • The payload test does not establish that each selected package installed its payload
  • Registry verification accepts a wrong installation location

@idass1990

Copy link
Copy Markdown
Contributor Author

Quick surface level review, dumping state before end of week.

I also reviewed with an agent using this prompt:

› Please review #8669. In particular:

  • github actions style / security, e.g. set -euo pipefail shouldn't be needed since github applies similar modes to bash run: steps by default
  • why do the tests themselves have tests? did we do something similar for native linux packaging...? that seems very weird
  • workflow design and location iteration: the build_tools/packaging/windows/native_windows_package_install_test.py script downloads the packages. What about running the test locally? I would prefer to download once and then be able to run and re-run each test using what I've downloaded
  • are the tests themselves meaningful? what potential issues are they guarding against? do they miss important cases?

which produced https://github.com/ScottTodd/rocm-workspace/blob/main/reviews/pr_TheRock_8669.md

it found a few notable gaps in there which I haven't verified:

  • Uninstall selects a product without validating the requested version
  • The payload test does not establish that each selected package installed its payload
  • Registry verification accepts a wrong installation location

the native linux packaging does indeed unit test its tests for install and uninstall so this was modelled after that for consistency sake.. However, I do see your point on how this is not the cleanest approach and indeed is a product of excess logic in tests. I will refactor this following the "keep logic out of tests" paradigm

idass1990 added a commit that referenced this pull request Oct 3, 2026
Addresses ScottTodd's review of the MSI install test (#8669):

Tests no longer touch the network. A new download_msi_packages.py downloads a
run's MSIs once into a directory; the install/uninstall tests consume them via
--msi-dir / MSI_DIR, so CI and developers can re-run the tests against the same
files. The workflow gained a separate "Download MSI packages" step.

Logic moved out of the tests into generate_msi_wxs.py (with its own unit tests),
so the test scripts are thin enough not to need tests-of-tests:
parse_major_minor, expected_msi_filename, install_subdir_for (and
discovery_registry_key, added in the shared-registry-key change). The two
native_windows_package_*_ut_test.py files are deleted; their reusable-logic
coverage lives in generate_msi_wxs_test.py and the real behavior is exercised by
the live install test on a Windows runner.

Fixes the three correctness gaps raised in review:
- Payload is verified per-product: the install test resolves each package's
  ProductCode and asks Windows Installer (MsiEnumComponents /
  MsiGetComponentPath via ctypes) which files it installed, then requires those
  files to exist under the install dir. One package's files can no longer
  satisfy another's check.
- Uninstall validates the installed version before removal: the product's
  recorded InstallLocation (or its component files) must belong to the requested
  version's directory, so a different version is never removed.
- The discovery-key check compares InstallDir against the expected install
  directory and requires it to exist, instead of only checking it is non-empty.

Also: treat msiexec exit 3010 (reboot required) as success; drop the redundant
set -euo pipefail from the workflow run: steps; align the test workflow's
release_type default with the MSI build workflow (ci).

NOTE: depends on the shared-registry-key change (discovery_registry_key) and the
System32 opt-in change; keep this PR in draft until both merge, then rebase.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
@idass1990
idass1990 marked this pull request as draft October 3, 2026 21:23
@idass1990

Copy link
Copy Markdown
Contributor Author

Restructured per the review. Marked draft: this now depends on #8723 (shared registry discovery key) and #8724 (System32 opt-in) — the tests import discovery_registry_key from the generator and assert the new behavior, so the module won't import standalone until those land. Will rebase and mark ready once both merge.

How the review points were addressed:

  • No network in tests / download once, re-run. New download_msi_packages.py downloads a run's MSIs into a directory; the tests consume them via --msi-dir / MSI_DIR and never touch the network. The workflow has a separate "Download MSI packages" step, so CI and developers download once and re-run the tests against the same files.

  • No tests-for-tests. Reusable logic moved into generate_msi_wxs.py with its own unit tests (parse_major_minor, expected_msi_filename, install_subdir_for, discovery_registry_key). Both native_windows_package_*_ut_test.py files are deleted; real behavior is exercised by the live install test on a Windows runner.

  • Uninstall version validation. Uninstall resolves the ProductCode, then confirms that product's recorded InstallLocation (or its component files) belongs to the requested version's install directory before removing it. Verified: uninstalling --rocm-version 9.9.0 while 10.2 is installed is refused.

  • Package-specific payload. The payload check resolves each package's ProductCode and asks Windows Installer which files it installed (MsiEnumComponents / MsiGetComponentPath via ctypes), then requires those files to exist under the install dir — so one package's files can't satisfy another's check. Packages are validated standalone.

  • Registry path validation. The discovery-key check compares InstallDir against the expected install directory and requires it to exist, rather than only checking non-empty.

  • Minor: msiexec 3010 (reboot required) is treated as success; dropped set -euo pipefail from the workflow run: steps; aligned the test workflow's release_type default with the MSI build workflow (ci).

Verified end-to-end on Windows against a real MSI built from the combined (#8723 + #8724) generator: install (7 payload files found via the Installer API, System32 scrubbed, discovery key InstallDir/Version match), uninstall + teardown, and the negative wrong-version case all behave as expected.

idass1990 and others added 2 commits October 14, 2026 15:04
Adds a standalone, dispatchable install test for the Windows MSI packages,
mirroring the Linux native-package install-test pattern (issue #1987 comment):
it consumes an arbitrary BUILD run's MSIs, is callable by ci/release, and is
runnable locally for iteration.

- native_windows_package_install_test.py: downloads each package's MSI from the
  public {run_id}-windows/packages/msi/ prefix (filenames derived from the
  generator's PACKAGES table, so no S3 listing and no credentials -- fork
  friendly), installs via `msiexec /i /qn`, and verifies the install dir,
  payload DLLs, System32 legacy DLLs, and the HKLM registry key. Dual CLI +
  pytest + env-var entry like the Linux script; --msi-dir skips the download for
  local iteration against a freshly built MSI.
- native_windows_package_uninstall_test.py: looks up the ProductCode by
  DisplayName, runs `msiexec /x /qn`, and verifies the install dir + registry
  key are gone (shared System32 DLLs are ref-counted, so not asserted).
- test_native_windows_packages_install.yml: workflow_dispatch + workflow_call on
  windows-2022, public download (no creds/guard -> forks run it), conditional
  uninstall step gated on run_uninstall. zizmor- and actionlint-clean.
- Unit tests for both scripts (mock msiexec/winreg/downloads/filesystem) run on
  any platform; the live modules import winreg lazily and the pytest entries
  skip on non-Windows, so Linux whole-tree collection stays green.

Style: argparse + type hints + pathlib, specific exceptions (no bare sys.exit),
SHA-pinned actions, inputs via env (no template injection), safe release_type
dispatch default (ci), set -euo pipefail. Test filenames are globally unique.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Addresses ScottTodd's review of the MSI install test (#8669):

Tests no longer touch the network. A new download_msi_packages.py downloads a
run's MSIs once into a directory; the install/uninstall tests consume them via
--msi-dir / MSI_DIR, so CI and developers can re-run the tests against the same
files. The workflow gained a separate "Download MSI packages" step.

Logic moved out of the tests into generate_msi_wxs.py (with its own unit tests),
so the test scripts are thin enough not to need tests-of-tests:
parse_major_minor, expected_msi_filename, install_subdir_for (and
discovery_registry_key, added in the shared-registry-key change). The two
native_windows_package_*_ut_test.py files are deleted; their reusable-logic
coverage lives in generate_msi_wxs_test.py and the real behavior is exercised by
the live install test on a Windows runner.

Fixes the three correctness gaps raised in review:
- Payload is verified per-product: the install test resolves each package's
  ProductCode and asks Windows Installer (MsiEnumComponents /
  MsiGetComponentPath via ctypes) which files it installed, then requires those
  files to exist under the install dir. One package's files can no longer
  satisfy another's check.
- Uninstall validates the installed version before removal: the product's
  recorded InstallLocation (or its component files) must belong to the requested
  version's directory, so a different version is never removed.
- The discovery-key check compares InstallDir against the expected install
  directory and requires it to exist, instead of only checking it is non-empty.

Also: treat msiexec exit 3010 (reboot required) as success; drop the redundant
set -euo pipefail from the workflow run: steps; align the test workflow's
release_type default with the MSI build workflow (ci).

NOTE: depends on the shared-registry-key change (discovery_registry_key) and the
System32 opt-in change; keep this PR in draft until both merge, then rebase.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
@idass1990
idass1990 force-pushed the users/idass1990/windows_msi_install_test branch from 65e3d89 to 7cb1b3a Compare October 14, 2026 19:26
@idass1990
idass1990 marked this pull request as ready for review October 14, 2026 19:31
@idass1990

Copy link
Copy Markdown
Contributor Author

Rebased onto main now that #8723 and #8724 have merged, and marked ready for review.

89 packaging tests pass; pre-commit / pr-bot green.

Two review comments (ScottTodd):

- download_msi_packages.py: resolve the bandit B310 finding. Validate the URL
  scheme is https (and has a netloc) before urlretrieve, raising ValueError
  otherwise, so the downloader can't be pointed at a file:// or other-scheme
  path (CWE-22). Mark the call `# nosec B310` now that the scheme is checked,
  mirroring the pattern in #8877.

- install/uninstall tests: drop the unused `raise_instead_of_exit` parameter and
  branch from parse_cli_arguments. Both call sites now use argparse's default
  exit-on-error behavior.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
@idass1990
idass1990 merged commit 44b26b5 into main Oct 15, 2026
28 checks passed
@idass1990
idass1990 deleted the users/idass1990/windows_msi_install_test branch October 15, 2026 18:54
@github-project-automation github-project-automation Bot moved this from TODO to Done in TheRock Triage Oct 15, 2026
idass1990 added a commit that referenced this pull request Oct 15, 2026
…stone 4)

The MSI build workflow (multi_arch_build_native_windows_packages.yml) existed but
had no callers — the gap behind the #8617 regression. Wire it into both Windows
workflows per Milestone 4 (#1987), with the build/publish distinction each needs:

- multi_arch_ci_windows.yml (nightly): add build_native_msi_packages, which
  builds the runtime MSI from this run's artifacts and uploads it to the run's
  artifacts bucket (a build artifact). No release-bucket publish in CI.

- multi_arch_release_windows.yml (release cut): add build_native_msi_packages,
  make publish_to_release_buckets depend on it (so the MSIs exist before publish
  copies them to the release bucket — the dependency edge that prevents #8617),
  and add trigger_test_native_windows_install, which dispatches the MSI install
  test against this run's artifacts prefix. Both jobs added to
  notify_quartz_completed's needs.

Only the runtime package is wired (packages: runtime); core's Windows artifacts
are not yet published in nightly runs.

DRAFT: depends on #8704 (publish_native_windows_packages) and #8669
(test_native_windows_packages_install.yml), which are not yet on main. The
workflow_dispatch_inputs test correctly flags the install-test dispatch until
#8669 merges. Rebase and validate once both land.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants