ROCm-on-WSL: support discrete Radeon (RDNA 3/4) in WSL, not just Strix Halo - #6915
Conversation
…A arch install_rocm_wsl_strixhalo.sh hardcoded gfx1151, so its verify step died on discrete Radeon cards even though the ROCm + librocdxg setup is arch-agnostic. Auto-detect the GPU arch from rocminfo (override via UNSLOTH_WSL_GFX), verify any GPU agent enumerates over DXG, and map the arch to AMD's per-arch wheel family for the optional smoke test (injecting librocdxg into torch/lib so torch's bundled ROCr finds the DXG bridge). Verified on gfx1200 (Radeon RX 9060 XT) in WSL2 + Ubuntu 24.04 -- torch.cuda now enumerates the GPU.
_maybe_bootstrap_rocm_wsl only fired for Strix APUs (matched via /proc/cpuinfo, which discrete cards don't appear in). Add _wsl_amd_gpu_name() -- queries the Windows host via WMI -- and broaden the trigger gate plus the 'already-usable ROCm' rocminfo check from gfx1151-only to any real GPU agent (gfxNNNN, excluding the gfx11-generic fallback ISA). The generalized bootstrap then auto-detects the arch. Enables 'curl install.sh | sh' to set up ROCm-on-WSL on discrete Radeon RX 7000/9000 in WSL2 + Ubuntu 24.04, not just Strix Halo/Point.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request generalizes the ROCm-on-WSL installation and bootstrapping process to support any ROCDXG-capable AMD GPU (including discrete Radeon RX 7000/9000 series and Strix Point/Halo APUs) rather than being hardcoded for Strix Halo (gfx1151). It introduces dynamic GPU architecture detection via rocminfo and falls back to querying the Windows host via PowerShell if the GPU is not visible in /proc/cpuinfo. Feedback is provided to filter out the 'generic' fallback ISA during architecture detection and to wrap the PowerShell host query in a bounded execution helper to prevent potential hangs.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…e host GPU query - install_rocm_wsl_strixhalo.sh: exclude the gfx11-generic fallback ISA in arch detection (grep -v generic), matching install.sh's rocminfo check, so a generic agent listed before the real one can't be picked as the arch. - install.sh: wrap the powershell.exe Win32_VideoController query in _run_bounded (10s timeout) so an unstable WSL-interop / busy host can't hang the installer.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request generalizes the ROCm-on-WSL bootstrap and installation scripts to support any AMD GPU (including discrete Radeon RX 7000/9000 cards and Strix Point/Halo APUs) rather than being restricted to Strix Halo (gfx1151). It queries the Windows host via PowerShell for the GPU name when it is not visible in /proc/cpuinfo, dynamically detects the GPU architecture from rocminfo, and maps it to the correct AMD wheel index. The feedback suggests appending || true to a command substitution to prevent script abortion under set -eo pipefail when no GPU agent is found, and replacing a non-empty string check with a directory existence check (-d) before copying librocdxg.so to the PyTorch library path.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…(review) - _detected_gfx: append '|| true' so a no-GPU rocminfo (empty pipeline, non-zero under pipefail) doesn't abort the assignment before the '[ -z ]' branch prints the diagnostic + die message. - smoke-test librocdxg copy: gate on '[ -d "$_tlib" ]' instead of '[ -n ]' so a non-directory value can't make cp rename librocdxg to 'lib'.
|
ci passing |
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d1834afcb5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…est locator) - Exclude gfx000 (the CPU agent) from the WSL 'usable ROCm' check and the bootstrap arch-detect: match gfx[1-9] (nonzero arch), so a partial ROCm install that only reports the CPU ISA no longer short-circuits the librocdxg setup. (P2) - Reuse the Ubuntu-24.04 reroute for discrete Radeon: broaden _maybe_reroute_strixhalo_to_2404's gate with the same _wsl_amd_gpu_name (WMI) fallback, so a discrete card on 26.04 reroutes to a 24.04 distro like Strix does instead of falling to CPU. Moved _wsl_amd_gpu_name above the reroute and made it self-contained + 10s-bounded (it runs before _run_bounded is defined). (P2) - Update TestInstallShDropinPersistence to locate the gate by its unique '!/generic/' clause now that the gfx1151 literal is gone. (P1)
|
All outstanding review items here were correct and are addressed at head ( Gemini
Codex
Also condensed the new comments in both files (comments only, no behavior change): |
|
@codex review |
1 similar comment
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 03ff51cec1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if ! grep -qiE 'Ryzen AI Max|Radeon 80[0-9]0S|Strix Halo' /proc/cpuinfo 2>/dev/null \ | ||
| && ! _wsl_amd_gpu_name >/dev/null 2>&1; then |
There was a problem hiding this comment.
When a WSL host has an AMD integrated adapter alongside a usable NVIDIA GPU, this WMI fallback returns true before the installer ever reaches the NVIDIA guard in _maybe_bootstrap_rocm_wsl or get_torch_index_url. On a non-24.04 distro with an Ubuntu-24.04 distro present, the installer reroutes there and exits after the child install, so the current distro never gets the CUDA path even though NVIDIA would work. Please skip this AMD reroute when a usable NVIDIA GPU is present, or make the WMI fallback distinguish the intended AMD-only target.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in be8cf07: _maybe_reroute_strixhalo_to_2404 now probes nvidia-smi -L (awk-consumed, so pipefail-safe) before the AMD/WMI signal and returns early when a usable NVIDIA GPU is present, so a hybrid AMD+NVIDIA host keeps the CUDA path on this distro instead of rerouting for AMD. Added a regression test.
| die "rocminfo did not enumerate any GPU agent. Most common cause: the Windows AMD driver predates production ROCDXG -- update Adrenalin (install.ps1 offers this), reboot, and re-run." | ||
| fi | ||
| # If the caller pinned an arch honour it (sanity-check), otherwise adopt what we found. | ||
| if [ -n "$GFX" ] && ! printf '%s\n' "$_rocminfo_out" | grep -qE "Name:[[:space:]]*${GFX}([^0-9]|$)"; then |
There was a problem hiding this comment.
grep -q under pipefail for the GFX override
When UNSLOTH_WSL_GFX is set and rocminfo output is large, this grep -q can find the match and exit before printf finishes; because the script runs with set -o pipefail, the upstream SIGPIPE makes the pipeline fail and the ! branch reports that the requested arch was not enumerated. That makes the documented arch override fail even when the GPU is present, so use a consuming awk/non--q check like the earlier probe.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in be8cf07: the UNSLOTH_WSL_GFX override sanity-check now uses a consuming grep -E ... >/dev/null instead of grep -q, so an early match cannot SIGPIPE printf under pipefail and misreport the arch. Reproduced the false 'not enumerated' die on large rocminfo output and confirmed the fix; added a regression test.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be8cf07d6e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if command -v nvidia-smi >/dev/null 2>&1 && \ | ||
| $_rr_nvsmi -L 2>/dev/null | awk '/^GPU[[:space:]]+[0-9]+:/{f=1} END{exit !f}'; then | ||
| return 0 |
There was a problem hiding this comment.
When a mixed AMD+NVIDIA WSL host sets CUDA_VISIBLE_DEVICES to "" or -1 to force the AMD path, the later _has_usable_nvidia_gpu deliberately treats NVIDIA as unavailable, but this early inline nvidia-smi probe still returns here as soon as nvidia-smi -L lists a card. On a non-24.04 distro that prevents the AMD reroute/skip logic from running, and the later ROCm bootstrap can proceed in the unsupported origin distro instead of rerouting to Ubuntu 24.04.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 9b7a358: the inline NVIDIA guard now honors CUDA_VISIBLE_DEVICES=''/-1 (treats NVIDIA as hidden and falls through to the AMD reroute), matching _has_usable_nvidia_gpu. Verified the four CVD states (unset and 0,1 skip the reroute; '' and -1 fall through) and extended the regression test.
|
@codex review |
1 similar comment
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9b7a35891a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| elif command -v nvidia-smi >/dev/null 2>&1; then | ||
| _rr_nvsmi="nvidia-smi"; command -v timeout >/dev/null 2>&1 && _rr_nvsmi="timeout 10 nvidia-smi" | ||
| if $_rr_nvsmi -L 2>/dev/null | awk '/^GPU[[:space:]]+[0-9]+:/{f=1} END{exit !f}'; then | ||
| return 0 | ||
| fi | ||
| fi |
There was a problem hiding this comment.
On hybrid AMD+NVIDIA WSL hosts where nvidia-smi is not on PATH (or times out) but the NVIDIA driver is still exposed via /proc/driver/nvidia/gpus, this guard misses the usable CUDA GPU and the new WMI AMD signal below can reroute the installer to Ubuntu 24.04 unnecessarily. The later _has_usable_nvidia_gpu helper explicitly treats that proc path as a usable NVIDIA fallback, so this early inline probe should mirror it before deciding the AMD/ROCm reroute is needed.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 4dcbb74: rather than keep mirroring pieces of _has_usable_nvidia_gpu inline, the reroute now reuses it directly. Moved _has_usable_nvidia_gpu (and its _run_bounded / _cvd_hides_nvidia deps) above the reroute call site, so the guard now covers the /proc/driver/nvidia sysfs fallback, the /usr/bin/nvidia-smi path, CUDA_VISIBLE_DEVICES, and the bounded probe, identical to the bootstrap guard. Updated the regression test.
|
@codex review |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
2 participants
What
Generalize the ROCm-on-WSL setup so
curl install.sh | shworks on discrete Radeon cards (RX 7000/9000) in WSL2, not only Strix Halo/Point APUs. The path is arch-agnostic: it detects any AMD Radeon GPU and auto-detects its gfx arch, so it supports RDNA 3/4 in principle. Actual per-card coverage depends on AMD's ROCm-on-WSL / librocdxg driver support for that GPU (see the test matrix below).Why
WSL2 exposes the GPU only over
/dev/dxg(no/dev/kfd), so ROCm needs the librocdxg DXG↔HSA bridge. The existing path (scripts/install_rocm_wsl_strixhalo.sh+_maybe_bootstrap_rocm_wslininstall.sh) only fired for Strix APUs (matched via/proc/cpuinfo) and hardcodedgfx1151. Discrete Radeon cards don't appear in/proc/cpuinfoand have a different arch, so they fell through to CPU even though the ROCm + librocdxg setup itself is arch-agnostic.Changes
scripts/install_rocm_wsl_strixhalo.sh): auto-detect the GPU arch fromrocminfo(overrideUNSLOTH_WSL_GFX), verify any GPU agent enumerates over DXG, map arch → AMD wheel family for the optional smoke test (+ inject librocdxg into torch/lib). No longer Strix-locked.install.sh: add_wsl_amd_gpu_name()(queries the Windows host via WMI, since a discrete GPU's arch is unknowable in WSL until ROCm is up), and broaden the bootstrap trigger + the "already-usable ROCm"rocminfocheck fromgfx1151-only to any real GPU agent (excluding thegfx11-genericfallback ISA).Verified end to end (RX 9060 XT / gfx1200, WSL2 + Ubuntu 24.04)
rocminfoenumerates gfx1200 over DXG; arch auto-detected.install.sh: detectedAMD ROCm (gfx1200), routed torch to repo.radeon.com (rocm-rel-7.2.1) since pytorch.org's CDN TLS-fails in WSL →torch 2.9.1+rocm7.2.1, bitsandbytes prerelease, unsloth. Studio installed.torch.cuda.is_available()= True, device = "AMD Radeon RX 9060 XT".import unslothOK; a real 4-bit LoRA fine-tune trained on the GPU: loss4.49 → 4.02 → 3.58 → 3.29(no NaN), final3.97.Test matrix (help wanted)
Same code path; each needs someone with the card to confirm ROCm-on-WSL enumerates it.
Strix Halo (gfx1151) behaviour unchanged.