Repository navigation
feat(amdsmi): add WSL GPU backend support [Temporary] - #9725
Merged
Merged
Conversation
Introduces a feature-flagged WSL/WDDM GPU backend (originally PR #8745, reverted by #9527) so amd-smi can answer GPU queries through a WSL/WDDM (D3DKMT via /dev/dxg) path instead of native DRM/sysfs, without duplicating the tool or scattering wsl-vs-native branches. - IGPUBackend interface with a WSLGPUBackend implementation dispatched through gpu_device->backend(); unsupported queries return NOT_SUPPORTED. - AMDSmiWslGPUDevice auto-detects WSL via /dev/dxg at amdsmi_init() and populates real GPU handles via the HSAKMT enumeration path. - Symbol table (g_wsl_syms) loads rocdxg_smi_*/hsaKmt* function pointers via dlopen("librocdxg.so.1"); the binary never hard-links the WSL library. - rocdxg_smi.h/cpp in libhsakmt: the rocdxg_smi_* C ABI bridging the WDDM device layer to amdsmi queries (asic, board, vram, power, temp, clocks, PCIe, driver, vbios, process list). - Hooked APIs: asic/board/vram/driver/vbios/cache/fw/pcie/clock info, GPU metrics, power cap, fan speed/rpm, ECC count, KFD info, VCN busy percent, and fabric telemetry. - Gated two ways: compile-time ENABLE_WSL_BACKEND (default OFF), and runtime auto-detection of /dev/dxg. - Test coverage under tests/amd_smi_test/{unit,functional}/wsl/; excluded from the main binary when librocdxg is not present at build time (e.g. TheRock CI stages that build before wsl-rocdxg). - Adds a how-to doc (docs/how-to/amdsmi-wsl-mode.md). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bill-shuzhou-liu
requested review from
cfreeamd,
chrispaquot,
dayatsin-amd,
fcui-amd,
gandryey and
kentrussell
as code owners
August 5, 2026 18:51
github-actions
Bot
added
documentation
✅ All Policy Checks Passed
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
3 tasks
chrispaquot
approved these changes
Aug 7, 2026
adam360x
approved these changes
Aug 7, 2026
gabrpham
approved these changes
Aug 7, 2026
dayatsin-amd
requested changes
Aug 10, 2026
dayatsin-amd
changed the title
dayatsin-amd
self-requested a review
August 10, 2026 17:34
dayatsin-amd
approved these changes
Aug 10, 2026
Delete dead empty namespace, restore full MIT headers on 3 new WSL files, fix stale file-location comments, fix a shadowed variable, restore deleted "why" comments in native device population, dedupe the HSAKMT status-mapping helper, drop an unused CMake compile definition, add the librocdxg.so.1 prerequisite to the WSL docs, and bound-check both source and destination arrays in GetGpuCacheInfo/ GetFwInfo to prevent OOB reads/writes.
…untime amdsmi_get_gpu_compute_process_info(_by_pid/_gpus) and the TTM pages-limit APIs lacked a WSL backend check, so they fell through to native sysfs/KFD code paths that don't work under WSL, returning ambiguous errors or zeroed-out data instead of AMDSMI_STATUS_NOT_SUPPORTED. Guard them with WSLGPUBackend::IsActive() so behavior is correct at runtime regardless of privilege level, matching the pattern used by amdsmi_get_npm_info. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…lify ROCDXG guard Reorder feature-support checks ahead of nullptr checks across the WSL backend dispatch in amd_smi.cc and WSLGPUBackend methods, so unsupported queries consistently return AMDSMI_STATUS_NOT_SUPPORTED even when called with a null output pointer. Drop the now-always-true AMDSMI_HAS_ROCDXG_SMI compile guard in amd_smi_wsl_device.cc in favor of a CMake-time check that hsakmt/rocdxg_smi.h exists when ENABLE_WSL_BACKEND=ON, failing the configure step early instead of silently degrading every WSL query to NOT_SUPPORTED. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
cfreeamd
reviewed
Aug 16, 2026
cfreeamd
left a comment
cfreeamd
left a comment
Contributor
There was a problem hiding this comment.
🤖 AI-assisted pre-review (for @cfreeamd)
Reviewed the rocr portion (projects/rocr-runtime/libhsakmt/ — the new librocdxg SMI shim + dxg plumbing). The backend is carefully written: the D3DKMT_ENUMPROCESSES struct is pinned to the decoded dxgkrnl ioctl size via static_assert, outputs are zeroed up front, null params are rejected, and the enum_processes copy-back is bounds-safe. The debug.cpp/queues.cpp changes are pure relocations of NOT_SUPPORTED stubs (no duplicate definitions), and all 9 new symbols are exported in librocdxg.ver with correct SPDX headers on the new files. Inline notes are mostly Questions about the Wkmi/dxgkrnl ABI (PMLog data-vs-limits indexing, PID-as-handle) plus one data-quality concern in get_gpu_metrics_info. amdsmi files are out of my rocr review scope.
- Fix PMLog sensor limits indexing: PmlogSensorLimits.limits[] is id-indexed like PmlogQueryResult, not slot-indexed, so power_limit now reads the correct sensor slot instead of a mismatched one. - Make PCIe info resilient to QueryChipsetId failure: still report dynamic PMLog-derived width/speed even when static chipset fields are unavailable on older librocdxg versions. - Populate subvendor_id/subsystem_id via D3DKMTQueryAdapterInfo(KMTQAITYPE_PHYSICALADAPTERDEVICEIDS). - Guard GetGpuMetricsInfo fields against the UINT32_MAX sentinel rocdxg returns for absent sensors so it isn't truncated into a bogus uint16_t reading. - Return NOT_SUPPORTED for amdsmi_get_gpu_process_list under the WSL backend: the WSL process-enumeration ioctl reports Linux PIDs, but the per-process VRAM query needs a real Windows process handle. - Document the process-list limitation in amdsmi-wsl-mode.md. Verified on WSL test hardware: SUBVENDOR_ID/SUBSYSTEM_ID populated, power_limit/power_cap correct, PCIe width reported despite QueryChipsetId failure, amdsmitst clean with/without sudo. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
marifamd
approved these changes
Aug 17, 2026
1 task
amd-justchen
added a commit
that referenced
this pull request
Sep 2, 2026
…mers (#10034) ## Motivation `librocdxg` already exposes the WSL GPU topology through the standard KMT entry points: * `hsaKmtOpenKFD()` * `hsaKmtAcquireSystemProperties()` * `hsaKmtGetNodeProperties()` * `hsaKmtReleaseSystemProperties()` * `hsaKmtCloseKFD()` This PR does not introduce a new topology interface. It fixes the lifetime and ownership issues that arise when multiple components in the same process use the existing DXG topology path. ROCr is already one consumer, AMD SMI added a WSL GPU backend in #9725, and #7016 adds rocprofiler-sdk as another. AMD SMI, for example, acquires the topology in `WSLGPUBackend::TryPopulate()` and holds it until `Shutdown()`. The issues addressed here are: * the DXG topology snapshot was process-global and unreferenced, so one consumer could release topology still in use by another; * `hsaKmtOpenKFD()` reset the fragment allocator on every open, including when allocations from another consumer were still live; * `hsaKmtCheckRuntimeDebugSupport()` acquired a topology snapshot without releasing it. This PR was originally broader. At the reviewers' request, changes not required for topology access have been split into two separate PRs so this one can be reviewed as the topology prerequisite alone: * independent DXG fixes: `NumCUPerArray`, `KFDGpuID`, failed-open descriptor cleanup, and the `librocdxg.pc` prefix; * failed-initialization unwind: `KfdLifecycle`, `Runtime::Load()` rollback, thunk API validation, and DTIF ownership. ## Technical Details ### Topology snapshot reference counting A new `src/dxg/snapshot_refcount.hpp` isolates the reference-count transitions so they can be tested without a GPU, DXCore adapter, or open thunk. `hsaKmtAcquireSystemProperties()` now increments the reference count when reusing an existing snapshot and initializes the count when publishing a newly built snapshot. `hsaKmtReleaseSystemProperties()` destroys the snapshot only when the final reference is released. An unmatched release is rejected rather than tearing down topology still owned by another consumer. Failed topology construction is fully unwound so a later acquire cannot reuse partially initialized state. The final `hsaKmtCloseKFD()` also releases any remaining snapshot before DXCore is destroyed, because the snapshot contains `WDDMDevice` objects owned by that session. ### `hsaKmtOpenKFD()` open count and allocator lifetime Both the first-open and already-open paths maintain `dxg_open_count`, so it tracks the number of live consumers rather than saturating at one. The already-open increment is pre-existing code and appears only as unchanged context between two diff hunks, which can make the change look one-sided when reviewing the patch. The functional change is the placement of `reset_suballocator()`. It now runs only on the first open. Previously, a second consumer could reset allocator bookkeeping while allocations from the first consumer were still live. ### Fork handling `child_fork_handler()` clears the inherited open count and snapshot references using plain stores only, keeping the atfork child handler minimal. `clear_after_fork()` later abandons the parent's `WDDMDevice` pointers without running their destructors, since doing so would issue D3DKMT calls using handles created by the parent process. The last-close path follows the same ownership rule in a forked child. `hsaKmtReleaseSystemProperties()` now verifies that DXG is open before releasing a reference, preventing a child from releasing topology it never acquired. `hsaKmtOpenKFD()` also calls `clear_after_fork()` before taking the runtime lock. Because `clear_after_fork()` may replace the runtime object, taking the lock first could leave the `lock_guard` referring to freed storage. ### Debug support query `hsaKmtCheckRuntimeDebugSupport()` previously called `hsaKmtAcquireSystemProperties()` without releasing the acquired reference. It now takes no snapshot reference of its own and returns `NOT_SUPPORTED` when the device list is empty, leaving topology ownership to the caller. This change must land together with the explicit acquisition in `KfdDriver::Init()`: the debug probe was where ROCr previously acquired its snapshot reference as a side effect. Removing that behavior without adding the explicit acquisition would leave ROCr holding no reference at all. ### ROCr: accepting an already-open DXG thunk `KfdDriver::Open()` now treats `HSAKMT_STATUS_KERNEL_ALREADY_OPENED` as success when `thunkLoader()->IsDXG()`. This is valid for DXG because `librocdxg` reference-counts opens, so the already-open path has acquired this consumer's handle reference. The native thunk retains the existing behavior. `KfdDriver::Init()` now explicitly acquires the driver's long-lived topology reference before runtime enablement instead of relying on the former side effect of the debug-support query. Failure paths release exactly what they acquired. ## Relationship to [#7016](#7016) #7016 is the rocprofiler-sdk side of the WSL profiling work. It obtains topology through the same five existing KMT entry points and requires no new topology API. The dependency on this PR is the snapshot lifetime mechanism. On `develop`, `hsaKmtReleaseSystemProperties()` destroys the single global snapshot unconditionally. With multiple consumers, that means one consumer can release topology still held by ROCr or AMD SMI. The normal `rocprofv3` path in #7016 is safe through initialization ordering: its open/acquire/release/close sequence completes from the library constructor before `main()`, and therefore before the application's `hsa_init()` or `amdsmi_init()`. Late attach has no equivalent ordering guarantee. For that reason, #7016 temporarily disables WSL enumeration when loaded through `rocprof-attach`. That restriction can be removed once this PR is available. There is no CI, submodule, or build-time mechanism enforcing merge order between #7016 and this PR. They may merge in either order. The only file currently modified by both is `libhsakmt/CMakeLists_wsl.txt`: this PR adds the `BUILD_ROCDXG_TESTS` wiring, while #7016 adds PM4 frame-ring test wiring. The changes merge cleanly today, as verified with `git merge-tree`. ## JIRA ID JIRA ID: ROCM-23912 ## Test Plan ## Test Result ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
AMD SMI targets native Linux, reading GPU telemetry through the amdgpu kernel driver via DRM ioctls and sysfs. Under Windows Subsystem for Linux 2 (WSL2), neither interface exists — there is no /sys/class/drm GPU tree and no /dev/kfd. The GPU is only reachable through the Windows WDDM display driver via the D3DKMT interface exposed by /dev/dxg.
This PR reintroduces a feature-flagged WSL/WDDM GPU backend (originally #8745, reverted by #9527) so amd-smi can answer GPU queries through this WDDM path instead of native DRM/sysfs, without duplicating the tool or scattering wsl-vs-native branches throughout the codebase.
Technical Details
Note: The librocdxg.so part is a temporary workaround and will be removed in the future, where the amdsmi will link to libwkmi.a direclty.
Issue Tracking
JIRA ID: ROCM-28065
Test Plan
Both configurations (ENABLE_WSL_BACKEND=OFF and =ON) build cleanly end-to-end;
Also run sanity test on wsl machine.
Test Result
The WSL-enabled binary embeds the expected /dev/dxg / librocdxg dlopen path while the default build contains none of it.
Submission Checklist