Repository navigation
refactor(amdsmi): backport unified ABI changes to release/therock-10.0 - #9864
Merged
marbre merged 5 commits intoAug 18, 2026
Merged
Conversation
Widen five amdsmi_gpu_metrics_t accumulator counter fields from uint32_t to uint64_t to match the amdgpu pmfw MetricsTable_t definition and prevent overflow: - gfx_activity_acc - mem_activity_acc - pcie_nak_sent_count_acc - pcie_nak_rcvd_count_acc - pcie_lc_perf_other_end_recovery This is a breaking ABI change: the struct layout and field offsets shift, so callers reading these fields must be recompiled. The change cascades through the ROCm SMI headers (rocm_smi.h, rocm_smi_dyn_gpu_metrics.h) and the C++ implementation, and the ctypes and Rust wrappers are regenerated to match. A 7.15.0 CHANGELOG entry documents the widening. --------- Signed-off-by: Bindhiya Kanangot Balakrishnan <Bindhiya.KanangotBalakrishnan@amd.com> Signed-off-by: Maisam Arif <Maisam.Arif@amd.com> (cherry picked from commit aeb9da9)
A static code scan flagged an insecure pattern in the driver-reload path. The functionality was already removed from the amd-smi CLI, so remove the API entirely rather than leave a vulnerable, unused entry point. - include/amd_smi/amdsmi.h: drop the amdsmi_gpu_driver_reload() declaration and its tagDriverControl doxygen group. - src/amd_smi/amd_smi.cc: delete the implementation. - rocm_smi: remove the underlying driver-reload plumbing (rocm_smi.cc, rocm_smi_device.cc, rocm_smi_utils.cc and their headers). - py-interface: remove the Python binding and __init__ export; regenerate amdsmi_wrapper.py and the Rust wrapper to match. - Callers now reload the driver out of band with `sudo modprobe -r amdgpu && sudo modprobe amdgpu` to apply memory partition changes; examples, docs/conceptual/partition.md, and hw_compatibility_status.py are updated accordingly. - tests: rework memorypartition_read_write_test.cc and test_partition.py for the removed reload step and add test_memory_partition_lifecycle.py. - CHANGELOG: add the 7.15.0 Removed entry. Signed-off-by: gabrpham <Gabriel.Pham@amd.com> Signed-off-by: Maisam Arif <Maisam.Arif@amd.com> (cherry picked from commit 706209f)
The fabric (UALoE/IFoE) telemetry, fabric-info, and NIC firmware APIs were introduced in the in-progress ROCm 7.14.0 release but their declarations had drifted from the "unified" ABI that downstream consumers build against. The public surface carried inconsistencies that would force source churn on every caller and leave dead symbols in the header: a string-returning telemetry helper that couldn't report failure, a misspelled fabric-type enumerator (UALLINK), an intermediate versioning struct that added an extra access level, a NIC firmware typedef whose name didn't reflect that it describes a single entry, a duplicate telemetry-category enumerator, and a cluster of unreferenced max-count macros. Because the feature is still unreleased, this is the last clean opportunity to reconcile the headers with the agreed ABI before the symbols are frozen, avoiding a future breaking change once external code depends on them. - `amdsmi_fabric_telem_id_to_string()` signature changed from `const char* f(uint64_t)` to `amdsmi_status_t f(uint64_t, const char**)`, so the function can surface lookup failures via a status code and return the name through an out-parameter. Callers (the GTest, the example, and the Python interface) were updated to pass a `const char**` and check the status. - Flattened `amdsmi_fabric_info_t` by removing the intermediate `amdsmi_fabric_info_ver_t` type. Its payload union is now the `fabric_info` member of `amdsmi_fabric_info_t`, and its version field is exposed directly as the top-level `fabric_version`. Field access simplifies from `fabric_info.fabric_version.v1.<field>` to `fabric_info.v1.<field>`, and `fabric_info.version` becomes `fabric_version`. The change is ABI-preserving: field offsets and overall struct size are unchanged, and the Python `amdsmi_get_gpu_fabric_info()` dictionary keys are unchanged. - Typedef rename: `amdsmi_nic_fw_t` → `amdsmi_nic_fw_entry_t` (it models one firmware entry within `amdsmi_nic_fw_info_t::fw[]`), with the `AMDSmiAINICDevice` member updated to match. - Enumerator rename: `AMDSMI_FABRIC_TYPE_UALLINK` → `AMDSMI_FABRIC_TYPE_UALINK`, including the `UALoeLinkTypeMap` entry. - Removed symbols: the duplicate `AMDSMI_FABRIC_TELEMETRY_CATEGORY_UNKNOWN` enumerator (identical `0xFFFFFFFF` value to `..._INVALID`), and eight unreferenced public macros (`AMDSMI_MAX_VF_COUNT`, `AMDSMI_MAX_DRIVER_NUM`, `AMDSMI_DFC_FW_NUMBER_OF_ENTRIES`, `AMDSMI_MAX_WHITE_LIST_ELEMENTS`, `AMDSMI_MAX_BLACK_LIST_ELEMENTS`, `AMDSMI_MAX_TA_WHITE_LIST_ELEMENTS`, `AMDSMI_MAX_ERR_RECORDS`, `AMDSMI_MAX_PROFILE_COUNT`). - Cascade: the ctypes wrapper (`amdsmi_wrapper.py`) and Rust bindings (`amdsmi_wrapper.rs`) were regenerated; `amdsmi_interface.py` was updated by hand; the CHANGELOG records the Changed/Removed entries. JIRA ID : AILITOOLS-270 C++ functional tests Python CLI test All tests passed as expected --------- Signed-off-by: josnarlo <joseph.narlo@amd.com> Signed-off-by: Maisam Arif <Maisam.Arif@amd.com> Signed-off-by: Galantsev, Dmitrii <dmitrii.galantsev@amd.com> Co-authored-by: Maisam Arif <Maisam.Arif@amd.com> Co-authored-by: Galantsev, Dmitrii <dmitrii.galantsev@amd.com> Co-authored-by: Mika Laitio <mika.laitio@amd.com> (cherry picked from commit 923767a)
Signed-off-by: Maisam Arif <Maisam.Arif@amd.com>
github-actions
Bot
added
documentation
Contributor
Author
…mi path (#9464) ## Motivation `RCCL_USE_AMD_SMI_LIB=1` silently disabled UALoE fabric on hardware where it is present and `ready`, while the default sysfs path worked. At the same time, the default path printed a warning telling users to set that very variable to "enable fabric support" — advice that would have made things worse. These two together sent readers (and agents) chasing a configuration problem that did not exist. ## Technical Details **Version check (`src/misc/amdsmi_wrap.cc`)** `amdsmi_get_gpu_fabric_info()` returns `SUCCESS` with a correct payload but never writes `fabric_info.version`, which stays at the sentinel `amdsmi.h` documents for fields it cannot source from sysfs. Strict equality against `AMDSMI_FABRIC_INFO_CURRENT_VERSION` could therefore never pass against `libamd_smi.so.26`, so every device was dropped with a warning. A probe showed `version = 0xFFFFFFFF` alongside `fabric_type = UALOE`, `accel_state = READY` and an `accel_id`/`vpod` matching sysfs; pre-setting version is overwritten, so it is not an input field. The sentinel now means "unreported", and v1 is assumed, while a version the library actually claims is still rejected. **False warning (`src/misc/amdsmi_wrap.cc`)** The "RCCL_USE_AMD_SMI_LIB not set, but HIP_FABRIC_API is defined" warning predates sysfs fabric discovery via `ARSMI_get_fabric_info()`, which populates `amdsmiFabricDevices` in the same way. With the env var defaulting to 0 and `HIP_FABRIC_API` auto-detected on modern ROCm, it printed on every rank of every job, and it conflated `HIP_FABRIC_API` (a `hipMemFabricHandle_t` build flag) with which library does discovery. Replaced by an `INFO` naming the active path. **Struct layout compatibility (`src/include/amdsmi_wrap.h`)** amd-smi 27.0 flattened `amdsmi_fabric_info_t`: the version moved from `fabric_info.version` to a top-level `fabric_version`, and the payload from `fabric_info.fabric_version.v1` to `fabric_info.v1`. Field offsets did not move (version at byte 8, v1 at 12), so this is a source-level rename only. A template-based shape detector (`amdSmiFabricInfoIsFlat`) with `if constexpr` accessor functions handles either layout at compile time. `static_assert` guards pin the struct sizes (244/320 bytes) and `reserved` offset (256) to catch future ABI drift. **Telemetry ABI (`src/misc/amdsmi_wrap.cc`)** `amdsmi_fabric_telem_id_to_string` changed from returning the name to writing it through an out-parameter in 27.0. The two are not call-compatible and we reach it through `dlopen`, so the choice follows the version the loaded runtime reports. An unknown version assumes the out-parameter form: calling a 27+ implementation the old way would leave it writing through an uninitialized pointer, whereas calling an older one the new way merely leaves the name unwritten. **Minor** amd-smi's version string was printed as a fourth dotted component, rendering as `version 26.5.0.26.5.0`. Now it will be printed as `version 26.5.0 (build 26.5.0)`. ## JIRA ID AICOMRCCL-1656 Signed-off-by: Maisam Arif <Maisam.Arif@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Maisam Arif <Maisam.Arif@amd.com> Signed-off-by: Maisam Arif <Maisam.Arif@amd.com>
dmitrii-galantsev
approved these changes
Aug 12, 2026
1 task done
JeniferC99
approved these changes
Aug 13, 2026
astyrrian1
requested changes
Aug 14, 2026
astyrrian1
left a comment
astyrrian1
left a comment
There was a problem hiding this comment.
Please specify a Jira ID that aligns to a qualified ticket.
Contributor
Author
AIDREL-13 & AIDREL-19 |
astyrrian1
approved these changes
Aug 17, 2026
astyrrian1
left a comment
astyrrian1
left a comment
There was a problem hiding this comment.
please address question from marius
alexander-sannikov
approved these changes
Aug 18, 2026
Member
Every relevant information belongs into the PR description to propagate into the commit message, not into a comment. |
marbre
added a commit
to ROCm/TheRock
that referenced
this pull request
Aug 19, 2026
Bumps rocm-systems to include cherry-picks * ROCm/rocm-systems#9864 - JIRA ID AIRDEL-13, AIRDEL-19 * ROCm/rocm-systems#10315 - JIRA ID AIRDEL-24
marbre
added a commit
to ROCm/TheRock
that referenced
this pull request
Aug 19, 2026
Bumps rocm-systems to include cherry-picks * ROCm/rocm-systems#9864 - JIRA ID AIRDEL-13, AIRDEL-19 * ROCm/rocm-systems#10315 - JIRA ID AIRDEL-24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Bring the unified ABI changes onto the
release/therock-10.0release branch. These already-mergeddevelopcommits change the amd_smi ABI (64-bit metrics counters, driver-reload removal, unified struct sync) and must land together so the 10.0 header, wrapper and interface stay consistent with each other — and so 10.0 does not ship an ABI that differs fromdevelop.The two earlier attempts are closed: #9852 stacked these changes on the Python packaging series, and #9734 was that packaging series. With #9734 closed, the packaging rework is not coming to 10.0, so this PR applies the ABI changes to the raw release branch.
Technical Details
Five commits on the
release/therock-10.0tip (6ccde5dbc27). Four are cherry-picks of mergeddevelopcommits (-xprovenance preserved); one is a changelog retarget.15432e923ffix(amdsmi): Bump GPU metrics accumulator counters (#8210)aeb9da985b7ff1686439refactor(amdsmi): remove driver reload API and related functions (#8575)706209f4f523ec5edc37refactor(amdsmi): Sync with unified with known abi changes (#7844)923767a9539030ac095cdocs(amdsmi): set changelog version to 10.07.15.0to10.094cec8fda6fix(rccl): correct UALoE fabric detection and diagnostics on the amdsmi path (#9464)0f674c16e8Why the RCCL commit is part of this PR
Commit 5 is not incidental: #7844 does not build without it.
#7844flattensamdsmi_fabric_info_t, deletingamdsmi_fabric_info_ver_tand moving the version to a top-levelfabric_version. RCCL on the release branch hardcodes the old nested spelling:Both identifiers cease to exist after commit 3, so RCCL fails to compile against the new header.
#9464replaces those accesses with aamdSmiFabricInfoIsFlatSFINAE trait that compiles against both the flat and nested layouts, and fixes the UALoE detection bugs described in that PR. It is a large diff (8 files, +1140/−29, including new RCCL tests), so it is called out explicitly here.These five commits must land as a unit. Splitting the PR, or reverting the RCCL commit on its own, breaks the RCCL build.
Conflicts resolved in #7844
Three conflicts, all because the release branch lacks changes
#7844was written on top of:py-interface/CMakeLists.txt— kept the release version.#7844's only hunk here reworks the packaging site-packages install block, which does not exist on this branch (it came from the packaging series in the now-closed Cherry-pick AMDSMI commits into ROCm 10.0 #9734).py-interface/amdsmi_interface.py— keep-both: added theAMDSMI_GPU_CUID_SIZEalias (context ondevelop, which has feat(amdsmi): replace UUID with CUID #9273) alongside the release branch's_AMDSMI_STRING_LENGTHconstants.CHANGELOG.md— dropped the packaging-rework entry, which describes work not on this branch.#7844's ABI entries merged cleanly.Issue Tracking
JIRA ID: AILITOOLS-270, ROCM-1553, ROCM-26658
JIRA ID: AIDREL-13
JIRA ID: AIDREL-19
Test Plan
release/therock-10.0tip (6ccde5dbc27)amdsmitst)amdsmi_wrapper.pyand diffed against the committed filedeveloporiginalamdsmi_fabric_info_tflattening is ABI-preserving, by compiling both headers and comparingsizeof/offsetofTest Result
developoriginals (modulo hunk offsets); commit 3 differs only in the three conflicts documented abovesizeof(amdsmi_fabric_info_t) = 320before and after, with every offset unchanged —bdf=0, version=8,v1=12,v1.accel_state=252,reserved=256amdsmi_gpu_driver_reloadhas no remaining references outside the CHANGELOG; none of the 16 removed/renamed symbols has a stale consumer anywhere underprojects/projects/rdcreadsamdsmi_gpu_metrics_tand the widened accumulators and is not modified here — a-fsyntax-onlyA/B ofRdcMetricFetcherImpl.ccagainst the old and new headers produces the identical error set (pre-existing, ESMI-gated), so nothing regresses. The fields only widen, and they feed auint64_t-valued map.projects/rocshmemkeeps a private dlopen mirror ofamdsmi_fabric_info_t; it does not includeamd_smi/amdsmi.h, and the layout is unchanged, so it is unaffected.projects/rocprofiler-systemsusesamdsmi_gpu_metrics_tand builds green in this PR's TheRock CI.projects/rocm-smi-libhas its own independentrocm_smi.h; unaffected.Known issues inherited from
developFaithful cherry-picks, so these come along unchanged; they are present on
developtoday and are not introduced by this backport. Listed so the release branch owners know what is shipping. Fixes belong ondevelopfirst.gpu_metricsaccumulators lose their "unset" sentinel (from fix(amdsmi): [ROCM-1553] Bump GPU metrics accumulator counters #8210). The counters widened to 64-bit but the amdgpu table still reports 32, so the0xFFFFFFFFsentinel zero-extends and stops matching the 64-bit sentinel consumers check.amdsmi_get_gpu_metrics_info()reportsgfx_activity_acc/mem_activity_accas a literal4294967295instead ofN/Aon hardware that does not report them (reproduced on gfx1030). Unchanged on parts that do report real values.tests/python/cli/test_set.pyfails instead of skipping (from refactor(amdsmi): Sync with unified with known abi changes #7844).cmds_need_permissionis assigned insetUpClassafter the_initializedearly-return, soTestSetraisesAttributeError; and the filter that replaced the old blanket skip covers only 3 of the privilegedsetsubcommands, so an unprivileged run also fails on--perf-level,--profile,--perf-determinism,--clk-leveland others.amd-smi set --jsonandamd-smi static --partition --jsondisagree on one key (from refactor(amdsmi): Sync with unified with known abi changes #7844).setnow emitsaccelerator_partition_mem_alloc_modewhilestaticand the--compute-partition-mem-alloc-modeflag keepcompute_partition_mem_alloc_mode._AMDSMI_STRING_LENGTHand_AMDSMI_MAX_STRING_LENGTHwith zero references (every call site moved to the publicAMDSMI_MAX_STRING_LENGTH).Submission Checklist