[dlamd1dai]

This PR is the rocSHMEM half of the CPX GIN fix. It also adds anvil_engine_map.hpp to RCCL's --rocshmem-gin copy list, so that header lands with anvil.hpp.

#12742 depends on this one and must merge after it: gin_anvil_sdma_factory.cc calls disconnectDevice() and its test calls oamMapEngineNeedsFold() and pciFunctionBus(), all added here. Those call sites sit inside #ifdef ENABLE_ROCSHMEM_GIN, so default builds of #12742 are unaffected in the meantime. The engine foldOamMapEngine() picks is pinned only by this PR's anvil_engine_map_gtest, not by RCCL's tests, because #13237 changes it.

Summary

GIN Anvil SDMA (NCCL_GIN_TYPE=7) could not initialize on MI300X in CPX mode. Two defects live in rocSHMEM.

1. getOamId read a sysfs node that does not exist for a partition. A CPX partition is a HIP alias at PCI function .1–.7 with no sysfs node of its own, so the xgmi_physical_id open failed. Fall back to the physical function-0 BDF, and fold the PCI function into the engine id so same-device peers do not all land on engine 0.

2. Engine selection used an SPX-only engine map. mi300xOamMap indexes the 14 xGMI SDMA engines of an unpartitioned MI300X; a CPX partition owns 2 host engines and 0 xGMI engines, so every map id was out of range and hsaKmtCreateQueueExt returned INVALID_PARAMETER. Fold the undoubled map value plus the PCI function onto an even engine in the reported range, and report failure through SdmaQueue::valid() so connect() fails and the caller can drop the backend instead of aborting. A rejected engine-pinned queue is not retried as a generic HSA_QUEUE_SDMA, because that retry cannot choose the engine or doorbell slot.

Queue budget. KFD caps user SDMA queues per engine. This PR counts only the even engines it can use: 1 on CPX (8 queues), 2 on QPX (16), 4 on DPX (32), and 8 on SPX (64), before the queues ROCm already holds. #13237 raises this to the GPU-reachable slots of every engine (12 / 24 / 48 / 96). Usage is counted across those engines and an over-subscribed request is refused up front rather than letting KFD return NO_MEMORY part way through. In RCCL, an over-subscribed mesh fails ncclCommInitRank rather than degrading to a non-GIN path; that behaviour is pre-existing for any connect() failure, and an RCCL-side graceful fallback is a follow-up listed on #12742.

New API. disconnectDevice(int dstDeviceId) releases the queues held for one destination and decrements the budget by only the valid queues it removed. The channel map is process-global and a second communicator reuses a previous create's queues, so a failed create must not take disconnect()'s whole-process path. sdmaHostInit's failure path and sdmaHostStop both use it, so no rocSHMEM path clears the process-global map any more.

rocSHMEM init fails fast on an SDMA connect failure. sdmaEnabled also selects the alltoall algorithm, so turning it off on one rank while its peers keep SDMA leads to mismatched synchronization and a hang. A connect failure now aborts init instead. #13235 (AICOMRCCL-2795) replaces this with a fallback that every node-local rank agrees on.

Testability. The pure engine-selection and budget helpers (isSdmaPartition, oamMapEngineNeedsFold, foldOamMapEngine, pciFunctionBus, isUsableSdmaEngine, usableSdmaEngines, preferredUsableEngine, queueBudgetExceeded) live in anvil_engine_map.hpp, which has no HSA/KFD dependency, and are covered by an unconditional rocSHMEM gtest (anvil_engine_map_gtest.cpp). AnvilLib::queuesUsed() exposes the budget count for tests.

Even engines here; slot-aware use of odd engines is #13237

ROCM-32598 is a doorbell-slot defect, not a blanket failure of every odd engine. In every MI300 compute-partition mode, GPU-written doorbells never reach slots 0–3 of an odd-numbered SDMA engine. Slots 4–7 of those engines work, and every slot of an even engine works. Anvil rings every doorbell from device code.

This PR cannot choose the slot KFD assigns, so it conservatively places every queue on an even engine:

• an HSA preferred mask uses its lowest even engine, or the even engine below an odd-only mask
• the OAM-map fold spreads peers across the even engines the node reports; CPX has only engine 0
• connect() refuses an odd id
• a rejected pinned create fails instead of retrying as a generic queue

That closes the earlier DPX hang, where pairs spanning four or more OAMs folded onto engines 1, 5, and 7, and the CPX hang where peers folded onto engine 1 (AICOMRCCL-2880).

#13237 is the follow-up that recovers the usable odd-engine capacity. After each pinned create it checks the doorbell address, parks a queue whose slot a GPU write cannot reach, and lets the next create on that engine take a reachable slot. This PR does not depend on that change.

A full-node 16-rank DPX run can still hang intermittently with -w 2. That is a separate ring exception on a reachable slot (ROCM-32719), documented in #13237, and not caused by the engine selection here.

Test plan

• AnvilEngineMap.*: 19/19 pass on banff-cyxtera-cx56-5 in SPX, DPX, QPX, and CPX, at b47b4f7ccf.
• Functional alltoall (-a 19 -w 1) with ROCSHMEM_SDMA_ENABLED=1 on that node. Traced queues used even engines only: SPX 0/2/4/6/8/10/12/14, DPX 0/2/4/6, QPX 0/2, CPX 0. SPX 8 ranks reached 1.70–1.74 TB/s; DPX 16 ranks at 1–2 channels reached 1.05–1.15 TB/s; CPX 8 ranks at 1 channel reached 320–324 GB/s. No generic-queue retry was logged.
• Over-budget and rejected pinned creates abort init instead of hanging. CPX at 2+ channels and QPX at 32 ranks hit the even-engine queue budget.
• rccl-UnitTestsGinAnvilSdmaFactory (the test binary in fix(rccl): GIN gate and cuMem window registration for MI300X CPX #12742) on a CPX node, run on the combined change before the split: 12/12 pass, none skipped (17/17 including the AnvilOamMapEngine cases). Run on dell300x-ccs-aus-k13-09 (8x MI300X) switched to CPX. Create_Success_1Rank asserts tryCreate(...) == 0, which is only reachable once createStatus() == HSAKMT_STATUS_SUCCESS and valid(), so an SDMA queue really was created on a partition.
• install.sh -l --rocshmem-gin builds clean (0 errors) against ROCm 7.13.

Review history for the combined change lives on #12603 (closed) and #12742.

JIRA ID: AICOMRCCL-2387

[dlamd1dai]

Carried over from #12742

projects/rocshmem/src/sdma/anvil.cpp moved to this PR during the CODEOWNERS split, so two unresolved Argus findings against it no longer have a diff to attach to on #12742. Reposting both here so they stay actionable. Neither is a correctness bug; both are testability asks on code that is in this PR.

---

Low — pciFunctionBus is unreachable from a test TU
Original: #12742 (comment)

oamMapEngineNeedsFold and foldOamMapEngine moved to anvil.hpp this round and now have five cases on them, and pciFunctionBus is equally pure but still unreachable from a test TU. It decides the fallback: a tail outside '0'-'7' leaves function at -1 and physBusId equal to busId, which collapses nCandidates to 1 at line 662 so the physical-BDF read never runs. The logic itself checks out, since PCI function numbers are three bits. Could it move beside the other two, so the digit range, the empty input and the .1 rewrite get asserted?

---

Low — the abandon() teardown path has never executed under test
Original: #12742 (comment)

Every negative case in GinAnvilSdmaFactory_test.cpp returns at gin_anvil_sdma_factory.cc:58, :66 or :72, before abandon() at :97 is reachable, so neither this check nor the teardown it guards has ever executed under test. Forcing most of these needs hardware, and an invalid device id would route through getBusId into std::exit, so the one thing that would help is an observable rather than a log line. Would a uint32_t queuesUsed() const on AnvilLib let a hardware-gated case assert the count is unchanged after a refused connect and zero after a failed create?

Note that the second one spans both PRs: abandon() and GinAnvilSdmaFactory_test.cpp live in #12742, while the queuesUsed() observable it proposes would go on AnvilLib here. The disconnectDevice() method this PR adds is the fix for the related High finding on that same teardown path.

[therock-pr-bot]

✅ All Policy Checks Passed

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

[dlamd1dai]

Both carried-over findings implemented

Pushed as e58d5c5c5e on this PR, with the tests as dfe5f76190 on #12742.

pciFunctionBus was unreachable from a test. It was a file-static in anvil.cpp. It is pure, and it picks the same fallback as oamMapEngineNeedsFold and foldOamMapEngine — a tail outside '0'–'7' leaves function at -1, which collapses getOamId's candidate list to one entry and skips the physical-BDF read — so it now sits beside them in anvil.hpp as inline rather than hidden in the translation unit.

The queue budget was only observable through a log line. That left connect()'s refusal and disconnectDevice()'s teardown with nothing to assert against: a refused connect has to leave the count unchanged, and a fully abandoned create has to return it to where it started. AnvilLib::queuesUsed() makes that readable without widening anything else.

No functional change on this PR — both are visibility only.

Coverage (in projects/rccl/test/gin/GinAnvilSdmaFactory_test.cpp, since that is where the existing AnvilOamMapEngine cases live):

• AnvilPciFunctionBus — a .1 tail yields function 1 with the function-0 BDF of the same device; every digit '0'–'7' parses; function 0 collapses getOamId to one candidate; '8' and '9' are refused. Those last two are the ones that matter, since a std::isdigit test would accept them and name a function PCI cannot express.
• ParsesTheLastCharacterOnlyNotTheBdfShape — pins the parse as last-character-only, so a BDF with no function field is read as function 0. getBusId always writes a ".N" suffix so this does not arise today; the case records which invariant getOamId leans on, because losing that suffix would silently stop the fallback read from happening rather than fail.
• AnvilQueueBudget.FailedConnectDoesNotChargeTheBudget — an over-subscribed connect leaves queuesUsed() where it was, covering both of connect()'s failure exits.

24/24 pass on an MI300X node (gfx942, ROCm 7.0.1).

Note the merge order: this PR has to land before #12742, since the tests there reference sdma_anvil::pciFunctionBus and queuesUsed().

[dlamd1dai]

therock-pr-bot cannot pass on RCCL or rocSHMEM PRs — requesting a policy override

Per the FAQ, the override path is "contact CODEOWNERS or supporters channel", so flagging it here. This is not specific to this PR — it affects any PR touching only projects/rccl/** or projects/rocshmem/**.

Root cause. tools/systems_pr_bot/policy.yml declares one required external check:

checks:
  required_check_runs:
    - pre-commit

No workflow creates a pre-commit check run for these paths. The pre-commit job name comes from the per-project formatting workflows, and each is path-filtered to projects that are not these two:

Workflow	Paths
cuid-formatting.yml	projects/cuid/**
lstt-formatting.yml	projects/rdc/**, projects/amdsmi/**, projects/rocm-smi-lib/**
pre-formatting.yml	only PRs targeting branch testbranch
rocjitsu-formatting.yml, rocprofiler-*-formatting.yml, hip-formatting.yml, profiler-hub-formatting.yml	their own projects

Grepping all nine for projects/rccl or projects/rocshmem returns zero matches. So policy_check.py polls for a check run that is never created, and after its 900-second budget exits 1 with:

❌ Timed out waiting for required checks to complete.
{
  "therock-pr-bot": "null",
}

The dump lists every check run that does exist on the SHA; pre-commit is absent from it, including on the original push-triggered run.

Why rerunning does not help. I tried gh run rerun --failed and a full-workflow rerun. Both reproduce the above verbatim. The documented remedy of an empty commit does not help either, since pre-commit still will not run.

Worth noting the policy content passes — the bot's own comment reports "✅ All Policy Checks Passed" (the one Unit Test item is a ⚠️ warning, not a failure). It is only the gate step that times out, and policy.yml:151 already warns about a related circular-dependency hazard in this same list.

Suggested fix (for whoever owns the bot): either extend a formatting workflow to cover projects/rccl/** and projects/rocshmem/** so pre-commit actually runs, or treat an absent required check as not-applicable rather than pending.

@ROCm/rccl-leads — flagging for the override.

[dlamd1dai]

@argus review

[amd-argus]

ARGUS

Bottom line: Worth settling before approval: connect() can still exit the process through getHipGpuAgent at anvil.cpp:531 when a HIP device has no mapped HSA agent, so the new fail-instead-of-abort path does not cover that case.

_{Comment @argus review to re-run, or @argus re-review after pushing.}

[dlamd1dai]

Pushed 086eaf338e, addressing all six findings from the last review:

• Medium (anvil.cpp:531): connect() now returns false when the source HIP device has no mapped HSA agent, so createSdmaQueue never reaches getHipGpuAgent's exit.
• Low (anvil.cpp:228): the queue-create error now logs the generic retry's own status (genericRetry=).
• Low (anvil.cpp:594): dropped the queueBudget > 0 guard on the post-create budget hint; the pre-flight check keeps it.
• Low (sdma_policy.cpp:79): a failed connect now breaks out of the loop instead of returning, so deviceHandles_d is still allocated (all null after disconnect()).
• Low (anvil.cpp:694): isSdmaPartition() in anvil.hpp is now the single copy of the predicate, used by both oamMapEngineNeedsFold and the log selection.
• Nit (anvil.hpp:53): alignment fixed.

oamMapEngineNeedsFold/foldOamMapEngine/pciFunctionBus signatures are unchanged, so #12742's tests are unaffected.

@argus re-review

[amd-argus]

ARGUS

Findings

2 finding(s) that could not be posted as inline comments

• projects/rocshmem/src/sdma/anvil.cpp:195 (Low) The constructor's alloc and map failures still abort, so the valid() contract does not cover the whole graceful path At anvil.cpp:195 and :197, hsaKmtAllocMemory and hsaKmtMapMemoryToGPU still go through CHECK_HSAKMT_SUCCESS, which is LOG_ERROR_EXIT, so the constructor can still kill the process before reaching the graceful path at line 231 and the valid() contract in anvil.hpp:131 does not cover them. Both lines are outside this diff, so purely a question: is finishing that path in scope here, or a follow-up?
• projects/rocshmem/src/sdma_policy.cpp:37 (Low) No in-repo build configuration sets USE_SDMA, so the new anvil code gets zero compile coverage USE_SDMA defaults OFF at projects/rocshmem/CMakeLists.txt:50, src/CMakeLists.txt:110 gates the sdma directory on it, CMakeLists.txt:337 forces it back OFF when libnuma is missing, and none of the scripts under scripts/build_configs enables it, including all_backends. So the new anvil code has no compile coverage in any configuration visible here. Is there a job outside this tree that builds with USE_SDMA=ON?

Bottom line: Worth settling before approval: the partition || fold in oamMapEngineNeedsFold is the fix itself and nothing in this repo exercises it, though tests/unit_tests already has the if (USE_GDA) pattern to host one.

_{Comment @argus review to re-run, or @argus re-review after pushing.}

[dlamd1dai]

Pushed b400b23c47 for the re-review findings. The four inline threads have replies. On the two general questions:

• Constructor alloc/map aborts (anvil.cpp:195, :197). Thanks for flagging these. I'd like to keep them as a follow-up: this PR makes the CPX path itself fail gracefully, and turning hsaKmtAllocMemory/hsaKmtMapMemoryToGPU into valid() == false exits is a separate hardening step for the constructor. Tracked as AICOMRCCL-2793.
• USE_SDMA compile coverage. Correct, no in-repo configuration builds USE_SDMA=ON. The only path is RCCL's --rocshmem-gin build (projects/rccl/cmake/ROCSHMEM.cmake), and the RCCL precheckin does not use that flag. That is why the engine-selection helpers now live in an HSA-free header with an unconditional gtest, so the CPX fix is covered regardless. Adding an SDMA build config to CI is out of scope here.

CI note: mci/rocm-libraries/precheckin(rocshmem) failed on node hyd-7c-zt05-02 in both of the last two runs, both times in the Docker stage before any build: a DNS timeout on registry-sc-harbor.amd.com, then a registry-1.docker.io timeout. The other nodes built and passed, so this looks like a networking issue on that node rather than this change.

@argus re-review

[amd-argus]

ARGUS

Bottom line: Worth settling before approval: sdma_policy.cpp:78 takes disconnect()'s whole-process path on a failed connect, which anvil.hpp:114 says a failed create must not do.

_{Comment @argus review to re-run, or @argus re-review after pushing.}

[amd-argus]

@dlamd1dai the current head (b400b23c) already has a review from me above; nothing has changed since then for me to look at again. Push a change and I'll pick it up, or ask again once you have.

[dlamd1dai]

Pushed two commits for the latest re-review:

• 3293f59031 addresses all six threads (replies inline).
• d5df570f34 applies the same scoping to normal shutdown: sdmaHostStop now calls disconnectDevice() for each local peer instead of disconnect(), so no rocSHMEM path clears the process-global channel map any more.

@argus re-review

[amd-argus]

ARGUS

Bottom line: Worth settling before approval: the per-rank sdmaEnabled fallback at sdma_policy.cpp:81 lets ranks pick incompatible alltoall algorithms and hang.

_{Comment @argus review to re-run, or @argus re-review after pushing.}

[dlamd1dai]

Pushed 0ac054782e for the High finding: rocSHMEM init now fails fast on an SDMA connect failure instead of falling back on one rank, and the cross-rank fallback is tracked in AICOMRCCL-2795. Reply inline.

@argus re-review

[amd-argus]

ARGUS

Bottom line: LGTM.

_{Comment @argus review to re-run, or @argus re-review after pushing.}

[dlamd1dai]

RCCL --rocshmem-gin build breaks with this PR's anvil.hpp

RCCL builds the Anvil SDMA host code by copying rocSHMEM sources into its own build tree. In projects/rccl/src/CMakeLists.txt (line 1198 at this PR's head f1742deb4f), the copy list is fixed:

foreach(_f anvil.cpp anvil.hpp)

This PR adds #include "anvil_engine_map.hpp" to projects/rocshmem/src/sdma/anvil.hpp (line 41, from b400b23c47). That header is not in the list, so it is never copied. install.sh --rocshmem-gin then fails while compiling RCCL's copy of anvil.cpp:

projects/rccl/build/release/gin/sdma/anvil.hpp:41:10: fatal error: 'anvil_engine_map.hpp' file not found
make[2]: *** [src/CMakeFiles/rccl.dir/build.make:5852: src/CMakeFiles/rccl.dir/__/gin/sdma/anvil.cpp.o] Error 1

Seen on banff-cyxtera-cx56-5 (MI300X, ROCm 7.2.0, gfx942). The tree was develop with this PR's rocSHMEM changes (as carried on users/dondai/aicomrccl-2880-anvil-cpx-engine0) plus the RCCL half from #12742. Neither this PR nor #12742 changes the copy list, and develop has the same list, so whichever lands the include also needs the RCCL change.

anvil_engine_map.hpp includes only <cstdint> and <string>, so it needs none of the sed rewrites that the copy step applies. Adding it to the list was enough for RCCL to build:

foreach(_f anvil.cpp anvil.hpp anvil_engine_map.hpp)

projects/rccl/ is owned by the RCCL reviewers, so this one-line change may fit better in #12742, as long as #12742 merges no later than this PR.

[abouteiller]

Minor nits on comments and log level, ready for merge after that

[abouteiller]

There may be a build issue with rccl dues to new file engine_anvil_map.hpp being required as a dependent in the rccl import recipe. rccl side adaptations would need to land at the same time.

[dlamd1dai]

Pushed aad8f6b.

• Removed the machine-specific comment and dropped the partition-engine log to LOG_TRACE.
• anvil_engine_map.hpp is now in RCCL's --rocshmem-gin copy list in this PR, so the header lands with anvil.hpp and a develop build does not depend on fix(rccl): GIN gate and cuMem window registration for MI300X CPX #12742.

therock-pr-bot will still time out waiting for a pre-commit check that no workflow creates for projects/rocshmem or projects/rccl. That is the same false failure as before; it needs the CODEOWNER policy override.

[amd-argus]

ARGUS

Bottom line: LGTM.

_{Comment @argus review to re-run, or @argus re-review after pushing.}

[amd-argus]

ARGUS

Bottom line: LGTM.

_{Comment @argus review to re-run, or @argus re-review after pushing.}

[systems-assistant]

TheRock Submodule Bump Activity

Newest first

• 2026-10-19 — Included in TheRock via ROCm/TheRock#9353.