[Kapil-Shyam-Pawar]

Motivation

On gfx950 CPX+NPS2 (64 ranks), all_gather_perf aborts during comm init even after #7766.

Issues identified:

• A bad-pointer Bad address in the topology bootstrap AllGather

NCCL WARN ncclOsSocketProgressOpt: Call to send to --- failed : Bad address

• A segfault from WarpSpeed over-allocating channels.

NCCL INFO WarpSpeed enabled: warpSpeedChannelMultiplier 4, maxNchannels 1760, nc 1760

Technical Details

• bootstrap.cc: ring slice offset slice * size overflowed 32-bit int with the ~102 MB per-rank topology buffer at 64 ranks
• connect.cc: single-node WarpSpeed channel count was unclamped (nc=1760), overrunning comm->channels[MAXCHANNELS]

JIRA ID

JIRA ID : ROCM-27078

Test Plan

• Run all_gather_perf on gfx950 CPX+NPS2 (64 GPUs)
• Add host unit tests for the 64-bit offset and the channel clamp

Test Result

• AllGather on CPX+NPS2 passes end-to-end with no bad address/segfault
• New unit tests (BootstrapBidir.SliceOffsetIsComputedIn64Bit, Rcclwrap.WarpSpeedChannelClampCapsAtHalfMaxChannels) pass

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[therock-pr-bot]

❌ PR Check — Action Required

Check	Status	Details
🌿 Branch Name	✅ Pass	—
📝 PR Title/Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	❌ Fail	Error: Source/code files changed without an accompanying unit test. Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*). Current: code file(s) changed: projects/rccl/src/bootstrap.cc, projects/rccl/src/graph/connect.cc, projects/rccl/test/WarpSpeedMPITests.cpp; no test file found
🔎 pre-commit	⏳ Pending	⏳ Still running…
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

⚠️ 1 policy check(s) failed. Please address the issues above before this PR can be Reviewed.

🚫 Please fix the failed policies

• ❌ Unit Test

The Not ready to Review label was added to this PR. Once all policies pass, the label is removed automatically.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

[therock-pr-bot]

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

• ❌ Unit Test

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

[Copilot]

Pull request overview

This PR addresses two comm-init crashes seen on gfx950 CPX+NPS2 at 64 ranks: (1) a bootstrap AllGather “Bad address” caused by 32-bit overflow in slice-offset pointer arithmetic, and (2) a WarpSpeed single-node channel over-allocation that can overrun the fixed comm->channels[MAXCHANNELS] array.

Changes:

• Compute ring slice indices as size_t in bootstrap bidirectional AllGather paths to ensure slice * size is evaluated in 64-bit.
• Clamp WarpSpeed single-node channel count to MAXCHANNELS/2 in ncclTopoPostset to prevent comm->channels[] overflow.
• Add two host-side regression tests intended to guard the above issues.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
projects/rccl/src/bootstrap.cc	Switches bidirectional ring AllGather slice indices to size_t to avoid 32-bit offset overflow.
projects/rccl/src/graph/connect.cc	Caps single-node WarpSpeed channel count at MAXCHANNELS/2 to prevent channel array overruns.
projects/rccl/test/BootstrapBidirTests.cpp	Adds a host regression test for 64-bit slice-offset arithmetic (currently not exercising production code).
projects/rccl/test/RcclWrapTests.cpp	Adds a host regression test for WarpSpeed channel clamping (currently not exercising production code).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

[i-kosarev]

LGTM