Skip to content

feat(stinkytofu): Reduce cluster barrier for long unrolled loops. - #13303

Merged
hcman2 merged 5 commits into
ROCm:developfrom
hcman2:users/hcman2/one_cluster_br_only
Oct 27, 2026
Merged

hcman2 merged 5 commits into
ROCm:developfrom
hcman2:users/hcman2/one_cluster_br_only

Conversation

@hcman2

@hcman2 hcman2 commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

JIRA ID: https://amd-hub.atlassian.net/browse/AIHPBLAS-4915

Reduce cluster barrier for multiple unrolled loop.

Technical Details

  • Limit Rule 3 cluster handshakes to one per enclosing loop. HalfPLR and ExpandPointerSwap copies that fall through inside the same LoopBegin share that handshake. A different loop head (tail loop, another path, or a wave-split clone) still gets its own.
  • Add the ReduceClusterBarrierPerLoop module option and scheduling knob. <0 leaves it unset, 0 handshakes every loop trigger, and >0 uses the reduced cadence. An explicit value wins. A missing or degenerate main loop stays off.
  • When the knob is on, insert the first handshake at the loop's first Rule 3 trigger, then another after each 2000 estimated cycles. Copies closer than that still share one handshake. A second pass treats an existing cluster wait as the spacing baseline.
  • Turn the knob on from the main-loop stats when sumWmmaLatencyCycles > 2500 and sumWmmaLatencyCycles / dsLoadCount >= 5. Division is integer division.
    StinkyTofuDebugPass: "InsertClusterBarrierPass" prints each reduce-mode decision. StinkyTofuDebugPass: "SchedulingKnobHeuristics" prints the resolved knob and its source (user, policy, or static).

Test Plan

check perforamance.

Test Result

Submission Checklist

@therock-pr-bot

therock-pr-bot Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ⚠️ Warning Error: Source/code files changed without an accompanying unit test.
Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*).
Current: code file(s) changed: shared/stinkytofu/include/stinkytofu/bindings/python/Module.hpp, shared/stinkytofu/include/stinkytofu/transforms/asm/InsertClusterBarrierPass.hpp, shared/stinkytofu/include/stinkytofu/transforms/asm/dag/SchedulingKnobHeuristics.hpp, shared/stinkytofu/src/pipeline/backend/Gfx1250Backend.cpp, shared/stinkytofu/src/transforms/asm/InsertClusterBarrierPass.cpp (+3 more); no test file found
🔎 pre-commit ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —
🤖 therock-pr-bot ✅ Pass —

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

therock-pr-bot Bot commented Oct 8, 2026

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

@codecov

codecov Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

❌ Your project check has failed because the head coverage (48.93%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #13303   +/-   ##
========================================
  Coverage    75.95%   75.95%           
========================================
  Files         2880     2880           
  Lines       479004   479050   +46     
  Branches     71955    71965   +10     
========================================
+ Hits        363816   363852   +36     
- Misses       90329    90340   +11     
+ Partials     24859    24858    -1     
Flag Coverage Δ *Carryforward flag
TensileLite-CPP 48.93% <ø> (+0.03%) ⬆️
TensileLite-Unit 83.68% <ø> (-0.01%) ⬇️
hipBLAS 90.62% <ø> (ø) Carriedforward from 751c770
hipBLASLt 64.43% <ø> (ø)
hipCUB 82.68% <ø> (ø) Carriedforward from 751c770
hipDNN 86.78% <ø> (ø) Carriedforward from 751c770
hipFFT 42.00% <ø> (ø) Carriedforward from 751c770
hipRAND 76.12% <ø> (ø) Carriedforward from 751c770
hipSOLVER 69.67% <ø> (ø) Carriedforward from 751c770
hipSPARSE 86.99% <ø> (ø) Carriedforward from 751c770
rocBLAS 68.73% <ø> (ø) Carriedforward from 751c770
rocFFT 48.10% <ø> (ø) Carriedforward from 751c770
rocRAND 57.30% <ø> (ø) Carriedforward from 751c770
rocSOLVER 76.74% <ø> (ø) Carriedforward from 751c770
rocSPARSE 76.92% <ø> (ø) Carriedforward from 751c770
rocThrust 93.15% <ø> (ø) Carriedforward from 751c770

*This pull request uses carry forward flags. Click here to find out more.
see 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hcman2
hcman2 force-pushed the users/hcman2/one_cluster_br_only branch 4 times, most recently from 12f3bb0 to 46456e0 Compare October 19, 2026 01:37
@hcman2
hcman2 requested a review from ThanHenderson October 19, 2026 01:38
@hcman2

hcman2 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

[----------] Global test environment tear-down
[==========] 68925 tests from 67 test suites ran. (891716 ms total)
[ PASSED ] 68753 tests. passed on B0 machine

@hcman2
hcman2 force-pushed the users/hcman2/one_cluster_br_only branch from 46456e0 to 3ad3863 Compare October 19, 2026 05:47
@hcman2

hcman2 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

tox passed on b0 machine w/o sk cases

@hcman2
hcman2 force-pushed the users/hcman2/one_cluster_br_only branch from 3ad3863 to e0d1eb1 Compare October 21, 2026 08:11
@hcman2 hcman2 changed the title feat(stinkytofu): Only 1 cluster barrier for unrolled loops. feat(stinkytofu): Reduce cluster barrier for long unrolled loops. Oct 22, 2026
@hcman2
hcman2 force-pushed the users/hcman2/one_cluster_br_only branch from c3f450f to 751c770 Compare October 22, 2026 10:18
@hcman2
hcman2 merged commit b1f7280 into ROCm:develop Oct 27, 2026
168 of 171 checks passed
@assistant-librarian

Copy link
Copy Markdown
Contributor

TheRock Submodule Bump Activity

Newest first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants