Skip to content

fix(rocrand): fix race condition in poisson distribution - #14419

Merged
stanleytsang-amd merged 3 commits into
developfrom
users/NguyenNhuDi/fix-rocrand-race-condition
Oct 29, 2026
Merged

stanleytsang-amd merged 3 commits into
developfrom
users/NguyenNhuDi/fix-rocrand-race-condition

Conversation

@NguyenNhuDi

@NguyenNhuDi NguyenNhuDi commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

QA reported a races condition in poisson distribution. This PR aims to fix this.

JIRA ID: ROCM-32132

Technical Details

Using TSAN QA reported that inside the poisson distribution when calling std::lgamma inside calculate_poisson_probabilities there was a race condition. This is happening because std::lgamma stores the sign of the output inside a global variable. To fix this we can use lgamma_r which allows us to store the sign locally without storing it in a global variable.

Since lgamma_r is not part of the stdlib and is only implemented in glibc, it is not available for windows (msvc). This is the reason why we need to check if we are currently on windows or not.

Test Plan

Standard CI test suite. Also local testing on both linux and windows.

Also re-run with TSAN to see if race condition is still reported.

Test Result

Local unit test ran on linux and windows shows no failures. TSAN run shows no race condition originated from rocrand.

Submission Checklist

@NguyenNhuDi
NguyenNhuDi marked this pull request as ready for review October 26, 2026 20:19
@NguyenNhuDi
NguyenNhuDi requested a review from a team as a code owner October 26, 2026 20:19
@therock-pr-bot

therock-pr-bot Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ⚠️ Warning Error: Source/code files changed without an accompanying unit test.
Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*).
Current: code file(s) changed: projects/rocrand/library/src/rng/distribution/poisson.hpp; no test file found
🔎 pre-commit ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —
🤖 therock-pr-bot ✅ Pass —

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

@stanleytsang-amd

Copy link
Copy Markdown
Contributor

Change looks fine with me, please update changelog and cmake packaging version to reflect this bugfix for ROCm 10.3

@NguyenNhuDi
NguyenNhuDi requested a review from a team as a code owner October 26, 2026 21:04
@NguyenNhuDi

NguyenNhuDi commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Change looks fine with me, please update changelog and cmake packaging version to reflect this bugfix for ROCm 10.3

It is done in 07246ec. Thanks @stanleytsang-amd

@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop   #14419      +/-   ##
===========================================
+ Coverage    76.89%   76.92%   +0.04%     
===========================================
  Files         2886     2886              
  Lines       481037   480811     -226     
  Branches     72377    72377              
===========================================
+ Hits        369847   369850       +3     
+ Misses       86832    86601     -231     
- Partials     24358    24360       +2     
Flag Coverage Δ *Carryforward flag
TensileLite-CPP 48.93% <ø> (ø) Carriedforward from 07246ec
TensileLite-Unit 83.71% <ø> (ø) Carriedforward from 07246ec
hipBLAS 90.62% <ø> (ø) Carriedforward from 07246ec
hipBLASLt 66.20% <ø> (ø) Carriedforward from 07246ec
hipCUB 82.68% <ø> (ø) Carriedforward from 07246ec
hipDNN 86.85% <ø> (ø) Carriedforward from 07246ec
hipFFT 38.02% <ø> (ø) Carriedforward from 07246ec
hipRAND 76.12% <ø> (ø) Carriedforward from 07246ec
hipSOLVER 69.67% <ø> (ø) Carriedforward from 07246ec
hipSPARSE 86.99% <ø> (ø) Carriedforward from 07246ec
rocBLAS 68.72% <ø> (ø) Carriedforward from 07246ec
rocFFT 46.05% <ø> (ø) Carriedforward from 07246ec
rocRAND 57.31% <100.00%> (+<0.01%) ⬆️
rocSOLVER 76.74% <ø> (ø) Carriedforward from 07246ec
rocSPARSE 83.51% <ø> (ø) Carriedforward from 07246ec
rocSPARSE-integration ?
rocSPARSE-unit ?
rocThrust 93.15% <ø> (ø) Carriedforward from 07246ec

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
...s/rocrand/library/src/rng/distribution/poisson.hpp 77.16% <100.00%> (+0.06%) ⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@spolifroni-amd spolifroni-amd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just need a period in the changelog entry.

Comment thread projects/rocrand/CHANGELOG.md Resolved
Co-authored-by: spolifroni-amd <Sandra.Polifroni@amd.com>
@stanleytsang-amd
stanleytsang-amd merged commit f968e16 into develop Oct 29, 2026
273 of 281 checks passed
@stanleytsang-amd
stanleytsang-amd deleted the users/NguyenNhuDi/fix-rocrand-race-condition branch October 29, 2026 15:20
assistant-librarian Bot pushed a commit to ROCm/rocRAND that referenced this pull request Oct 29, 2026
fix(rocrand): fix race condition in poisson distribution
 (#14419)

## Motivation

QA reported a races condition in `poisson` distribution. This PR aims to
fix this.

JIRA ID: ROCM-32132

## Technical Details

Using `TSAN` QA reported that inside the `poisson` distribution when
calling `std::lgamma` inside `calculate_poisson_probabilities` there was
a race condition. This is happening because `std::lgamma` stores the
sign of the output inside a global variable. To fix this we can use
`lgamma_r` which allows us to store the sign locally without storing it
in a global variable.

Since `lgamma_r` is not part of the `stdlib` and is only implemented in
`glibc`, it is not available for windows (`msvc`). This is the reason
why we need to check if we are currently on windows or not.

## Test Plan

Standard CI test suite. Also local testing on both linux and windows.

Also re-run with `TSAN` to see if race condition is still reported.
## Test Result

Local unit test ran on linux and windows shows no failures. `TSAN` run
shows no race condition originated from rocrand.

## Submission Checklist

- [ ] Look over the contributing guidelines at
https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests.
@assistant-librarian

Copy link
Copy Markdown
Contributor

TheRock Submodule Bump Activity

Newest first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants