Repository navigation
Conversation
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
newling
force-pushed
the
tensilelite-rj-race-ci-2
branch
from
June 29, 2026 22:19
594d914 to
1f5ad38
Compare
newling
force-pushed
the
tensilelite-rj-race-ci-2
branch
4 times, most recently
from
June 30, 2026 21:10
a94625c to
d36cfec
Compare
talumbau
self-requested a review
July 6, 2026 19:25
newling
force-pushed
the
tensilelite-rj-race-ci-2
branch
9 times, most recently
from
July 10, 2026 17:05
dcf56cb to
9152373
Compare
newling
force-pushed
the
tensilelite-rj-race-ci-2
branch
from
July 10, 2026 18:06
9152373 to
0e9ebbb
Compare
Contributor
Author
|
Superseded by #9363 |
newling
added a commit
that referenced
this pull request
Jul 23, 2026
## Motivation Adds minimal rocjitsu race detection for GEMMs on gfx1151, gfx950, and gfx942/gfx94X, run through `hipblaslt-bench` and `tensilelite-client`. This is the third step in the evolution of race detection for hipBLASLt: 1. #7951 did not use TheRock. It used a free GitHub runner, checked out rocm-libraries and rocm-systems, built them, and ran rocjitsu with hipBLASLt. 2. #8929 used TheRock, but only ran `hipblaslt-bench` on gfx950. 3. This PR uses TheRock, adds gfx1151 and gfx942/gfx94X coverage, adds a reduced `tensilelite-client` path, and runs rocjitsu on a CPU build runner instead of consuming a GPU test runner. JIRA ID : AIHPBLAS-3605 https://amd-hub.atlassian.net/browse/AIHPBLAS-3605 ## High Level Overview For hipBLASLt TheRock Linux rows on supported GPU families, this adds a post-build rocjitsu race-check job. The flow is: 1. The normal TheRock build produces the ROCm runtime, hipBLASLt, and TensileLite test artifacts. 2. The race-check job fetches those artifacts from the same TheRock run. 3. The job builds rocjitsu from a pinned rocm-systems revision. 4. The job runs small `hipblaslt-bench` and `tensilelite-client` workloads under rocjitsu race detection. 5. The script reports a failure if a workload fails or rocjitsu reports a race. The check is initially advisory. Failures remain visible in the job logs and uploaded reports, but do not fail the required TheRock CI summary while the job's reliability is evaluated. The goal is to test GPU-targeted hipBLASLt and TensileLite artifacts under CPU-side emulation without requiring GPU runner capacity. ### Workspace Layout The job uses three repository checkouts in one workspace: - `ROCm/TheRock` is checked out at the workspace root. Its actions and scripts fetch the artifacts produced by the current TheRock run. - The current `ROCm/rocm-libraries` revision is checked out under `rocm-libraries/`. A sparse checkout provides the race-check driver script from `.github/scripts`. - `ROCm/rocm-systems` is checked out under `rocm-systems/` at a pinned revision. A sparse checkout provides the rocjitsu sources and machine-readable ISA data needed to build rocjitsu. These checkouts provide the workflow logic and test tooling. The ROCm runtime, compiler, hipBLASLt, TensileLite, and test payload being exercised still come from the artifacts produced by the associated TheRock build. ## End-to-end CI Proof A recent CI run has successful rocjitsu race-check jobs for all three target rows: - gfx1151: https://github.com/ROCm/rocm-libraries/actions/runs/29942116838/job/89005639878 - gfx950-dcgpu: https://github.com/ROCm/rocm-libraries/actions/runs/29942116838/job/89010903066 - gfx94X-dcgpu: https://github.com/ROCm/rocm-libraries/actions/runs/29942116838/job/89015610352 ## Current Limitations - rocjitsu is still built from a pinned rocm-systems source checkout. This should move to a packaged TheRock component artifact. - The check is initially advisory while its reliability is evaluated. - The TensileLite YAMLs are embedded in the script. They should move to checked-in test data or be replaced by existing product-owned workloads. ## Next Steps And Ongoing Work - Evaluate the job's reliability and decide when it should become blocking. - Move rocjitsu consumption to a complete TheRock runnable artifact. - Move the inline TensileLite YAMLs into checked-in test data or reuse existing workloads. - Continue the TheRock-side work toward a cleaner component path: - ROCm/TheRock#6517 makes rocjitsu available as a runnable artifact. - ROCm/TheRock#6525 adds a component-level rocjitsu smoke test. - ROCm/TheRock#6575 separates artifact selection from runner placement so GPU-targeted artifacts can be tested on CPU runners.
bghimireamd
pushed a commit
that referenced
this pull request
Jul 28, 2026
## Motivation Adds minimal rocjitsu race detection for GEMMs on gfx1151, gfx950, and gfx942/gfx94X, run through `hipblaslt-bench` and `tensilelite-client`. This is the third step in the evolution of race detection for hipBLASLt: 1. #7951 did not use TheRock. It used a free GitHub runner, checked out rocm-libraries and rocm-systems, built them, and ran rocjitsu with hipBLASLt. 2. #8929 used TheRock, but only ran `hipblaslt-bench` on gfx950. 3. This PR uses TheRock, adds gfx1151 and gfx942/gfx94X coverage, adds a reduced `tensilelite-client` path, and runs rocjitsu on a CPU build runner instead of consuming a GPU test runner. JIRA ID : AIHPBLAS-3605 https://amd-hub.atlassian.net/browse/AIHPBLAS-3605 ## High Level Overview For hipBLASLt TheRock Linux rows on supported GPU families, this adds a post-build rocjitsu race-check job. The flow is: 1. The normal TheRock build produces the ROCm runtime, hipBLASLt, and TensileLite test artifacts. 2. The race-check job fetches those artifacts from the same TheRock run. 3. The job builds rocjitsu from a pinned rocm-systems revision. 4. The job runs small `hipblaslt-bench` and `tensilelite-client` workloads under rocjitsu race detection. 5. The script reports a failure if a workload fails or rocjitsu reports a race. The check is initially advisory. Failures remain visible in the job logs and uploaded reports, but do not fail the required TheRock CI summary while the job's reliability is evaluated. The goal is to test GPU-targeted hipBLASLt and TensileLite artifacts under CPU-side emulation without requiring GPU runner capacity. ### Workspace Layout The job uses three repository checkouts in one workspace: - `ROCm/TheRock` is checked out at the workspace root. Its actions and scripts fetch the artifacts produced by the current TheRock run. - The current `ROCm/rocm-libraries` revision is checked out under `rocm-libraries/`. A sparse checkout provides the race-check driver script from `.github/scripts`. - `ROCm/rocm-systems` is checked out under `rocm-systems/` at a pinned revision. A sparse checkout provides the rocjitsu sources and machine-readable ISA data needed to build rocjitsu. These checkouts provide the workflow logic and test tooling. The ROCm runtime, compiler, hipBLASLt, TensileLite, and test payload being exercised still come from the artifacts produced by the associated TheRock build. ## End-to-end CI Proof A recent CI run has successful rocjitsu race-check jobs for all three target rows: - gfx1151: https://github.com/ROCm/rocm-libraries/actions/runs/29942116838/job/89005639878 - gfx950-dcgpu: https://github.com/ROCm/rocm-libraries/actions/runs/29942116838/job/89010903066 - gfx94X-dcgpu: https://github.com/ROCm/rocm-libraries/actions/runs/29942116838/job/89015610352 ## Current Limitations - rocjitsu is still built from a pinned rocm-systems source checkout. This should move to a packaged TheRock component artifact. - The check is initially advisory while its reliability is evaluated. - The TensileLite YAMLs are embedded in the script. They should move to checked-in test data or be replaced by existing product-owned workloads. ## Next Steps And Ongoing Work - Evaluate the job's reliability and decide when it should become blocking. - Move rocjitsu consumption to a complete TheRock runnable artifact. - Move the inline TensileLite YAMLs into checked-in test data or reuse existing workloads. - Continue the TheRock-side work toward a cleaner component path: - ROCm/TheRock#6517 makes rocjitsu available as a runnable artifact. - ROCm/TheRock#6525 adds a component-level rocjitsu smoke test. - ROCm/TheRock#6575 separates artifact selection from runner placement so GPU-targeted artifacts can be tested on CPU runners.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Superseded by #9363
Add a first rocjitsu race-detection smoke lane for hipBLASLt changes in TheRock Linux CI.
This adds a
rocjitsu-race-smoke-linuxjob totherock-ci-linux.yml. The job runs only for the existinggfx950Linux test target and only when the selected test projects includehipblaslt.The smoke test:
--blas --testsROCm/rocm-systemssparsely foremulation/rocjitsuhipblaslt-benchgfx950 GEMM underRJ_RACE=1RACErecordsBackground
This follows up on the earlier proof-of-concept in #7951, which explored running race detection for hipBLASLt/TensileLite changes in
rocm-libraries.The main differences from that PoC are:
hipblaslt-benchsmoke command to validate the CI wiring and race-report behavior before broadening coverage.Follow-up
Longer term, this should move toward a fully TheRock-native component that fetches rocjitsu via TheRock artifacts, for example with
--blas --tests --rocjitsu, rather than building rocjitsu inside this job.Testing
Local lightweight checks only:
bash -n .github/scripts/run_rocjitsu_hipblaslt_race_smoke.shgit diff --check -- .github/scripts/run_rocjitsu_hipblaslt_race_smoke.sh .github/workflows/therock-ci-linux.yml.github/workflows/therock-ci-linux.ymlwith PyYAMLJIRA ID : AIHPBLAS-3605
https://amd-hub.atlassian.net/browse/AIHPBLAS-3605