Repository navigation
Conversation
KKyang
force-pushed
the
users/kkyang/var2reg
branch
2 times, most recently
from
July 31, 2025 03:43
c7b4c09 to
96262ba
Compare
KKyang
force-pushed
the
users/kkyang/var2reg
branch
from
July 31, 2025 03:50
96262ba to
4e94508
Compare
minsukim-amd
pushed a commit
that referenced
this pull request
Jul 31, 2025
[ROCm/hipBLAS commit: c3ec688]
Contributor
|
This pull request has been inactive for 25 days and will be marked as stale. If you would like to keep this PR open, please:
This PR will be automatically closed in 5 days if no further activity occurs. |
Contributor
|
This pull request has been automatically closed due to inactivity (30 days with no updates). If you'd like to continue working on this, feel free to reopen the PR or create a new one. |
sa-faizal
added a commit
that referenced
this pull request
Oct 8, 2026
ISSUE ID: #12727 ## Motivation The hipBLASLt per-PR ASAN lane has been hanging under full ASAN with `HSA_XNACK=1`. This switches it to host-only ASAN with XNACK off, matching the coverage already described in `TESTING.md`. GPU correctness tests still run, and ASAN still checks CPU memory accesses. The trade-off is that no gfx90a device-ASAN runtime tests remain in CI. Before restoring that coverage, we need to review runner capacity and full-ASAN stability with DevOps, potentially for a dedicated nightly pool. The cause of the hangs remains under investigation. [#12728](#12728) added shorter timeouts and diagnostics. [#12813](#12813) offers an alternative runner-pool workaround. The investigation is tracked in [TheRock-Infra #978](ROCm/TheRock-Infra#978). ## Technical Details - Use `HOST_ASAN` artifacts and `HSA_XNACK=0` in the focused lane. - Stop requiring `xnack+` targets for host-only ASAN; preserve existing full-ASAN behavior. - Update `TESTING.md` and remove the stale gfx942 `ci:asan` row. ## Test Plan Run the existing gfx90a build and smoke tests, plus whitespace and applicable pre-commit checks. ## Test Result [Build and all 950 smoke tests passed](https://github.com/ROCm/rocm-libraries/actions/runs/37402899925) on `c92deaa`. The documentation follow-up passes whitespace checks; pre-commit completed with no applicable hooks. CI for the latest commit must pass before merge. ## Submission Checklist - [x] Reviewed the [contributing guidelines](https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests). ## Risk level Medium (3/5): retains host-ASAN and GPU correctness checks, but removes device-ASAN coverage from the gfx90a lane. A passing run does not prove the hangs are fixed. --------- Co-authored-by: Sahil Faizal <197828758+sa-faizal@users.noreply.github.com> Co-authored-by: Sahil Faizal <safaizal@amd.com>
rkayaith
added a commit
that referenced
this pull request
Nov 6, 2026
…ARSELt (#14240) JIRA ID : AIHPBLAS-5081 ## Motivation The hipBLASLt ASAN workflow runs only hipBLASLt `*smoke*` tests on gfx90a, so host-side leaks in hipSPARSELt, on gfx942/gfx950, and in tests outside `*smoke*` go unseen. This PR moves hipBLASLt host-ASAN CI from gfx90a to gfx942/gfx950, adds hipSPARSELt, and adds a nightly run of the broader `standard` tests. Dropping gfx90a slightly reduces architecture-specific sanitizer coverage, but avoids the persistent gfx90a runner hangs that make this CI flaky ([TheRock-Infra #978](ROCm/TheRock-Infra#978 (comment))). Regular gfx90a Math CI coverage remains. The leaks this coverage found are fixed in the PRs below this one in the stack, and in [#14800](#14800) above it. ## Technical Details - **Add hipBLASLt and hipSPARSELt on gfx942/gfx950 to the per-PR host ASAN CI, with extended test coverage in a nightly run** (`hipblaslt-asan-ci.yml`, `test_hipblaslt.py`, `test_hipsparselt.py`, `TESTING.md`): - Per-PR (`quick`, `*smoke*`): 2 build jobs (gfx942 ~1h43m, gfx950 ~1h14m) and 4 test jobs (hipBLASLt gfx942 24m, hipBLASLt gfx950 19m, hipSPARSELt gfx942 16m, hipSPARSELt gfx950 13m), replacing the existing gfx90a build and hipBLASLt gfx90a test. - Nightly at 07:00 UTC (`standard`, `*smoke*:*quick*:*pre_checkin*-*known_bug*`): 2 build jobs (gfx942 ~1h24m, gfx950 ~1h01m) and 4 test jobs (hipBLASLt gfx942 1h34m, gfx950 2h00m; hipSPARSELt gfx942 1h05m, gfx950 59m). - Manual dispatch requires a `test_type` input. - Pull requests touching either library, either test driver, the suppression file or the workflow trigger the per-PR run. - hipSPARSELt `quick` excludes `*known_bug*` and the M=N=128 `spmm_strided_batched` smoke cases, which are much slower under ASAN (43-67 minutes per GPU); `standard` still runs them. - Timeouts are set per architecture, library and test type. - **Remove unused suppressions:** drop the `rocr::AMD::`, `hsaKmt` and `libhsa-runtime64.so` rules, which matched nothing in these runs. - **Suppress [ROCM-30419](https://amd-hub.atlassian.net/browse/ROCM-30419):** add `leak:^rocsparselt_matmul_descr_set_attribute$`, and stop setting `print_suppressions=0`, so each run logs which suppressions matched. - **Update the shared build image to ccache 4.12.1** (`.github/actions/ci-env/action.yml`): use the image published by [TheRock #9506](ROCm/TheRock#9506), whose ccache keeps the `-Xarch_host` sanitizer flags. The image also updates CMake to 3.31.12 and sets `USER root`. This image is shared with component CI. ## Test Plan Run `hipBLASLt ASAN CI` on this PR (`quick`), and dispatch it with `test_type: standard`. ## Test Result The gfx942/gfx950 jobs passed in [`quick`](https://github.com/ROCm/rocm-libraries/actions/runs/41685433208) and [`standard`](https://github.com/ROCm/rocm-libraries/actions/runs/42086928980) runs. Times are whole-job times. Builds are the same for both test types. Test jobs start only after both builds complete. The nightly `standard` run is new. The per-PR `quick` run replaces the gfx90a jobs with gfx942/gfx950 jobs for both libraries. | Job | `quick` (per-PR) | `standard` (nightly, new) | |---|---|---| | **(removed)** Build gfx90a | ~25m (~3m hipBLASLt / <1m hipSPARSELt) | — | | **(new)** Build gfx942 | ~1h43m (~42m hipBLASLt / ~18m hipSPARSELt) | ~1h24m (~40m hipBLASLt / ~18m hipSPARSELt) | | **(new)** Build gfx950 | ~1h14m (~35m hipBLASLt / ~1m hipSPARSELt) | ~1h01m (~36m hipBLASLt / ~1m hipSPARSELt) | | **(removed)** Test hipBLASLt gfx90a | 4m (2m test time) | — | | **(new)** Test hipBLASLt gfx942 | 24m (14m test time) | 1h34m (1h25m test time) | | **(new)** Test hipBLASLt gfx950 | 19m (5m test time) | 2h00m (1h39m test time) | | **(new)** Test hipSPARSELt gfx942 | 16m (8m test time) | 1h05m (56m test time) | | **(new)** Test hipSPARSELt gfx950 | 13m (4m test time) | 59m (39m test time) | ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests. ## Risk level Low (2/5): CI, test-driver and suppression changes, plus the shared build image update. The workflow is not a required check. [ROCM-30419]: https://amd-hub.atlassian.net/browse/ROCM-30419?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In
rocisapass, thevar2regfunction was insidebuildGraph.Now it's moved before the
buildGraphfunction.