kfdtest: Add simple AMD Infinity Storage (AIS) test by hkasivis · Pull Request #1234 · ROCm/rocm-systems

hkasivis · 2025-10-03T18:33:15Z

Add AISReadTest which invokes direct transfer (read) from the filesystem to GPU VRAM. The test carries out the following:

create a file in /tmp directory. Checks if the file is an NVME drive, if not, fail.
fill the file with different patterns at different offset
direct transfer (read) from (above file) each offset to a different VRAM buffer
cross-check if each VRAM buffer contains the correct pattern

Motivation

This is a unit test to test AMD Infinity Storage (AIS) feature

Test needs a machine that /tmp mounted on a NVME drive. The NVME drive and the GPU should able to do a direct transfer which means they should be in same PCI root complex.

Add AISReadTest which invokes direct transfer (read) from the filesystem to GPU VRAM. The test carries out the following: - create a file in /tmp directory. Checks if the file is an NVME drive, if not, fail. - fill the file with different patterns at different offset - direct transfer (read) from (above file) each offset to a different VRAM buffer - cross-check if each VRAM buffer contains the correct pattern Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>

kurtmcmillan · 2025-10-22T19:37:00Z

+            return -1;
+        }
+    } else {
+        LOG() << "File not backed by PCI device" << std::endl;


If we land here its only because readlink failed. No?

Yes correct. But readlink is not to the path of the file but to path below. This should succeed if major(st.st_dev), minor(st.st_dev) points to PCI device.

snprintf(path, sizeof(path), "/sys/dev/block/%u:%u", major(st.st_dev), minor(st.st_dev));

Right. But I don't think having an directory entry (or symlink) in /sys/dev/block/ indicates a PCIe device. Non-NVMe/Non-PCIe block devices would also have an entry in /sys/dev/block

To the point, I think the error message could be changed to say:

File is not backed by a block device

kurtmcmillan · 2025-10-22T19:39:00Z

+
+/*
+ * Check if the buffers copied from the file has the correct pattern
+ * To keep it simple only the first and the last buffer is checked


To keep it simple only the first and the last buffer is checked

Maybe this should read

To keep it simple only the first and last elements of each buffer are checked

Yes. Correct. Will fix it.

kurtmcmillan · 2025-10-22T20:29:10Z

+    ASSERT_EQ(checkBuffersWithPatterns(defaultGPUNode), 0);
+
+    freeBuffers();
+    deleteFileInNVME();


Is it important to avoid leaking files? It looks like the file is only deleted if the test passes. Maybe unlink the file just after creating it so that it is always deleted when the test program exits.

Thanks for this. Will fix this also.

riley-dixon · 2025-10-27T19:13:44Z

+/*
+ * Fill the file with patterns
+ *  [0 - BUFFER_SIZE-1]             <-- START_PATTERN
+ *  [BUFFER_SIZE - 2*BUFFER_SIZE - ]<-- START_PATTERN + 1


Did you mean to document [BUFFER_SIZE - 2*BUFFER_SIZE-1]?
^^

## Associated JIRA ticket number/Github issue number  ## What type of PR is this? (check all applicable) - [ ] Refactor - [ ] Feature - [ ] Bug Fix - [ ] Optimization - [ ] Documentation Update - [ ] Continuous Integration ## What were the changes?  ## Why are these changes needed?  ## Updated CHANGELOG?  - [ ] Yes - [ ] No, Does not apply to this PR. ## Added/Updated documentation? - [ ] Yes - [ ] No, Does not apply to this PR. ## Additional Checks - [ ] I have added tests relevant to the introduced functionality, and the unit tests are passing locally. - [ ] Any dependent changes have been merged.

## Motivation Fix intersphinx links broken in 7.14.0 Update v_exp_f32 and v_log_f32 for base 2 ## Technical Details  ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation Fix ref.ref and ref.docs errors. ## Technical Details  ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation  ## Technical Details  ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation Fix ROCr DXG Linux path issues around memory import, PM4 enablement, and fork child reinitialization. ## Technical Details - Reset the inherited `hsakmt_mutex` in the fork child to avoid deadlock on the next `hsaKmtOpenKFD()` call. - Fix DXG memory import handling by preserving imported shared memory flags. - Report sysmem fd size during graphics handle registration. - Avoid duplicate GPU VA mappings during imported memory mapping. - Force PM4 usage on Linux for the DXG path. ## Issue Tracking    ## Test Plan rocrtst64 --gtest_filter=rocrtstFunc.IPC rocrtst64 --gtest_filter=rocrtstFunc.VirtMemory_Basic_Test rocrtst64 --gtest_filter=rocrtstFunc.VirtMemory_Interprocess_DevicePool_Test ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Signed-off-by: Flora Cui <flora.cui@amd.com>

## Motivation Add fcui-amd as a maintainer for DXG directory ## Technical Details  ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

…flows (#8783) ## Motivation  ## Technical Details  ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

## Motivation  This PR is a part of PR series to remove dependency on Timemory `filepath` utilities. This particular PR: - replaces `tim::filepath::realpath()` with existing in-tree `path::realpath()` - refactors the mentioned in-tree `path::realpath()` to use modern `std::filesystem` approach ## Technical Details  Existing in-tree `path.hpp:realpath()` function is simplified using non-throwing overload of `std::filesystem::canonical()`. Behavior is unchanged: return result on success, return original input on error. ## Issue Tracking    JIRA ID : AIPROFSYST-528 ## Test Plan  Existing CTest suite should pass ## Test Result  ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation  `ROCprofiler-SDK`'s #6276 has made a CI warning that would previously crash some `selective-region` tests non-fatal. Now we can re-enable the remaining tests. ## Technical Details Remove the following from the common TheRock exclusion list on `gfx950` workflow AND `test_categories.yaml`: ``` selective-region-region-1-filter.* selective-region-region-2-and-3.* selective-region-no-marker-region-1-filter.* ```  ## Issue Tracking    Jira ID: AIPROFSYST-521 ## Test Plan  Workflows + TheRock CI ## Test Result  `gfx950` tests pass: https://github.com/ROCm/rocm-systems/actions/runs/29582264277/job/87890685718?pr=8759 TheRock CI tests pass: https://github.com/ROCm/rocm-systems/actions/runs/29584647543/job/87915973657?pr=8759 ``` Start 488: selective-region-no-filter-start-stop-sys-run 355/473 Test #488: selective-region-no-filter-start-stop-sys-run ................................................................................................................................... Passed 6.85 sec Start 489: selective-region-no-filter-start-stop-sampling 356/473 Test #489: selective-region-no-filter-start-stop-sampling .................................................................................................................................. Passed 6.80 sec Start 490: selective-region-no-filter-push-pop-sys-run 357/473 Test #490: selective-region-no-filter-push-pop-sys-run ..................................................................................................................................... Passed 6.82 sec Start 491: selective-region-no-filter-push-pop-sampling 358/473 Test #491: selective-region-no-filter-push-pop-sampling .................................................................................................................................... Passed 6.82 sec Start 492: selective-region-region-1-filter-start-stop-sys-run 359/473 Test #492: selective-region-region-1-filter-start-stop-sys-run ............................................................................................................................. Passed 6.09 sec Start 493: selective-region-region-1-filter-start-stop-sampling 360/473 Test #493: selective-region-region-1-filter-start-stop-sampling ............................................................................................................................ Passed 6.11 sec Start 494: selective-region-region-1-filter-push-pop-sys-run 361/473 Test #494: selective-region-region-1-filter-push-pop-sys-run ............................................................................................................................... Passed 6.12 sec Start 495: selective-region-region-1-filter-push-pop-sampling 362/473 Test #495: selective-region-region-1-filter-push-pop-sampling .............................................................................................................................. Passed 6.12 sec Start 496: selective-region-region-2-and-3-filter-start-stop-sys-run 363/473 Test #496: selective-region-region-2-and-3-filter-start-stop-sys-run ....................................................................................................................... Passed 5.93 sec Start 497: selective-region-region-2-and-3-filter-start-stop-sampling 364/473 Test #497: selective-region-region-2-and-3-filter-start-stop-sampling ...................................................................................................................... Passed 5.94 sec Start 498: selective-region-region-2-and-3-filter-push-pop-sys-run 365/473 Test #498: selective-region-region-2-and-3-filter-push-pop-sys-run ......................................................................................................................... Passed 5.93 sec Start 499: selective-region-region-2-and-3-filter-push-pop-sampling 366/473 Test #499: selective-region-region-2-and-3-filter-push-pop-sampling ........................................................................................................................ Passed 5.95 sec Start 500: selective-region-pause-no-filter-inside-start-stop-sys-run 367/473 Test #500: selective-region-pause-no-filter-inside-start-stop-sys-run ...................................................................................................................... Passed 4.79 sec Start 501: selective-region-pause-no-filter-inside-start-stop-sampling 368/473 Test #501: selective-region-pause-no-filter-inside-start-stop-sampling ..................................................................................................................... Passed 4.76 sec Start 502: selective-region-pause-no-filter-inside-push-pop-sys-run 369/473 Test #502: selective-region-pause-no-filter-inside-push-pop-sys-run ........................................................................................................................ Passed 4.80 sec Start 503: selective-region-pause-no-filter-inside-push-pop-sampling 370/473 Test #503: selective-region-pause-no-filter-inside-push-pop-sampling ....................................................................................................................... Passed 4.80 sec Start 504: selective-region-pause-no-filter-before-start-stop-sys-run 371/473 Test #504: selective-region-pause-no-filter-before-start-stop-sys-run ...................................................................................................................... Passed 4.80 sec Start 505: selective-region-pause-no-filter-before-start-stop-sampling 372/473 Test #505: selective-region-pause-no-filter-before-start-stop-sampling ..................................................................................................................... Passed 4.81 sec Start 506: selective-region-pause-no-filter-before-push-pop-sys-run 373/473 Test #506: selective-region-pause-no-filter-before-push-pop-sys-run ........................................................................................................................ Passed 4.79 sec Start 507: selective-region-pause-no-filter-before-push-pop-sampling 374/473 Test #507: selective-region-pause-no-filter-before-push-pop-sampling ....................................................................................................................... Passed 4.79 sec Start 508: selective-region-pause-no-filter-outside-start-stop-sys-run 375/473 Test #508: selective-region-pause-no-filter-outside-start-stop-sys-run ..................................................................................................................... Passed 4.81 sec Start 509: selective-region-pause-no-filter-outside-start-stop-sampling 376/473 Test #509: selective-region-pause-no-filter-outside-start-stop-sampling .................................................................................................................... Passed 4.77 sec Start 510: selective-region-pause-no-filter-outside-push-pop-sys-run 377/473 Test #510: selective-region-pause-no-filter-outside-push-pop-sys-run ....................................................................................................................... Passed 4.82 sec Start 511: selective-region-pause-no-filter-outside-push-pop-sampling 378/473 Test #511: selective-region-pause-no-filter-outside-push-pop-sampling ...................................................................................................................... Passed 4.81 sec Start 512: selective-region-pause-filtered-inside-start-stop-sys-run 379/473 Test #512: selective-region-pause-filtered-inside-start-stop-sys-run ....................................................................................................................... Passed 4.55 sec Start 513: selective-region-pause-filtered-inside-start-stop-sampling 380/473 Test #513: selective-region-pause-filtered-inside-start-stop-sampling ...................................................................................................................... Passed 4.67 sec Start 514: selective-region-pause-filtered-inside-push-pop-sys-run 381/473 Test #514: selective-region-pause-filtered-inside-push-pop-sys-run ......................................................................................................................... Passed 4.55 sec Start 515: selective-region-pause-filtered-inside-push-pop-sampling 382/473 Test #515: selective-region-pause-filtered-inside-push-pop-sampling ........................................................................................................................ Passed 4.54 sec Start 516: selective-region-pause-filtered-before-start-stop-sys-run 383/473 Test #516: selective-region-pause-filtered-before-start-stop-sys-run ....................................................................................................................... Passed 4.53 sec Start 517: selective-region-pause-filtered-before-start-stop-sampling 384/473 Test #517: selective-region-pause-filtered-before-start-stop-sampling ...................................................................................................................... Passed 4.57 sec Start 518: selective-region-pause-filtered-before-push-pop-sys-run 385/473 Test #518: selective-region-pause-filtered-before-push-pop-sys-run ......................................................................................................................... Passed 4.53 sec Start 519: selective-region-pause-filtered-before-push-pop-sampling 386/473 Test #519: selective-region-pause-filtered-before-push-pop-sampling ........................................................................................................................ Passed 4.68 sec Start 520: selective-region-pause-filtered-outside-start-stop-sys-run 387/473 Test #520: selective-region-pause-filtered-outside-start-stop-sys-run ...................................................................................................................... Passed 4.53 sec Start 521: selective-region-pause-filtered-outside-start-stop-sampling 388/473 Test #521: selective-region-pause-filtered-outside-start-stop-sampling ..................................................................................................................... Passed 4.56 sec Start 522: selective-region-pause-filtered-outside-push-pop-sys-run 389/473 Test #522: selective-region-pause-filtered-outside-push-pop-sys-run ........................................................................................................................ Passed 4.55 sec Start 523: selective-region-pause-filtered-outside-push-pop-sampling 390/473 Test #523: selective-region-pause-filtered-outside-push-pop-sampling ....................................................................................................................... Passed 4.52 sec Start 524: selective-region-pause-no-marker-inside-start-stop-sys-run 391/473 Test #524: selective-region-pause-no-marker-inside-start-stop-sys-run ...................................................................................................................... Passed 3.37 sec Start 525: selective-region-pause-no-marker-inside-start-stop-sampling 392/473 Test #525: selective-region-pause-no-marker-inside-start-stop-sampling ..................................................................................................................... Passed 3.46 sec Start 526: selective-region-pause-no-marker-inside-push-pop-sys-run 393/473 Test #526: selective-region-pause-no-marker-inside-push-pop-sys-run ........................................................................................................................ Passed 3.35 sec Start 527: selective-region-pause-no-marker-inside-push-pop-sampling 394/473 Test #527: selective-region-pause-no-marker-inside-push-pop-sampling ....................................................................................................................... Passed 3.52 sec Start 528: selective-region-pause-no-marker-before-start-stop-sys-run 395/473 Test #528: selective-region-pause-no-marker-before-start-stop-sys-run ...................................................................................................................... Passed 3.40 sec Start 529: selective-region-pause-no-marker-before-start-stop-sampling 396/473 Test #529: selective-region-pause-no-marker-before-start-stop-sampling ..................................................................................................................... Passed 3.36 sec Start 530: selective-region-pause-no-marker-before-push-pop-sys-run 397/473 Test #530: selective-region-pause-no-marker-before-push-pop-sys-run ........................................................................................................................ Passed 3.37 sec Start 531: selective-region-pause-no-marker-before-push-pop-sampling 398/473 Test #531: selective-region-pause-no-marker-before-push-pop-sampling ....................................................................................................................... Passed 3.36 sec Start 532: selective-region-pause-no-marker-outside-start-stop-sys-run 399/473 Test #532: selective-region-pause-no-marker-outside-start-stop-sys-run ..................................................................................................................... Passed 3.35 sec Start 533: selective-region-pause-no-marker-outside-start-stop-sampling 400/473 Test #533: selective-region-pause-no-marker-outside-start-stop-sampling .................................................................................................................... Passed 3.39 sec Start 534: selective-region-pause-no-marker-outside-push-pop-sys-run 401/473 Test #534: selective-region-pause-no-marker-outside-push-pop-sys-run ....................................................................................................................... Passed 3.49 sec Start 535: selective-region-pause-no-marker-outside-push-pop-sampling 402/473 Test #535: selective-region-pause-no-marker-outside-push-pop-sampling ...................................................................................................................... Passed 3.51 sec Start 536: selective-region-no-marker-region-1-filter-start-stop-sys-run 403/473 Test #536: selective-region-no-marker-region-1-filter-start-stop-sys-run ................................................................................................................... Passed 4.81 sec Start 537: selective-region-no-marker-region-1-filter-start-stop-sampling 404/473 Test #537: selective-region-no-marker-region-1-filter-start-stop-sampling .................................................................................................................. Passed 4.82 sec ``` ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation  The `rocprofiler-compute-tarball.yml` and `rocprofiler-compute-rhel8.yml` workflows were failing due to their respective Docker images being removed from Dockerhub. Since these dockers were originally meant for rocprofiler-systems, we want to publish and maintain our own images to use instead. ## Technical Details  - Updated `.github/workflows/rocprofiler-compute-ghcr.yml` - Adds a new "base" option, which uses `Dockerfile.*.ci` but without downloading TheRock tarball - Enables workflow_dispatch input to specify which Dockers to update (helpful for manual testing) - Updated `.github/workflows/rocprofiler-compute-rhel-8.yml` and renamed it to `.github/workflows/rocprofiler-compute-rhel.yml` - Original name implied only RHEL8 being tested, but we also use RHEL9 so this keeps it more aligned with the actual workflow - Uses new "base" images hosted on GHCR - Made some minor changes to accommodate new base images - Updated `.github/workflows/rocprofiler-compute-tarball.yml` - Use new "base" images, updated spacing to make it more readable - Updated `projects/rocprofiler-compute/docker/containers-ci.yml` - Added an explicit RHEL 9.4 option - The default RHEL9 tag uses RHEL 9.8, which does not have ROCm repositories set up for it ## Issue Tracking    JIRA ID : AIPROFCOMP-711 ## Test Plan  - Manually uploaded new "base" images to GHCR - Confirmed new images are used in workflows and that they pass as intended ## Test Result  - No regressions with changed workflow files ## Submission Checklist - [X] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…fx archs (#9052) ## Motivation  - `TestTransposeGPUPerfCounters` only skipped `gfx1151`, but the GPU perf counter (device counting service) validation is unreliable on a broader set of architectures, causing spurious failures on those GPUs. - The list is taken from the SDK test skip list - https://github.com/ROCm/rocm-systems/blob/develop/projects/rocprofiler-sdk/tests/rocprofv3/counter-collection/input1/validate.py#L83C4-L83C82 - The skip list needs to cover `gfx1101`, `gfx1102`, `gfx1150`, `gfx1151`, `gfx1152`, and `gfx1153`. ## Technical Details  - Added `UNSUPPORTED_PERF_COUNTER_GFX` module constant in `tests/pytest/rocprofsys/gpu.py` listing the archs where GPU perf counter validation is unsupported: `gfx1101`, `gfx1102`, `gfx1150`, `gfx1151`, `gfx1152`, `gfx1153`. - Added a `GPUInfo.unsupported_perf_counter_archs` property that returns the intersection of detected architectures with that set (empty set = all detected GPUs supported). - Updated `TestTransposeGPUPerfCounters.test` in `test_transpose.py` to skip via the new property and report exactly which detected arch(es) triggered the skip, replacing the single hard-coded `gfx1151` check. - Added unit tests in `test_pytest_impl.py::TestGPUInfo` covering both the "match present" and "all supported" cases. ## Issue Tracking    Jira ID: AIPROFSYST-659 ## Test Plan  - Run the new unit tests on a configured build: `pytest tests/pytest/test_pytest_impl.py -k unsupported_perf_counter`. - On an affected GPU (e.g. gfx1102/gfx115x), confirm `TestTransposeGPUPerfCounters` is skipped with the arch named in the reason; on a supported GPU (e.g. gfx942), confirm it runs. ## Test Result  ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

#8911) …tchMemOp - Add STREAM_CAPTURE hooks to hipStreamWaitValue32/64, hipStreamWriteValue32/64, and hipStreamBatchMemOp so they record BatchMemOp graph nodes during capture instead of executing immediately. - Declare and implement capture handlers in hip_graph_capture.hpp/hip_graph.cpp. - Fix hipGraphBatchMemOpNode to deep-copy paramArray to prevent dangling pointer on graph replay. - Fix batchMemOps() globalWorkSize from count to 1 so ops execute sequentially in array order per CUDA spec. - Inline sequential for-loop in __amd_rocclr_batchMemOp in blitcl.cpp to be self-contained until the fix lands in upstream llvm-project amdblit.cl. ## Motivation  ## Technical Details  ## Issue Tracking JIRA ID: AIRUNTIME-2528    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

#8935) …ression) PR #8400 (7c23d18) removed KernelTimestamp.queue_index when fixing the kernelNamesOwner UAF, causing all graph kernel dispatches to be reported on the launch stream queue_id instead of the actual parallel stream they ran on. Profiling traces showed 1 stream for paths4/full4 graphs regardless of their topology. Restore: - KernelTimestamp struct with start/end/queue_index fields - queue_index parameter in addTimestamps() (UINT32_MAX = unknown) - signal->queue_index_ passed through addTimestamps() in rocvirtual.cpp - Per-packet queue_id assignment in ReportActivity so each kernel lands on its actual internal parallel stream Verified with GraphBench paths4/full4 — now shows 4 distinct stream queue_ids with correct event distribution. ## Motivation  ## Technical Details  ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

… nightly tarball (#9059) ## Motivation The rocprofiler-sdk-build-ci-docker-images workflow builds the base images all CI runs in. Recently, gfx94X/gfx950 images ballooned from ~3 GB to ~40 GB, and rocdecode-tracing / rocjpeg-tracing tests started failing intermittently at startup with: `ASan runtime does not come first in initial library list...` ## Technical Details Restrict selection to the standard release stream: - `--prefix "v4/tarball/"` -> excludes the v4/tarball-asan/ directory - `&& !contains(Key, 'asan')` -> defense in depth - `&& !contains(Key, '-tests-')` -> drop the unneeded test bundle ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation  The comm->symmetricSupport check disabled DDA entirely on gfx942/950 (e.g. ROCm 7.13) even with no symmteric window registered. Remove it as we use per-call isSymmetricKernelRequested() guard (as in allreduce/allgather/reducescatter). No guard needed for alltoall as there is nor symmetric kernel support for alltoall. ## Technical Details  ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation The previous fix for a heap buffer overflow in `VendorSpecificAqlToPm4` introduced a regression in a PyTorch Transformers text-generation workload using Llama 3.1 8B Instruct in FP16 mode. This PR reverts that fix and replaces it with a safer PM4 command buffer size check. pid:8989 tid:0x7ffe07fff6c0 [KernelDispatchAqlToPm4] queue 0x7fe400660000 kernel dispatch head=b02 setup=3 wx=80 wy=1 wz=1 gx=1a00 gy=1 gz=1 ps=0 gs=0 ko=7ff18724cac0 ka=0x7fe401206580 cs=0 pid:8989 tid:0x7ffe07fff6c0 [KernelDispatchAqlToPm4] kernel object property=408 entry=0x7ff187491500 lds=0+0 pid:8989 tid:0x7ffe07fff6c0 [KernelDispatchAqlToPm4] PM4 command buffer overflow in KernelDispatch: cumulative 592 bytes exceeds frame limit 560 bytes pid:8989 tid:0x7ffe07fff6c0 [AqlToPm4Thread] process compute queue fail status = 00001008 pid:8989 tid:0x7ffe07fff6c0 [AqlToPm4Thread] aql to pm4 thread 0x7fe400660000 exit ## Technical Details - Revert the previous `VendorSpecificAqlToPm4` heap buffer overflow fix. - Calculate the required PM4 command buffer frame size before writing commands. - Skip vendor packet processing when the required size exceeds `cmdbuf_aql_frame_size`. ## Issue Tracking    ## Test Plan Perform PyTorch Transformers text-generation workload with Llama 3.1 8B Instruct, FP16 precision, and batch size 1. The workload exercises Hugging Face `model.generate()` on ROCm using default text prompts and reports standard inference latency and throughput metrics. ## Test Result test pass. ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Signed-off-by: Flora Cui <flora.cui@amd.com>

## Motivation # rocminfo WSL environment detected. Cannot load librocdxg.so, failed:librocdxg.so: cannot open shared object file: No such file or directory GetExportAddress failed: /opt/venv/lib/python3.14/site-packages/_rocm_sdk_devel/bin/../lib/libhsa-runtime64.so.1: undefined symbol: hsaKmtOpenKFD hsa_init Failed, possibly no supported GPU devices ## Technical Details Add a fallback path for loading thunk libraries when the normal platform library search fails. The fallback locates the directory containing the currently loaded ROCR runtime library and attempts to load the thunk library from the same directory. ## Issue Tracking    ## Test Plan rocminfo pass without specified LD_LIBRARY_PATH setting ## Test Result rocminfo pass ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. Signed-off-by: Flora Cui <flora.cui@amd.com>

#9063) …sable Several correctness fixes for the segmented HIP graph executor, centered on node enable/disable and hipGraphExecKernelNodeSetParams: - Relocate embedded sync patches during filtered-list rebuild. When a node carrying an embedded completion signal or dependency wait is disabled, its packet is filtered out of the dispatch buffer, leaving the patch pointing at a dropped packet. rebuildFilteredLists now moves such patches onto the last enabled packet, and reserves a fallback barrier for segments that become empty, preventing deadlocks on disabled segment boundaries. - Materialize updated packets in UpdateAQLPacket for disabled nodes. A disabled node's CreateCommand emits no command, so re-capturing on SetParams produced zero packets and the packet-count-change path deleted the node's dispatch slot entirely, so the kernel never ran after re-enable. Force the node enabled only for the capture, then restore its dispatch-time disabled state. - Propagate errors through graph scheduling and path finding. Scheduling and path-finding routines now return hipError_t, detect cycles via topological ordering, and surface failures from segment creation. Multi-device child graph instantiation is rejected instead of silently misbehaving. - Harden command-creation and partial segmented-launch cleanup error paths. Adds regression coverage: cyclic graph instantiation, disabled-boundary synchronization, a collapse-correctness performance scenario, and a SetParams functional test that updates params while a node is disabled. ## Motivation  ## Technical Details  ## Issue Tracking   JIRA ID: AIRUNTIME-0 ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

…9186) ## Motivation  UCX sys-run tests were failing Perfetto validation because `merged.proto` was not produced while the tests were still forcing legacy Perfetto. ## Technical Details  Updated the UCX pytest environment to use cached Perfetto by setting `ROCPROFSYS_TRACE_LEGACY=OFF` while keeping `ROCPROFSYS_PERFETTO_COMBINE_TRACES=ON`. ## Issue Tracking    JIRA ID - AIPROFSYST-428 ## Test Plan  Ran representative UCX sys-run validation and the full UCX CTest bucket. ## Test Result  `ucx-mpi-allreduce-sys-run` passed, and the full UCX CTest bucket passed: 100% tests passed, 0 failed out of 24. <img width="393" height="263" alt="image" src="https://github.com/user-attachments/assets/6d3d7327-bc4f-43be-8122-53add3e9eb8d" /> ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

…from tests (#9039) ## Motivation - Legacy Perfetto tracing is deprecated; cached mode is the default and is fully supported, so there is no value in pinning tests to the legacy path. - `ROCPROFSYS_TRACE_CACHED` was already removed as an environment variable, so any test still setting it is a dead/no-op. - Standardizing every test on the default (cached) tracing mode keeps the suite aligned with real-world usage. ## Technical Details - Removed `ROCPROFSYS_TRACE_LEGACY` from the workload environment fixtures in `test_transferbench.py`, `test_annotate.py`, `test_hpc.py` (`test_usm`), `test_pthread.py`, `test_ucx.py`, and `test_nic_perf.py`, so they run in the default cached mode. - Removed both `ROCPROFSYS_TRACE_LEGACY: OFF` and the dead `ROCPROFSYS_TRACE_CACHED: ON` from `test_rccl.py`. - Refactored `test_roctx.py`: dropped the `ROCPROFSYS_TRACE_LEGACY` env var, deleted the now-unused `roctx_legacy_labels/count/depth` helpers. - Fixed a stale "legacy trace" docstring in `test_sdma.py` (the fixture never actually set the legacy env var). - Intentionally kept legacy references that guard the still-supported setting/flag: the `ROCPROFSYS_TRACE_LEGACY -> tracing.legacy` env→JSON mapping in `test_binaries.py` and the `-L, --trace-legacy` help assertion in `test_cli_help.py`. ## Issue Tracking    Jira ID: AIPROFSYST-658 ## Test Plan  - Test-only change; no accompanying new unit test required (all modified files are `test_*.py` files). - Verified no dangling references remain: `ROCPROFSYS_TRACE_CACHED` is gone from all tests, and the only remaining `ROCPROFSYS_TRACE_LEGACY` reference is the intentional config-plumbing mapping in `test_binaries.py`. - Manual tests performed. ## Test Result  Pass. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Co-authored-by: Milan Radosavljevic <milan.radosavljevic@amd.com>

## Motivation  Bump dyninst submodule from 5dff47bf6fb53e51cdc5e98c8ef6656080931ede to 81cd27fc9ae4984df307bc41d5cb8c1ce9bf761e ## Technical Details  This bump picks up ROCm/dyninst#23, which is the complete implementation of `DT_RELR` support (ELF's newer relocation format) ## Issue Tracking    Jira ID: AIPROFSYST-676 ## Test Plan  Workflows ## Test Result  See workflows ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation  When a hip application tries swap operations using incorrectly aligned addresses in memory_async_batch_copy API, rocr would send back an expected error code. Hip layers plans to add a fallback logic to catch this negative case later on but it is recommended to note this alignment requirement around the API documentation. ## Technical Details  ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

On gfx94x/gfx125x the device-memory AQL ring buffer was force-enabled by default. On large-BAR configurations where ROCr rejects the device-resident ring buffer (e.g. inside some containers), hsa_amd_queue_create fails, the HIP default stream's backing queue cannot be created, and Device::NullStream() returns null. The first default-stream kernel launch then dereferenced that null hip::Stream* in UpdateNumClustersFromKernel, crashing with SIGSEGV. - rocsettings: make DEBUG_CLR_AQL_DEV_QUEUE the single source of truth for aql_device_ring_buf_ (default 0 = off, 1 = on) instead of force-enabling the device-memory ring buffer on gfx94x/gfx125x. Restores the pre-regression system-memory ring buffer behavior by default; opt-in for device memory. - hip_module/hip_graph: enforce the invariant that a resolved hip::Stream* is non-null at the getStream() launch boundaries. ihipModuleLaunchKernel and ihipGraphLaunch now return hipErrorInvalidResourceHandle instead of passing a null stream downstream, so a failed default stream fails loudly rather than crashing. ## Motivation  ## Technical Details  ## Issue Tracking   JIRA ID: AIRUNTIME-2569 ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation We'd like to put example text in the pull request template without having that text pass the policy checks itself. This can be achieved through HTML/markdown comments, so long as the policy bot does not evaluate the text in comments. * Follow-up to #8497 which added `` to `.github/pull_request_template.md` * Progress on #7842 ## Test Plan `python tools\systems_pr_bot\test_policy_check_ut.py` (not yet running on CI, or fully passing) passes the new tests ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

## Motivation We'd like to put example text in the pull request template without having that text pass the policy checks itself. This can be achieved through HTML/markdown comments, so long as the policy bot does not evaluate the text in comments. * Follow-up to ROCm/rocm-systems#8497 which added `` to `.github/pull_request_template.md` in rocm-systems. * Progress on #8829 ## Test Plan `python tools\libraries_pr_bot\test_policy_check_ut.py` (not yet running on CI, or fully passing) passes the new tests ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

#9322) The TearDown safety-net cancelled a nonblocking collective (MPI_Ibarrier) request on timeout, which the MPI standard forbids. OpenMPI raises MPI_ERR_REQUEST and, because MPI_COMM_WORLD uses MPI_ERRORS_ARE_FATAL, that MPI_Cancel call itself aborts the job with a misleading error before the intended MPI_Abort runs. Remove the MPI_Cancel/MPI_Request_free and call MPI_Abort directly (it tears the job down anyway). Also raise the teardown-barrier timeout 3s -> 60s (overridable via RCCL_TEST_TEARDOWN_TIMEOUT_SEC) so benign rank skew in heavy multi-phase MPI tests (GrowMPITest, NetIbMPITest CtsDepthStress) no longer trips a false 'out of sync' abort. Test-harness only; no librccl behavior change. Recovers Grow_MultiRankJump and NetIB_CtsDepthStress (ib-cast) from false failures. ## Motivation Fix an illegal MPI_Cancel on a nonblocking-barrier request that crashed MPI test teardown, causing passing tests to be reported as failures. ## Technical Details MPIEnvironment::TearDown() uses an MPI_Ibarrier + poll-with-timeout as a safety net. On timeout it called MPI_Cancel(&barrier_req) — but cancelling a nonblocking collective request is forbidden by the MPI standard. OpenMPI returns MPI_ERR_REQUEST, and since MPI_COMM_WORLD uses the default MPI_ERRORS_ARE_FATAL handler, that MPI_Cancel call itself aborts the job (before the intended MPI_Abort). Fix: drop the MPI_Cancel/MPI_Request_free and call MPI_Abort directly. Also raised the barrier timeout 3s→60s (env-overridable via RCCL_TEST_TEARDOWN_TIMEOUT_SEC) so benign rank skew in heavy multi-phase tests doesn't trigger a false abort. Test-harness only; no librccl change. ## Issue Tracking    ## Test Plan  ## Test Result  ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. Co-authored-by: Cursor <cursoragent@cursor.com>

hkasivis requested review from kentrussell and xiaogang-chen-amd as code owners October 3, 2025 18:33

github-actions Bot added the project: rocr-runtime label Oct 3, 2025

systems-assistant Bot added the organization: ROCm label Oct 3, 2025

hkasivis force-pushed the users/hkasivis/add-kfdtest-for-ais branch from 8915276 to 819a3ba Compare October 6, 2025 13:41

hkasivis requested review from derobins, gaoikawa, kurtmcmillan, riley-dixon and sbates130272 October 7, 2025 20:10

hkasivis force-pushed the users/hkasivis/add-kfdtest-for-ais branch from 819a3ba to d82bb39 Compare October 22, 2025 16:29

kurtmcmillan approved these changes Oct 22, 2025

View reviewed changes

riley-dixon approved these changes Oct 27, 2025

View reviewed changes

saleelk closed this in 7cd2ed3 May 7, 2026

Navigation Menu

Repository navigation

kfdtest: Add simple AMD Infinity Storage (AIS) test - #1234

kfdtest: Add simple AMD Infinity Storage (AIS) test#1234
hkasivis wants to merge 1 commit into
developfrom
users/hkasivis/add-kfdtest-for-ais

hkasivis commented Oct 3, 2025

Uh oh!

kurtmcmillan Oct 22, 2025

Uh oh!

hkasivis Oct 23, 2025

Uh oh!

kurtmcmillan Oct 23, 2025

Uh oh!

kurtmcmillan Oct 22, 2025

Uh oh!

hkasivis Oct 23, 2025

Uh oh!

kurtmcmillan Oct 22, 2025

Uh oh!

hkasivis Oct 23, 2025

Uh oh!

riley-dixon Oct 27, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

hkasivis commented Oct 3, 2025

Motivation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants