Repository navigation
Conversation
systems-assistant Bot
pushed a commit
that referenced
this pull request
Aug 5, 2025
Update rocm_ci_caller.yml to use amd-master , until amd-mainline is aligned Signed-off-by: Choudhary, Rahul <Rahul.Choudhary@amd.com>
systems-assistant Bot
pushed a commit
that referenced
this pull request
Aug 5, 2025
- Fixes the broken links in rocm_smi.h - Uses automodule instead of autofunction in docs/reference/python_api.rst - Fixes some warnings during docs build - Update some of the versions in requirements.txt
jayhawk-commits
pushed a commit
that referenced
this pull request
Aug 5, 2025
[ROCm/rocprofiler-register commit: 16b142b]
jayhawk-commits
pushed a commit
that referenced
this pull request
Aug 5, 2025
[ROCm/rocprofiler commit: 0bb6be9]
jayhawk-commits
pushed a commit
that referenced
this pull request
Aug 5, 2025
[ROCm/roctracer commit: 3d3b9ea]
|
I think this PR can be closed? Looks like it's up to date with develop |
ammallya
pushed a commit
that referenced
this pull request
Aug 7, 2025
* SWDEV-492625: Track free memory HSA functions to help determine total amount of memory allocated on the system at any one time * Minor fixes to address comments * Update allocation size description * Moved get function back to specialization, minor typo fixes * Removed memory_operation_type field, removed memory_pool allocation enum, converted starting address to hex string for json format. * Made conversion to hex_string a function, changed address to use union rocprofiler_address_t type, changed VMEM descriptors * Removed as_hex from the global namespace * Formatting * Removed TRACK_EVENT for memory allocation, now TRACK_COUNTER for memory allocation is being performed * Check if address was recorded before retrieving allocation size in generate Perfetto * Formatting * Update source/lib/output/generatePerfetto.cpp * Explicitly disable app-abort tests * Remove excluding app-abort test from workflow CI - redundant bc these tests are explicitly marked as disabled now --------- Co-authored-by: Madsen, Jonathan <Jonathan.Madsen@amd.com> Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com> [ROCm/rocprofiler-sdk commit: 79006bb]
jayhawk-commits
pushed a commit
that referenced
this pull request
Aug 8, 2025
- Fixes the broken links in rocm_smi.h - Uses automodule instead of autofunction in docs/reference/python_api.rst - Fixes some warnings during docs build - Update some of the versions in requirements.txt [ROCm/rocm_smi_lib commit: e5c5a1d]
jayhawk-commits
pushed a commit
that referenced
this pull request
Aug 8, 2025
- Fixes the broken links in rocm_smi.h - Uses automodule instead of autofunction in docs/reference/python_api.rst - Fixes some warnings during docs build - Update some of the versions in requirements.txt [ROCm/rocm_smi_lib commit: fc61e40]
jayhawk-commits
pushed a commit
that referenced
this pull request
Aug 11, 2025
[ROCm/rocminfo commit: 61eb689]
jayhawk-commits
pushed a commit
that referenced
this pull request
Aug 11, 2025
Update rocm_ci_caller.yml to use amd-master , until amd-mainline is aligned Signed-off-by: Choudhary, Rahul <Rahul.Choudhary@amd.com> [ROCm/ROCR-Runtime commit: 16cd712]
jayhawk-commits
pushed a commit
that referenced
this pull request
Aug 18, 2025
[ROCm/hipother commit: ed4a99b]
ammallya
pushed a commit
that referenced
this pull request
Jan 21, 2026
MI300X RO Support [ROCm/rocshmem commit: efa075f]
stellaraccident
added a commit
that referenced
this pull request
Feb 26, 2026
## Summary - When `kpack_get_kernel()` fails with `KPACK_ERROR_KERNEL_NOT_FOUND` after finding a matching archive+arch, the loader now returns immediately instead of falling through to probe the next ISA (e.g., `gfx9-4-generic`) - The previous behavior produced misleading "no matching architecture found" errors and wasted time probing archives that can't help ## Evidence CI log for rocblas showed: ``` kpack: opened and cached archive: .../blas_lib_gfx942.kpack kpack: architecture: gfx942 kpack: kernel not found in this archive <-- should stop here kpack: trying architecture: gfx9-4-generic <-- wrong: kept probing kpack: no matching architecture found in any archive ``` The kernel (`#45`) wasn't in the archive due to the multi-CCOB bug (PR #10). But regardless, once we match an archive and architecture, the result is definitive. ## Test plan - [x] New test `LoadCodeObject_KernelNotFound`: archive found, arch matches, but `co_index=999` not in TOC → `KPACK_ERROR_KERNEL_NOT_FOUND` - [x] 101/101 CTest pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <noreply@anthropic.com>
1 task
jeffqjiangNew
added a commit
that referenced
this pull request
Mar 13, 2026
## Motivation This PR adds entry and exit logs for core functions. ## Technical Details - Info level (3) logs are added at core function entry and exit points. - Monotonic clock stamp, process id, thread id, and file name are added to the logs. - The logs now have similar format to HIP logs for better viewing when both are present. - Direct error or warning logs to std::cerr. Other logs still go to std::cout. ## JIRA ID <!-- If applicable, mention the JIRA ID resolved by this PR (Example: Resolves SWDEV-12345). --> <!-- Do not post any JIRA links here. --> ## Test Plan Set env variable ROCDEC_LOG_LEVEL to control log output. All CTests and conformance tests must pass. Example summary of the test results: - CTests: Start 1: video_decodeRaw-HEVC 1/15 Test #1: video_decodeRaw-HEVC ............. Passed 2.05 sec Start 2: video_decodeRaw-AVC 2/15 Test #2: video_decodeRaw-AVC .............. Passed 1.89 sec Start 3: video_decodeRaw-AV1 3/15 Test #3: video_decodeRaw-AV1 .............. Passed 2.31 sec Start 4: video_decodeRaw-VP9 4/15 Test #4: video_decodeRaw-VP9 .............. Passed 2.04 sec Start 5: rocdec_Decode-HEVC 5/15 Test #5: rocdec_Decode-HEVC ............... Passed 0.92 sec Start 6: rocDecode_Negative_API_Tests 6/15 Test #6: rocDecode_Negative_API_Tests ..... Passed 0.78 sec Start 7: video_decode-HEVC 7/15 Test #7: video_decode-HEVC ................ Passed 2.74 sec Start 8: video_decode-AVC 8/15 Test #8: video_decode-AVC ................. Passed 2.64 sec Start 9: video_decode-AV1 9/15 Test #9: video_decode-AV1 ................. Passed 2.74 sec Start 10: video_decode-VP9 10/15 Test #10: video_decode-VP9 ................. Passed 2.78 sec Start 11: video_decodePerf-HEVC 11/15 Test #11: video_decodePerf-HEVC ............ Passed 2.92 sec Start 12: video_decodeBatch 12/15 Test #12: video_decodeBatch ................ Passed 4.34 sec Start 13: video_decodeRGB-HEVC 13/15 Test #13: video_decodeRGB-HEVC ............. Passed 33.51 sec Start 14: video_decodeMem-HEVC 14/15 Test #14: video_decodeMem-HEVC ............. Passed 2.09 sec Start 15: video_decodeRGB-Resize 15/15 Test #15: video_decodeRGB-Resize ........... Passed 33.94 sec 100% tests passed, 0 tests failed out of 15 - AVC conformance test: Conformance test completed on the 127 streams: - The number of passing streams is 127 - The number of failing streams is 0 - The number of streams that did not finish decoding is 0 - HEVC conformance test: Conformance test completed on the 135 streams: - The number of passing streams is 135 - The number of failing streams is 0 - The number of streams that did not finish decoding is 0 - AV1 conformance test: Conformance test completed on the 162 streams: - The number of passing streams is 162 - The number of failing streams is 0 - The number of streams that did not finish decoding is 0 - VP9 conformance test: Conformance test completed on the 108 streams: - The number of passing streams is 108 - The number of failing streams is 0 - The number of streams that did not finish decoding is 0 ## Test Result Function entry/exit logs should show up when ROCDEC_LOG_LEVEL >= 3. All conformance tests should pass. ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
Nikhil-Nunna
pushed a commit
that referenced
this pull request
Mar 13, 2026
## Motivation This PR adds entry and exit logs for core functions. ## Technical Details - Info level (3) logs are added at core function entry and exit points. - Monotonic clock stamp, process id, thread id, and file name are added to the logs. - The logs now have similar format to HIP logs for better viewing when both are present. - Direct error or warning logs to std::cerr. Other logs still go to std::cout. ## JIRA ID <!-- If applicable, mention the JIRA ID resolved by this PR (Example: Resolves SWDEV-12345). --> <!-- Do not post any JIRA links here. --> ## Test Plan Set env variable ROCDEC_LOG_LEVEL to control log output. All CTests and conformance tests must pass. Example summary of the test results: - CTests: Start 1: video_decodeRaw-HEVC 1/15 Test #1: video_decodeRaw-HEVC ............. Passed 2.05 sec Start 2: video_decodeRaw-AVC 2/15 Test #2: video_decodeRaw-AVC .............. Passed 1.89 sec Start 3: video_decodeRaw-AV1 3/15 Test #3: video_decodeRaw-AV1 .............. Passed 2.31 sec Start 4: video_decodeRaw-VP9 4/15 Test #4: video_decodeRaw-VP9 .............. Passed 2.04 sec Start 5: rocdec_Decode-HEVC 5/15 Test #5: rocdec_Decode-HEVC ............... Passed 0.92 sec Start 6: rocDecode_Negative_API_Tests 6/15 Test #6: rocDecode_Negative_API_Tests ..... Passed 0.78 sec Start 7: video_decode-HEVC 7/15 Test #7: video_decode-HEVC ................ Passed 2.74 sec Start 8: video_decode-AVC 8/15 Test #8: video_decode-AVC ................. Passed 2.64 sec Start 9: video_decode-AV1 9/15 Test #9: video_decode-AV1 ................. Passed 2.74 sec Start 10: video_decode-VP9 10/15 Test #10: video_decode-VP9 ................. Passed 2.78 sec Start 11: video_decodePerf-HEVC 11/15 Test #11: video_decodePerf-HEVC ............ Passed 2.92 sec Start 12: video_decodeBatch 12/15 Test #12: video_decodeBatch ................ Passed 4.34 sec Start 13: video_decodeRGB-HEVC 13/15 Test #13: video_decodeRGB-HEVC ............. Passed 33.51 sec Start 14: video_decodeMem-HEVC 14/15 Test #14: video_decodeMem-HEVC ............. Passed 2.09 sec Start 15: video_decodeRGB-Resize 15/15 Test #15: video_decodeRGB-Resize ........... Passed 33.94 sec 100% tests passed, 0 tests failed out of 15 - AVC conformance test: Conformance test completed on the 127 streams: - The number of passing streams is 127 - The number of failing streams is 0 - The number of streams that did not finish decoding is 0 - HEVC conformance test: Conformance test completed on the 135 streams: - The number of passing streams is 135 - The number of failing streams is 0 - The number of streams that did not finish decoding is 0 - AV1 conformance test: Conformance test completed on the 162 streams: - The number of passing streams is 162 - The number of failing streams is 0 - The number of streams that did not finish decoding is 0 - VP9 conformance test: Conformance test completed on the 108 streams: - The number of passing streams is 108 - The number of failing streams is 0 - The number of streams that did not finish decoding is 0 ## Test Result Function entry/exit logs should show up when ROCDEC_LOG_LEVEL >= 3. All conformance tests should pass. ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
1 task
jeffqjiangNew
added a commit
that referenced
this pull request
Apr 2, 2026
…4229) ## Motivation This PR adds a new rocDecode feature: support for user specified output surface format. ## Technical Details In certain use cases, the user requests a decoded image format that is not the native format based on the stream content. For example, 8-bit output is needed on a 10-bit stream. This PR enables the output format selection as long as it is supported by the VA-API driver. - Added a special output format define to indicate if the output format is user set or native. - VA-API decoder now sets the surface format either native or by user override. - Added the output format command option to rocdecdecode sample for feature verification. ## JIRA ID <!-- If applicable, mention the JIRA ID resolved by this PR (Example: Resolves SWDEV-12345). --> <!-- Do not post any JIRA links here. --> ## Test Plan - Run CTests and decode conformance tests to catch any regressions. - Run rocdecdecode samples to verify user set output. ## Test Result All CTests and conformance tests pass. Example summary of the test results: CTests: Start 1: video_decodeRaw-HEVC 1/15 Test #1: video_decodeRaw-HEVC ............. Passed 2.05 sec Start 2: video_decodeRaw-AVC 2/15 Test #2: video_decodeRaw-AVC .............. Passed 1.89 sec Start 3: video_decodeRaw-AV1 3/15 Test #3: video_decodeRaw-AV1 .............. Passed 2.31 sec Start 4: video_decodeRaw-VP9 4/15 Test #4: video_decodeRaw-VP9 .............. Passed 2.04 sec Start 5: rocdec_Decode-HEVC 5/15 Test #5: rocdec_Decode-HEVC ............... Passed 0.92 sec Start 6: rocDecode_Negative_API_Tests 6/15 Test #6: rocDecode_Negative_API_Tests ..... Passed 0.78 sec Start 7: video_decode-HEVC 7/15 Test #7: video_decode-HEVC ................ Passed 2.74 sec Start 8: video_decode-AVC 8/15 Test #8: video_decode-AVC ................. Passed 2.64 sec Start 9: video_decode-AV1 9/15 Test #9: video_decode-AV1 ................. Passed 2.74 sec Start 10: video_decode-VP9 10/15 Test #10: video_decode-VP9 ................. Passed 2.78 sec Start 11: video_decodePerf-HEVC 11/15 Test #11: video_decodePerf-HEVC ............ Passed 2.92 sec Start 12: video_decodeBatch 12/15 Test #12: video_decodeBatch ................ Passed 4.34 sec Start 13: video_decodeRGB-HEVC 13/15 Test #13: video_decodeRGB-HEVC ............. Passed 33.51 sec Start 14: video_decodeMem-HEVC 14/15 Test #14: video_decodeMem-HEVC ............. Passed 2.09 sec Start 15: video_decodeRGB-Resize 15/15 Test #15: video_decodeRGB-Resize ........... Passed 33.94 sec 100% tests passed, 0 tests failed out of 15 AVC conformance test: Conformance test completed on the 127 streams: The number of passing streams is 127 The number of failing streams is 0 The number of streams that did not finish decoding is 0 HEVC conformance test: Conformance test completed on the 135 streams: The number of passing streams is 135 The number of failing streams is 0 The number of streams that did not finish decoding is 0 AV1 conformance test: Conformance test completed on the 162 streams: The number of passing streams is 162 The number of failing streams is 0 The number of streams that did not finish decoding is 0 VP9 conformance test: Conformance test completed on the 108 streams: The number of passing streams is 108 The number of failing streams is 0 The number of streams that did not finish decoding is 0 <!-- Briefly summarize test outcomes. --> rocdecdecode sample dumps the clean output YUV clips of the specified format (NV12 or P016). ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
gandryey
pushed a commit
that referenced
this pull request
Apr 6, 2026
…4229) ## Motivation This PR adds a new rocDecode feature: support for user specified output surface format. ## Technical Details In certain use cases, the user requests a decoded image format that is not the native format based on the stream content. For example, 8-bit output is needed on a 10-bit stream. This PR enables the output format selection as long as it is supported by the VA-API driver. - Added a special output format define to indicate if the output format is user set or native. - VA-API decoder now sets the surface format either native or by user override. - Added the output format command option to rocdecdecode sample for feature verification. ## JIRA ID <!-- If applicable, mention the JIRA ID resolved by this PR (Example: Resolves SWDEV-12345). --> <!-- Do not post any JIRA links here. --> ## Test Plan - Run CTests and decode conformance tests to catch any regressions. - Run rocdecdecode samples to verify user set output. ## Test Result All CTests and conformance tests pass. Example summary of the test results: CTests: Start 1: video_decodeRaw-HEVC 1/15 Test #1: video_decodeRaw-HEVC ............. Passed 2.05 sec Start 2: video_decodeRaw-AVC 2/15 Test #2: video_decodeRaw-AVC .............. Passed 1.89 sec Start 3: video_decodeRaw-AV1 3/15 Test #3: video_decodeRaw-AV1 .............. Passed 2.31 sec Start 4: video_decodeRaw-VP9 4/15 Test #4: video_decodeRaw-VP9 .............. Passed 2.04 sec Start 5: rocdec_Decode-HEVC 5/15 Test #5: rocdec_Decode-HEVC ............... Passed 0.92 sec Start 6: rocDecode_Negative_API_Tests 6/15 Test #6: rocDecode_Negative_API_Tests ..... Passed 0.78 sec Start 7: video_decode-HEVC 7/15 Test #7: video_decode-HEVC ................ Passed 2.74 sec Start 8: video_decode-AVC 8/15 Test #8: video_decode-AVC ................. Passed 2.64 sec Start 9: video_decode-AV1 9/15 Test #9: video_decode-AV1 ................. Passed 2.74 sec Start 10: video_decode-VP9 10/15 Test #10: video_decode-VP9 ................. Passed 2.78 sec Start 11: video_decodePerf-HEVC 11/15 Test #11: video_decodePerf-HEVC ............ Passed 2.92 sec Start 12: video_decodeBatch 12/15 Test #12: video_decodeBatch ................ Passed 4.34 sec Start 13: video_decodeRGB-HEVC 13/15 Test #13: video_decodeRGB-HEVC ............. Passed 33.51 sec Start 14: video_decodeMem-HEVC 14/15 Test #14: video_decodeMem-HEVC ............. Passed 2.09 sec Start 15: video_decodeRGB-Resize 15/15 Test #15: video_decodeRGB-Resize ........... Passed 33.94 sec 100% tests passed, 0 tests failed out of 15 AVC conformance test: Conformance test completed on the 127 streams: The number of passing streams is 127 The number of failing streams is 0 The number of streams that did not finish decoding is 0 HEVC conformance test: Conformance test completed on the 135 streams: The number of passing streams is 135 The number of failing streams is 0 The number of streams that did not finish decoding is 0 AV1 conformance test: Conformance test completed on the 162 streams: The number of passing streams is 162 The number of failing streams is 0 The number of streams that did not finish decoding is 0 VP9 conformance test: Conformance test completed on the 108 streams: The number of passing streams is 108 The number of failing streams is 0 The number of streams that did not finish decoding is 0 <!-- Briefly summarize test outcomes. --> rocdecdecode sample dumps the clean output YUV clips of the specified format (NV12 or P016). ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
ammallya
pushed a commit
that referenced
this pull request
Apr 9, 2026
ammallya
pushed a commit
that referenced
this pull request
Apr 9, 2026
[ROCm/hipFile commit: 3ff68e7]
1 task done
kcossett-amd
added a commit
that referenced
this pull request
Jul 14, 2026
…gs (#6992) ## Motivation Closes #8543 <!-- Explain the purpose of this PR and the goals it aims to achieve. --> In this PR, we introduce tiered filtering, where we first filter by objects (shared libs), then modules (generally DWARF compilation units), before fetching the remaining functions. - Regarding modules, Dyninst as a fallback generates a fallback module named after the object file if no DWARF info is available, which is *usually* the case for shared libs. This prevents the costly `app_image->getProcedures()` from slowing down instrumentation when the user does not want to analyze every shared library. Useful for `runtime-instrument`. - Note that we keep `app_image->getProcedures()` as a fallback. Within object filtering, we introduce `--exe-only` and `--max-library-functions`: - `exe-only`: Filters out all objects excluding the main executable furing `runtime-instrumentation`. - `--max-library-functions` (DEFAULT val of `20000`). Libraries with more than `20000` functions get filtered out by default. This is done to drastically reduce default instrumentation time. ## Technical Details <!-- Explain the changes along with any relevant GitHub links. --> - Add `filter_objects` - Fix filtering logic of `filter_modules`, which now respects `--internal-function-include`. - Add `get_modules` (and modify `get_procedures`), which take in a list of filtered objects/modules and return the corresponding modules/procedures. - Added more timing verbprints - Modified verbprints to be more descriptive - Removed `--parse-all-modules` as half of its functionality (going through modules for functions), is now the default. The remaining functionality of excluding all internal library paths is covered with the new `--exclude-internal-lib-paths`. - Added `--exe-only`: Filters out all shared libraries during `runtime-instrumentation`. Has no effect in `binary-rewrite` mode as that mode only handles the main application by default. - Added `--max-library-functions`: Filters out libraries with more than `20000` functions by default. This is done to drastically reduce default instrumentation time. - Removed `libclang-cpp.so` and `libLLVM.so` from internal libraries. With a default `--min-instructions` of `20000`, they are no longer instrumented by default, and will NOT cause a timeout on TheRock. - Force `libomptarget.so` through for `openmp-fortran-offload-runtime-instrument` test. That way we have coverage. **Note**: Regarding the default values of `--max-library-functions` (Following data is gathered on a system that was only running `runtime-instrument`. On a system with many processes running at once, the time may get worse). Some data shows that the `get_procedures` call takes (averaged across 3 runs): ~0.23 **ms** per function for an OpenMP host program that had to get 4067 procedures. - Total time taken: ~4.4 **seconds** ~0.51 **ms** per function for an OpenMP offload program that had to get 57587 procedures. - Total time taken: ~30.0 **seconds** For `get_procedures`, there seems to be more or less overhead depending on the complexity of the functions. ------------------------------------------------------------------------------------------------------------------------ Regarding the time taken to catalog these functions assuming `--dump-info` is **NOT** set (this entire section https://github.com/ROCm/rocm-systems/blob/develop/projects/rocprofiler-systems/source/bin/rocprof-sys-instrument/rocprof-sys-instrument.cpp#L1482) ~0.51 **ms** per function for an OpenMP host program that had to process 4067 procedures. - Total time taken: ~2.1 **seconds** ~0.54 **ms** per function for an OpenMP offload program that had to process 57587 procedures. - Total time taken: ~30 **seconds** ------------------------------------------------------------------------------------------------------------------------ Based off these numbers, 20000 sounds fine. It should allow most small libraries through whilst excluding super large ones (`libomptarget.so, libLLVM.so, libclang-cpp.so`) ## JIRA ID <!-- If applicable, mention the JIRA ID resolved by this PR (Example: Resolves SWDEV-12345). --> <!-- Do not post any JIRA links here. --> AIPROFSYST-412 ## Test Plan <!-- Explain any relevant testing done to verify this PR. --> Workflows. Added two new tests: ``` rocprofiler-systems-instrument-exe-only rocprofiler-systems-instrument-max-library-functions ``` ## Test Result <!-- Briefly summarize test outcomes. --> Tests pass: ``` Start 11: rocprofiler-systems-instrument-exe-only 7/43 Test #11: rocprofiler-systems-instrument-exe-only ................ Passed 3.20 sec Start 12: rocprofiler-systems-instrument-max-library-functions 8/43 Test #12: rocprofiler-systems-instrument-max-library-functions ... Passed 3.35 sec ``` See Workflows ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests. --------- Co-authored-by: David Galiffi <David.Galiffi@amd.com> Co-authored-by: Claude <noreply@anthropic.com>
pvallem
added a commit
that referenced
this pull request
Aug 16, 2026
The plan section-5 scheduler, built on the Phase-3 Rendezvous. Host-validated (33 pytest pass on Windows + WSL); no GPU needed. - lib/pipeline.py PipelineScheduler: loader thread (launches ONLY pipeline entries, under a BoundedSemaphore init-pool; serial entries are handed straight to the executor and never loader-launched), one watcher per in-flight pipeline entry, and a single executor thread. Enforces INITIALIZING+READY <= init_pool and EXECUTING <= 1. - Correctness details from the plan: idempotent finish_entry with a claim (terminal) / settle split so completion is signalled only after teardown (#11); atomic release_slot_if_owned and atomic READY->EXECUTING handoff (#5); per-unit executor exception boundary with INFRA_ERROR distinct from a test FAILED (#4/#10); loader finally-finalizes every unvisited entry (#3); shutdown()/request_stop() where an executor-owned entry is only terminated (its executor records the result). Model A: every entry -> exactly one terminal. - tests/test_scheduler.py: proves the pool + execution bounds post-hoc from runner-observed [t_launch,t_go) / [t_go,t_exit) interval overlap, mixed pipeline+serial non-overlap, init-overlaps-exec, exit-before-READY, exec timeout, launch-fault containment, cancellation race, idempotent finish. Simple release stream for now; legacy per-parent sibling ordering and the coordinator eligibility priority structure land with sub-entry expansion.
prasanna-amd
added a commit
that referenced
this pull request
Aug 26, 2026
…ntegration Add monotonic generation counter per pool slot to detect stale callbacks from reclaimed collectives, resolving the 0==0 race in pool reclaim and all callback-lifetime concerns (threads #4, #6, #8, #9). Add atomic ref-count to acclCommContext so acclPluginFinalize defers free() until all outstanding callbacks have drained (thread #2). Wire accl plugin into the parent RCCL CMake build (thread #20), add CXX_VISIBILITY_PRESET/VISIBILITY_INLINES_HIDDEN (thread #1), and bump ACCL_MAX_CHANNELS from 64 to 256 (thread #11). Fix acclShouldFinalize: nChannels==0 collectives (onerank path) finalize immediately on coll stop; guard channelId against nChannels (thread #10, #12). Free leaked ProxyOp when coll's proxyOpIndices is full (thread #13). Rename firstCpuStart to firstKernelCpuStartUs for clarity (thread #14). Fix warmup off-by-one (sn <= warmup → sn < warmup, thread #7), use try/finally for output file handle (thread #15), fix test fixtures to start at coll_sn=0 (thread #16), remove stale ACCL_PROFILER_WARMUP_ITERS docs (thread #17), sync bottleneck categories with classify() (thread #18), add NVIDIA Apache-2.0 header to accl_shim.h (thread #19). Add tests: ProxyStep lifecycle, channel bounds, kernel timing assertions (threads #3, #12, #21).
1 task done
venkat1361
pushed a commit
that referenced
this pull request
Sep 1, 2026
…10809) ## Summary Addresses a 15-point static re-review of the KFD dispatch-log signal-less path (on top of `review-fixes`). Each finding was evaluated against the code and the prior design history; this PR **fixes the genuine correctness/safety/resource issues** (potential misattribution, UAF on terminal teardown, unbounded pairer-map growth, plus FATAL-handling, env-race, and robustness items) and **documents the ones intentionally left as accepted residuals or pre-existing**, with rationale. `+365 / −49` across 9 files. ## Fixed - **#4 (misattribution) — validate identity before claiming an unknown-START EOP.** The unknown-START path in `dispatch_hub.hpp::record_kernel_end` took the sole first-owner entry with no timestamp check, and terminal-time validation ran only *after* the entry was consumed/retired — so a stale EOP could claim a newer dispatch's entry. Now rejects an EOP whose `end_ticks <= window.t_open` (a kernel cannot end at/before its window opened; same GPU-tick domain the known-START path already uses) **before** taking the entry, so a rejected EOP consumes/retires nothing. Fail-first verified. - **#3 + #2 (UAF) — the terminal fences are now truly terminal (fail-closed).** `signal_less_fence_completions()` previously quiesced visible work but did not stop the reader on success, so a still-PENDING kernel could complete after the client's callback/buffer state (detach/finalize, F23) or the HSA runtime (final `hsa_shut_down`, F24) was freed. A new `signal_less_terminal_stop()` (factored from teardown) latches disable → stops+joins the reader/processor → drains+ledgers PENDING, and is called from the F23/F24 sites. Verified deadlock-free (never joins its own task group, holds no lock across the join, joined threads never block on the joiner) and 5/5 clean under a shutdown watchdog. The existing `get_fini_status()` re-entry guard (the earlier crash fix) is preserved. - **#11 (unbounded growth) — hard size cap on the pairer START map.** `pair_state::pending_starts` was bounded only by a 30-minute age timer. Added `ROCPROFILER_KFD_DISPATCH_LOG_MAX_PENDING_STARTS` (default 2M, shared reject-and-default parser) with oldest-first eviction (recurring key exempted so the `ambiguous` latch is preserved). - **#7 (coverage) + #10 (liveness) — FATAL handling.** FATAL now latches admission and waits for the processor to consume the already-published pipe **before** draining the hub (so a fatal session's final EOP and healthy-GPU EOPs behind it aren't dropped), and the FATAL status is re-probed while the empty-wake backstop is tripped (so a fatal stream that produces no records is still detected). - **#5 (robustness) — master switch is a strict fail-closed opt-in.** `ROCPROFILER_KFD_DISPATCH_LOG_SIGNAL_LESS` now accepts only explicit true values; anything else is disabled with a one-time warning (no throw on malformed input). - **#9 (arg-order) — deterministic GC-gate operand ordering.** `gc_gate_open`'s two operands are read into named locals in the intended order rather than relying on unspecified argument-evaluation order. - **#12 (race) — env knobs primed at serialized init.** The age/cap/quiesce/switch env reads are populated during reader start-up rather than lazily on a worker thread. - **#14(b) — fork child no longer hangs in `interposition_fini()`.** A normal (non-exec) child now short-circuits `interposition_fini()` under the existing `fork_stale()` guard, avoiding a wlock on a registry a vanished parent thread may hold. ## Intentionally not changed (accepted residuals / design decisions / pre-existing) - **#1 — teardown coverage loss** (a kernel in flight at exit may emit no record): the accepted D8/F1 coverage residual. The *UAF* variant of this concern is #3, which is fixed. - **#8 — equal-tick START/EOP**: the accepted D1 residual. The code emits a **no-timing / zero-duration drop**, not a fabricated 1 ns; no misattribution. - **#9-main — GC gate "race"**: the copied-but-still-in-ring window is the accepted **R10** residual (an EOP still in the ring is intentionally not gated), and cap-eviction's bypass of the in-flight gate is D9's documented coverage-loss residual. The operand-ordering nit is fixed (#9-arg above); the residual is by design (coverage loss, never misattribution). - **#13 — cross-batch inversion**: the explicitly-accepted D1 residual (coverage loss under queue churn, never misattribution). A cross-batch reorder buffer is a larger design change weighed against measured coverage impact, which the benchmarks do not show hitting. - **#6 — completion-callback self-deadlock**: a **pre-existing** reentrancy defect on the generic signal path (a client callback that calls its own finalize). The new terminal-stop path was specifically designed **not** to add another instance of it. - **#14(a) — session mid-construction skipped by a concurrent fork**: a child-lifetime resource leak (not a hang); the parent is unaffected and the child drops it at exit. The "fix" would introduce a formal data race on an `smap` pointer a vanished thread was mid-write on, to reclaim memory about to be freed — not worth it. Left as-is. - **#15 — eager arming on a registered (vs active) context**: an efficiency consideration, not a correctness issue; arming stays gated on a registered kernel-dispatch context. ## Known residuals in the fixes (documented) - #4 adds no upper time bound (an implausibly-future EOP is still taken and rejected by the finalizer as no-timing, never misattributed). - #10 restores FATAL *detection* under the backstop; a *persistently failing* status ioctl still isn't fail-closed (separate follow-up). - #11 eviction is O(live) per over-cap insert (same tradeoff as the D9 cap; only reached under pathological EOP loss). ## Testing (MI350, host ROCm 7.2.1, build `fd06e1407c`) - Build clean at the project warning level — **0 new warnings**. - `ctest -L kfd-dlog` — **90/90 pass** (incl. 2 new tests: pairer-map cap, strict opt-in). - ThreadSanitizer, all 8 kfd-dlog suites — **90/90, zero reports** (covers the terminal-stop join, FATAL drain-wait, GC-gate ordering). - Shutdown crash/hang reproducer (1040), **5 runs under a watchdog — 5/5 clean, no hang, no crash** (the decisive check for the terminal-stop rework). - 1041-d (20k kernels ×3), 1031, 1041-a, 1041-c — all pass, no attribution regression. - graph-perf — **−44.21%** overhead reduction, `register-refused=0`/block (no throughput regression). ## Tracking JIRA ID: AIPROFSDK-1012 AIPROFSDK-1040 AIPROFSDK-1041
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copy the following changes::
2190964: ROCm/rocm-libraries#691
a63c5d2: ROCm/rocm-libraries#732