Skip to content

[EX CI] Sync .github and .azuredevops folders from rocm-libraries - #11

Closed
jharryma wants to merge 3 commits into
developfrom
jharryma/create-empty-summary
Closed

jharryma wants to merge 3 commits into
developfrom
jharryma/create-empty-summary

Conversation

@jharryma

@jharryma jharryma commented Jul 29, 2025 •

Copy link
Copy Markdown
Collaborator

systems-assistant Bot pushed a commit that referenced this pull request Aug 5, 2025
Update rocm_ci_caller.yml to use amd-master , until amd-mainline is aligned

Signed-off-by: Choudhary, Rahul <Rahul.Choudhary@amd.com>
systems-assistant Bot pushed a commit that referenced this pull request Aug 5, 2025
systems-assistant Bot pushed a commit that referenced this pull request Aug 5, 2025
- Fixes the broken links in rocm_smi.h
- Uses automodule instead of autofunction in docs/reference/python_api.rst
- Fixes some warnings during docs build
- Update some of the versions in requirements.txt
systems-assistant Bot pushed a commit that referenced this pull request Aug 5, 2025
systems-assistant Bot pushed a commit that referenced this pull request Aug 5, 2025
jayhawk-commits pushed a commit that referenced this pull request Aug 5, 2025
[ROCm/rocprofiler-register commit: 16b142b]
jayhawk-commits pushed a commit that referenced this pull request Aug 5, 2025
jayhawk-commits pushed a commit that referenced this pull request Aug 5, 2025
[ROCm/roctracer commit: 3d3b9ea]
@danielsu-amd

Copy link
Copy Markdown

I think this PR can be closed? Looks like it's up to date with develop

ammallya pushed a commit that referenced this pull request Aug 7, 2025
* SWDEV-492625: Track free memory HSA functions to help determine total amount of memory allocated on the system at any one time

* Minor fixes to address comments

* Update allocation size description

* Moved get function back to specialization, minor typo fixes

* Removed memory_operation_type field, removed memory_pool allocation enum, converted starting address to hex string for json format.

* Made conversion to hex_string a function, changed address to use union rocprofiler_address_t type, changed VMEM descriptors

* Removed as_hex from the global namespace

* Formatting

* Removed TRACK_EVENT for memory allocation, now TRACK_COUNTER for memory allocation is being performed

* Check if address was recorded before retrieving allocation size in generate Perfetto

* Formatting

* Update source/lib/output/generatePerfetto.cpp

* Explicitly disable app-abort tests

* Remove excluding app-abort test from workflow CI

- redundant bc these tests are explicitly marked as disabled now

---------

Co-authored-by: Madsen, Jonathan <Jonathan.Madsen@amd.com>
Co-authored-by: Jonathan R. Madsen <jonathanrmadsen@gmail.com>

[ROCm/rocprofiler-sdk commit: 79006bb]
jayhawk-commits pushed a commit that referenced this pull request Aug 8, 2025
- Fixes the broken links in rocm_smi.h
- Uses automodule instead of autofunction in docs/reference/python_api.rst
- Fixes some warnings during docs build
- Update some of the versions in requirements.txt

[ROCm/rocm_smi_lib commit: e5c5a1d]
jayhawk-commits pushed a commit that referenced this pull request Aug 8, 2025
- Fixes the broken links in rocm_smi.h
- Uses automodule instead of autofunction in docs/reference/python_api.rst
- Fixes some warnings during docs build
- Update some of the versions in requirements.txt

[ROCm/rocm_smi_lib commit: fc61e40]
jayhawk-commits pushed a commit that referenced this pull request Aug 11, 2025
[ROCm/rocminfo commit: 61eb689]
jayhawk-commits pushed a commit that referenced this pull request Aug 11, 2025
Update rocm_ci_caller.yml to use amd-master , until amd-mainline is aligned

Signed-off-by: Choudhary, Rahul <Rahul.Choudhary@amd.com>

[ROCm/ROCR-Runtime commit: 16cd712]
jayhawk-commits pushed a commit that referenced this pull request Aug 18, 2025
@jharryma jharryma closed this Dec 3, 2025
ammallya pushed a commit that referenced this pull request Jan 21, 2026
ammallya pushed a commit that referenced this pull request Jan 21, 2026
MI300X RO Support

[ROCm/rocshmem commit: efa075f]
stellaraccident added a commit that referenced this pull request Feb 26, 2026
## Summary
- When `kpack_get_kernel()` fails with `KPACK_ERROR_KERNEL_NOT_FOUND`
after finding a matching archive+arch, the loader now returns
immediately instead of falling through to probe the next ISA (e.g.,
`gfx9-4-generic`)
- The previous behavior produced misleading "no matching architecture
found" errors and wasted time probing archives that can't help

## Evidence
CI log for rocblas showed:
```
kpack: opened and cached archive: .../blas_lib_gfx942.kpack
kpack:   architecture: gfx942
kpack:   kernel not found in this archive          <-- should stop here
kpack: trying architecture: gfx9-4-generic         <-- wrong: kept probing
kpack: no matching architecture found in any archive
```

The kernel (`#45`) wasn't in the archive due to the multi-CCOB bug (PR
#10). But regardless, once we match an archive and architecture, the
result is definitive.

## Test plan
- [x] New test `LoadCodeObject_KernelNotFound`: archive found, arch
matches, but `co_index=999` not in TOC → `KPACK_ERROR_KERNEL_NOT_FOUND`
- [x] 101/101 CTest pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
jeffqjiangNew added a commit that referenced this pull request Mar 13, 2026
## Motivation

This PR adds entry and exit logs for core functions.

## Technical Details

- Info level (3) logs are added at core function entry and exit points.
- Monotonic clock stamp, process id, thread id, and file name are added
to the logs.
- The logs now have similar format to HIP logs for better viewing when
both are present.
- Direct error or warning logs to std::cerr. Other logs still go to
std::cout.

## JIRA ID

<!-- If applicable, mention the JIRA ID resolved by this PR (Example:
Resolves SWDEV-12345). -->
<!-- Do not post any JIRA links here. -->

## Test Plan

Set env variable ROCDEC_LOG_LEVEL to control log output.

All CTests and conformance tests must pass. Example summary of the test
results:
- CTests:

      Start  1: video_decodeRaw-HEVC
 1/15 Test  #1: video_decodeRaw-HEVC .............   Passed    2.05 sec
      Start  2: video_decodeRaw-AVC
 2/15 Test  #2: video_decodeRaw-AVC ..............   Passed    1.89 sec
      Start  3: video_decodeRaw-AV1
 3/15 Test  #3: video_decodeRaw-AV1 ..............   Passed    2.31 sec
      Start  4: video_decodeRaw-VP9
 4/15 Test  #4: video_decodeRaw-VP9 ..............   Passed    2.04 sec
      Start  5: rocdec_Decode-HEVC
 5/15 Test  #5: rocdec_Decode-HEVC ...............   Passed    0.92 sec
      Start  6: rocDecode_Negative_API_Tests
 6/15 Test  #6: rocDecode_Negative_API_Tests .....   Passed    0.78 sec
      Start  7: video_decode-HEVC
 7/15 Test  #7: video_decode-HEVC ................   Passed    2.74 sec
      Start  8: video_decode-AVC
 8/15 Test  #8: video_decode-AVC .................   Passed    2.64 sec
      Start  9: video_decode-AV1
 9/15 Test  #9: video_decode-AV1 .................   Passed    2.74 sec
      Start 10: video_decode-VP9
10/15 Test #10: video_decode-VP9 .................   Passed    2.78 sec
      Start 11: video_decodePerf-HEVC
11/15 Test #11: video_decodePerf-HEVC ............   Passed    2.92 sec
      Start 12: video_decodeBatch
12/15 Test #12: video_decodeBatch ................   Passed    4.34 sec
      Start 13: video_decodeRGB-HEVC
13/15 Test #13: video_decodeRGB-HEVC .............   Passed   33.51 sec
      Start 14: video_decodeMem-HEVC
14/15 Test #14: video_decodeMem-HEVC .............   Passed    2.09 sec
      Start 15: video_decodeRGB-Resize
15/15 Test #15: video_decodeRGB-Resize ...........   Passed   33.94 sec

100% tests passed, 0 tests failed out of 15

- AVC conformance test:
Conformance test completed on the 127 streams:
     - The number of passing streams is 127
     - The number of failing streams is 0
     - The number of streams that did not finish decoding is 0

- HEVC conformance test:
Conformance test completed on the 135 streams:
     - The number of passing streams is 135
     - The number of failing streams is 0
     - The number of streams that did not finish decoding is 0

- AV1 conformance test:
Conformance test completed on the 162 streams:
     - The number of passing streams is 162
     - The number of failing streams is 0
     - The number of streams that did not finish decoding is 0

- VP9 conformance test:
Conformance test completed on the 108 streams:
     - The number of passing streams is 108
     - The number of failing streams is 0
     - The number of streams that did not finish decoding is 0

## Test Result

Function entry/exit logs should show up when ROCDEC_LOG_LEVEL >= 3.

All conformance tests should pass.

## Submission Checklist

- [ ] Look over the contributing guidelines at
https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
Nikhil-Nunna pushed a commit that referenced this pull request Mar 13, 2026
## Motivation

This PR adds entry and exit logs for core functions.

## Technical Details

- Info level (3) logs are added at core function entry and exit points.
- Monotonic clock stamp, process id, thread id, and file name are added
to the logs.
- The logs now have similar format to HIP logs for better viewing when
both are present.
- Direct error or warning logs to std::cerr. Other logs still go to
std::cout.

## JIRA ID

<!-- If applicable, mention the JIRA ID resolved by this PR (Example:
Resolves SWDEV-12345). -->
<!-- Do not post any JIRA links here. -->

## Test Plan

Set env variable ROCDEC_LOG_LEVEL to control log output.

All CTests and conformance tests must pass. Example summary of the test
results:
- CTests:

      Start  1: video_decodeRaw-HEVC
 1/15 Test  #1: video_decodeRaw-HEVC .............   Passed    2.05 sec
      Start  2: video_decodeRaw-AVC
 2/15 Test  #2: video_decodeRaw-AVC ..............   Passed    1.89 sec
      Start  3: video_decodeRaw-AV1
 3/15 Test  #3: video_decodeRaw-AV1 ..............   Passed    2.31 sec
      Start  4: video_decodeRaw-VP9
 4/15 Test  #4: video_decodeRaw-VP9 ..............   Passed    2.04 sec
      Start  5: rocdec_Decode-HEVC
 5/15 Test  #5: rocdec_Decode-HEVC ...............   Passed    0.92 sec
      Start  6: rocDecode_Negative_API_Tests
 6/15 Test  #6: rocDecode_Negative_API_Tests .....   Passed    0.78 sec
      Start  7: video_decode-HEVC
 7/15 Test  #7: video_decode-HEVC ................   Passed    2.74 sec
      Start  8: video_decode-AVC
 8/15 Test  #8: video_decode-AVC .................   Passed    2.64 sec
      Start  9: video_decode-AV1
 9/15 Test  #9: video_decode-AV1 .................   Passed    2.74 sec
      Start 10: video_decode-VP9
10/15 Test #10: video_decode-VP9 .................   Passed    2.78 sec
      Start 11: video_decodePerf-HEVC
11/15 Test #11: video_decodePerf-HEVC ............   Passed    2.92 sec
      Start 12: video_decodeBatch
12/15 Test #12: video_decodeBatch ................   Passed    4.34 sec
      Start 13: video_decodeRGB-HEVC
13/15 Test #13: video_decodeRGB-HEVC .............   Passed   33.51 sec
      Start 14: video_decodeMem-HEVC
14/15 Test #14: video_decodeMem-HEVC .............   Passed    2.09 sec
      Start 15: video_decodeRGB-Resize
15/15 Test #15: video_decodeRGB-Resize ...........   Passed   33.94 sec

100% tests passed, 0 tests failed out of 15

- AVC conformance test:
Conformance test completed on the 127 streams:
     - The number of passing streams is 127
     - The number of failing streams is 0
     - The number of streams that did not finish decoding is 0

- HEVC conformance test:
Conformance test completed on the 135 streams:
     - The number of passing streams is 135
     - The number of failing streams is 0
     - The number of streams that did not finish decoding is 0

- AV1 conformance test:
Conformance test completed on the 162 streams:
     - The number of passing streams is 162
     - The number of failing streams is 0
     - The number of streams that did not finish decoding is 0

- VP9 conformance test:
Conformance test completed on the 108 streams:
     - The number of passing streams is 108
     - The number of failing streams is 0
     - The number of streams that did not finish decoding is 0

## Test Result

Function entry/exit logs should show up when ROCDEC_LOG_LEVEL >= 3.

All conformance tests should pass.

## Submission Checklist

- [ ] Look over the contributing guidelines at
https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
jeffqjiangNew added a commit that referenced this pull request Apr 2, 2026
…4229)

## Motivation

This PR adds a new rocDecode feature: support for user specified output
surface format.

## Technical Details

In certain use cases, the user requests a decoded image format that is
not the native format based on the stream content. For example, 8-bit
output is needed on a 10-bit stream. This PR enables the output format
selection as long as it is supported by the VA-API driver.
- Added a special output format define to indicate if the output format
is user set or native.
- VA-API decoder now sets the surface format either native or by user
override.
- Added the output format command option to rocdecdecode sample for
feature verification.
## JIRA ID

<!-- If applicable, mention the JIRA ID resolved by this PR (Example:
Resolves SWDEV-12345). -->
<!-- Do not post any JIRA links here. -->

## Test Plan

- Run CTests and decode conformance tests to catch any regressions. 
- Run rocdecdecode samples to verify user set output.
## Test Result

All CTests and conformance tests pass. Example summary of the test
results:

CTests:
      Start  1: video_decodeRaw-HEVC
1/15 Test #1:
video_decodeRaw-HEVC ............. Passed 2.05 sec
      Start  2: video_decodeRaw-AVC
2/15 Test #2:
video_decodeRaw-AVC .............. Passed 1.89 sec
      Start  3: video_decodeRaw-AV1
3/15 Test #3:
video_decodeRaw-AV1 .............. Passed 2.31 sec
      Start  4: video_decodeRaw-VP9
4/15 Test #4:
video_decodeRaw-VP9 .............. Passed 2.04 sec
      Start  5: rocdec_Decode-HEVC
5/15 Test #5:
rocdec_Decode-HEVC ............... Passed 0.92 sec
      Start  6: rocDecode_Negative_API_Tests
6/15 Test #6:
rocDecode_Negative_API_Tests ..... Passed 0.78 sec
      Start  7: video_decode-HEVC
7/15 Test #7: video_decode-HEVC
................ Passed 2.74 sec
      Start  8: video_decode-AVC
8/15 Test #8: video_decode-AVC
................. Passed 2.64 sec
      Start  9: video_decode-AV1
9/15 Test #9: video_decode-AV1
................. Passed 2.74 sec
      Start 10: video_decode-VP9
10/15 Test #10:
video_decode-VP9 ................. Passed 2.78 sec
      Start 11: video_decodePerf-HEVC
11/15 Test #11:
video_decodePerf-HEVC ............ Passed 2.92 sec
      Start 12: video_decodeBatch
12/15 Test #12:
video_decodeBatch ................ Passed 4.34 sec
      Start 13: video_decodeRGB-HEVC
13/15 Test #13:
video_decodeRGB-HEVC ............. Passed 33.51 sec
      Start 14: video_decodeMem-HEVC
14/15 Test #14:
video_decodeMem-HEVC ............. Passed 2.09 sec
      Start 15: video_decodeRGB-Resize
15/15 Test #15:
video_decodeRGB-Resize ........... Passed 33.94 sec

100% tests passed, 0 tests failed out of 15

AVC conformance test:
Conformance test completed on the 127 streams:

The number of passing streams is 127
The number of failing streams is 0
The number of streams that did not finish decoding is 0
HEVC conformance test:
Conformance test completed on the 135 streams:

The number of passing streams is 135
The number of failing streams is 0
The number of streams that did not finish decoding is 0
AV1 conformance test:
Conformance test completed on the 162 streams:

The number of passing streams is 162
The number of failing streams is 0
The number of streams that did not finish decoding is 0
VP9 conformance test:
Conformance test completed on the 108 streams:

The number of passing streams is 108
The number of failing streams is 0
The number of streams that did not finish decoding is 0
<!-- Briefly summarize test outcomes. -->

rocdecdecode sample dumps the clean output YUV clips of the specified
format (NV12 or P016).

## Submission Checklist

- [ ] Look over the contributing guidelines at
https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
gandryey pushed a commit that referenced this pull request Apr 6, 2026
…4229)

## Motivation

This PR adds a new rocDecode feature: support for user specified output
surface format.

## Technical Details

In certain use cases, the user requests a decoded image format that is
not the native format based on the stream content. For example, 8-bit
output is needed on a 10-bit stream. This PR enables the output format
selection as long as it is supported by the VA-API driver.
- Added a special output format define to indicate if the output format
is user set or native.
- VA-API decoder now sets the surface format either native or by user
override.
- Added the output format command option to rocdecdecode sample for
feature verification.
## JIRA ID

<!-- If applicable, mention the JIRA ID resolved by this PR (Example:
Resolves SWDEV-12345). -->
<!-- Do not post any JIRA links here. -->

## Test Plan

- Run CTests and decode conformance tests to catch any regressions. 
- Run rocdecdecode samples to verify user set output.
## Test Result

All CTests and conformance tests pass. Example summary of the test
results:

CTests:
      Start  1: video_decodeRaw-HEVC
1/15 Test #1:
video_decodeRaw-HEVC ............. Passed 2.05 sec
      Start  2: video_decodeRaw-AVC
2/15 Test #2:
video_decodeRaw-AVC .............. Passed 1.89 sec
      Start  3: video_decodeRaw-AV1
3/15 Test #3:
video_decodeRaw-AV1 .............. Passed 2.31 sec
      Start  4: video_decodeRaw-VP9
4/15 Test #4:
video_decodeRaw-VP9 .............. Passed 2.04 sec
      Start  5: rocdec_Decode-HEVC
5/15 Test #5:
rocdec_Decode-HEVC ............... Passed 0.92 sec
      Start  6: rocDecode_Negative_API_Tests
6/15 Test #6:
rocDecode_Negative_API_Tests ..... Passed 0.78 sec
      Start  7: video_decode-HEVC
7/15 Test #7: video_decode-HEVC
................ Passed 2.74 sec
      Start  8: video_decode-AVC
8/15 Test #8: video_decode-AVC
................. Passed 2.64 sec
      Start  9: video_decode-AV1
9/15 Test #9: video_decode-AV1
................. Passed 2.74 sec
      Start 10: video_decode-VP9
10/15 Test #10:
video_decode-VP9 ................. Passed 2.78 sec
      Start 11: video_decodePerf-HEVC
11/15 Test #11:
video_decodePerf-HEVC ............ Passed 2.92 sec
      Start 12: video_decodeBatch
12/15 Test #12:
video_decodeBatch ................ Passed 4.34 sec
      Start 13: video_decodeRGB-HEVC
13/15 Test #13:
video_decodeRGB-HEVC ............. Passed 33.51 sec
      Start 14: video_decodeMem-HEVC
14/15 Test #14:
video_decodeMem-HEVC ............. Passed 2.09 sec
      Start 15: video_decodeRGB-Resize
15/15 Test #15:
video_decodeRGB-Resize ........... Passed 33.94 sec

100% tests passed, 0 tests failed out of 15

AVC conformance test:
Conformance test completed on the 127 streams:

The number of passing streams is 127
The number of failing streams is 0
The number of streams that did not finish decoding is 0
HEVC conformance test:
Conformance test completed on the 135 streams:

The number of passing streams is 135
The number of failing streams is 0
The number of streams that did not finish decoding is 0
AV1 conformance test:
Conformance test completed on the 162 streams:

The number of passing streams is 162
The number of failing streams is 0
The number of streams that did not finish decoding is 0
VP9 conformance test:
Conformance test completed on the 108 streams:

The number of passing streams is 108
The number of failing streams is 0
The number of streams that did not finish decoding is 0
<!-- Briefly summarize test outcomes. -->

rocdecdecode sample dumps the clean output YUV clips of the specified
format (NV12 or P016).

## Submission Checklist

- [ ] Look over the contributing guidelines at
https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
ammallya pushed a commit that referenced this pull request Apr 9, 2026
kcossett-amd added a commit that referenced this pull request Jul 14, 2026
…gs (#6992)

## Motivation

Closes #8543

<!-- Explain the purpose of this PR and the goals it aims to achieve.
-->

In this PR, we introduce tiered filtering, where we first filter by
objects (shared libs), then modules (generally DWARF compilation units),
before fetching the remaining functions.
- Regarding modules, Dyninst as a fallback generates a fallback module
named after the object file if no DWARF info is available, which is
*usually* the case for shared libs.

This prevents the costly `app_image->getProcedures()` from slowing down
instrumentation when the user does not want to analyze every shared
library. Useful for `runtime-instrument`.
 - Note that we keep `app_image->getProcedures()` as a fallback.

Within object filtering, we introduce `--exe-only` and
`--max-library-functions`:
- `exe-only`: Filters out all objects excluding the main executable
furing `runtime-instrumentation`.
- `--max-library-functions` (DEFAULT val of `20000`). Libraries with
more than `20000` functions get filtered out by default. This is done to
drastically reduce default instrumentation time.

## Technical Details

<!-- Explain the changes along with any relevant GitHub links. -->

 - Add `filter_objects`
- Fix filtering logic of `filter_modules`, which now respects
`--internal-function-include`.
- Add `get_modules` (and modify `get_procedures`), which take in a list
of filtered objects/modules and return the corresponding
modules/procedures.
 - Added more timing verbprints
 - Modified verbprints to be more descriptive
- Removed `--parse-all-modules` as half of its functionality (going
through modules for functions), is now the default. The remaining
functionality of excluding all internal library paths is covered with
the new `--exclude-internal-lib-paths`.
- Added `--exe-only`: Filters out all shared libraries during
`runtime-instrumentation`. Has no effect in `binary-rewrite` mode as
that mode only handles the main application by default.
- Added `--max-library-functions`: Filters out libraries with more than
`20000` functions by default. This is done to drastically reduce default
instrumentation time.
- Removed `libclang-cpp.so` and `libLLVM.so` from internal libraries.
With a default `--min-instructions` of `20000`, they are no longer
instrumented by default, and will NOT cause a timeout on TheRock.
- Force `libomptarget.so` through for
`openmp-fortran-offload-runtime-instrument` test. That way we have
coverage.

**Note**: Regarding the default values of `--max-library-functions`

(Following data is gathered on a system that was only running
`runtime-instrument`. On a system with many processes running at once,
the time may get worse).

Some data shows that the `get_procedures` call takes (averaged across 3
runs):
~0.23 **ms** per function for an OpenMP host program that had to get
4067 procedures.
 - Total time taken: ~4.4 **seconds**

~0.51 **ms** per function for an OpenMP offload program that had to get
57587 procedures.
 - Total time taken: ~30.0 **seconds**

For `get_procedures`, there seems to be more or less overhead depending
on the complexity of the functions.


------------------------------------------------------------------------------------------------------------------------

Regarding the time taken to catalog these functions assuming
`--dump-info` is **NOT** set (this entire section
https://github.com/ROCm/rocm-systems/blob/develop/projects/rocprofiler-systems/source/bin/rocprof-sys-instrument/rocprof-sys-instrument.cpp#L1482)

~0.51 **ms** per function for an OpenMP host program that had to process
4067 procedures.
 - Total time taken: ~2.1 **seconds**

~0.54 **ms** per function for an OpenMP offload program that had to
process 57587 procedures.
 - Total time taken: ~30 **seconds**


------------------------------------------------------------------------------------------------------------------------

Based off these numbers, 20000 sounds fine. It should allow most small
libraries through whilst excluding super large ones (`libomptarget.so,
libLLVM.so, libclang-cpp.so`)

## JIRA ID

<!-- If applicable, mention the JIRA ID resolved by this PR (Example:
Resolves SWDEV-12345). -->
<!-- Do not post any JIRA links here. -->

AIPROFSYST-412

## Test Plan

<!-- Explain any relevant testing done to verify this PR. -->

Workflows. Added two new tests:
```
rocprofiler-systems-instrument-exe-only
rocprofiler-systems-instrument-max-library-functions
```

## Test Result

<!-- Briefly summarize test outcomes. -->

Tests pass:
```
      Start  11: rocprofiler-systems-instrument-exe-only
 7/43 Test  #11: rocprofiler-systems-instrument-exe-only ................   Passed    3.20 sec
      Start  12: rocprofiler-systems-instrument-max-library-functions
 8/43 Test  #12: rocprofiler-systems-instrument-max-library-functions ...   Passed    3.35 sec
```

See Workflows

## Submission Checklist

- [x] Look over the contributing guidelines at
https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

---------

Co-authored-by: David Galiffi <David.Galiffi@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
pvallem added a commit that referenced this pull request Aug 16, 2026
The plan section-5 scheduler, built on the Phase-3 Rendezvous. Host-validated
(33 pytest pass on Windows + WSL); no GPU needed.

- lib/pipeline.py PipelineScheduler: loader thread (launches ONLY pipeline
  entries, under a BoundedSemaphore init-pool; serial entries are handed straight
  to the executor and never loader-launched), one watcher per in-flight pipeline
  entry, and a single executor thread. Enforces INITIALIZING+READY <= init_pool
  and EXECUTING <= 1.
- Correctness details from the plan: idempotent finish_entry with a claim
  (terminal) / settle split so completion is signalled only after teardown
  (#11); atomic release_slot_if_owned and atomic READY->EXECUTING handoff (#5);
  per-unit executor exception boundary with INFRA_ERROR distinct from a test
  FAILED (#4/#10); loader finally-finalizes every unvisited entry (#3);
  shutdown()/request_stop() where an executor-owned entry is only terminated
  (its executor records the result). Model A: every entry -> exactly one terminal.
- tests/test_scheduler.py: proves the pool + execution bounds post-hoc from
  runner-observed [t_launch,t_go) / [t_go,t_exit) interval overlap, mixed
  pipeline+serial non-overlap, init-overlaps-exec, exit-before-READY, exec
  timeout, launch-fault containment, cancellation race, idempotent finish.

Simple release stream for now; legacy per-parent sibling ordering and the
coordinator eligibility priority structure land with sub-entry expansion.
prasanna-amd added a commit that referenced this pull request Aug 26, 2026
…ntegration

Add monotonic generation counter per pool slot to detect stale callbacks
from reclaimed collectives, resolving the 0==0 race in pool reclaim and
all callback-lifetime concerns (threads #4, #6, #8, #9).

Add atomic ref-count to acclCommContext so acclPluginFinalize defers free()
until all outstanding callbacks have drained (thread #2).

Wire accl plugin into the parent RCCL CMake build (thread #20), add
CXX_VISIBILITY_PRESET/VISIBILITY_INLINES_HIDDEN (thread #1), and bump
ACCL_MAX_CHANNELS from 64 to 256 (thread #11).

Fix acclShouldFinalize: nChannels==0 collectives (onerank path) finalize
immediately on coll stop; guard channelId against nChannels (thread #10, #12).
Free leaked ProxyOp when coll's proxyOpIndices is full (thread #13).
Rename firstCpuStart to firstKernelCpuStartUs for clarity (thread #14).

Fix warmup off-by-one (sn <= warmup → sn < warmup, thread #7), use
try/finally for output file handle (thread #15), fix test fixtures to
start at coll_sn=0 (thread #16), remove stale ACCL_PROFILER_WARMUP_ITERS
docs (thread #17), sync bottleneck categories with classify() (thread #18),
add NVIDIA Apache-2.0 header to accl_shim.h (thread #19).

Add tests: ProxyStep lifecycle, channel bounds, kernel timing assertions
(threads #3, #12, #21).
bwelton added a commit that referenced this pull request Aug 27, 2026
bwelton added a commit that referenced this pull request Aug 31, 2026
bwelton added a commit that referenced this pull request Aug 31, 2026
bwelton added a commit that referenced this pull request Sep 1, 2026
bwelton added a commit that referenced this pull request Sep 1, 2026
venkat1361 pushed a commit that referenced this pull request Sep 1, 2026
…10809)

## Summary
Addresses a 15-point static re-review of the KFD dispatch-log
signal-less path (on top of `review-fixes`). Each finding was evaluated
against the code and the prior design history; this PR **fixes the
genuine correctness/safety/resource issues** (potential misattribution,
UAF on terminal teardown, unbounded pairer-map growth, plus
FATAL-handling, env-race, and robustness items) and **documents the ones
intentionally left as accepted residuals or pre-existing**, with
rationale. `+365 / −49` across 9 files.

## Fixed

- **#4 (misattribution) — validate identity before claiming an
unknown-START EOP.** The unknown-START path in
`dispatch_hub.hpp::record_kernel_end` took the sole first-owner entry
with no timestamp check, and terminal-time validation ran only *after*
the entry was consumed/retired — so a stale EOP could claim a newer
dispatch's entry. Now rejects an EOP whose `end_ticks <= window.t_open`
(a kernel cannot end at/before its window opened; same GPU-tick domain
the known-START path already uses) **before** taking the entry, so a
rejected EOP consumes/retires nothing. Fail-first verified.
- **#3 + #2 (UAF) — the terminal fences are now truly terminal
(fail-closed).** `signal_less_fence_completions()` previously quiesced
visible work but did not stop the reader on success, so a still-PENDING
kernel could complete after the client's callback/buffer state
(detach/finalize, F23) or the HSA runtime (final `hsa_shut_down`, F24)
was freed. A new `signal_less_terminal_stop()` (factored from teardown)
latches disable → stops+joins the reader/processor → drains+ledgers
PENDING, and is called from the F23/F24 sites. Verified deadlock-free
(never joins its own task group, holds no lock across the join, joined
threads never block on the joiner) and 5/5 clean under a shutdown
watchdog. The existing `get_fini_status()` re-entry guard (the earlier
crash fix) is preserved.
- **#11 (unbounded growth) — hard size cap on the pairer START map.**
`pair_state::pending_starts` was bounded only by a 30-minute age timer.
Added `ROCPROFILER_KFD_DISPATCH_LOG_MAX_PENDING_STARTS` (default 2M,
shared reject-and-default parser) with oldest-first eviction (recurring
key exempted so the `ambiguous` latch is preserved).
- **#7 (coverage) + #10 (liveness) — FATAL handling.** FATAL now latches
admission and waits for the processor to consume the already-published
pipe **before** draining the hub (so a fatal session's final EOP and
healthy-GPU EOPs behind it aren't dropped), and the FATAL status is
re-probed while the empty-wake backstop is tripped (so a fatal stream
that produces no records is still detected).
- **#5 (robustness) — master switch is a strict fail-closed opt-in.**
`ROCPROFILER_KFD_DISPATCH_LOG_SIGNAL_LESS` now accepts only explicit
true values; anything else is disabled with a one-time warning (no throw
on malformed input).
- **#9 (arg-order) — deterministic GC-gate operand ordering.**
`gc_gate_open`'s two operands are read into named locals in the intended
order rather than relying on unspecified argument-evaluation order.
- **#12 (race) — env knobs primed at serialized init.** The
age/cap/quiesce/switch env reads are populated during reader start-up
rather than lazily on a worker thread.
- **#14(b) — fork child no longer hangs in `interposition_fini()`.** A
normal (non-exec) child now short-circuits `interposition_fini()` under
the existing `fork_stale()` guard, avoiding a wlock on a registry a
vanished parent thread may hold.

## Intentionally not changed (accepted residuals / design decisions /
pre-existing)

- **#1 — teardown coverage loss** (a kernel in flight at exit may emit
no record): the accepted D8/F1 coverage residual. The *UAF* variant of
this concern is #3, which is fixed.
- **#8 — equal-tick START/EOP**: the accepted D1 residual. The code
emits a **no-timing / zero-duration drop**, not a fabricated 1 ns; no
misattribution.
- **#9-main — GC gate "race"**: the copied-but-still-in-ring window is
the accepted **R10** residual (an EOP still in the ring is intentionally
not gated), and cap-eviction's bypass of the in-flight gate is D9's
documented coverage-loss residual. The operand-ordering nit is fixed
(#9-arg above); the residual is by design (coverage loss, never
misattribution).
- **#13 — cross-batch inversion**: the explicitly-accepted D1 residual
(coverage loss under queue churn, never misattribution). A cross-batch
reorder buffer is a larger design change weighed against measured
coverage impact, which the benchmarks do not show hitting.
- **#6 — completion-callback self-deadlock**: a **pre-existing**
reentrancy defect on the generic signal path (a client callback that
calls its own finalize). The new terminal-stop path was specifically
designed **not** to add another instance of it.
- **#14(a) — session mid-construction skipped by a concurrent fork**: a
child-lifetime resource leak (not a hang); the parent is unaffected and
the child drops it at exit. The "fix" would introduce a formal data race
on an `smap` pointer a vanished thread was mid-write on, to reclaim
memory about to be freed — not worth it. Left as-is.
- **#15 — eager arming on a registered (vs active) context**: an
efficiency consideration, not a correctness issue; arming stays gated on
a registered kernel-dispatch context.

## Known residuals in the fixes (documented)
- #4 adds no upper time bound (an implausibly-future EOP is still taken
and rejected by the finalizer as no-timing, never misattributed).
- #10 restores FATAL *detection* under the backstop; a *persistently
failing* status ioctl still isn't fail-closed (separate follow-up).
- #11 eviction is O(live) per over-cap insert (same tradeoff as the D9
cap; only reached under pathological EOP loss).

## Testing (MI350, host ROCm 7.2.1, build `fd06e1407c`)
- Build clean at the project warning level — **0 new warnings**.
- `ctest -L kfd-dlog` — **90/90 pass** (incl. 2 new tests: pairer-map
cap, strict opt-in).
- ThreadSanitizer, all 8 kfd-dlog suites — **90/90, zero reports**
(covers the terminal-stop join, FATAL drain-wait, GC-gate ordering).
- Shutdown crash/hang reproducer (1040), **5 runs under a watchdog — 5/5
clean, no hang, no crash** (the decisive check for the terminal-stop
rework).
- 1041-d (20k kernels ×3), 1031, 1041-a, 1041-c — all pass, no
attribution regression.
- graph-perf — **−44.21%** overhead reduction,
`register-refused=0`/block (no throughput regression).

## Tracking
JIRA ID: AIPROFSDK-1012 AIPROFSDK-1040 AIPROFSDK-1041
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants