[DO NOT MERGE] Modify Project Workflows by amd-hsivasun · Pull Request #8 · ROCm/rocm-systems

amd-hsivasun · 2025-07-24T20:09:30Z

No description provided.

github-advanced-security · 2025-07-25T15:18:52Z

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

ammarwa · 2025-07-25T20:51:29Z


 jobs:
  git-mirror:
-    runs-on: AMD-ROCm-Internal-dev1


This file is no longer needed

ammarwa · 2025-07-25T20:51:37Z


 jobs:
  git-mirror:
-    runs-on: AMD-ROCm-Internal-dev1


This file is no longer needed

ammarwa · 2025-07-25T20:53:42Z

@@ -19,7 +19,7 @@ jobs:
    #   - https://gh.io/supported-runners-and-hardware-resources
    #   - https://gh.io/using-larger-runners (GitHub.com only)
    # Consider using larger runners or machines with greater resources for possible analysis time improvements.


It seems you have CodeQL in the parent Directory, if so and if it checks for CPP, Actions and Python then no need for this file as well

ammarwa · 2025-07-25T20:54:01Z


 jobs:
  git-mirror:
-    runs-on: AMD-ROCm-Internal-dev1


no need for this file

ammarwa · 2025-07-25T20:54:13Z


 jobs:
  git-mirror:
-    runs-on: AMD-ROCm-Internal-dev1


no need for this file

ammarwa · 2025-07-25T20:54:45Z

 jobs:
  git-mirror:
-    runs-on: AMD-ROCm-Internal-dev1
+    runs-on: ubuntu-latest


no need for this file

ammarwa · 2025-07-25T21:02:24Z

Delete the following workflows as they are not needed anymore:

In AQLProfile:

In ROCProfiler Register:

In ROCProfiler SDK:

…SSION_THRESHOLD Adds a standalone gate rule ``trace_diff_regression_rule`` to ``perfxpert.runtime.gate_cascade`` so the correctness cascade, the ``perfxpert ci`` subcommand, and any future agentic caller share the same regression-detection engine. Behavior: * Thin wrapper over ``tools.trace_diff.diff_runs``. * Verdict ``fail`` when ``wall_delta_pct > threshold``; otherwise ``pass``. * Threshold precedence: explicit ``threshold_pct=`` > env var ``PERFXPERT_CI_REGRESSION_THRESHOLD`` > :data:`DEFAULT_CI_REGRESSION_THRESHOLD_PCT` (5.0%). Invalid env values fall back to the default (no exceptions escape to the caller). * Returns a machine-readable dict including the full ``diff_runs`` result so gate callers can show the same tables the CLI does. The existing 5-gate ``evaluate()`` cascade is intentionally untouched — this is a complementary standalone rule for the "compare two runs without a candidate patch" flow that Confluence rows #7 + #8 need. Tests: 8 new — threshold-resolution precedence (4) + verdict correctness on baseline/tail-hurt/improved fixture pairs (4). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Ships the CI-gating wrapper (Confluence row #8). Thin shim over ``perfxpert diff`` — same engine, same output shapes, but the verdict is enforced by the process exit code. Behavior: * ``perfxpert ci <baseline.db> <new.db>`` runs the same ``trace_diff`` engine as ``perfxpert diff``. * rc=1 when ``wall_delta_pct > --threshold`` (default 5%; env override ``PERFXPERT_CI_REGRESSION_THRESHOLD``; CLI flag wins). * rc=0 when the delta is within threshold (or improved). * Stderr one-liner (✅ / ❌) so CI log scanners can pick it up. * Stdout carries the full diff report so humans and tooling can inspect the numbers without re-running the command. * Formats: ``text`` (default, human), ``json`` (CI-parseable), or ``markdown``. Webview intentionally omitted — not a CI-native format. Tests: 5 new — rc=0 on improvement, rc=1 above threshold, env override semantics, CLI-flag-wins-over-env, text format regression summary. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…mped to 42 tools Documents Confluence rows #5 (severity refinement), #7 (trace_diff tool + subcommand + analyze splice), and #8 (perfxpert ci wrapper): - docs/guides/getting-started.md: new §7 "Compare runs + CI gating (``perfxpert diff`` / ``perfxpert ci``)" with GitHub Actions + GitLab CI snippets; remaining numbered sections shifted by +1. - docs/guides/python-api.md: document ``api.trace_diff_diff_runs`` mirror + expected output shape. - docs/integration/mcp-server.md: tool count 41 → 42; add ``trace_diff.diff_runs`` to the classifier/knowledge snapshot (34 → 35). - docs/contributing/schemas.md: describe the 0.3.1 ``trace_diff`` top-level field (shape, emission rules, own ``schema_version``); note severity + severity_label additions to ``hotspots[i].source_locations``. - docs/contributing/output_formats.md: contract checklist gains the conditional ``trace_diff`` top-level key; adds a severity- coloring section (bucket table + webview/text/markdown propagation rules) and a diff-reports contract requiring every format to surface regression-vs-improvement direction. Validation: - scripts/lint.sh --strict rc=0 - scripts/link-checker.py --strict rc=0 - scripts/test-samples.py --strict rc=0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

- Guard sentinel values before mW-to-W arithmetic to prevent overflow - Add offsetof static_asserts for layout compatibility checks - Replace GTEST_SKIP with continue in device loop - Document mW vs W distinction in APU power section comments - Reorder APU macros in rocm_smi.h to match amdsmi.h - Complete CHANGELOG sentinel docs for all field types

…al config Add get_sampling_signals_integration_test.cpp: 9 tests that call the real get_sampling_signals(tid) from core/config.cpp to verify ROCPROFSYS_SAMPLING_REALTIME_TIDS / _CPUTIME_TIDS allow-list filtering (R-3 / AC-4). Uses configure_settings(false) in a GTest Environment to pre-initialize the settings map without triggering tim::timemory_init(), which would cause a recursive-static-init error (__gnu_cxx::recursive_init_error) on first test run. Standalone binary links rocprofiler-systems-core-library directly; no config stubs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@var

Translation parser (#3, #6): Replace the residual ROCP_FATAL on wave_id mismatch in the GFX1250 V2/V4 and stochastic copySample paths with ROCP_ERROR + returning a value-initialized record so it is filtered by is_invalid_sample() instead of aborting the process. Drop the redundant std::memset() that followed value-initialization of the invalid-sample sentinel in the GFX9/GFX12/GFX1250 V2 and GFX1250 V4 paths, and document the timestamp-zero invariant. query_agent_configurations_v2 (#5): Replace the dead "sort preferred-method configs first" pass with the same PREFER semantics already implemented by configure_v2: if any preferred-method config exists, return only those; otherwise fall back to returning all configs. REQUIRE remains a hard filter. Callback signature: Change rocprofiler_available_pc_sampling_configurations_v2_cb_t to deliver configurations as a const T** array of pointers, matching the convention used by rocprofiler_query_available_agents_cb_t. Update the only consumer (samples/pc_sampling_v2/pcs.cpp) accordingly. Documentation (#7, #8): Remove the V5 reference (only V0-V4 exist today); expand the api_flags doc with PREFER/REQUIRE behavior for both query and configure entry points and add per-enumerator @var blocks; add an Example 4 to configure_v2 showing conflicting flags being rejected; mark configuration_v2_t as output-only; add per-flag filtering rules and three usage examples to query_v2's doc; expand _get_parse_func_for_record_kind's docstring to specify when nullptr is returned. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

@var

Translation parser (#3, #6): Replace the residual ROCP_FATAL on wave_id mismatch in the GFX1250 V2/V4 and stochastic copySample paths with ROCP_ERROR + returning a value-initialized record so it is filtered by is_invalid_sample() instead of aborting the process. Drop the redundant std::memset() that followed value-initialization of the invalid-sample sentinel in the GFX9/GFX12/GFX1250 V2 and GFX1250 V4 paths, and document the timestamp-zero invariant. query_agent_configurations_v2 (#5): Replace the dead "sort preferred-method configs first" pass with the same PREFER semantics already implemented by configure_v2: if any preferred-method config exists, return only those; otherwise fall back to returning all configs. REQUIRE remains a hard filter. Callback signature: Change rocprofiler_available_pc_sampling_configurations_v2_cb_t to deliver configurations as a const T** array of pointers, matching the convention used by rocprofiler_query_available_agents_cb_t. Update the only consumer (samples/pc_sampling_v2/pcs.cpp) accordingly. Documentation (#7, #8): Remove the V5 reference (only V0-V4 exist today); expand the api_flags doc with PREFER/REQUIRE behavior for both query and configure entry points and add per-enumerator @var blocks; add an Example 4 to configure_v2 showing conflicting flags being rejected; mark configuration_v2_t as output-only; add per-flag filtering rules and three usage examples to query_v2's doc; expand _get_parse_func_for_record_kind's docstring to specify when nullptr is returned. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

## Motivation On certain test systems we do not have a shared filesystem, the deadlock_analysis script harness would depend on a shared filesystem to collect traces and artifacts. Remove that limitation. ## Technical Details Use MPI stdout redirection to coalesce the trace artifacts to the headnode in a headnode local output directory, perform analysis on the headnode. Prior filesystem based mode remains available and default. ## JIRA ID ## Test Plan Intentionally added a deadlock to a functional test, compiled with Debug. Run mpiexec_deadlock_attach script and observe artifacts. ## Test Result ``` ROCSHMEM_DEBUG_LEVEL=trace:stats ~/ompi/bin/mpirun -n 2 tests/functional_tests/rocshmem_functional_tests -n 1 -a 26 & (sleep 2; /home/abouteil/rocm-systems/projects/rocshmem/scripts/debug/mpiexec_deadlock_analysis.sh rocshmem_functional_tests --stats --color --no-shared-fs ) ### Creating Test: Blocking WG level Puts B=ipc PE=2 W=1 Z=64 ### Deploying deadlock analysis... Executable: rocshmem_functional_tests Output directory: ./rocshmem_deadlock_analysis_20260530_013437 Filesystem mode: no shared fs (--output-filename) Stats: yes Color: --color Found 4 process(es) matching 'rocshmem_functional_tests': 717472 717473 751554 751555 Output directory: /tmp/rocshmem_deadlock_20260530_013437 Per-process timeout: 120s Analyzing PID 717472 (PE 0) ... OK Analyzing PID 717473 (PE 1) ... OK (no GPU threads) Analyzing PID 751554 (PE 0) ... OK Analyzing PID 751555 (PE 1) ... OK (no GPU threads) === Attach and Analyze Summary === Processes analyzed: 4 / 4 PE 0 PID 751554 groups=1 \033[1;31m1\033[0m wf(s) in rocSHMEM; \033[1;36mWaiting for remote PE memory update (barrier/sync or user rocshmem_wait_until). Check if the remote PE is alive and making progress. (at: #0 0x0000792efa5af3c8 in rocshmem::Context::wait_until<long> (this=0x792eed407000, ivars=0x792efac04008, cmp=0, val=1) at src/context_tmpl_device.hpp:224)\033[0m PE 1 PID 751555 groups=? (no GPU threads found) === BEGIN FILE: pe0_pid_717472.txt === [New LWP 717493] [New LWP 717490] [New LWP 717488] [New LWP 717485] [New LWP 717481] [New LWP 717480] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". 0x000077a09e06695f in ?? () from /opt/rocm-7.2.1/lib/libhsa-runtime64.so.1 === rocSHMEM Deadlock Analysis === Process(es): 717472 Total GPU wavefronts analyzed: 1 Unique backtrace groups: 1 --- Group 1 (1 wavefront(s)) --- WFs: (0,0,0)/0 Backtrace: #0 0x0000776005fafce0 in rocshmem::uncached_load<long volatile> (src=0x776006604008) at src/assembly.hpp:125 #1 rocshmem::Context::test<long> (this=0x775ff8e07000, ivars=0x776006604008, cmp=0, val=1) at src/context_tmpl_device.hpp:432 #2 rocshmem::Context::wait_until<long> (this=0x775ff8e07000, ivars=0x776006604008, cmp=0, val=1) at src/context_tmpl_device.hpp:224 #3 rocshmem::IPCContext::internal_direct_barrier (this=0x775ff8e07000, pe=0, PE_start=0, stride=1, n_pes=2, pSync=0x776006604000) at build/src/ipc/context_ipc_device_coll.cpp:44 #4 0x0000776005fb7c24 in rocshmem::IPCContext::internal_sync_wg (this=0x775ff8e07000, pe=0, PE_start=0, stride=1, PE_size=2, pSync=0x776006604000) at build/src/ipc/context_ipc_device_coll.cpp:115 #5 0x000077600603d018 in rocshmem::IPCContext::sync_all_wg (this=0x775ff8e07000) at build/src/ipc/context_ipc_device_coll.cpp:168 #6 0x000077600603d558 in rocshmem::IPCContext::barrier_all_wg (this=0x775ff8e07000) at build/src/ipc/context_ipc_device_coll.cpp:187 #7 0x000077600603dedc in rocshmem::Context::barrier_all_wg (this=0x775ff8e07000) at build/src/context_device.cpp:151 #8 0x000077600603e33c in rocshmem::rocshmem_barrier_all_wg () at build/src/rocshmem_gpu.cpp:885 #9 0x00007760060437cc in WorkGroupPrimitiveTest (loop=1, skip=10, start_time=0x775ff8e31000, end_time=0x775ff8e32000, source=0x776013c00480 "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqr"..., dest=0x776013d00500 "1", size=1, type=WGPutTestType, ctx_type=4) at /home/abouteil/rocm-systems/projects/rocshmem/tests/functional_tests/workgroup_primitives.cpp:72 [HINT] Waiting for remote PE memory update (barrier/sync or user rocshmem_wait_until). Check if the remote PE is alive and making progress. (at: #2 rocshmem::Context::wait_until<long> (this=0x775ff8e07000, ivars=0x776006604008, cmp=0, val=1) at src/context_tmpl_device.hpp:224) ... === END FILE: pe1_pid_751555.txt === Reconstructing per-PE output files from mpiexec output... Reconstructed output from 1 rank file(s). Running cross-rank coalescing pass... === Cross-Rank Deadlock Analysis === Directory: ./rocshmem_deadlock_analysis_20260530_013437 Ranks with analysis files: 2 (0, 1) Unique backtrace patterns: 2 --- Pattern 1 [UNIQUE: rank 0] --- Ranks: 0 Total wavefronts: 1 Backtrace: #0 0x0000776005fafce0 in rocshmem::uncached_load<long volatile> (src=0x776006604008) at src/assembly.hpp:125 #1 rocshmem::Context::test<long> (this=0x775ff8e07000, ivars=0x776006604008, cmp=0, val=1) at src/context_tmpl_device.hpp:432 #2 rocshmem::Context::wait_until<long> (this=0x775ff8e07000, ivars=0x776006604008, cmp=0, val=1) at src/context_tmpl_device.hpp:224 #3 rocshmem::IPCContext::internal_direct_barrier (this=0x775ff8e07000, pe=0, PE_start=0, stride=1, n_pes=2, pSync=0x776006604000) at build/src/ipc/context_ipc_device_coll.cpp:44 #4 0x0000776005fb7c24 in rocshmem::IPCContext::internal_sync_wg (this=0x775ff8e07000, pe=0, PE_start=0, stride=1, PE_size=2, pSync=0x776006604000) at build/src/ipc/context_ipc_device_coll.cpp:115 #5 0x000077600603d018 in rocshmem::IPCContext::sync_all_wg (this=0x775ff8e07000) at build/src/ipc/context_ipc_device_coll.cpp:168 #6 0x000077600603d558 in rocshmem::IPCContext::barrier_all_wg (this=0x775ff8e07000) at build/src/ipc/context_ipc_device_coll.cpp:187 #7 0x000077600603dedc in rocshmem::Context::barrier_all_wg (this=0x775ff8e07000) at build/src/context_device.cpp:151 #8 0x000077600603e33c in rocshmem::rocshmem_barrier_all_wg () at build/src/rocshmem_gpu.cpp:885 #9 0x00007760060437cc in WorkGroupPrimitiveTest (loop=1, skip=10, start_time=0x775ff8e31000, end_time=0x775ff8e32000, source=0x776013c00480 "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqr"..., dest=0x776013d00500 "1", size=1, type=WGPutTestType, ctx_type=4) at /home/abouteil/rocm-systems/projects/rocshmem/tests/functional_tests/workgroup_primitives.cpp:72 [HINT] Waiting for remote PE memory update (barrier/sync or user rocshmem_wait_until). Check if the remote PE is alive and making progress. (at: #2 rocshmem::Context::wait_until<long> (this=0x775ff8e07000, ivars=0x776006604008, cmp=0, val=1) at src/context_tmpl_device.hpp:224) === Per-Rank rocSHMEM Wavefront Count === Rank 0: 2 wf(s) inside rocSHMEM Rank 1: 0 wf(s) inside rocSHMEM === Summary === 0 pattern(s) common to all 2 rank(s) 1 pattern(s) present in exactly one rank Most common hint: Waiting for remote PE memory update (barrier/sync or user rocshmem_wait_until). Check if the remote PE is alive and making progress. (at: #2 rocshmem::Context::wait_until<long> (this=0x775ff8e07000, ivars=0x776006604008, cmp=0, val=1) at src/context_tmpl_device.hpp:224) ```

… progress tracker (#6616) ## Motivation When the program deadlocks it can be hard to know how much work was performed thus far, for example did it deadlock after only a couple of communication calls or after a long time, is there only 1 call of 1 type? etc. ## Technical Details builds with PROFILE=ON can produce call statistics (slowing down execution btw), this can be triggered from the gdb attachement to get the stat dump. ## JIRA ID   ## Test Plan modified a functional test to intentionally deadlock. build with Debug and PROFILE=ON. ``` ROCSHMEM_DEBUG_LEVEL=trace:stats ~/ompi/bin/mpirun -n 2 tests/functional_tests/rocshmem_functional_tests -n 1 -a 26 & (sleep 2; ../scripts/debug/attach_deadlock_analysis.sh rocshmem_functional_tests --stats ) ``` Output file ``` === rocSHMEM Deadlock Analysis === Process(es): 845512 Total GPU wavefronts analyzed: 1 Unique backtrace groups: 1 --- Group 1 (1 wavefront(s)) --- WFs: (0,0,0)/0 Backtrace: #0 0x00007ae55f5afcf0 in rocshmem::uncached_load<long volatile> (src=0x7ae55fc04008) at src/assembly.hpp:125 #1 rocshmem::Context::test<long> (this=0x7ae552407000, ivars=0x7ae55fc04008, cmp=0, val=1) at src/context_tmpl_device.hpp:432 #2 rocshmem::Context::wait_until<long> (this=0x7ae552407000, ivars=0x7ae55fc04008, cmp=0, val=1) at src/context_tmpl_device.hpp:224 #3 rocshmem::IPCContext::internal_direct_barrier (this=0x7ae552407000, pe=0, PE_start=0, stride=1, n_pes=2, pSync=0x7ae55fc04000) at build/src/ipc/context_ipc_device_coll.cpp:44 #4 0x00007ae55f5b7c24 in rocshmem::IPCContext::internal_sync_wg (this=0x7ae552407000, pe=0, PE_start=0, stride=1, PE_size=2, pSync=0x7ae55fc04000) at build/src/ipc/context_ipc_device_coll.cpp:115 #5 0x00007ae55f63d018 in rocshmem::IPCContext::sync_all_wg (this=0x7ae552407000) at build/src/ipc/context_ipc_device_coll.cpp:168 #6 0x00007ae55f63d558 in rocshmem::IPCContext::barrier_all_wg (this=0x7ae552407000) at build/src/ipc/context_ipc_device_coll.cpp:187 #7 0x00007ae55f63dedc in rocshmem::Context::barrier_all_wg (this=0x7ae552407000) at build/src/context_device.cpp:151 #8 0x00007ae55f63e33c in rocshmem::rocshmem_barrier_all_wg () at build/src/rocshmem_gpu.cpp:885 #9 0x00007ae55f6437cc in WorkGroupPrimitiveTest (loop=1, skip=10, start_time=0x7ae552431000, end_time=0x7ae552432000, source=0x7ae56d200480 "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqr"..., dest=0x7ae56d300500 "1", size=1, type=WGPutTestType, ctx_type=4) at /home/abouteil/rocm-systems/projects/rocshmem/tests/functional_tests/workgroup_primitives.cpp:72 [HINT] Waiting for remote PE memory update (barrier/sync or user rocshmem_wait_until). Check if the remote PE is alive and making progress. (at: #2 rocshmem::Context::wait_until<long> (this=0x7ae552407000, ivars=0x7ae55fc04008, cmp=0, val=1) at src/context_tmpl_device.hpp:224) === rocSHMEM Stats === PE 0 (counters reflect activity since process start) Device: quiet 1 put_wg 704 Host: (all zero) === Summary === 1 wavefront(s) inside rocSHMEM 0 wavefront(s) in user code / other ``` Note the == rocshmem Stats == section contains the number of put and quiet calls. ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests. --------- Signed-off-by: Aurelien Bouteiller <aurelien.bouteiller@amd.com> Signed-off-by: Aurelien Bouteiller <abouteil@amd.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

…8515) ## Motivation Enable roofline in any ctest runs in test_profile_roofline* on gfx115x systems. ## Technical Details `skip_unsupported_roofline_soc()` was blocking gfx115x from running becasue it was marked as unsupported soc until we could fully validate and enable roofline on these systems. We are confident in their passing so we are removing the gfx115x systems from this list. ## JIRA ID AIPROFCOMP-677 ## Test Plan ``` ctest --output-on-failure -R test_profile_general ctest --output-on-failure -R test_profile_roofline ``` ## Test Result On gfx1151 gorgon halo system: ``` Start 28: test_roofline_calc 1/3 Test #28: test_roofline_calc ............... Passed 2.71 sec Start 29: test_roofline_analyze 2/3 Test #29: test_roofline_analyze ............ Passed 28.19 sec Start 30: test_roofline_main 3/3 Test #30: test_roofline_main ............... Passed 2.67 sec Start 8: test_profile_roofline_1 1/2 Test #8: test_profile_roofline_1 .......... Passed 708.39 sec Start 9: test_profile_roofline_2 2/2 Test #9: test_profile_roofline_2 .......... Passed 226.78 sec ``` ## Submission Checklist - [X] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests. --------- Signed-off-by: Carrie Fallows <Carrie.Fallows@amd.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: vedithal-amd <Vignesh.Edithal@amd.com>

… real headers - Fix double-deref in KernelCh stop: combine per-channel and kernel-completion ref drops into a single atomic operation under the coll mutex, preventing negative refcount and use-after-free (#1) - Add mutex to ProxyOpInfo, protect step timing accumulation from concurrent proxy thread writes (#2) - Lock ProxyOp before memcpy into coll->proxyOps to prevent reading partially-written step data (#3) - Move all pools (coll, proxyOp, proxyStep) from process-global statics into per-communicator acclCommContext, eliminating cross-comm pool starvation (#4) - Fix fmt_size to show fractional values (1.5M instead of 1M) (#6) - Fix bottleneck classifier denominator to use launch + max(kernel, proxy) instead of exec + launch, preventing percentages exceeding 100% due to kernel/proxy overlap (#7) - Add --output flag to 'single' subcommand and clean up hasattr guard (#8) - Switch output file from append to write mode to avoid stale data from PID reuse (#9) - Replace vendored nccl/profiler.h with accl_shim.h that provides minimal types and includes RCCL's real profiler_v5.h; add CMakeLists.txt for in-tree builds (#10)

Defaulting --jobs to 4 broke the Jenkins precheckin Test stage. That pipeline invokes the runner as test_runner.py -c configs/ci-precheckin.json --skip-mpi-check \ --rerun-failed --stop-on-rerun-failure --verbose with no --jobs, so validate_arg_combinations rejected --rerun-failed against the new default and exited 1. Build #8 failed in 833 ms / 1405 ms / 2167 ms across the three arch legs -- an argument-validation exit, not a test failure -- where build #7 ran the identical command and passed in 428-1880 s. Reverting rather than relaxing the guard, because on the evidence the default bought nothing. The only current consumers of this runner are that precheckin job and .github/workflows/ainic_test_runner.yml; CoCo has migrated off it (see its own scripts/convert_test_runner_config.py, which reads the "legacy test_runner/configs/*.json format"). ainic.json is now entirely serial_only, and 62 of ci-precheckin's 66 tests are single-process gtests that enumerate all 8 GPUs, so each fills the co-tenancy budget and runs alone regardless -- only 4 of its tests could ever share the node. Concurrency stays available behind an explicit --jobs N for anyone running a full functional config directly, which is where the measured 13.7% (1 node) / 15.9% (2 nodes) actually came from. Keeps two things from the reverted commit that stand on their own: -j/--jobs and --max-parallel-gpus are now listed in the README options table, which they were not before. JIRA ID: AICOMRCCL-2053

…ntegration Add monotonic generation counter per pool slot to detect stale callbacks from reclaimed collectives, resolving the 0==0 race in pool reclaim and all callback-lifetime concerns (threads #4, #6, #8, #9). Add atomic ref-count to acclCommContext so acclPluginFinalize defers free() until all outstanding callbacks have drained (thread #2). Wire accl plugin into the parent RCCL CMake build (thread #20), add CXX_VISIBILITY_PRESET/VISIBILITY_INLINES_HIDDEN (thread #1), and bump ACCL_MAX_CHANNELS from 64 to 256 (thread #11). Fix acclShouldFinalize: nChannels==0 collectives (onerank path) finalize immediately on coll stop; guard channelId against nChannels (thread #10, #12). Free leaked ProxyOp when coll's proxyOpIndices is full (thread #13). Rename firstCpuStart to firstKernelCpuStartUs for clarity (thread #14). Fix warmup off-by-one (sn <= warmup → sn < warmup, thread #7), use try/finally for output file handle (thread #15), fix test fixtures to start at coll_sn=0 (thread #16), remove stale ACCL_PROFILER_WARMUP_ITERS docs (thread #17), sync bottleneck categories with classify() (thread #18), add NVIDIA Apache-2.0 header to accl_shim.h (thread #19). Add tests: ProxyStep lifecycle, channel bounds, kernel timing assertions (threads #3, #12, #21).

Add Test #3 -- when the network reports an in-progress put done, ncclRmaProxyPollNonPersistCompletion returns the credit (inflightRequests--), nulls the request, publishes opSeq into doneSeq, dequeues the descriptor, and destroys it. Adds the PushInProgressPutSignal fixture helper the completion-side tests (#4/#8/#9) build on. Co-authored-by: Pi Agent

Add Test #6 -- when credit is available for every op in a PutSignalGroup, ncclRmaProxyPollNonPersistDesc issues all of them in one pass, moves the group descriptor to the in-progress queue, and advances the consumer index once. Adds the PushPendingPutGroup fixture helper the group-path tests (#7/#8) build on. Co-authored-by: Pi Agent

…10809) ## Summary Addresses a 15-point static re-review of the KFD dispatch-log signal-less path (on top of `review-fixes`). Each finding was evaluated against the code and the prior design history; this PR **fixes the genuine correctness/safety/resource issues** (potential misattribution, UAF on terminal teardown, unbounded pairer-map growth, plus FATAL-handling, env-race, and robustness items) and **documents the ones intentionally left as accepted residuals or pre-existing**, with rationale. `+365 / −49` across 9 files. ## Fixed - **#4 (misattribution) — validate identity before claiming an unknown-START EOP.** The unknown-START path in `dispatch_hub.hpp::record_kernel_end` took the sole first-owner entry with no timestamp check, and terminal-time validation ran only *after* the entry was consumed/retired — so a stale EOP could claim a newer dispatch's entry. Now rejects an EOP whose `end_ticks <= window.t_open` (a kernel cannot end at/before its window opened; same GPU-tick domain the known-START path already uses) **before** taking the entry, so a rejected EOP consumes/retires nothing. Fail-first verified. - **#3 + #2 (UAF) — the terminal fences are now truly terminal (fail-closed).** `signal_less_fence_completions()` previously quiesced visible work but did not stop the reader on success, so a still-PENDING kernel could complete after the client's callback/buffer state (detach/finalize, F23) or the HSA runtime (final `hsa_shut_down`, F24) was freed. A new `signal_less_terminal_stop()` (factored from teardown) latches disable → stops+joins the reader/processor → drains+ledgers PENDING, and is called from the F23/F24 sites. Verified deadlock-free (never joins its own task group, holds no lock across the join, joined threads never block on the joiner) and 5/5 clean under a shutdown watchdog. The existing `get_fini_status()` re-entry guard (the earlier crash fix) is preserved. - **#11 (unbounded growth) — hard size cap on the pairer START map.** `pair_state::pending_starts` was bounded only by a 30-minute age timer. Added `ROCPROFILER_KFD_DISPATCH_LOG_MAX_PENDING_STARTS` (default 2M, shared reject-and-default parser) with oldest-first eviction (recurring key exempted so the `ambiguous` latch is preserved). - **#7 (coverage) + #10 (liveness) — FATAL handling.** FATAL now latches admission and waits for the processor to consume the already-published pipe **before** draining the hub (so a fatal session's final EOP and healthy-GPU EOPs behind it aren't dropped), and the FATAL status is re-probed while the empty-wake backstop is tripped (so a fatal stream that produces no records is still detected). - **#5 (robustness) — master switch is a strict fail-closed opt-in.** `ROCPROFILER_KFD_DISPATCH_LOG_SIGNAL_LESS` now accepts only explicit true values; anything else is disabled with a one-time warning (no throw on malformed input). - **#9 (arg-order) — deterministic GC-gate operand ordering.** `gc_gate_open`'s two operands are read into named locals in the intended order rather than relying on unspecified argument-evaluation order. - **#12 (race) — env knobs primed at serialized init.** The age/cap/quiesce/switch env reads are populated during reader start-up rather than lazily on a worker thread. - **#14(b) — fork child no longer hangs in `interposition_fini()`.** A normal (non-exec) child now short-circuits `interposition_fini()` under the existing `fork_stale()` guard, avoiding a wlock on a registry a vanished parent thread may hold. ## Intentionally not changed (accepted residuals / design decisions / pre-existing) - **#1 — teardown coverage loss** (a kernel in flight at exit may emit no record): the accepted D8/F1 coverage residual. The *UAF* variant of this concern is #3, which is fixed. - **#8 — equal-tick START/EOP**: the accepted D1 residual. The code emits a **no-timing / zero-duration drop**, not a fabricated 1 ns; no misattribution. - **#9-main — GC gate "race"**: the copied-but-still-in-ring window is the accepted **R10** residual (an EOP still in the ring is intentionally not gated), and cap-eviction's bypass of the in-flight gate is D9's documented coverage-loss residual. The operand-ordering nit is fixed (#9-arg above); the residual is by design (coverage loss, never misattribution). - **#13 — cross-batch inversion**: the explicitly-accepted D1 residual (coverage loss under queue churn, never misattribution). A cross-batch reorder buffer is a larger design change weighed against measured coverage impact, which the benchmarks do not show hitting. - **#6 — completion-callback self-deadlock**: a **pre-existing** reentrancy defect on the generic signal path (a client callback that calls its own finalize). The new terminal-stop path was specifically designed **not** to add another instance of it. - **#14(a) — session mid-construction skipped by a concurrent fork**: a child-lifetime resource leak (not a hang); the parent is unaffected and the child drops it at exit. The "fix" would introduce a formal data race on an `smap` pointer a vanished thread was mid-write on, to reclaim memory about to be freed — not worth it. Left as-is. - **#15 — eager arming on a registered (vs active) context**: an efficiency consideration, not a correctness issue; arming stays gated on a registered kernel-dispatch context. ## Known residuals in the fixes (documented) - #4 adds no upper time bound (an implausibly-future EOP is still taken and rejected by the finalizer as no-timing, never misattributed). - #10 restores FATAL *detection* under the backstop; a *persistently failing* status ioctl still isn't fail-closed (separate follow-up). - #11 eviction is O(live) per over-cap insert (same tradeoff as the D9 cap; only reached under pathological EOP loss). ## Testing (MI350, host ROCm 7.2.1, build `fd06e1407c`) - Build clean at the project warning level — **0 new warnings**. - `ctest -L kfd-dlog` — **90/90 pass** (incl. 2 new tests: pairer-map cap, strict opt-in). - ThreadSanitizer, all 8 kfd-dlog suites — **90/90, zero reports** (covers the terminal-stop join, FATAL drain-wait, GC-gate ordering). - Shutdown crash/hang reproducer (1040), **5 runs under a watchdog — 5/5 clean, no hang, no crash** (the decisive check for the terminal-stop rework). - 1041-d (20k kernels ×3), 1031, 1041-a, 1041-c — all pass, no attribution regression. - graph-perf — **−44.21%** overhead reduction, `register-refused=0`/block (no throughput regression). ## Tracking JIRA ID: AIPROFSDK-1012 AIPROFSDK-1040 AIPROFSDK-1041

Sivasuntharampillai, Haresh added 16 commits July 24, 2025 15:41

Updated Trigger Conditions for Code_Coverage

d441c00

Updated Runner information for all Workflos

04df37a

Refactored RocProfiler-SDK CodeQL

d2ee399

Added Sparse Checkout

519f956

Grab submodules from rocprofiler-sdk

d2932cb

Update Submodules

a413ba9

Moved gitmodules to root

3032583

Update the submodule path of fmt

02ceb6d

Remove fetch for submodules

866cfc4

Added fmt to gitmodules

86c7d01

Added glog to submodules

c75cefc

Added remaining submodules for rocprofiler-sdk

e819aab

Added Filesystem to Submodules

31a8f82

Added remaining submodules

62553a0

Update Requirements File

3efdf53

Updated docs.yml

f8d6fd0

Sivasuntharampillai, Haresh added 6 commits July 25, 2025 12:25

Added Workflows to Sparse Checkout

b285076

Fixed Sparse Checkout

a7114d4

Removed Workflows from checkout

a04cdeb

Checkout specific actions files

c013787

Refactored docs.yml

7856bc7

Update rocprofiler-sdk-docs.yml

c1809fe

ammarwa reviewed Jul 25, 2025

View reviewed changes

Sivasuntharampillai, Haresh and others added 5 commits July 28, 2025 13:58

Removed Unnecessary Workflows

91fefe2

Added RocProfiler-SDK Restrictions

55ef356

Added rocprofiler-sdk Code Coverage

4cff19c

Added rocprofiler-sdk CI Workflow

b2a0a97

Update rocprofiler-sdk-continuous_integration.yml

86ab671

vlaindic mentioned this pull request May 18, 2026

PC sampling API refactoring #4548

Closed

1 task

bwelton mentioned this pull request Aug 27, 2026

fix(rocprofiler-sdk): KFD dispatch-log signal-less re-review fixes #10809

Merged

SakaSitharammurthy mentioned this pull request Oct 1, 2026

feat(amdsmi): add CPU apb_status, xgmi_link_width, df_pstate_range metrics #7521

Open

Navigation Menu

Repository navigation

[DO NOT MERGE] Modify Project Workflows - #8

[DO NOT MERGE] Modify Project Workflows#8
amd-hsivasun merged 56 commits into
developfrom
amd/hsivasun/modify-workflows

amd-hsivasun commented Jul 24, 2025

Uh oh!

github-advanced-security AI commented Jul 25, 2025

Uh oh!

ammarwa Jul 25, 2025

Uh oh!

ammarwa Jul 25, 2025

Uh oh!

ammarwa Jul 25, 2025

Uh oh!

ammarwa Jul 25, 2025

Uh oh!

ammarwa Jul 25, 2025

Uh oh!

ammarwa Jul 25, 2025

Uh oh!

ammarwa commented Jul 25, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

amd-hsivasun commented Jul 24, 2025

Uh oh!

github-advanced-security AI commented Jul 25, 2025

Uh oh!

ammarwa Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

ammarwa Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

ammarwa Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

ammarwa Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

ammarwa Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

ammarwa Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

ammarwa commented Jul 25, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants