Skip to content

fix(rocprofiler-sdk): reject --kernel-replay-beta-enabled without --replay-mode kernel - #11432

Open
amd-vkale wants to merge 4 commits into
developfrom
users/vkale/kernel-replay-beta-flag-with-cursor-fixes-help
Open

amd-vkale wants to merge 4 commits into
developfrom
users/vkale/kernel-replay-beta-flag-with-cursor-fixes-help

Conversation

@amd-vkale

@amd-vkale amd-vkale commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

rocprofv3 already fails when --replay-mode kernel arrives without
--kernel-replay-beta-enabled. The reverse was accepted silently.

That asymmetry matters because of the flag's name. --kernel-replay-beta-enabled reads as though
it enables kernel replay, and that is the natural thing to assume from it; in fact it only
acknowledges that the feature is beta, and --replay-mode kernel is what selects it. A command
line carrying only the acknowledgement ran application replay — the whole application re-run
once per counter group — while the user believed they were running kernel replay.

Nothing makes that mistake self-correcting. The run succeeds, no diagnostic is printed, no output
field records which replay strategy was used, and the counter values application replay produces
are indistinguishable from replayed ones. The user gets plausible numbers collected by a strategy
they did not ask for, and any conclusion they draw about replay overhead, or about the
determinism the two strategies differ on, is drawn from the wrong run.

This is easy to hit in practice: the acknowledgement is the memorable half of the pair, and it is
the half that mentions kernel replay by name.

Technical Details

Both existing validation sites gain the symmetric check.

run() — the check is the else of the existing if kernel_replay_mode: block. args there
is the fully merged per-run view, so it sees the acknowledgement and the mode together regardless
of which source supplied either one. It fires well before os.execvpe, so nothing is launched.

main() — the check is guarded on nothing anywhere having selected kernel replay:

if (
    getattr(cmd_args, "kernel_replay_beta_enabled", None)
    and not replay_enabled
    and not any(getattr(inp, "replay_mode", None) == "kernel" for inp in inp_args)
):

The guard is load-bearing. The acknowledgement can only be given on the command line, but an input
file job may be the thing asking for replay, so --kernel-replay-beta-enabled alongside a job
with "replay_mode": "kernel" has to stay valid. Without the inp_args term this check would
reject a legitimate and already-supported combination.

The diagnostic names the flag the user actually needs, and both ways out rather than only one:

--kernel-replay-beta-enabled acknowledges that kernel replay is a beta feature but does not
select it. Add --replay-mode kernel to use kernel replay, or drop the acknowledgement to use
application replay

The flag's --help text now states that it does not select kernel replay on its own and that
passing it alone is rejected rather than ignored, which is the root cause rather than the symptom.
The four documentation pages that describe the flag are updated to match:
quick-reference/rocprofv3-cli-options.rst, how-to/using-kernel-replay-rocprofv3.rst,
how-to/using-rocprofv3.rst, and conceptual/kernel_replay/kernel_replay_callback_api.md — each
of which previously stated the requirement in one direction only.

This is a behaviour change for a command line that used to be accepted, so it is recorded under
Resolved issues in the changelog. It cannot break a working kernel replay invocation, since any
such invocation already passes --replay-mode kernel; it can only fail a command line that was
not doing what its author intended.

Issue Tracking

No GitHub issue or JIRA ID — found while validating the kernel replay CLI surface from
#7960. Happy to file one and link it if that is
preferred for release tracking.

Test Plan

Three tests added to tests/rocprofv3/kernel-replay/test_kernel_replay_cli.py, which imports
rocprofv3.py and replaces run so no application is executed and no environment is touched. It
needs neither a GPU nor a built SDK.

  1. test_beta_acknowledgement_without_replay_mode_is_rejected — the case this PR is about:
    --pmc SQ_WAVES --kernel-replay-beta-enabled must exit nonzero.
  2. test_beta_acknowledgement_is_accepted_for_an_input_file_job_that_asks_for_replay — the
    combination the main() guard exists to protect: the acknowledgement on the command line with
    "replay_mode": "kernel" in an input file job must still run.
  3. test_application_replay_is_unaffected_without_the_acknowledgement — a plain --pmc run,
    confirming the rejection is about the unaccompanied flag rather than about --pmc.

The second and third exist because a validation check is only as good as the things it declines to
reject; they pin the two neighbouring command lines that must keep working.

Test Result

All 30 tests in the file pass, comprising the 3 new ones and the 27 that were already there —
including test_replay_without_beta_acknowledgement_is_rejected, which covers the original
direction and confirms the two checks do not interfere:

test_kernel_replay_cli.py::test_replay_without_beta_acknowledgement_is_rejected PASSED
test_kernel_replay_cli.py::test_replay_without_counter_collection_is_rejected PASSED
test_kernel_replay_cli.py::test_beta_acknowledgement_without_replay_mode_is_rejected PASSED
test_kernel_replay_cli.py::test_beta_acknowledgement_is_accepted_for_an_input_file_job_that_asks_for_replay PASSED
test_kernel_replay_cli.py::test_application_replay_is_unaffected_without_the_acknowledgement PASSED
30 passed in 0.05s

The new rejection emits the intended diagnostic in the captured output rather than a bare exit
code, so the failure mode a user sees is what is being tested.

Submission Checklist

@github-actions github-actions Bot added documentation Improvements or additions to documentation project: rocprofiler-sdk labels Sep 10, 2026
@therock-pr-bot

Copy link
Copy Markdown

❌ PR Check — Action Required

Check Status Details
📝 PR Description ❌ Fail Error: PR description must reference a JIRA ID, ISSUE ID, or a GitHub closing keyword.
Expected: include a JIRA ID / ISSUE ID line (separator : or -, or omitted; value may be a JIRA key, a number with/without #, or a link), OR a closing keyword + issue reference. Accepted examples:
• JIRA ID : TESTAUTO-6039
• JIRA ID - #330
• JIRA ID #330
• JIRA ID (on separate line)
ROCM-25757
• ISSUE ID : TESTUTO-3334
• ISSUE ID #3334
• ISSUE ID - TESTAUTO-3433
• ISSUE ID (on separate line)
AIRUNTIME-2352
• ISSUE ID : https://github.com/<org_name>/<repo_name>/issues/1234
• Closes #10
• Fixes octo-org/octo-repo#100
• Resolves: #123
• #123
• https://github.com/<org_name>/<repo_name>/issues/123
Current: no valid JIRA/ISSUE/closing-keyword reference found
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass —
🔎 pre-commit ⏳ Pending ⏳ Still running…
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —

⚠️ 1 policy check(s) failed. Please address the issues above before this PR can be Reviewed.

🚫 Please fix the failed policies

  • ❌ PR Description

The Not ready to Review label was added to this PR. Once all policies pass, the label is removed automatically.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

  • ❌ PR Description

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

@amd-vkale
amd-vkale marked this pull request as ready for review September 11, 2026 02:57
@amd-vkale
amd-vkale requested review from a team as code owners September 11, 2026 02:57
Copilot AI lite review requested due to automatic review settings September 11, 2026 02:57
@amd-vkale amd-vkale changed the title fix(rocprofiler-sdk): reject the kernel replay beta acknowledgement o… fix(rocprofiler-sdk): reject --kernel-replay-beta-enabled without --replay-mode kernel Sep 11, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved mixed-input validation and insufficient direct coverage for the run-level rejection remain.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates rocprofv3 to reject --kernel-replay-beta-enabled unless kernel replay is selected, while preserving supported input-file behavior.

Changes:

  • Adds symmetric validation and diagnostics.
  • Adds CLI regression tests.
  • Updates documentation and changelog entries.
File summaries
File Summary
projects/rocprofiler-sdk/tests/rocprofv3/kernel-replay/test_kernel_replay_cli.py Adds kernel-replay CLI tests.
projects/rocprofiler-sdk/source/docs/quick-reference/rocprofv3-cli-options.rst Updates flag documentation.
projects/rocprofiler-sdk/source/docs/how-to/using-rocprofv3.rst Clarifies acknowledgement behavior.
projects/rocprofiler-sdk/source/docs/how-to/using-kernel-replay-rocprofv3.rst Documents kernel-replay requirements.
projects/rocprofiler-sdk/source/docs/conceptual/kernel_replay/kernel_replay_callback_api.md Updates kernel-replay guidance.
projects/rocprofiler-sdk/source/bin/rocprofv3.py Adds validation and updated help text.
projects/rocprofiler-sdk/CHANGELOG.md Records the behavior change.
Review details

Suppressed comments (2)

projects/rocprofiler-sdk/source/docs/how-to/using-kernel-replay-rocprofv3.rst:74

  • This wording is too absolute for the supported input-file path covered by the new test: with the acknowledgement on the command line and "replay_mode": "kernel" in a job, no --replay-mode kernel appears on the command line but the invocation is accepted. Qualify the rejection condition so the documentation matches the inp_args exception in main().
``--kernel-replay-beta-enabled`` only acknowledges the beta; it does not select kernel replay.
Passing it without ``--replay-mode kernel`` is rejected rather than ignored, so a command line that
asks only for the acknowledgement cannot quietly fall back to application replay.

projects/rocprofiler-sdk/source/docs/how-to/using-rocprofv3.rst:1181

  • The new note omits the input-file exception implemented and tested here: a command-line acknowledgement is valid when an input-file job supplies "replay_mode": "kernel", even though the command line lacks --replay-mode kernel. Please qualify this sentence so users of input files are not told that their supported configuration is rejected.
   - ``--replay-mode kernel`` requires ``--pmc`` and ``--kernel-replay-beta-enabled``. The reverse also holds: ``--kernel-replay-beta-enabled`` acknowledges the beta but does not select kernel replay, so passing it without ``--replay-mode kernel`` is rejected rather than ignored.
  • Files reviewed: 6/7 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +250 to +253
try:
launched_runs("--pmc", "SQ_WAVES", "--kernel-replay-beta-enabled")
except SystemExit as exc:
assert exc.code != 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in eb63cdc1: added a test that calls the real run() path and asserts the complete stderr diagnostic.

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

This pull request has been inactive for 25 days and will be marked as stale.

If you would like to keep this PR open, please:

  • Add new commits
  • Add a comment explaining why it should remain open

This PR will be automatically closed in 5 days if no further activity occurs.

@github-actions github-actions Bot added the Stale PR has no activity for 25+ days label Oct 7, 2026
@amd-vkale
amd-vkale requested a lite review from Copilot October 7, 2026 05:53
@amd-vkale

Copy link
Copy Markdown
Contributor Author

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply the suggestion exactly.

Do not make changes beyond what is described in the linked review thread.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Validation can permit partial execution or bypass acknowledgement requirements, and tests do not cover the run-level check or diagnostic text.

Review effort: Lite
Findings: 1 Medium severity · 2 Low severity

Open (3)

Comment on lines +72 to +74
``--kernel-replay-beta-enabled`` only acknowledges the beta; it does not select kernel replay.
Passing it without ``--replay-mode kernel`` is rejected rather than ignored, so a command line that
asks only for the acknowledgement cannot quietly fall back to application replay.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in eb63cdc1: the guide now documents that an input-file job selecting replay_mode: kernel is a valid alternative selector.

.. note::

- ``--replay-mode kernel`` requires ``--pmc`` and ``--kernel-replay-beta-enabled``.
- ``--replay-mode kernel`` requires ``--pmc`` and ``--kernel-replay-beta-enabled``. The reverse also holds: ``--kernel-replay-beta-enabled`` acknowledges the beta but does not select kernel replay, so passing it without ``--replay-mode kernel`` is rejected rather than ignored.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in eb63cdc1: the note now qualifies rejection to the case where no input-file job selects kernel replay.

Copilot AI commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply ...

Addressed in eb63cdc1: per-job preflight prevents mixed-input partial launches, the real run() test asserts the full diagnostic, and both cited docs now describe the input-file selector exception.

@github-actions github-actions Bot removed the Stale PR has no activity for 25+ days label Oct 8, 2026
cursor Bot pushed a commit that referenced this pull request Oct 16, 2026
… integration

Co-authored-by: Vivek Kale <vivek.kale@amd.com>
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

This pull request has been inactive for 25 days and will be marked as stale.

If you would like to keep this PR open, please:

  • Add new commits
  • Add a comment explaining why it should remain open

This PR will be automatically closed in 5 days if no further activity occurs.

@github-actions github-actions Bot added the Stale PR has no activity for 25+ days label Nov 3, 2026
cursoragent and others added 3 commits November 5, 2026 08:21
…n its own

rocprofv3 already fails when --replay-mode kernel arrives without
--kernel-replay-beta-enabled, but not the other way round. The flag's name reads
as if it enabled replay, so a command line carrying only the acknowledgement ran
application replay while the user believed they were running kernel replay, and
the counter values application replay produces are indistinguishable from
replayed ones.

Fail with a diagnostic that names --replay-mode kernel instead. An input file
job may be the thing asking for replay, and the acknowledgement can only be
given on the command line, so the check only fires when nothing anywhere
selected kernel replay.

Co-authored-by: Vivek Kale <vivek.kale@amd.com>
Co-authored-by: amd-vkale <291155109+amd-vkale@users.noreply.github.com>
Co-authored-by: amd-vkale <291155109+amd-vkale@users.noreply.github.com>
@cursor
cursor Bot force-pushed the users/vkale/kernel-replay-beta-flag-with-cursor-fixes-help branch from ab93c89 to 8813cae Compare November 5, 2026 08:21
@github-actions github-actions Bot removed the Stale PR has no activity for 25+ days label Nov 6, 2026
cursor Bot pushed a commit that referenced this pull request Nov 7, 2026
…to the kernel replay integration

Co-authored-by: Vivek Kale <vivek.kale@amd.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants