[abchoudh-amd]

Motivation

vLLM V1 can execute GPU kernels outside the profiled entry process, leaving ROCm Compute Profiler results without dispatch or counter data. Document the verified single-process workaround and its performance caveat.

Technical Details

• Add vLLM profiling guidance and example
• Document V1 multiprocessing environment workaround
• Add FAQ entry for empty counter data

JIRA ID

JIRA ID: AIPROFCOMP-182

Test Plan

• Run workload: python -m sphinx -q -b html docs docs/_build/html

Test Result

• workload: python -m sphinx -q -b html docs docs/_build/html passes

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[Copilot]

Pull request overview

Documents a workaround for profiling vLLM V1 workloads with ROCm Compute Profiler when kernel dispatch/counter data is missing due to multiprocessing, and adds an FAQ entry to help users diagnose empty counter output.

Changes:

• Add a new “Profile vLLM workloads” section with an example command and a performance caveat.
• Add an FAQ entry explaining why vLLM profiling can produce empty performance counter data and linking to the workaround.
• Extend FAQ metadata keywords to include vLLM for discoverability.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
projects/rocprofiler-compute/docs/reference/faq.rst	Adds an FAQ entry for empty vLLM counter data and links to the documented workaround.
projects/rocprofiler-compute/docs/how-to/profile/mode.rst	Adds a new how-to section describing the vLLM profiling workaround and warning about performance impacts.

Suppressed comments (1)

projects/rocprofiler-compute/docs/how-to/profile/mode.rst:1200

• Similarly, the workaround instruction reads as if it applies to all vLLM workloads; since it toggles a V1-specific multiprocessing behavior, it would be clearer to say "vLLM V1 workloads" here too.

To profile vLLM workloads, set ``VLLM_ENABLE_V1_MULTIPROCESSING=0`` before
launching ROCm Compute Profiler. The vLLM workload then inherits the setting
and runs the model in the profiled process. For example:

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[therock-pr-bot]

✅ All Policy Checks Passed

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	PR does not contain code files — Unit Test auto-passed
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

[vedithal-amd]

/docs-preview

[vedithal-amd]

Ran /docs-preview to generate the documentation build

[systems-assistant]

Docs preview for rocprofiler-compute is building on Read the Docs: https://rocm.docs.amd.com/projects/rocprofiler-compute/en/users-abchoudh-vllm_bug/ (Note: If a pull request includes multiple projects, only the preview for the first project will be built.)

[systems-assistant]

Docs preview for rocprofiler-compute is building on Read the Docs: https://rocm.docs.amd.com/projects/rocprofiler-compute/en/users-abchoudh-vllm_bug/ (Note: If a pull request includes multiple projects, only the preview for the first project will be built.)

[vedithal-amd]

VLLM_ENABLE_V1_MULTIPROCESSING=0 should not be required in latest rocm build - lets test vllm with latest rocm build and see if that works.

This flag was required because rocprofiler-sdk hangs during finalization in the multiprocess case but that is a bug which was fixed

[vedithal-amd]

The workaround is correct, but the reason given for it is not, and users will draw the wrong conclusion. The profiler does profile the kernels in the vLLM worker process and records its dispatches. The results come out empty because vLLM kills that worker on shutdown, and the profiler only writes counter data during a normal exit.

Longer term we plan to stop relying on the LD_PRELOADed SDK tool and move the features we need into our own collector, at which point the env var stops being necessary. Details inline.

[vedithal-amd]

Comments addressed, thanks. I pushed one small edit: the FAQ answer repeated the how-to text almost verbatim, so it now states the cause in a line and links profile-vllm-workloads for the workaround and its limits.

[vedithal-amd]

Re-targeting to develop so that this documentation only clarification PR lands into ROCm 10.1 release

[systems-assistant]

TheRock Submodule Bump Activity

Newest first

• 2026-09-08 — Included in TheRock via ROCm/TheRock#7974.