Repository navigation
[hardware, profile] fix: improve existing pytorch profiling to support other pytorch supported device - #8151
Merged
wuxibin89 merged 4 commits intoOct 9, 2026
Conversation
…ctive platform ### What does this PR do? `get_torch_profiler` knew exactly one device activity, `ProfilerActivity.CUDA`, and `TorchProfilerToolConfig` accepted exactly one device keyword, `"cuda"`. On an accelerator that is not CUDA there was then no way to record device kernels: the default `contents=[]` asks for CUDA, torch logs "CUDA is not available, disabling CUDA profiling" and the trace holds CPU ops only, while naming the real device in `contents` fails config validation. torch names each `ProfilerActivity` member after the device type it profiles, and a single profiler session records exactly one of them. So the active platform's `device_name` is both the activity to collect and the `contents` keyword that requests it. Resolve it that way instead of hardcoding CUDA, and take the accepted device keywords from `torch.profiler.ProfilerActivity` rather than from a verl-maintained list. No new `PlatformBase` API: `device_name` is already an abstract property. ### No change for existing platforms `PlatformCUDA.device_name` is `"cuda"`, and `PlatformROCm`, `PlatformMetaX` and `PlatformIluvatar` inherit it, so all four keep building the same `activities` list. Platforms with no matching member (`npu`, `mlu`, `musa`, `gcu`, `tpu`, `supa`) still get a CPU-only trace, exactly as they do today when torch disables the unavailable CUDA activity -- the warning explaining it is now verl's, and it names the device. ### Tests `test_get_torch_profiler` and `test_cpu_activity_added_for_device_only_contents` assert the CUDA activity, which until now was hardcoded and so held on any machine. They now patch in a CUDA platform, stating the assumption they always had -- without it they fail when the suite is run on non-CUDA hardware. Verified by running the file with `VERL_PLATFORM` forced to a non-CUDA platform.
`getattr(torch.profiler.ProfilerActivity, device.upper(), None)` depends on torch capitalizing every member as the uppercased device type, which `PrivateUse1` already breaks, and `DEVICE_CONTENTS` lowercased every member name -- so `contents: ["privateuse1"]` passed config validation and was then never recorded, since no platform reports "privateuse1" as its `device_name`. Resolve the activity through an explicit lowercased device-type -> activity map instead, and build the accepted `contents` keywords from it. `CPU` and `PrivateUse1` are left out of the map: neither is a device type a platform reports, CPU activity is collected in every trace regardless of `contents`, and "cpu" stays an accepted keyword. An out-of-tree backend whose torch activity is not named after its device type can add its own entry to the map, rather than verl carrying a hook for it. Tests: every entry in the map resolves to its own activity (so the lookup holds for whatever members a torch build exposes), and "privateuse1" is rejected by config validation.
kahlun
marked this pull request as ready for review
October 8, 2026 12:45
wuxibin89
approved these changes
Oct 9, 2026
7 of 8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
get_torch_profilerknew exactly one device activity,ProfilerActivity.CUDA, andTorchProfilerToolConfigaccepted exactly one device keyword,"cuda". On an accelerator that isnot CUDA there was then no way to record device kernels: the default
contents=[]asks for CUDA,torch logs
CUDA is not available, disabling CUDA profilingand the trace holds CPU ops only, whilenaming the real device in
contentsfails config validation.torch exposes one
ProfilerActivitymember per backend it can profile, named after the device typeit profiles, and a single profiler session records exactly one of them. So the active platform's
device_nameis both the activity to collect and thecontentskeyword that requests it. This PRresolves it that way instead of hardcoding CUDA, and takes the accepted device keywords from
torch.profiler.ProfilerActivityrather than from a verl-maintained list.No new
PlatformBaseAPI —device_nameis already an abstract property.No change for existing platforms
PlatformCUDA.device_nameis"cuda", andPlatformROCm,PlatformMetaXandPlatformIluvatarinherit it (torch exposes HIP as
torch.cuda), so all four build exactly the sameactivitieslistas before. Platforms with no matching member (
npu,mlu,musa,gcu,tpu,supa) still get aCPU-only trace, exactly as they do today when torch disables the unavailable CUDA activity — the
warning explaining it is now verl's, and it names the device.
xpu,hpuandmtiastart working with no further verl change and no plugin code.Out-of-tree backends whose activity is not named after the device type
DEVICE_ACTIVITIESis a module-leveldevice type -> ProfilerActivitymap, so a backend that needsa different pairing can add one entry from its plugin:
This is deliberately data rather than API: it keeps the mapping out of
PlatformBasewhile leavingprivateuse1-based backends a one-line override. In practice those backends all select a vendor
profiler through
global_profiler.toolinstead, which this PR does not touch.CPUandPrivateUse1are excluded from the map and"privateuse1"is not an acceptedcontentskeyword: CPU activity is collected in every trace regardless of
contents, andPrivateUse1istorch's slot for an out-of-tree backend rather than a device type any platform reports as its
device_name, so accepting it would validate a value that is then silently never recorded."cpu"remains accepted.Why this is not a duplicate
PlatformBasehooks,torch_profiler_activity()andtorch_profiler_content_name(). This PR drops both:device_namealready carries the information, which removes the paired-hook invariant between them, removes the
requirement for every plugin to implement them, and leaves
platform_base.pyuntouched.gh pr list --repo verl-project/verl --state open --search "profiler activity"— nothing else openin this area.
Test
New tests:
TestDeviceActivityFromPlatform— a non-CUDA device is recorded and CUDA is not added alongsideit (torch checks CUDA before every other device, so leaving it in would disable device profiling
altogether); CUDA platforms are unchanged; every entry in
DEVICE_ACTIVITIESresolves to its ownactivity, so the lookup holds for whatever members a given torch build exposes; a device with no
member warns; a CUDA-written
contentsrun on another accelerator warns and records no deviceactivity;
contentsnaming only profiler options warns nothing; a CPU platform does not add CPUtwice.
The non-CUDA cases take their device from whatever
torch.profiler.ProfilerActivitythe runningbuild exposes rather than hardcoding one accelerator, and skip if a build offers none.
TestDeviceContentsConfigValidation— every keyword inDEVICE_CONTENTSvalidates, profileroptions still validate, an unknown keyword and
"privateuse1"are both rejected.Two pre-existing tests (
test_get_torch_profiler,test_cpu_activity_added_for_device_only_contents)assert the CUDA activity, which until now was hardcoded and so held on any machine. They now patch in
a CUDA platform, stating the assumption they always had — without it they fail when the suite runs on
non-CUDA hardware.
No new e2e CI test: CI has no non-CUDA device. Manually verified end to end on Intel Arc Pro B60
(device kernels present in the chrome trace, which was not the case before this change).
API and Usage Example
No API change. On a non-CUDA accelerator, the default config now records device kernels:
Design & Code Changes
verl/utils/profiler/config.py—DEVICE_ACTIVITIESmap built fromtorch.profiler.ProfilerActivity.__members__(keyed by lowercased device type, so the lookup doesnot depend on how torch capitalizes a member);
DEVICE_CONTENTSderived from it for configvalidation, replacing the hardcoded
["cuda", "cpu", "memory", "shapes", "stack"].verl/utils/profiler/torch_profile.py—get_torch_profilerresolves the active platform'sdevice_namethrough that map; warns when the platform has no activity, and whencontentsnamesa device activity belonging to a different backend.
docs/perf/torch_profiling.md— documents the device keyword per accelerator.tests/utils/test_torch_profile.py— as above.Checklist
[{modules}] {type}: {description}.pre-commit/ ruff clean on changed files.docs/perf/torch_profiling.md).ci-requestping if CI needs a nudge.