[kahlun]

What does this PR do?

Two independent changes, both making verl's device handling less CUDA-specific.

1. A profiler_markers() hook on PlatformBase. Lets a hardware backend supply its own tracing markers -- (mark_start_range, mark_end_range, mark_annotate, marked_timer) -- instead of nvtx or mstx. Default is None, so existing platforms are unaffected.

This also fixes marker resolution. verl/utils/profiler/__init__.py checked is_nvtx_available() before consulting the platform, and nvtx ships in the verl-core extra, so on any install where nvtx imports the platform was never asked. Order is now platform, nvtx, mstx, generic, and stays lazy so the platform need not exist at import time.

Relatedly, engine_workers.py converts tool_config for any configured profiler.tool rather than an allowlist of the five built-in names. omega_conf_to_dataclass returns None for a missing or empty config, so a tool with no tool_config entry still works, and an out-of-tree tool can name its own dataclass through _target_.

2. The attention padding helpers become device-neutral. npu_flash_attn_utils.py was already a pure torch/einops port of flash-attn's bert_padding, with nothing NPU-specific in it, so it is renamed attention_padding_utils.py and used as the fallback for any install where flash_attn will not import -- not just NPU. That covers non-CUDA accelerators and CPU-only installs such as unit tests and dev boxes. The duplicate _fallback_* helpers in attention_utils.py are deleted, and torch_functional.py's two remaining hardcoded flash_attn / get_device_name() == "cuda"/"npu" branches now go through the same dispatcher.

No public API changes.

Consumed by verl-hardware-plugin's PlatformXPU (#22).

Why this is not duplicating an existing PR

Searched open and closed PRs for profiler_markers, PlatformBase hook, marked_timer, mark_start_range, attention_utils, npu_flash_attn_utils, index_first_axis.

For the profiler hook, nothing else lets a platform override the markers. #7988 (also mine) adds a different PlatformBase hook -- an optional torch.profiler activity list, i.e. what the built-in torch profiler records, not which marker functions the trainer calls. It overlaps only in touching platform_base.py; I will rebase whichever lands second. #6086 (merged) created the platform abstraction both extend.

For the attention change, #7098 overlaps substantially and predates this work. It reaches the same conclusion from the same reasoning -- that the NPU copy only uses torch and einops, so dispatching on hardware was unnecessary -- and both PRs delete the dispatch and touch tests/utils/test_attention_utils.py, so they conflict. The differences are small: #7098 inlines the helpers into attention_utils.py and deletes the module, where this PR keeps it as attention_padding_utils.py; #7098 also normalizes indices to int64, which this PR does not. #7098 has the stronger test evidence of the two. I did not see it before implementing, and I am happy to drop part 2 here and review #7098 instead if you would rather land that -- part 1 is independent and does not depend on it.

Test

python -m pytest -q tests/utils/test_profiler_markers.py
python -m pytest -q tests/utils/test_attention_utils.py
python -m pytest -q tests/plugin/test_platform_abstraction.py

Target	Result
tests/utils/test_profiler_markers.py	2 passed, 2 skipped
tests/utils/test_attention_utils.py	5 passed
tests/plugin/test_platform_abstraction.py	14 passed

The 2 skips are the nvtx and mstx paths; neither package is installed in this image. Both new unit test files inject a fake platform, so no accelerator is needed.

Against a real out-of-tree platform supplying Intel ITT markers, _resolve_markers() selects them even with is_nvtx_available() forced to True -- the regression the fix targets. profiler.tool resolution was checked for three cases: an out-of-tree tool name resolves to its implementation, a built-in name still reaches core's dispatch, and an unknown name still falls through to _NoOpProfiler.

Existing tests touched

Four existing test files change only in routing their unpad_input / pad_input imports through verl.utils.attention_utils -- no logic change:

• tests/models/test_transformer.py
• tests/models/test_transformers_ulysses.py
• tests/utils/test_activation_offload.py
• tests/special_distributed/test_fsdp_ckpt.py

They are covered by the existing CUDA/NPU CI (model.yml, model_ascend.yml). Before this change they failed at import on a device without a flash-attn build (unpad_input unbound). Results on 4x Intel Arc Pro B60 24GB:

• test_transformer.py -- 4/4 passed
• test_transformers_ulysses.py (torchrun --nproc_per_node=4 -m pytest) -- all sp_size<=4 configs pass; sp_size=8 needs 8 GPUs
• test_activation_offload.py -- fsdp2 cases pass; FSDP1 cases hit a Level Zero runtime error, an XPU-side issue unrelated to this change
• test_fsdp_ckpt.py -- the default config exceeds 24GB per device; with a reduced batch, fsdp2 save/load round-trips bit-exactly

Design & Code Changes

File	Change
verl/plugin/platform/platform_base.py	profiler_markers(), None by default
verl/utils/profiler/__init__.py	resolution order and the nvtx-shadowing fix
verl/workers/engine_workers.py	allowlist replaced with an any-tool check
verl/utils/npu_flash_attn_utils.py -> verl/utils/attention_padding_utils.py	rename only
verl/utils/attention_utils.py	falls back to the padding utils whenever flash_attn will not import; duplicate _fallback_* helpers deleted
verl/utils/torch_functional.py	two hardcoded device branches routed through the dispatcher
tests/utils/test_profiler_markers.py, tests/utils/test_attention_utils.py	new

Checklist Before Submitting

• Read the Contribute Guide.
• Apply pre-commit checks.
• Add / Update the documentation.
• Add unit or end-to-end test(s) to the CI workflow. The hardware that exposes the original collision is not in CI; the in-tree platforms are covered by unit tests and the table above documents that their behavior is unchanged.
• Once your PR is ready for CI, send a message in the ci-request channel.
• If your PR is related to the recipe submodule — not applicable.

[wuxibin89]

If XPU doesn't support ReduceOp.AVG, then how do you do gradient reduce_scatter and all_reduce in HSDP?
https://github.com/pytorch/pytorch/blob/v2.13.0/torch/distributed/fsdp/_fully_shard/_fsdp_collectives.py#L846

[kahlun]

On coverage — the fix is validated on FSDP2 with a single-mesh setup (fsdp_size=-1): 6 runs with it and 6 without, back-to-back on the same node, clean pass every time with it and the same failure every time without. What that doesn't cover is the 2D mesh, so the all-reduce leg is from reading the source rather than measured. I'd rather keep this PR to the platform hooks themselves, since they're device-neutral and every default preserves current CUDA behavior. The XPU-side validation is a separate change, and I'll send it as its own PR: a 2×2 mesh run, plus a CPU/gloo test asserting SUM on both a 1D and a 2D mesh so it's covered in CI without needing XPU hardware.

[wuxibin89]

From intel/torch-xpu-ops#3020, I think XCCL do support ReduceOp.AVG but seems to have double-division bug? If this bug still exists, it may also affect FSDP gradient reduce_scatter and all_reduce in 1/2D mesh, so disable it in verl side alone is not enough. Please check whether this bug has been fixed.

[kahlun]

Hi, pytorch xpu ops 3020 fixed. It is fixed, when using pytorch2.13 will not encounter average divison bug.
the Average that i mention not support is more of the AVG Reduction in Reduce (Scale-up & Scale-out)

this bug will be fix in oneccl2022.2 = pytorch2.15.

but for the FSDP gradient reduce_scatter and all_reduce in 1/2D mesh question or concern, i will put in the comment

On HSDP coverage: you're right that it wasn't covered, and running it surfaced a real bug — but it's not a Ray bug, and I want to be precise about what's proven versus what isn't.

Root cause: worker.py:273 only pins a device with set_device() when a RAY_EXPERIMENTAL_NOSET_* env var happens to be set. In the default config it never pins at all — it silently assumes Ray's visibility mask has already relabeled devices so index 0 is correct. That assumption doesn't hold on XPU (a separate torch-xpu issue discards the mask after lazy init), so every rank lands on the same physical GPU. Ray itself isn't at fault here — I checked ray-project/ray#64440, it's correct.

I have a fix on a branch (not yet in this PR): pin unconditionally, and resolve the index against what the process can actually see rather than trusting the mask blindly. Verified on real 4x B60 hardware against verl's own create_device_mesh() / collective path — 1 of 4 distinct GPUs before the fix, 4 of 4 after, 0 AVG failures across both 1-D and 2-D meshes.

What I haven't yet gotten to complete: a full real GRPO training run under this PR's exact colocated-RM + FSDP2 + transfer_queue configuration hit a separate, unrelated stall — verl's shared-memory weight-transfer fallback (BucketedWeightSender, used because XPU doesn't support CUDA-style IPC) deterministically stalls partway through sending weights into the rollout engine. Reproduced twice, same byte offset both times, independent of oneCCL env settings, so it isn't a flaky resource issue. Separate bug, still open, will report on its own.

So: the device-pinning fix is verified correct at the mechanism level — real hardware, real verl code path, not a mock. A full end-to-end HSDP training run under this specific config is still blocked by that second, unrelated issue.
I will change the PR description:
narrow the claim from "xccl does NOT support ReduceOp.AVG" to the accurate version — oneCCL implements AVG on its SYCL-kernel path but not its scheduler path, path selection is internal and not capability-aware, so a framework can't guarantee an AVG request will be serviceable at runtime. Forcing SUM is the only way to guarantee it for pre oneccl 2022.2

[kahlun]

the full description for the question on, quite lengthy or detail

From https://github.com/intel/torch-xpu-ops/issues/3020, 
I think XCCL do support ReduceOp.AVG but seems to have double-division bug?
If this bug still exists, it may also affect FSDP gradient reduce_scatter and all_reduce in 1/2D mesh, 
so disable it in verl side alone is not enough. Please check whether this bug has been fixed.

Checked #3020 — it's real, and it's fixed on the stack I'm running. It's AVG double-division on
XCCL's small-message protocol (the non-monotonic size pattern in that issue). The fix shipped in
oneAPI DLE 2026.0 rather than in a torch release; torch.version.xpu reports 20260000 on my
image, which is how I can tell the measurements below are on the fixed build. Stack: 4× Arc Pro
B60, torch 2.13.0+xpu, oneCCL 2022.1.

So I measured your case instead of reasoning from the source. 4 ranks under torchrun, one rank
per physical GPU (verified by device uuid), real fully_shard. I wrapped
_get_gradient_divide_factors to record what FSDP2 actually requested, and checked gradients
against a reference that uses no AVG anywhere — each rank's local gradient, ReduceOp.SUM over
gloo on CPU, divided by world size by hand:

mesh	ops FSDP2 requested (pre, post, RS, AR)	grad ÷ reference
1-D (4,), default	None, None, AVG, AVG	1.000000
1-D (4,), force_sum	None, 4.0, SUM, SUM	1.000000
2-D (2,2), default	None, None, AVG, AVG	1.000000
2-D (2,2), force_sum	None, 4.0, SUM, SUM	1.000000

fp32; a surviving double division would show as 1/4 = 0.250000. The AVG and force-SUM arms are
bitwise identical (max|diff| = 0.000e+00) — expected, since world size 4 is a power of two, so
oneCCL's AVG and an explicit post_factor=4.0 are the same arithmetic. Raw
all_reduce/reduce_scatter AVG on the 2-D mesh's own size-2 subgroups is also correct (1.5
against a 3.0 SUM control). So no double division on either leg on either mesh — your reading was
right and my PR description is too broad.

There is a second, distinct failure, and it isn't a wrong number — it's a hard abort:
coll_param.cpp:491 validate: EXCEPTION: average operation is not supported for the scheduler path. oneCCL has two execution paths, and AVG is implemented on the SYCL-kernel path but not on
the scheduler path (confirmed with Intel's oneCCL team). ccl_coll_param::validate() throws
unconditionally when the collective is allreduce/reduce_scatter/reduce and the reduction is avg —
no size threshold and no partial-correctness window, which is how I know it isn't #3020. The
caller doesn't pick the path: you request a reduction and oneCCL selects internally, and that
selection isn't capability-aware, so an AVG request the API accepted can be dispatched to a path
that cannot service it — surfacing mid-collective inside a training step. I'm not claiming the
path choice is wrong; it may be right for the topology. What's missing is capability-awareness in
the dispatch, and I'm following up with Intel to ask for that, or for AVG emulated as
SUM-then-scale on that path.

On "disabling it in verl alone is not enough" — agreed on the principle, and that's why the change
isn't scoped to verl's own AVG call sites. is_reduce_avg_supported() also gates the FSDP2 side,
at fsdp_utils.py:615 in this PR: when it's False, apply_fsdp2 calls
set_force_sum_reduction_for_comms(True) on the root and every fully_shard()'d layer, since
that API doesn't recurse into children. That's the same knob FSDP2 consults for its internal
reduce_scatter/all_reduce, so one flag covers both verl's all_reduce_avg() helper and FSDP's own
collectives — the SUM, SUM rows above are that block firing. Worth being explicit about what the
flag can and can't express: it's static, while oneCCL's path selection happens at runtime, so
False is the only safe answer a capability query can give when the path isn't predictable. It's
also a backend property rather than a device one, and it lives in the out-of-tree XPU platform
class — so when oneCCL's dispatch becomes capability-aware, flipping it back is a one-line change
there with nothing to revert in verl. The test is written and passing, not a follow-up:
tests/utils/test_fsdp2_force_sum_reduction_on_cpu.py, 4 gloo ranks on CPU, parametrised over a
1-D (4,) mesh and a 2-D (2,2) ("ddp", "fsdp") mesh, calling the real apply_fsdp2; it
asserts both legs come back ReduceOp.SUM and that gradients match the default AVG path at
rtol=atol=1e-6. Both parametrisations pass.

What I'll change in the PR: narrow the claim from "xccl does NOT support ReduceOp.AVG" to the
accurate version — oneCCL implements AVG on its SYCL-kernel path but not its scheduler path, path
selection is internal and not capability-aware, so a framework can't guarantee an AVG request will
be serviceable at runtime. Forcing SUM is the only way to guarantee it, and per the table it's
numerically exact.

On size coverage, since #3020 was size-dependent: separately from FSDP I ran raw all_reduce and
reduce_scatter AVG at numel 0, 1, 8 and 1024 — the 0-dim scalar and size-1 cases that issue
lists as broken — and all return 1.5 against a 3.0 SUM control on two ranks. Double division
would give 0.75. So the small-message region the issue identifies as the risky one is correct on
this stack, and the FSDP runs above cover the gradient path itself up to 65536 elements including
a padded head.

---

if want the run log, i can put in gist if requested

[kahlun]

@wuxibin89 following up on our thread above — since your last comment I've
removed the is_reduce_avg_supported() hook entirely (AVG's reduce_avg
workaround now runs as a plugin-only patch, verl-hardware-plugin#32, per
the direction confirmed in verl-hardware-plugin#26), and
intel/torch-xpu-ops#3020 being fixed covers the double-division concern you
raised — the remaining AVG gap I was describing is specifically the
Reduce (scale-up/scale-out) path, not the FSDP gradient reduce_scatter/
all_reduce path you asked about.

This PR is now down to two hooks: attention_utils_module() and
profiler_markers() (+ dist_profiler_cls() for the rollout-server profiler
allowlist, which can't be done from the plugin). Would appreciate a look
when you have a chance —if something's still blocking, just let me know what it is.
So I can fix it quickly.

[wuxibin89]

verl.utils.npu_flash_attn_utils copy attention utils from Dao-AILab/flash-attention, can XPU reuse it?

[wuxibin89]

Can we unify this as below? Then we can avoid add a new dist_profiler_cls to PlatformBase:

if omega_profiler_config.get("tool", None) is not None:
      tool_config = omega_conf_to_dataclass(
          omega_profiler_config.get("tool_config", {}).get(omega_profiler_config.get("tool"))
      )

[wuxibin89]

Thanks for narrowing this down. After looking at how verl-hardware-plugin#22 actually implements these hooks, I think both attention_utils_module() and dist_profiler_cls() can be dropped from PlatformBase.

1. attention_utils_module()

PlatformXPU.attention_utils_module() returns "verl.utils.npu_flash_attn_utils", which is a verl-core module. That file is a pure torch + einops copy of flash-attn's bert_padding.py and has no torch_npu dependency, so nothing in it is NPU-specific. A hook that only points verl-core at another verl-core module isn't needed. Instead, attention_utils._get_attention_functions() can use npu_flash_attn_utils whenever flash_attn can't be imported, replacing the _fallback_* duplicates. Renaming the module to something device-neutral would make that clearer. XPU (and any future backend without flash-attn) then gets the same validated code path with no platform hook.

Please keep the call-site changes (torch_functional.py and the 4 test files routing through verl.utils.attention_utils). Those are real fixes.

2. dist_profiler_cls()

• engine_workers.py: convert tool_config for any non-None tool, as in my inline comment. omega_conf_to_dataclass returns None for an empty config, so tools without a tool_config entry still work, and a plugin tool can provide its own dataclass through _target_. The hardcoded allowlist goes away entirely.
• Rollout servers (vllm_async_server.py / async_sglang_server.py / trtllm_async_server.py): I'd revert these. There, profiler_controller is only used for gating (check_enable / check_this_rank / is_discrete_mode). Actual profiling goes through the engine's own profiler (engine.start_profile(), configured through build_vllm_profiler_args() from tool_config.contents). A plugin profiler instantiated there never has its start()/stop() called, so allowing vtune through doesn't enable VTune for rollout.
• DistProfiler dispatch: VtuneProfiler is essentially NsightSystemsProfiler with ITT markers instead of nvtx. start/stop already go through get_platform().profiler_start()/profiler_stop(), which are no-ops on XPU. If XPU really needs its own profiler.tool name, it can be registered from the plugin side, in line with the direction in verl-hardware-plugin#26. It doesn't need a new PlatformBase method.

Side note on profiler_markers() (the remaining hook): _resolve_markers() checks is_nvtx_available() first, and nvtx is a dependency of the verl-core extra. is_nvtx_available() only checks whether the package is importable, not the device. So on an XPU box with verl[verl-core] installed, the platform markers are never used. Asking the platform first and falling back to nvtx/mstx when it returns None keeps CUDA and NPU behavior unchanged.

That would reduce this PR to profiler_markers() plus the attention call-site cleanup.

[kahlun]

@wuxibin89 thanks — all three points are done in 72e089d. The PR is now 13 files, +369/−112, and platform_base.py is a single +14 hook.

1. attention_utils_module() — dropped. The five _fallback_* duplicates are deleted and verl/utils/attention_utils.py falls back to that module whenever flash_attn cannot be imported. Renamed it verl/utils/attention_padding_utils.py per your suggestion. Call-site changes in torch_functional.py and the tests kept.

2. dist_profiler_cls() — dropped. engine_workers.py:492 unified exactly as in your inline comment; the allowlist is gone. Rollout-server edits reverted. The profiler class is registered from the plugin side instead.

3. profiler_markers() — platform first, then nvtx → mstx → generic. Resolved lazily on first marker use, so importing the package does not probe hardware.

is_reduce_avg_supported() is no longer in this PR at all, so that thread is moot on the core side.

Tests

CPU-only container, PYTHONPATH pointed at this branch:

$ python -m pytest tests/utils/test_attention_utils.py
5 passed, 10 warnings in 6.70s

$ python -m pytest tests/utils/test_profiler_markers.py
2 passed, 2 skipped, 10 warnings in 7.36s

$ python -m pytest tests/plugin/test_platform_abstraction.py
14 passed, 12 warnings in 6.72s

The 2 skips are the nvtx and mstx fallback branches — importorskip on nvtx / torch_npu, neither of which is in that image. They should run in CI, since nvtx comes with the verl-core extra.

I also checked the cross-repo behaviour against verl-hardware-plugin#22, including the case your side note is about:

[PASS] before patch, tool='vtune' -> core no-op -> _NoOpProfiler
[PASS] after patch, tool='vtune' -> VtuneProfiler
[PASS] builtin tool='torch' untouched -> Profiler
[PASS] unknown tool -> core no-op -> _NoOpProfiler
[PASS] tool with no tool_config entry -> None
[PASS] tool_config with _target_ -> dataclass -> NsightToolConfig
[PASS] ITT markers win over an available nvtx

edited <4.20pm, for update the commit ID>