Repository navigation
[hardware] feat: add platform abstraction layer and plugin-based engine override system - #6086
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces FlagOS (FL) multi-chip support to the verl framework, enabling unified training and inference across diverse hardware platforms. Key additions include a hardware-agnostic platform abstraction layer, an environment manager for phase-specific configurations, and specialized FlagOS engines for Megatron and FSDP. Review feedback highlights a bug in the FlagCX enablement logic that could cause unnecessary crashes and notes that the FlagGems initialization function is currently a placeholder that needs implementation to fulfill the PR's objectives.
|
|
|
Currently, verl has native support for NVIDIA/AMD/Ascend hardware, other training backend(e.g DeepSpeed, Bubblebee) should be dynamically loaded by hook to avoid maintain in source tree. https://github.com/verl-project/verl/blob/main/verl/workers/engine/__init__.py # path style
trainer.custom_engine_path=/path/to/fsdp_fl/transformer_impl.py
# module style
trainer.custom_engine_path=flag_os.verl_plugin.fsdp_fl.transformer_implThen we can load this file by |
physics31415926
changed the title
|
verl relies on ray for device resource management: https://github.com/verl-project/verl/blob/main/verl/single_controller/ray/base.py#L137-L141 But Ray Accelerator has no support for other hardware platform, such as Cambricon MLU, Moore Threads MUSA. |
718ffd1 to
a3b9120
Compare
| return "cuda" | ||
|
|
||
| @property | ||
| def device_module(self) -> ModuleType: |
There was a problem hiding this comment.
I’ve seen identical implementations across numerous repos, including DeepSpeed and vLLM 😂. It’s probably high time we adopted the modern torch.accelerator module 😄. https://docs.pytorch.org/docs/2.9/accelerator.html
This feature has covered most common torch.cuda interfaces since PyTorch 2.9, including device management and memory management. A few capabilities are yet to be integrated: device related ccl backend, unified configuration of *_visible_devices, and the profiler module. Our team is also working on abstracting away these backend discrepancies natively at the PyTorch layer.
cc @wuxibin89
a1397c3 to
81d0609
Compare
| # ------------------------------------------------------------------ | ||
|
|
||
| @abc.abstractmethod | ||
| @contextmanager |
There was a problem hiding this comment.
The capability differences between different profiling backends have been reflected in DistProfiler. If we also need to implement this here, what should the cooperation relationship between it and DistProfiler be?
There was a problem hiding this comment.
They operate at different layers and don't overlap:
PlatformBase.profiler_start/stop: lowest level, pure device abstraction — how to control the hardware profiler
DistProfiler: highest level — when and where to profile (rank/step selection, tool dispatch)
DistProfiler delegates to tool-specific impls (e.g. NsightSystemsProfiler), which in turn call get_platform().profiler_start() for the actual device operation.
There was a problem hiding this comment.
This logic assumes a one-to-one correspondence between the DistProfilerand the platform.
What if there are many different profilers on one platform/device? For example, mstx_profile\torch_profile\precision_debuger_profiler are all supported on npu. I don’t think it’s a good idea to couple DistProfiler with the platform.
| DistProfilers | NPU | GPU |
|---|---|---|
| nvtx_profiler | ❌ | ✅ |
| torch_profiler | ✅ | ✅ |
| mstx_profiler | ✅ | ❌ |
| precisidon_debuger | ✅ | ✅ |
|
Thanks for your contribution. A good platform abstraction can facilitate the development of various backends. Looking forward to merging this PR. |
Hi @physics31415926 @wuxibin89 , mlu already supports Ray, so the device_name here also need to be handled for mlu. Could you help to modify the code to get device_name from ray_resource_name() https://github.com/flagos-ai/verl-FL/blob/feat/multi_chip_support/verl/plugin/platform/platform_base.py#L189. Thank you! |
Hi @uqyxx The verl repository only retains the platforms for nvidia-GPU/huawei-NPU, and other hardwares can be support via https://github.com/verl-project/verl-hardware-plugin. You can decide how to implement the |
Hi @heavyrain-lzy , yes we are developing in verl-project/verl-hardware-plugin#1, but code here https://github.com/verl-project/verl/blob/main/verl/single_controller/ray/base.py#L137-L141 only handles cuda and npu logic, other platforms are not covered. Even we implement 'get_device_name' in verl-hardware-plugin, the device name cannot be passed in here. So the code here should be like |
|
@uqyxx This PR has unified the ray resource name when creating placement, you can see |
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a unified multi-chip platform abstraction layer to support RL training across multiple hardware platforms (such as NVIDIA CUDA and Huawei Ascend NPU) through a plugin system, refactoring existing device utilities to delegate to this new layer. Key feedback points out several critical issues: a memory leak in _state_dict_utils.py due to a strong reference cycle in weakref.finalize, an assertion in EngineRegistry.register that prevents engine overrides, a bug in CUDA platform detection when nvidia-smi is missing, a potential ModuleNotFoundError if cupy is not installed, and an AssertionError on non-CUDA platforms when calling get_placement_groups with its default "cuda" argument.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| t.data_ptr(), | ||
| t.numel() * t.element_size(), | ||
| 1, # lines up with 'cudaHostRegisterPortable' | ||
| from verl.plugin.platform import get_platform |
There was a problem hiding this comment.
Don't change this file, it's only used torch<2.7
…orm hook and share the attention padding helpers (#7917) ### What does this PR do? Two independent changes, both making verl's device handling less CUDA-specific. **1. A `profiler_markers()` hook on `PlatformBase`.** Lets a hardware backend supply its own tracing markers -- `(mark_start_range, mark_end_range, mark_annotate, marked_timer)` -- instead of nvtx or mstx. Default is `None`, so existing platforms are unaffected. This also fixes marker resolution. `verl/utils/profiler/__init__.py` checked `is_nvtx_available()` before consulting the platform, and nvtx ships in the `verl-core` extra, so on any install where nvtx imports the platform was never asked. Order is now platform, nvtx, mstx, generic, and stays lazy so the platform need not exist at import time. Relatedly, `engine_workers.py` converts `tool_config` for any configured `profiler.tool` rather than an allowlist of the five built-in names. `omega_conf_to_dataclass` returns `None` for a missing or empty config, so a tool with no `tool_config` entry still works, and an out-of-tree tool can name its own dataclass through `_target_`. **2. The attention padding helpers become device-neutral.** `npu_flash_attn_utils.py` was already a pure torch/einops port of flash-attn's `bert_padding`, with nothing NPU-specific in it, so it is renamed `attention_padding_utils.py` and used as the fallback for *any* install where `flash_attn` will not import -- not just NPU. That covers non-CUDA accelerators and CPU-only installs such as unit tests and dev boxes. The duplicate `_fallback_*` helpers in `attention_utils.py` are deleted, and `torch_functional.py`'s two remaining hardcoded `flash_attn` / `get_device_name() == "cuda"/"npu"` branches now go through the same dispatcher. No public API changes. Consumed by `verl-hardware-plugin`'s `PlatformXPU` ([#22](verl-project/verl-hardware-plugin#22)). ### Why this is not duplicating an existing PR Searched open and closed PRs for `profiler_markers`, `PlatformBase hook`, `marked_timer`, `mark_start_range`, `attention_utils`, `npu_flash_attn_utils`, `index_first_axis`. For the profiler hook, nothing else lets a platform override the markers. #7988 (also mine) adds a different `PlatformBase` hook -- an optional `torch.profiler` activity list, i.e. what the built-in torch profiler records, not which marker functions the trainer calls. It overlaps only in touching `platform_base.py`; I will rebase whichever lands second. #6086 (merged) created the platform abstraction both extend. For the attention change, **#7098 overlaps substantially and predates this work.** It reaches the same conclusion from the same reasoning -- that the NPU copy only uses torch and einops, so dispatching on hardware was unnecessary -- and both PRs delete the dispatch and touch `tests/utils/test_attention_utils.py`, so they conflict. The differences are small: #7098 inlines the helpers into `attention_utils.py` and deletes the module, where this PR keeps it as `attention_padding_utils.py`; #7098 also normalizes indices to `int64`, which this PR does not. #7098 has the stronger test evidence of the two. I did not see it before implementing, and I am happy to drop part 2 here and review #7098 instead if you would rather land that -- part 1 is independent and does not depend on it. ### Test ```bash python -m pytest -q tests/utils/test_profiler_markers.py python -m pytest -q tests/utils/test_attention_utils.py python -m pytest -q tests/plugin/test_platform_abstraction.py ``` | Target | Result | | --- | --- | | `tests/utils/test_profiler_markers.py` | 2 passed, 2 skipped | | `tests/utils/test_attention_utils.py` | 5 passed | | `tests/plugin/test_platform_abstraction.py` | 14 passed | The 2 skips are the nvtx and mstx paths; neither package is installed in this image. Both new unit test files inject a fake platform, so no accelerator is needed. Against a real out-of-tree platform supplying Intel ITT markers, `_resolve_markers()` selects them even with `is_nvtx_available()` forced to `True` -- the regression the fix targets. `profiler.tool` resolution was checked for three cases: an out-of-tree tool name resolves to its implementation, a built-in name still reaches core's dispatch, and an unknown name still falls through to `_NoOpProfiler`. #### Existing tests touched Four existing test files change only in routing their `unpad_input` / `pad_input` imports through `verl.utils.attention_utils` -- no logic change: - `tests/models/test_transformer.py` - `tests/models/test_transformers_ulysses.py` - `tests/utils/test_activation_offload.py` - `tests/special_distributed/test_fsdp_ckpt.py` They are covered by the existing CUDA/NPU CI (`model.yml`, `model_ascend.yml`). Before this change they failed at import on a device without a flash-attn build (`unpad_input` unbound). Results on 4x Intel Arc Pro B60 24GB: - `test_transformer.py` -- 4/4 passed - `test_transformers_ulysses.py` (`torchrun --nproc_per_node=4 -m pytest`) -- all `sp_size<=4` configs pass; `sp_size=8` needs 8 GPUs - `test_activation_offload.py` -- fsdp2 cases pass; FSDP1 cases hit a Level Zero runtime error, an XPU-side issue unrelated to this change - `test_fsdp_ckpt.py` -- the default config exceeds 24GB per device; with a reduced batch, fsdp2 save/load round-trips bit-exactly ### Design & Code Changes | File | Change | | --- | --- | | `verl/plugin/platform/platform_base.py` | `profiler_markers()`, `None` by default | | `verl/utils/profiler/__init__.py` | resolution order and the nvtx-shadowing fix | | `verl/workers/engine_workers.py` | allowlist replaced with an any-tool check | | `verl/utils/npu_flash_attn_utils.py` -> `verl/utils/attention_padding_utils.py` | rename only | | `verl/utils/attention_utils.py` | falls back to the padding utils whenever `flash_attn` will not import; duplicate `_fallback_*` helpers deleted | | `verl/utils/torch_functional.py` | two hardcoded device branches routed through the dispatcher | | `tests/utils/test_profiler_markers.py`, `tests/utils/test_attention_utils.py` | new | ### Checklist Before Submitting - [x] Read the [Contribute Guide](https://github.com/verl-project/verl/blob/main/CONTRIBUTING.md). - [x] Apply [pre-commit checks](https://github.com/verl-project/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting). - [ ] Add / Update [the documentation](https://github.com/verl-project/verl/tree/main/docs). - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/verl-project/verl/tree/main/.github/workflows). The hardware that exposes the original collision is not in CI; the in-tree platforms are covered by unit tests and the table above documents that their behavior is unchanged. - [x] Once your PR is ready for CI, send a message in the `ci-request` channel. - [x] If your PR is related to the `recipe` submodule — not applicable. --------- Co-authored-by: wuxibin <wuxibin@bytedance.com>
) ### What does this PR do? `FSDPCheckpointManager` gates FSDP's sharded-state-dict CPU offload on `is_cuda_available`: ```python state_dict_cfg = ShardedStateDictConfig(offload_to_cpu=True if is_cuda_available else False) optim_cfg = ShardedOptimStateDictConfig(offload_to_cpu=True if is_cuda_available else False) ``` `is_cuda_available` is `torch.cuda.is_available()` evaluated once at import (`verl/utils/device.py`). On any non-CUDA accelerator it is permanently `False`, so the gathered model and optimizer shards stay resident in device memory for the whole `torch.save` window instead of being staged through host memory. CUDA gets the memory headroom; every other backend pays for it, with no way to opt in. This switches the four sites to `is_device_available()` — the platform-abstraction check (`get_platform().is_available()`) that this same class already uses to decide whether accelerator RNG state is reachable in `BaseCheckpointManager.get_rng_state`. Why this is a per-device-class oversight rather than a deliberate CUDA-only choice: the `offload_to_cpu` lines predate the platform abstraction layer (present at least as of 2026-04-20; `is_device_available` arrived 2026-06-04 in #6086), and the migration to it has been incremental — #7871 applied the same `is_device_available()` treatment to the Megatron engine's seed path in September. `offload_to_cpu` is a plain PyTorch FSDP feature with no CUDA-specific code behind it, so there was never a technical reason to exclude other backends. The measurement below confirms it works on a non-CUDA backend today. ### Checklist Before Starting - [x] Search for similar PRs: - https://github.com/verl-project/verl/pulls?q=offload_to_cpu - https://github.com/verl-project/verl/pulls?q=is_device_available - https://github.com/verl-project/verl/issues?q=offload_to_cpu I enumerated every open PR in the repo and checked its changed-file list: **#7286 is the only open PR that touches `verl/utils/checkpoint/fsdp_checkpoint_manager.py`.** **Not a duplicate of #7286** (`[ckpt] fix: load FSDP checkpoint shards onto CPU instead of the saved device`, open). That PR is the closest neighbour — same file, same "treat sharded state dicts as CPU-resident" theme — but a disjoint change: it adds `map_location="cpu"` to the three `torch.load` calls in `load_checkpoint`, which is wrong on CUDA too and fixes where tensors are *revived from disk*. This PR changes the `ShardedStateDictConfig` / `ShardedOptimStateDictConfig` gather configs, which is CUDA-privileged by construction and governs where tensors land *during the FSDP gather*, on both the save and load paths. The two touch different lines and neither subsumes the other; they merge independently in either order. Worth noting for whoever lands the second one: together they make the manager's CPU-residency story consistent end to end, which is the stated motivation of #7286. **Related precedent, not overlapping**: #7871 (merged) added an `is_device_available()` check for non-GPU/NPU devices in `verl/workers/engine/megatron/utils.py`. Same class of fix, different file; it did not touch the checkpoint managers. - [x] PR title formatted as `[{modules}] {type}: {description}` ### Test **Measured on real hardware: 2× Intel Arc Pro B60 (XPU), `torch 2.11.0`, `xccl` backend.** The standalone repro below wraps an 8-layer 4096-wide model in FSDP across 2 ranks, runs one `optimizer.step()` so optimizer state exists, then performs the same sharded gather twice — once with `offload_to_cpu=False` (today's behaviour on XPU) and once with `True` (this patch) — reporting the resulting tensor device and the peak device memory during each gather: ``` device=xpu world_size=2 free/total=22057000960/25669140480 [offload_to_cpu=False] sample tensor 'net.0.weight' device=xpu:0 peak_device_mem_during_gather=2816.8 MiB optim_sd_keys=16 [offload_to_cpu=True] sample tensor 'net.0.weight' device=cpu peak_device_mem_during_gather=2560.8 MiB optim_sd_keys=16 RESULT offload_False_device=xpu:0 offload_True_device=cpu RESULT peak_delta_MiB=256.1 PASS: offload_to_cpu mechanism confirmed functional on this device ``` Two things this establishes: 1. **`offload_to_cpu=True` works correctly on a non-CUDA backend.** The gathered shard lands on `cpu` and the optimizer state dict is intact (16 entries, matching the 16 parameters). Nothing about this FSDP feature is CUDA-specific, so the current gate is excluding a working code path. 2. **The cost of the current `False` default is real and scales with model size.** 256 MiB of extra peak device memory held through the gather, on the order of the per-rank sharded state-dict size for this model — i.e. proportional to model/optimizer size, not a fixed overhead. On a card already near capacity this is the difference between a checkpoint save that completes and one that OOMs, which is the failure mode #7286 describes from the load side. **On CUDA this patch is a no-op by construction**, which is why no CUDA regression run is included: `PlatformCUDA.is_available()` is `return torch.cuda.is_available()` — the identical expression the old module-level `is_cuda_available` captured. The only semantic difference is import-time vs. call-time evaluation, which cannot differ in practice since CUDA availability is fixed for a process. On a CPU-only host both spellings yield `False`, so the `*_on_cpu` checkpoint tests are unaffected. Lint: `ruff==0.12.2 check` and `ruff format --check` clean (pinned to the repo's `.pre-commit-config.yaml` rev; note `ruff format` reflows two now-shorter ternaries, included in the diff). **Not covered:** `tests/special_distributed/test_fsdp_ckpt.py` — the repo's own end-to-end `FSDPCheckpointManager` save/load round trip — has not been run with this patch. It needs 2 accelerators and a Qwen2.5 tokenizer, and on XPU additionally a verl image with `flash_attention_2` available. What is established here is the mechanism and its memory cost on real XPU silicon, not a full verl checkpoint round trip through the patched manager. That test is the natural place to settle it if you would like it settled before merging, and I am happy to run it on the B60s on request. **Known trade-off, stated explicitly:** this makes non-CUDA backends stage shards through host memory, so it trades device-memory pressure for host-memory pressure — exactly the trade CUDA has always made here. On nodes where host RAM is the scarcer resource this is not free. I considered exposing it as a `checkpoint.offload_to_cpu` config override instead, and deliberately did not: that adds a user-facing knob to work around what is currently a one-platform exclusion from an existing default, and the right first step is to make the default consistent across backends. If maintainers would prefer the knob (either instead of or on top of this), say so and I will follow up separately. <details> <summary>Repro script (2 accelerators, torchrun, no verl import needed)</summary> ```python #!/usr/bin/env python3 """Repro: FSDP sharded state_dict() offload_to_cpu behavior on XPU vs CUDA. Run with: torchrun --standalone --nproc_per_node=2 repro_offload_to_cpu.py """ import argparse import os import torch import torch.distributed as dist import torch.nn as nn from torch.distributed.fsdp import FullyShardedDataParallel as FSDP from torch.distributed.fsdp import ShardedOptimStateDictConfig, ShardedStateDictConfig, StateDictType def get_device_module(): if hasattr(torch, "xpu") and torch.xpu.is_available(): return "xpu", torch.xpu if torch.cuda.is_available(): return "cuda", torch.cuda raise RuntimeError("no accelerator available") class Toy(nn.Module): def __init__(self, hidden=4096, layers=8): super().__init__() self.net = nn.Sequential(*[nn.Linear(hidden, hidden) for _ in range(layers)]) def forward(self, x): return self.net(x) def gather_once(model, optimizer, offload_to_cpu, device_mod, rank): device_mod.reset_peak_memory_stats() cfg = ShardedStateDictConfig(offload_to_cpu=offload_to_cpu) optim_cfg = ShardedOptimStateDictConfig(offload_to_cpu=offload_to_cpu) with FSDP.state_dict_type(model, StateDictType.SHARDED_STATE_DICT, cfg, optim_cfg): model_sd = model.state_dict() optim_sd = FSDP.optim_state_dict(model, optimizer) dist.barrier() peak = device_mod.max_memory_allocated() sample_key = next(iter(model_sd)) sample_tensor = model_sd[sample_key] sample_device = getattr(sample_tensor, "device", None) if sample_device is None and hasattr(sample_tensor, "local_tensor"): sample_device = sample_tensor.local_tensor().device if rank == 0: print( f"[offload_to_cpu={offload_to_cpu}] sample tensor '{sample_key}' device={sample_device} " f"peak_device_mem_during_gather={peak / 1024**2:.1f} MiB " f"optim_sd_keys={len(optim_sd.get('state', {}))}" ) return sample_device, peak def main(): parser = argparse.ArgumentParser() parser.add_argument("--hidden", type=int, default=4096) parser.add_argument("--layers", type=int, default=8) args = parser.parse_args() device_name, device_mod = get_device_module() local_rank = int(os.environ["LOCAL_RANK"]) device_mod.set_device(local_rank) backend = "xccl" if device_name == "xpu" else "nccl" dist.init_process_group(backend=backend) rank = dist.get_rank() device = torch.device(f"{device_name}:{local_rank}") model = Toy(args.hidden, args.layers).to(device) model = FSDP(model, device_id=device) optimizer = torch.optim.AdamW(model.parameters(), lr=1e-4) x = torch.randn(8, args.hidden, device=device) loss = model(x).sum() loss.backward() optimizer.step() dist.barrier() if rank == 0: free, total = None, None try: free, total = device_mod.mem_get_info() except Exception: pass print(f"device={device_name} world_size={dist.get_world_size()} free/total={free}/{total}") dev_false, peak_false = gather_once(model, optimizer, False, device_mod, rank) dev_true, peak_true = gather_once(model, optimizer, True, device_mod, rank) if rank == 0: print(f"RESULT offload_False_device={dev_false} offload_True_device={dev_true}") print(f"RESULT peak_delta_MiB={(peak_false - peak_true) / 1024**2:.1f}") assert str(dev_true).startswith("cpu"), "offload_to_cpu=True did not land the shard on cpu" assert not str(dev_false).startswith("cpu"), "offload_to_cpu=False unexpectedly landed on cpu" print("PASS: offload_to_cpu mechanism confirmed functional on this device") dist.barrier() dist.destroy_process_group() if __name__ == "__main__": main() ``` </details> ### API and Usage Example No API change. No config change. Behaviour change is limited to non-CUDA accelerators, which now get the same CPU-offload default CUDA already had. ### Design & Code Changes `verl/utils/checkpoint/fsdp_checkpoint_manager.py`: - Import `is_device_available` instead of `is_cuda_available`. - `load_checkpoint`: `ShardedStateDictConfig` / `ShardedOptimStateDictConfig` now take `offload_to_cpu=is_device_available()`. - `save_checkpoint`: same for both configs. - Drops the redundant `True if … else False` wrapper; `ruff format` then reflows the two surrounding ternaries onto single lines. ### Checklist Before Submitting - [x] Read the Contribute Guide - [x] `ruff` / `ruff-format` clean at the pinned `v0.12.2` - [x] Docs — no user-facing surface changes, nothing to update - [ ] Tests — see the Test section for what was measured on 2× B60 and what was not run - [ ] `ci-request` Slack message - [x] Not related to the `recipe` submodule
What does this PR do?
Add a unified platform abstraction layer and plugin-based engine system to support multi-chip hardware (NVIDIA CUDA, Ascend NPU, Cambricon MLU, Moore Threads MUSA, etc.) in verl.
Key changes:
verl.plugin.platform): Hardware-agnostic device API with auto-detection, backward-compatible withverl.utils.device.@PlatformRegistry.register(platform="xxx")): Dynamic platform registration via decorator. Built-in platforms (CUDA, NPU) use the same mechanism. External plugins register new platforms without modifying the verl source tree.EngineRegistry): Last-writer-wins registration — plugins override default engines by registering with the same(model_type, backend, device)key. Both registration and lookup useget_device_name()for automatic hardware matching.verl-plugin-fl(https://github.com/physics31415926/verl-plugin-fl) provides FL-optimized FSDP/Megatron engines with FlagGems, FlagCX, TE-FL support.Plugin loading uses the existing
VERL_USE_EXTERNAL_MODULEShook — no new config fields needed.Checklist Before Starting
[{modules}] {type}: {description}Test
tests/plugin/test_platform_abstraction.py: Platform detection, creation, singleton, and PlatformRegistry tests.verl-plugin-flinstalled.API and Usage Example
For platform plugin developers (adding new hardware):
For engine plugin developers:
Design & Code Changes
How the platform + engine override works:
VERL_USE_EXTERNAL_MODULES=verl_plugin_flverl/__init__.pycallsimport_external_libs()which imports the plugin module@PlatformRegistry.register()(or uses built-in CUDA/NPU)@EngineRegistry.register(device=get_device_name())EngineRegistry.get_engine_cls()usesget_device_name()-> returns plugin engineCore changes:
verl/plugin/platform/platform_manager.py:PlatformRegistrywith@register(platform="xxx")decorator, auto-detection iterates registered platformsverl/plugin/platform/platform_cuda.py:@PlatformRegistry.register(platform="cuda")verl/plugin/platform/platform_npu.py:@PlatformRegistry.register(platform="npu")verl/workers/engine/base.py:EngineRegistry.register()allows overwriting (last writer wins);get_engine_cls()usesget_device_name()docs/hardware/multi_chip_support.rst: Architecture documentationexamples/grpo_trainer/run_qwen3-0.6b_fl.sh: Example FL training scriptChecklist Before Submitting
ci-requestchannel.