[kahlun]

Summary

Adds an xccl checkpoint engine to the XPU accelerator package so a disaggregated trainer/rollout (for example one-step-off or fully-async GRPO) can sync weights on Intel GPUs. It registers when the plugin loads on an XPU host and is selected with:

actor_rollout_ref.rollout.checkpoint_engine.backend=xccl

No verl core change. No Ray collective backend, cupy or vLLM change is needed.

Motivation

verl's nccl checkpoint engine cannot run on XPU: it imports cupy (CUDA only), hard-codes cuda, and broadcasts through ray.util.collective, which has no XCCL backend in upstream Ray. The engine never registers on an XPU image, so there is no usable weight-sync mode for a disaggregated trainer/rollout.

Changes

• verl_hardware_plugin/accelerators/xpu/engines/xccl_checkpoint_engine.py: the engine. Same bucketed all_gather + broadcast design, ZMQ metadata channel and rebuild_group option as verl's NCCL engine. The data path is a torch ProcessGroupXCCL: actor rank 0 hosts a TCPStore (OS-assigned port) and publishes its port and a per-round id in MasterMetadata; each rank builds its own process group over a PrefixStore, leaving the default torch.distributed world alone. Staging buffers are plain torch XPU tensors. Only _start_store, _create_process_group, _destroy_process_group and _broadcast touch the transport.
• verl_hardware_plugin/accelerators/xpu/registration.py: try-import so xccl registers on load (same pattern as the MLU checkpoint engines).
• tests/accelerators/xpu/test_xccl_checkpoint_engine.py: registration, defaults and rank layout (pure Python; skipped without a torch XCCL build).
• docs/accelerators/xpu/xccl_checkpoint_engine.md: usage, the settings that mattered, and how to swap the transport later.

multi_sender (relay ranks on the source node) is kept for parity with the NCCL engine but defaults to False: with a real FSDP weight generator and more than one bucket the relay path stalled after the first bucket.

Testing

Environment: 4x Intel Arc Pro B60, one node, image verl-intel-gpu:kahlunte-xpu-main-pinning-20261002 with this branch and upstream-main verl (7e14b6be, no XCCL engine in it) overlaid on PYTHONPATH, VERL_PLATFORM=intel.

• pytest plugin_tests/test_xccl_checkpoint_engine.py (this PR's test file, run inside that image): 5 passed.
• End to end: one-step-off GRPO (verl.experimental.one_step_off_policy.main_ppo), 2 trainer GPUs + 2 vLLM rollout GPUs, Qwen2.5-0.5B-Instruct on gsm8k, batch 32, 3 steps, 32 CPUs, 52 GiB host memory, bucket 1024 MB, vLLM gpu_memory_utilization=0.3, RAY_EXPERIMENTAL_NOSET_ONEAPI_DEVICE_SELECTOR unset.
  • xccl appeared in CheckpointEngineRegistry with no import error.
  • All 3 training steps completed (333 s, no errors).
  • Four weight syncs (initial plus one per step) took 1.85, 0.86, 0.73 and 0.73 s; the vLLM servers loaded the full model after each.
  • The plugin's ray_device_pin_patch pinned the trainers to xpu:0/xpu:1 and the checkpoint-engine workers to xpu:2/xpu:3.
• Pre-commit hooks run on the changed files: ruff, ruff-format, mypy, license header, copyright header, naming and compileall all pass.

Settings that mattered (details in the docs page):

• At least 32 CPUs on the job. With 16 the Ray rollout placement group never schedules (the task runner holds 10 CPUs and the trainer group reserves 6), so the run looks hung.
• update_weights_bucket_megabytes at least as large as the largest single weight (about 544 MB in fp32 for embed_tokens); 512 MB is refused by verl's own check.
• multi_sender=False (the default).
• One device per Ray worker. The pinning comes from the plugin's ray_device_pin_patch; with the older plugin bundled in the image over upstream-main core, a vLLM replica shared the trainer's card and the trainer died with UR_RESULT_ERROR_OUT_OF_RESOURCES.

Not verified: multi-node (cross-node XCCL has known problems on oneCCL 2022.x), more than 4 GPUs, models above 0.5B, repeated runs. A 2048 MB-bucket run with vLLM gpu_memory_utilization=0.5 on a node with a known faulty card aborted on a later update with oneCCL ZE_RESULT_ERROR_UNKNOWN; that node is not a clean data point and the combination was not rerun. The engine uses private torch APIs (ProcessGroupXCCL, ProcessGroup._set_default_backend, _register_backend) and mirrors the structure of verl's NCCL engine, so it has to follow upstream changes by hand.

• pytest tests/ -v passes (only the new test file was run; the full suite was not)
• Manually verified on target hardware (4x B60, see above)

Acceptance Baseline (for new hardware adaptation PRs)

This PR does not add a platform; it adds an optional checkpoint engine to an existing one. scripts/baseline_grpo_gsm8k.sh runs the colocated trainer (verl.trainer.main_ppo) on 8 devices and never goes through a disaggregated checkpoint engine, so it cannot exercise this change. It was not run here.

• Ran scripts/baseline_grpo_gsm8k.sh on target hardware (8 devices): not run, see above
• Training completed all epochs without error: n/a for this change
• critic/rewards/mean shows clear upward trend in first 100 steps: n/a
• Curve is consistent with the NVIDIA reference: n/a

Checklist

• Code follows the project's style and passes the pre-commit hooks on the changed files
• Documentation updated (docs/accelerators/xpu/xccl_checkpoint_engine.md)
• No secrets or credentials included

[Copilot]

🟡 Changes recommended

A subscription race can drop bucket metadata and stall updates, while unaligned mixed-dtype weight offsets can cause tensor reconstruction failures.

2 open issues

• Confirm ZMQ subscriptions before publishing bucket metadata · New
• Align mixed-dtype weight offsets before chunk views · New

What changed in this PR

Adds an optional xccl checkpoint engine for weight synchronization between disaggregated trainers and rollout workers on Intel GPUs, without changes to verl core.

Changes:

• Implements bucketed weight transfer using a dedicated XCCL process group and ZMQ metadata channel.
• Registers the engine through the XPU plugin, with single-sender mode enabled by default.
• Adds registration and topology tests, plus usage guidance and validation limits.

File	Description
verl_hardware_plugin/​accelerators/​xpu/​registration.py	Adds guarded import to register the XCCL engine.
verl_hardware_plugin/​accelerators/​xpu/​engines/​xccl_checkpoint_engine.py	Implements rendezvous, rank topology, and bucketed weight transfer.
tests/​accelerators/​xpu/​test_xccl_checkpoint_engine.py	Tests registration, defaults, and sender/receiver rank layouts.
docs/​accelerators/​xpu/​xccl_checkpoint_engine.md	Documents configuration, hardware validation, and known limitations.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

With multi_sender=False, excluded trainer ranks still allocate unused XPU buffers—2 GiB for a 1024 MiB bucket—reducing FSDP memory headroom and risking avoidable out-of-memory failures.

1 open issue

• Avoid allocating unused XPU buffers on excluded trainer ranks · New

2 resolved since last review

• Confirm ZMQ subscriptions before publishing bucket metadata
• Align mixed-dtype weight offsets before chunk views

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The nccl alias can override a non-Intel platform’s native checkpoint engine and break weight synchronization, while multi-bucket send/receive behavior lacks regression coverage.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Guard nccl alias registration to Intel platform

verl_hardware_plugin/​accelerators/​xpu/​engines/​xccl_checkpoint_engine.py:189

load_backends() imports every accelerator's registration module, so a torch build exporting ProcessGroupXCCL reaches this nccl alias even when Intel is not the selected platform. Registering it can replace the selected platform's native checkpoint engine with one that allocates XPU tensors, breaking disaggregated weight synchronization. Keep xccl independently registered, but register the nccl compatibility alias only when Intel is the selected platform.

Add end-to-end multi-bucket weight round-trip coverage

verl_hardware_plugin/​accelerators/​xpu/​engines/​xccl_checkpoint_engine.py:514

The automated round-trip test manually constructs a single bucket and only invokes verl's merge helper, so neither this packing loop nor the receiving loop is exercised by the new tests. Padding-triggered flushes, buffer reuse across multiple buckets, and tensors split across buckets therefore have no regression coverage. Please add a round-trip test that drives send_weights() and receive_weights() with XPU allocations and transport mocked, spans at least three buckets, and verifies all reconstructed tensor values.

1 resolved since last review

• Avoid allocating unused XPU buffers on excluded trainer ranks

_{Review effort: Balanced}