diff --git a/docs/accelerators/xpu/README.md b/docs/accelerators/xpu/README.md index 638b804c..8512fffa 100644 --- a/docs/accelerators/xpu/README.md +++ b/docs/accelerators/xpu/README.md @@ -1,6 +1,6 @@ # Intel XPU User Guide -Last updated: 10/09/2026. +Last updated: 10/30/2026. ## Introduction @@ -17,6 +17,7 @@ verl_hardware_plugin/accelerators/xpu/ ├── engines/fsdp_xpu.py # FSDP/FSDP2 actor + critic engines ├── engines/megatron_xpu.py # Megatron engine registration (Work in Progress — not yet validated end-to-end) ├── patches/reduce_avg_allreduce_patch.py # xccl ReduceOp.AVG workaround +├── patches/ray_device_pin_patch.py # pins each Ray worker to its assigned device (otherwise every rank lands on xpu:0) ├── profilers/itt_profile_xpu.py # ITT range markers + VtuneProfiler └── profilers/register_vtune.py # claims `profiler.tool: vtune` ``` @@ -50,11 +51,16 @@ docs/accelerators/xpu/ | Variable | Purpose | |----------|---------| -| `VERL_PLATFORM` | Set to `intel` to force platform selection instead of relying on auto-detection | +| `VERL_PLATFORM` | Optional. Auto-detection already selects `intel` once the plugin is installed and `torch.xpu` is available; set `intel` only to force the choice | | `VERL_USE_EXTERNAL_PLUGINS` | `auto` (default) discovers this plugin via entry_points; `none` disables discovery | | `ZE_AFFINITY_MASK` | Selects physical device indices visible to this process, e.g. `0,1` | | `ONEAPI_DEVICE_SELECTOR` | Must be left unset alongside `ZE_AFFINITY_MASK` — see [quick_start.md](./quick_start.md) | +## Docker + +An Intel GPU image (verl + this plugin + vLLM) is built from `docker/xpu/` in verl-core. A system oneAPI install is +not needed at runtime. Build and run commands are in the [installation guide](./install_guidance.md). + ## Related Documentation - [verl plugin system](../../development.md) diff --git a/docs/accelerators/xpu/install_guidance.md b/docs/accelerators/xpu/install_guidance.md index c5ffd3c7..0e3da5c8 100644 --- a/docs/accelerators/xpu/install_guidance.md +++ b/docs/accelerators/xpu/install_guidance.md @@ -7,13 +7,14 @@ in verl-core for the plugin pattern this follows. ## Prerequisites -- PyTorch with XPU support (`torch.xpu.is_available() == True`) -- vLLM with XPU kernels — either the prebuilt wheel from vLLM's XPU wheel - index (`pip install "vllm==+xpu" --extra-index-url - https://wheels.vllm.ai//xpu`) or built from source with - `VLLM_TARGET_DEVICE=xpu`. The default PyPI `pip install vllm` wheel does - not ship XPU kernels. -- oneCCL runtime for the `xccl` distributed backend +- An Intel GPU with the Level Zero driver stack on the host (compute-runtime >= 26.18 and Level Zero). + **A system oneAPI install is not required**: PyTorch's XPU wheel and vLLM's XPU wheel bring their own SYCL + runtime and oneCCL (the `xccl` backend). This was checked by hiding `/opt/intel/oneapi` and the oneAPI + environment variables: platform detection, an `xccl` all_reduce and a GRPO training step all still worked. +- PyTorch with XPU support (`torch.xpu.is_available() == True`). +- vLLM with XPU kernels, from vLLM's XPU wheel index (`pip install "vllm==+xpu" + --extra-index-url https://wheels.vllm.ai//xpu`) or built from source with + `VLLM_TARGET_DEVICE=xpu`. The default PyPI `pip install vllm` wheel does not ship XPU kernels. **verl-core version note:** the one `PlatformBase` hook this plugin implements, `profiler_markers()`, was merged into `verl-project/verl:main` on @@ -31,26 +32,95 @@ plugin-side monkeypatch over `torch.distributed.all_reduce` ## 1. Install verl and verl-hardware-plugin +Pick one. Both install the same set of packages (torch 2.14.0+xpu, vLLM 0.32.0+xpu, transformers 5.16.1): the Docker +image runs the commands of Option B. + +### Option A: Docker (recommended) + +The image is [`docker/xpu/Dockerfile.stable.vllm`](https://github.com/verl-project/verl/blob/main/docker/xpu/Dockerfile.stable.vllm) +in verl-core. It installs verl and this plugin together, so nothing else is needed. Pin the two checkouts with +`--build-arg VERL_REF= --build-arg PLUGIN_REF=`; for release builds add `--no-cache` so the system +certificates, `pip`, `certifi` and `uv` are fetched fresh. + ```bash -# Install verl (needs the profiler_markers() hook, on main since 2026-10-08 — see note above) -git clone https://github.com/verl-project/verl.git -cd verl -pip install -e . - -# Install verl-hardware-plugin -git clone https://github.com/verl-project/verl-hardware-plugin.git -cd verl-hardware-plugin -pip install -e . +git clone https://github.com/verl-project/verl.git && cd verl +docker build -f docker/xpu/Dockerfile.stable.vllm -t verl:xpu-vllm032 . +docker run --rm -it --device /dev/dri --ipc=host --privileged verl:xpu-vllm032 ``` -No environment variable is required beyond this. The plugin is -auto-discovered by verl through the `verl.plugins` setuptools entry_points +The image is built in stages (`drivers` -> `runtime-base` -> `builder` -> `runtime`); only the finished `/opt/venv` and +the verl / plugin checkouts (`/opt/verl`, `/opt/verl-hardware-plugin`) are shipped. It contains: + +- the Intel GPU user-mode driver (compute-runtime 26.27, IGC 2.38.2, Level Zero 1.32), matching vLLM 0.32.0's XPU image; +- gcc/g++ and the Python headers, `ffmpeg`, `libnuma-dev` and `libze-dev` (see the system packages under Option B); +- `CCL_SYCL_ALLGATHERV_TMP_BUF`, `CCL_SYCL_ALLREDUCE_TMP_BUF`, `CCL_SYCL_REDUCE_SCATTER_TMP_BUF` and + `CCL_SYCL_BROADCAST_TMP_BUF`, all set to `1`: they stop oneCCL from caching IPC handles of freed buffers, which + leaked host memory on multi-GPU runs. Set them yourself when you install from source. + +The build needs a verl checkout whose `requirements.txt` allows transformers >= 5.16.1 (verl `main` since PR #8390). +An older `VERL_REF` is rejected by a check at the end of the build. + +### Option B: From source + +verl pins its own versions, and `pip install -e .` on a verl checkout can replace the XPU build of torch with a +different one (it also moves `transformers` and `tensordict` to verl's pins). Install the dependencies as below and +add verl and the plugin with `--no-deps`. The Dockerfile above is the reference for these steps. + +System packages on the host: `g++`/`gcc` and the Python 3.12 headers (Triton compiles its launcher stubs at first +use), `libnuma-dev`, `libze-dev` (it provides the unversioned `libze_loader.so` that oneCCL opens; without it `xccl` +fails with "ze_data was not initialized"), and `ffmpeg` (the FFmpeg shared libraries that `torchcodec` needs to decode +video). The Docker image also installs `libhwloc15`, `libpciaccess0`, `libigsc1` and `intel-metrics-library` (from +Intel's `ppa:kobuk-team/intel-graphics`) for the Sysman PCI/NUMA query that pyzes makes; install them as well. + +```bash +# Use pip >= 26.2.2 and uv >= 0.13.4 (both carry the corrected CA bundle). +uv venv --python 3.12 --seed /opt/venv && source /opt/venv/bin/activate +export UV_INDEX_URL=https://download.pytorch.org/whl/xpu UV_EXTRA_INDEX_URL=https://pypi.org/simple \ + UV_INDEX_STRATEGY=unsafe-best-match VLLM_TARGET_DEVICE=xpu +export LD_LIBRARY_PATH=/opt/venv/lib:/usr/local/lib # without it oneCCL fails: "could not load any plugin" +export CCL_SYCL_ALLGATHERV_TMP_BUF=1 CCL_SYCL_ALLREDUCE_TMP_BUF=1 \ + CCL_SYCL_REDUCE_SCATTER_TMP_BUF=1 CCL_SYCL_BROADCAST_TMP_BUF=1 + +# vLLM XPU wheel (the PyPI wheel has no XPU kernels). --extra-index-url on the command line replaces +# UV_EXTRA_INDEX_URL, so PyPI is repeated. +uv pip install "vllm==0.32.0+xpu" \ + --extra-index-url https://wheels.vllm.ai/0.32.0/xpu --extra-index-url https://wheels.vllm.ai/xpu \ + --extra-index-url https://pypi.org/simple + +git clone https://github.com/verl-project/verl.git && cd verl +echo 'pyzes>=0.1.2' > /tmp/override-pyzes.txt # torch pins pyzes==0.1.1; the device-memory query needs >= 0.1.2 + +# --no-config: run inside a verl checkout, uv would otherwise read verl's pyproject [tool.uv] overrides +# (for the CUDA stack) and they would beat the XPU pins below. transformers is not pinned: it comes from verl's +# requirements.txt window and vLLM's (>= 5.16.1), and the [kernels] extra brings the matching kernels release. +uv --no-config pip install -r requirements.txt \ + "torch==2.14.0+xpu" "transformers[kernels]" \ + dpctl "pyzes>=0.1.2" triton-xpu "pip>=26.2.2" "certifi>=2026.10.19" qwen_vl_utils mathruler \ + --override /tmp/override-pyzes.txt +uv --no-config pip install --no-deps -e . + +# liger-kernel pulls the CUDA triton, which shares triton/ with triton-xpu: keep triton-xpu only. +TRITON_XPU="$(uv pip freeze | grep -i '^triton-xpu==')" +uv pip uninstall triton +uv pip install --reinstall --no-deps "${TRITON_XPU}" --extra-index-url https://download.pytorch.org/whl/xpu + +# This plugin, on top of that environment: +cd .. && git clone https://github.com/verl-project/verl-hardware-plugin.git && cd verl-hardware-plugin +pip install --no-deps -e . +``` + +`--no-deps` keeps the plugin install from touching the torch and verl already in place. The plugin needs no +environment variable: it is auto-discovered by verl through the `verl.plugins` setuptools entry_points group declared in this repo's `pyproject.toml` -(`[project.entry-points."verl.plugins"] hardware = "verl_hardware_plugin"`), -which verl loads by default (`VERL_USE_EXTERNAL_PLUGINS=auto`). Set -`VERL_USE_EXTERNAL_PLUGINS=none` to disable discovery, e.g. to isolate a bug -to this plugin, or `VERL_PLATFORM=intel` to force platform selection instead -of relying on auto-detection. +(`[project.entry-points."verl.plugins"] hardware = "verl_hardware_plugin"`), which verl loads by default +(`VERL_USE_EXTERNAL_PLUGINS=auto`). + +### Platform selection + +With the plugin installed, verl picks the Intel platform by itself on a host where `torch.xpu` is available. +`VERL_PLATFORM=intel` is only an override, for forcing the choice (for example on a host with several kinds of +accelerator). `VERL_USE_EXTERNAL_PLUGINS=none` disables discovery, e.g. to isolate a bug to this plugin; verl then +falls back to `nvidia`. ## 2. Verify the Install @@ -63,9 +133,10 @@ print('device:', p.device_name, '/ vendor:', p.vendor_name) ``` Expected on Intel GPU: `device: xpu / vendor: intel`. If it instead falls -back to `nvidia`, the plugin was not discovered — check that `pip install` -completed without error and that no `VERL_USE_EXTERNAL_PLUGINS=none` is set -in the environment. +back to `nvidia`, either the plugin was not discovered (check that `pip install` +completed without error and that no `VERL_USE_EXTERNAL_PLUGINS=none` is set) or +torch lost its XPU build (check `python3 -c "import torch; print(torch.__version__, torch.xpu.is_available())"`; +a plain version like `2.14.1` without `+xpu` means a dependency install replaced it). ## Next Steps diff --git a/docs/accelerators/xpu/profiling.md b/docs/accelerators/xpu/profiling.md index aa88d2fe..2f90c741 100644 --- a/docs/accelerators/xpu/profiling.md +++ b/docs/accelerators/xpu/profiling.md @@ -1,6 +1,6 @@ # Intel VTune Profiling Guide -Last updated: 10/09/2026. +Last updated: 10/30/2026. This guide describes Intel VTune (ITT) profiling support in `verl-hardware-plugin` for Intel XPU. There is no PyTorch-native profiler activity for Intel XPU, so @@ -31,15 +31,10 @@ needing to know about ITT or XPU itself, while `VtuneProfiler` is registered for (`hotspots` reads `perf_event_open`, so a container typically needs `kernel.perf_event_paranoid <= 2` or `CAP_PERFMON`). - Install `verl` (`main` includes `profiler_markers()` as of 2026-10-08) and - `verl-hardware-plugin` in editable mode, and enable the plugin in Ray - runtime: - - ```yaml - working_dir: ./ - excludes: ["/.git/"] - env_vars: - VERL_USE_EXTERNAL_MODULES: "verl_hardware_plugin" - ``` + `verl-hardware-plugin` (see [install_guidance.md](./install_guidance.md)). The plugin is auto-discovered in + every process, including Ray workers, through its `verl.plugins` entry point, so no Ray `runtime_env` setting is + needed. (`VERL_USE_EXTERNAL_MODULES: "verl_hardware_plugin"` is an equivalent explicit way to load it, and is + only useful if entry-point discovery is disabled.) ## Enabling VTune diff --git a/docs/accelerators/xpu/quick_start.md b/docs/accelerators/xpu/quick_start.md index f660cdfa..3307e44d 100644 --- a/docs/accelerators/xpu/quick_start.md +++ b/docs/accelerators/xpu/quick_start.md @@ -48,7 +48,8 @@ python3 examples/data_preprocess/gsm8k.py --local_save_dir ~/data/gsm8k ## 3. Launch Training ```bash -export ZE_AFFINITY_MASK=0,1 # select physical device indices +export ZE_AFFINITY_MASK=0 # select physical device indices; for two GPUs use 0,1 together with + # trainer.n_gpus_per_node=2 and +ray_kwargs.ray_init.num_gpus=2 unset ONEAPI_DEVICE_SELECTOR # must NOT be set python3 -m verl.trainer.main_ppo \ @@ -72,24 +73,26 @@ python3 -m verl.trainer.main_ppo \ actor_rollout_ref.rollout.enforce_eager=True \ trainer.n_gpus_per_node=1 \ trainer.nnodes=1 \ + actor_rollout_ref.actor.ppo_mini_batch_size=16 \ + actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=4 \ + actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=4 \ + actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=4 \ trainer.total_epochs=1 \ + trainer.total_training_steps=2 \ + trainer.test_freq=-1 \ + trainer.save_freq=-1 \ +ray_kwargs.ray_init.num_gpus=1 ``` -Expected (Qwen2.5-0.5B-Instruct, GSM8K, batch 16, 1 step, 1x Arc Pro B60): +This is a smoke test: `total_training_steps=2` stops after two steps (without it, `total_epochs=1` over GSM8K at +batch 16 is about 467 steps). Raise it for a longer run. A `DataLoader worker ... is killed by signal` traceback +printed at exit is shutdown noise after training has finished. -```text -timing_s/step: ~51 timing_per_token_ms/gen: ~0.35 -perf/throughput: ~148 tok/s -``` - -On 2 GPUs with the same batch size (throughput scales with larger batch — -this keeps the comparison apples-to-apples rather than showing peak -throughput): +Expected: two training steps are logged, each with metrics such as `timing_s/step` and `perf/throughput`, and the +process exits. Step times and throughput depend on the GPU and the software stack, so none are listed here. -```text -timing_s/step: ~93 perf/throughput: ~41 tok/s -``` +Two GPUs: set `ZE_AFFINITY_MASK=0,1`, `trainer.n_gpus_per_node=2` and `+ray_kwargs.ray_init.num_gpus=2`, and keep +everything else the same. `perf/throughput` is reported per GPU. ## 4. Run the Plugin Test Suite @@ -108,9 +111,9 @@ Intel GPU attached. | Inference engine | vLLM validated | SGLang: `update_weights` — Work in Progress | | Trainer backend | FSDP, FSDP2 | Megatron — Work in Progress | | Algorithms | GRPO, PPO, SFT | Validated on GSM8K / Qwen2.5 | -| Hardware (1-GPU) | Validated | Arc Pro B60: 51.2 s/step, 148.2 tok/s (batch 16, Qwen2.5-0.5B) | -| Hardware (2-GPU) | Validated | Arc Pro B60 x2: 93.0 s/step at same batch size | -| Multi-node | Work in Progress | — | +| Hardware (1-GPU) | Validated | Arc Pro B60 (the launch command above) | +| Hardware (2-GPU) | Validated | Arc Pro B60 x2; FSDP2 and sleep mode also run | +| Multi-node | Work in Progress | Two-node collectives work only with `CCL_ENABLE_SYCL_KERNELS=0` (then `ReduceOp.AVG` is unavailable); with oneCCL defaults a 64 MiB `all_reduce` across nodes hangs. A multi-node verl training run is not validated | | Profiler | Intel VTune (ITT) — Validated | `profiler_markers()` hook merged into verl-core `main` on 2026-10-08; VTune collector capture itself still untested, see [profiling.md](./profiling.md) | ## Next Steps