Skip to content

[recipe, hardware] fix: drop the DeepSeek-V4 recipe's vLLM B-preshuffle check that refuses every vLLM on gfx94x - #4

Open
xiaohong42 wants to merge 1 commit into
mainfrom
rocm-dsv4-veomni-main
Open

xiaohong42 wants to merge 1 commit into
mainfrom
rocm-dsv4-veomni-main

Conversation

@xiaohong42

@xiaohong42 xiaohong42 commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

What does this PR do?

Removes the vLLM check that verl-project#8371 added to examples/grpo_trainer/run_deepseek_v4_veomni.sh. Where the recipe turns on AITER (gfx94x), it exits unless the installed vLLM defines VLLM_ROCM_DSV4_BPRESHUFFLE. That variable was proposed in vllm-project/vllm#63604, which was closed without being merged, so no vLLM defines it and the recipe refuses every vLLM on gfx94x:

With AITER this vLLM B-preshuffles DeepSeek-V4's FP8 weights, so the rollouts would run on corrupted weights after the first weight sync. Use a vLLM with VLLM_ROCM_DSV4_BPRESHUFFLE (vllm-project/vllm#63604).

Redoing the B-preshuffle when weights are reloaded belongs in vLLM, so the recipe no longer inspects vLLM for it. The requirement in the recipe header goes with the check.

Checklist Before Starting

Test

On gfx942 with a vLLM that does not define VLLM_ROCM_DSV4_BPRESHUFFLE, with the trainer launch replaced by an echo: on main the recipe exits with the message above (exit code 1); with this PR it reaches the launch and still passes NCCL_DEBUG=VERSION and VLLM_ROCM_USE_AITER=1 to the workers. With VLLM_ROCM_USE_AITER=0, where the check does not run, main and this PR print the same launch arguments. bash -n passes.

API and Usage Example

No API or config change.

Design & Code Changes

  • examples/grpo_trainer/run_deepseek_v4_veomni.sh: remove the vLLM check and its requirement in the header. The worker environment block is unchanged.

Not changed here: verl/plugin/platform/platform_rocm.py still defaults VLLM_ROCM_DSV4_BPRESHUFFLE=0 for the rollout servers. No vLLM reads it, so it has no effect.

Checklist Before Submitting

@xiaohong42
xiaohong42 force-pushed the rocm-dsv4-veomni-main branch from a9be2dc to 0146022 Compare November 2, 2026 02:15
…le check that refuses every vLLM on gfx94x

Co-authored-by: Cursor <cursoragent@cursor.com>
@xiaohong42
xiaohong42 force-pushed the rocm-dsv4-veomni-main branch from 0146022 to 4e3670b Compare November 2, 2026 02:45
@xiaohong42 xiaohong42 changed the title [veomni, recipe, hardware] feat: move to VeOmni main and run the DeepSeek-V4 recipe on AMD ROCm [recipe, hardware] fix: drop the DeepSeek-V4 recipe's vLLM B-preshuffle check that refuses every vLLM on gfx94x Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant