Skip to content

[recipe, hardware] fix: drop the VLLM_ROCM_DSV4_BPRESHUFFLE workaround for DeepSeek-V4 on ROCm #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
xiaohong42 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[recipe, hardware] fix: drop the VLLM_ROCM_DSV4_BPRESHUFFLE workaround for DeepSeek-V4 on ROCm #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4e3670b
[recipe, hardware] fix: drop the DeepSeek-V4 recipe's vLLM B-preshuff…
xiaohong42 Nov 2, 2026
62b5519
[hardware] fix: stop defaulting VLLM_ROCM_DSV4_BPRESHUFFLE=0 for ROCm…
xiaohong42 Nov 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 1 addition & 10 deletions examples/grpo_trainer/run_deepseek_v4_veomni.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#
# Environment:
# - transformers==5.9.0
# - vllm==0.24.0; on AMD ROCm, a vLLM with VLLM_ROCM_DSV4_BPRESHUFFLE (vllm-project/vllm#63604)
# - vllm==0.24.0
# - veomni==0.1.12

set -xeuo pipefail
Expand Down Expand Up @@ -174,15 +174,6 @@ if python -c "import sys, torch; sys.exit(torch.version.hip is None)" 2>/dev/nul
gfx_version=$(awk '$1 == "gfx_target_version" && $2 > 0 {print $2; exit}' /sys/class/kfd/kfd/topology/nodes/*/properties 2>/dev/null || true)
if [ -n "${VLLM_ROCM_USE_AITER:-}" ] || [[ "$gfx_version" == 904?? ]]; then
add_worker_env VLLM_ROCM_USE_AITER "${VLLM_ROCM_USE_AITER:-1}"
aiter=${VLLM_ROCM_USE_AITER:-1}
# With AITER, a vLLM without VLLM_ROCM_DSV4_BPRESHUFFLE B-preshuffles DeepSeek-V4's FP8 weights once after
# loading, and the weights verl syncs later are not reshuffled. A vLLM that cannot be imported here is not checked.
if [[ $rollout_name == vllm && (${aiter,,} == 1 || ${aiter,,} == true) ]] &&
python -c "import sys, vllm.envs as e; sys.exit('VLLM_ROCM_DSV4_BPRESHUFFLE' in e.environment_variables)" 2>/dev/null; then
echo "With AITER this vLLM B-preshuffles DeepSeek-V4's FP8 weights, so the rollouts would run on corrupted weights" \
"after the first weight sync. Use a vLLM with VLLM_ROCM_DSV4_BPRESHUFFLE (vllm-project/vllm#63604)." >&2
exit 1
fi
fi
fi

Expand Down
6 changes: 0 additions & 6 deletions verl/plugin/platform/platform_rocm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,15 +63,9 @@ def rollout_env_vars(self) -> dict[str, str]:
# routes SGLang's non-attention kernels (RMSNorm/RoPE/MoE/quant) through AITER.
# Default to "1" but honor an explicit user override (e.g. SGLANG_USE_AITER=0
# to fall back to vLLM kernels).
#
# VLLM_ROCM_DSV4_BPRESHUFFLE: with AITER, vLLM B-preshuffles three DeepSeek-V4 FP8 weights
# (attention and shared experts) once after loading, so the weight syncs that follow silently
# corrupt the rollouts. verl updates the rollout weights at runtime, so keep the checkpoint layout.
# vLLM builds without this variable (vllm-project/vllm#63604) ignore it and always preshuffle.
return {
**super().rollout_env_vars(),
"SGLANG_USE_AITER": os.environ.get("SGLANG_USE_AITER", "1"),
"VLLM_ROCM_DSV4_BPRESHUFFLE": os.environ.get("VLLM_ROCM_DSV4_BPRESHUFFLE", "0"),
}

def ray_noset_envvars(self) -> list[str]:
Expand Down
Loading