[vllm] feat: Support LoRA for DeepSeek V4 & GLM5.3 Flash

[HollowMan6]

What does this PR do?

Depends on:

• [Bugfix] Fix DeepSeek V4 mHC broadcast buffer for dummy load vllm-project/vllm#51368
• [BugFix] lora_base_layer / routed_experts order in expert param mapping vllm-project/vllm#52552
• [Bugfix] Fix DeepSeek V4 mHC broadcast buffer for weight sync vllm-project/vllm#52626
• [LoRA] feat: Support LoRA for DeepSeek V4 vllm-project/vllm#53361
• [LoRA][Model] Add LoRA support for GLM-5.3-Flash vllm-project/vllm#56327
• feat(model): add GLM-5.3-Flash support NVIDIA-NeMo/Megatron-Bridge#6044
• GLM5.3 Flash (KDA + mHC + KPool DSA) support + FP8 NVIDIA/Megatron-LM#7054

Adds DeepSeek V4 (DSV4) LoRA support to verl's vLLM weight-sync path. Previously a merge=False LoRA base sync orphaned the .base_layer. suffix on wrapped modules and mishandled the mxfp4 quant path, so DSV4 LoRA diverged from the actor.

This reworks weight-name reconciliation to be runtime-probed (strict vs flat loader) instead of model-type-based, unwraps the LoRA-installed mxfp4 quant-method wrapper, and threads peft_config through the FP8/mxfp4 load path so the same reconciliation applies there.

Tested with Megatron backend, GPU E2E (DSV4 merge=F GRPO, 2-node): step1/step2 pearson 0.9970/0.9950, aligned to run15 merge=T 0.9952/0.9944.

Checklist Before Starting

• Search for similar PRs. Query: https://github.com/verl-project/verl/pulls?q=deepseek+v4+lora
• Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
  • {modules} include fsdp, megatron, veomni, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward, fully_async, one_step_off
  • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
  • {type} is in feat, fix, refactor, chore, test
  • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
  • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

pytest tests/utils/test_vllm_weight_name_normalization_on_cpu.py
# GPU E2E: DSV4 merge=F GRPO 2-node — step1/step2 pearson 0.9970/0.9950 vs run15 merge=T 0.9952/0.9944

Unit tests cover strict-vs-flat loader detection, per-expert routed leaves, and non-LoRA params on wrapped modules (e.g. DSV4 gate.tid2eid / gate.e_score_correction_bias).

API and Usage Example

No CLI/API change. Behavior is automatic: resolve_weight_name probes the inner model's load_weights at runtime and toggles .base_layer. accordingly. For DSV4 merge=False LoRA, no extra config is needed beyond enabling LoRA + merge=False.

# DSV4 merge=False LoRA (existing config; no new flags)
actor_rollout_ref.model.lora.enable=True
actor_rollout_ref.model.lora.merge=False
actor_rollout_ref.model.lora.target_modules='["linear_wkv","linear_wgate","linear_wq_b","linear_weights_proj","linear_kv_proj","linear_q_down_proj","linear_q_up_proj","linear_proj","linear_fc1","linear_fc2","router"]'

Design & Code Changes

Runtime-probed .base_layer. reconciliation (verl/utils/vllm/utils.py)

• Adds _inner_load_weights_is_strict: probes the inner transformer's load_weights bytecode (co_names) to decide strict (keeps .base_layer., DeepseekV2/DSV4) vs flat (strips it, Llama/Qwen3.5). Cached per class.
• resolve_weight_name toggles one .base_layer. segment loader-style-aware, re-adds the suffix for non-LoRA params on a wrapped module, and reconciles per-expert routed leaves (strict keep / flat strip, gated by _HAS_LORA_BASE_LAYER_PREFIX).

mxfp4 LoRA unwrap (verl/utils/vllm/vllm_fp4_utils.py)

• Adds _resolve_mxfp4_quant_method: unwraps the FusedMoEModularMethod wrapper that merge=False LoRA installs over Mxfp4MoEMethod (stored as old_quant_method) so staging, layout derivation, and process_weights_after_loading repack reach the legacy method — the modular wrapper inherits a no-op post-load and would otherwise skip the repack.

FP8/mxfp4 name reconciliation (verl/utils/vllm/vllm_quant_utils.py, verl/workers/rollout/vllm_rollout/utils.py)

• load_quanted_weights gains a peft_config parameter; when set (LoRA base sync), names run through resolve_weight_name against live params/buffers so .base_layer. is reconciled on the FP8/mxfp4 path too (e.g. DSV4 compressor.fused_wkv_wgate). Callers thread peft_config for both base and drafter loads.

LoRA MoE prefix (verl/utils/vllm/patch.py)

• Removes the eager lora_base_layer_prefix clearing block; the prefix is now reconciled through resolve_weight_name instead.

Tests (tests/utils/test_vllm_weight_name_normalization_on_cpu.py)

• Expanded coverage for strict-vs-flat loader detection, per-expert routed leaves, and non-LoRA params on wrapped modules.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

• Read the Contribute Guide.
• Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
• Add / Update the documentation.
• Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...
• Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace. (If not accessible, please try the Feishu group (飞书群).)
• If your PR is related to the recipe submodule, please also update the reference to the submodule commit via git submodule update --remote or cd recipe && git pull origin main.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.