Fix qwen3-vl model update in non-colocate mode

[andakai]

Summary

Try to solve this issue: #106

This PR fixes Qwen3-VL online weight update when Megatron training and vLLM
rollout run in non-colocate mode.

The main change is to make non-colocate vLLM weight update honor
--megatron-to-hf-mode bridge. When bridge mode is selected, the trainer now
uses Megatron-Bridge to export Hugging Face compatible weight names and tensors,
then sends those exported tensors to vLLM through the existing non-colocate NCCL
weight transfer path.

The PR also fixes the legacy Qwen3-VL raw converter's visual-tower name mapping
so raw mode remains usable for dense Qwen3-VL models.

Problem

Before this PR, colocate mode and non-colocate mode did not handle
--megatron-to-hf-mode bridge consistently.

colocate + bridge     -> Megatron-Bridge HF export
non-colocate + bridge -> legacy hand-written convert_to_hf path

This was fragile for Qwen3-VL because its visual tower uses names that differ
between Megatron internals and the HF/vLLM loader layout. For example, the old
raw converter could produce visual names under a Megatron-style
decoder.layers.* layout, while vLLM expects HF-style names under
model.visual.blocks.*.

The transport layer itself was not the root cause. Non-colocate NCCL transfer
only sends the names and tensors it receives. The problem was that the
non-colocate bridge configuration still bypassed Megatron-Bridge conversion and
could send vLLM parameter names that its Qwen3-VL loader did not recognize.

Changes

1. Use Megatron-Bridge in non-colocate bridge mode

File:

slime/backends/megatron_utils/update_weight/update_weight_from_distributed.py

Changes:

• Store weights_getter in UpdateWeightFromDistributed.
• Create HfWeightIteratorBase when args.megatron_to_hf_mode == "bridge".
• Dispatch _sync_weights_to_rollout_engines() to a bridge-specific sync path
  when the iterator exists.
• In the bridge sync path:
  • call self.weights_getter() to get Megatron local weights,
  • call self._hf_weight_iterator.get_hf_weight_chunks(...),
  • let Megatron-Bridge produce HF-compatible names and tensors,
  • send each exported chunk to vLLM using the existing non-colocate NCCL update
    helpers.

The bridge path is now:

weights_getter
  -> HfWeightIteratorBase / Megatron-Bridge export
  -> HF-compatible named tensors
  -> existing non-colocate NCCL transfer to vLLM

The raw path still uses the existing non-colocate flow:

named_params_and_buffers
  -> all_gather_param
  -> convert_to_hf
  -> existing non-colocate NCCL transfer to vLLM

2. Fix Qwen3-VL raw visual-tower mappings

File:

slime/backends/megatron_utils/megatron_to_hf/qwen3_vl.py

The legacy raw converter now maps Qwen3-VL visual parameters to the HF/vLLM
layout more accurately:

vision_model.decoder.layers.*                  -> model.visual.blocks.*
vision_model.decoder.deepstack_merger_list.*   -> model.visual.deepstack_merger_list.*
vision_model.merger.patch_norm.*               -> model.visual.merger.norm.*
vision_model.merger.linear_fc{1,2}.*           -> model.visual.merger.linear_fc{1,2}.*
vision_model.patch_embed.proj.*                -> model.visual.patch_embed.proj.*
vision_model.pos_embed.weight                  -> model.visual.pos_embed.weight

This keeps raw mode runnable for dense Qwen3-VL while bridge remains the
preferred path for Qwen3-VL non-colocate runs.

Why Megatron-Bridge

Megatron-Bridge is model-aware. For models like Qwen3-VL, the conversion from
Megatron parameter names to HF/vLLM names depends on multimodal model structure:
language layers, vision blocks, merger layers, patch embedding, multimodal
rotary embeddings, and model-specific layernorm names.

Maintaining all of that as hand-written slime conversion logic is brittle.
Using Megatron-Bridge in bridge mode makes Bridge the source of truth for
model-specific HF export, while slime remains responsible for transport to
vLLM.

This matches the same separation used by SkyRL-style Megatron weight sync:

• Bridge handles model-specific conversion and HF parameter naming.
• The rollout update path handles transport to the inference engine.

Behavior after this PR

colocate + bridge     -> Megatron-Bridge HF export
non-colocate + bridge -> Megatron-Bridge HF export
non-colocate + raw    -> legacy convert_to_hf path, with improved Qwen3-VL visual mappings

For Qwen3-VL geo3k scripts, the configured mode is bridge:

examples/geo3k_vlm/run_geo3k_vlm.sh      --megatron-to-hf-mode bridge
examples/geo3k_vlm/run_geo3k_vlm_sft.sh  --megatron-to-hf-mode bridge

Validation

Functional smoke: Qwen3-VL non-colocate bridge

Goal: verify that non-colocate vLLM rollout can update Qwen3-VL weights through
Megatron-Bridge.

Result:

Item	Result
Model	Qwen3-VL-2B-Instruct
Mode	--megatron-to-hf-mode bridge
Backend	Megatron train + non-colocate vLLM rollout
Train / rollout split	1 train GPU, 1 rollout GPU
Initial /update_weights	passed, HTTP 200
Post-train /update_weights	passed, HTTP 200
Rollout	passed, generated samples
Train	passed, actor train step 0 completed
Visual key mismatch	not observed

Key successful log signals:

colocate=False
Loading from ... Qwen3VLBridge
Using Megatron-Bridge HF weight export for non-colocate vLLM weight sync
POST /start_weight_update HTTP/1.1" 200 OK
POST /update_weights HTTP/1.1" 200 OK
POST /finish_weight_update HTTP/1.1" 200 OK
Converting to HuggingFace ... Qwen3VLBridge

Update-time comparison: Qwen3-4B

Goal: measure the overhead of bridge mode on a text-only dense model.

Setup:

• Model: Qwen3-4B
• Train / rollout split: 1 train GPU, 1 rollout GPU
• Repeats: 5 consecutive actor_model.update_weights() calls
• Backend: non-colocate vLLM rollout

Result:

Model	TP	EP	Mode	Times (s)
Qwen3-4B	1	1	raw	1.5, 0.3, 0.3, 0.3, 0.3
Qwen3-4B	1	1	bridge	1.7, 0.5, 0.5, 0.5, 0.5

Interpretation:

• Bridge adds about 0.2s steady-state overhead in this setup.
• Raw remains available for users that prefer the faster hand-written converter
  on models where it is known to be correct.

Update-time comparison: Qwen3-VL-4B-Instruct

Goal: measure the same overhead on the target multimodal model.

Setup:

• Model: Qwen3-VL-4B-Instruct
• HF checkpoint:
  /mnt/data2/dakai/huggingface_cache/hub/models--Qwen--Qwen3-VL-4B-Instruct/snapshots/ebb281ec70b05090aa6165b016eac8ec08e71b17
• Torch distributed checkpoint:
  /mnt/data2/dakai/vime-fix_qwen3vl_modelupdate/codex_tmp/Qwen3-VL-4B-Instruct_torch_dist
• Train / rollout split: 1 train GPU, 1 rollout GPU
• Repeats: 5 consecutive actor_model.update_weights() calls
• Backend: non-colocate vLLM rollout

Result:

Model	TP	EP	Mode	Times (s)
Qwen3-VL-4B-Instruct	1	1	raw	1.9, 0.5, 0.5, 0.4, 0.5
Qwen3-VL-4B-Instruct	1	1	bridge	3.1, 0.8, 0.8, 0.8, 0.8

Both modes completed:

• 5 successful /start_weight_update calls
• 5 successful /finish_weight_update calls
• 90 successful /update_weights chunk updates
• no traceback
• no KeyError

Interpretation:

• Bridge adds about 0.3s steady-state overhead in this setup.
• The overhead is acceptable for Qwen3-VL because bridge avoids fragile
  hand-written visual parameter mapping in the correctness-sensitive path.

TP compatibility

Goal: verify non-colocate update with Megatron TP>1.

Result:

Model	Train GPUs	Rollout GPUs	TP	EP	Mode	Result	update_weights time	Log path
Qwen3-4B	2	1	2	1	bridge	passed weight-sync smoke	1.8s	codex_tmp/qwen3_4b_update_compare_logs/bridge_tp2_eager_repeat1.log
Qwen3-VL-4B-Instruct	4	1	4	1	bridge	passed weight-sync smoke	2.1s	codex_tmp/qwen3vl_4b_update_compare_logs/bridge_tp4_repeat1.log
Qwen3-VL-4B-Instruct	4	1	4	1	raw	passed weight-sync smoke	2.3s	codex_tmp/qwen3vl_4b_update_compare_logs/raw_tp4_repeat1_retry.log

Both Qwen3-VL-4B-Instruct TP=4 runs completed one repeat update with:

• /start_weight_update 200
• /finish_weight_update 200
• 18 successful /update_weights chunk updates
• no traceback
• no KeyError

Checklist

• Non-colocate bridge path implemented.
• Qwen3-VL raw visual-tower mapping fixed.
• Qwen3-VL non-colocate bridge smoke passed.
• Qwen3-4B raw vs bridge update-time comparison completed.
• Qwen3-VL-4B-Instruct raw vs bridge update-time comparison completed.
• Qwen3-4B TP=2 bridge weight-sync smoke passed.
• Qwen3-VL-4B-Instruct TP=4 bridge weight-sync smoke passed.
• Qwen3-VL-4B-Instruct TP=4 raw weight-sync smoke passed.
• Target-file pre-commit checks passed.

[gemini-code-assist]

Code Review

This pull request adds weight conversion mappings for Qwen3-VL vision models to Hugging Face format and integrates Megatron-Bridge HF weight export for non-colocate vLLM weight synchronization. A critical feedback points out a potential deadlock issue when pipeline parallelism is greater than 1, caused by calling dist.barrier inside the chunk loop where different ranks may have a different number of chunks; moving the barrier outside the loop is recommended to resolve this.