Skip to content

[sglang-miles] Cherry-pick RL/VLM/Qwen3-MoE fixes - #27602

Open
ByronHsu wants to merge 50 commits into
sgl-project:sglang-milesfrom
ByronHsu:byron/cherry-pick-rl-vlm-qwen3-mtp-sglang-miles
Open

ByronHsu wants to merge 50 commits into
sgl-project:sglang-milesfrom
ByronHsu:byron/cherry-pick-rl-vlm-qwen3-mtp-sglang-miles

Conversation

@ByronHsu

@ByronHsu ByronHsu commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

Why?

Bring the RL, VLM token-id, and Qwen3-MoE MTP fixes that landed on OSS main onto the OSS sglang-miles branch.

How?

  • Cherry-pick the Kimi K2.5 weight hook forwarding fix and preserve the newer sglang-miles wrapper behavior.
  • Port the pre-tokenized VLM no-retokenize path, including Kimi image token-count handling.
  • Add Qwen3-MoE MTP model wiring while keeping existing sglang-miles DFlash/Qwen3.5 support.
  • Port the mixed optional-output alignment fix into the current scheduler output processor path instead of restoring deleted output_streamer.py.

Test plan

  • python3 -m py_compile python/sglang/srt/models/kimi_k25.py python/sglang/srt/multimodal/processors/base_processor.py python/sglang/srt/multimodal/processors/kimi_common.py python/sglang/srt/models/qwen3_moe.py python/sglang/srt/models/qwen3_moe_mtp.py python/sglang/srt/speculative/eagle_worker.py python/sglang/srt/speculative/eagle_worker_v2.py python/sglang/srt/managers/scheduler_output_processor_mixin.py python/sglang/srt/managers/tokenizer_manager.py test/registered/vlm/test_token_id_retokenize_e2e.py test/registered/rl/test_return_routed_experts.py
  • Could not run focused pytest locally: python3 -m pytest ... fails because this shell does not have pytest installed; import checks are also blocked by missing local dependency orjson.

Generated with Codex


CI States

Latest PR Test (Base): ❌ Missing run-ci label -- add it to run CI tests.
Latest PR Test (Extra): ❌ Blocked -- run-ci is required first.
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

hzh0425 and others added 30 commits May 15, 2026 14:30
Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
…g kv_args (sgl-project#25380)

Co-authored-by: JoeLee314 <liqichao@baidu.com>
…gl-project#24704)

Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: xuyongfei <xuyongfei.xyf@antgroup.com>
layer_mapping was changed from a compact list to a sparse array
containing None for out-of-stage layers by PP support (sgl-project#24704),
but the HiCache consumer was not updated to handle None entries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blackwell SM100/SM120 deepseek dependencies Pull requests that update a dependency file hicache Hierarchical Caching for SGLang lora memory-pool model-gateway Multi-modal multi-modal language model unified-radix-cache

Projects

None yet

Development

Successfully merging this pull request may close these issues.