[gcanlin]

Overview

GLM-5.3-Flash's checkpoint sets index_share_for_mtp_iteration: true: the MTP head computes the sparse indexer top-k on draft step 0 and reuses it on the later steps. Since #57387 the flag is no longer visible on the draft config, so sharing has been silently off. This restores it.

Not a duplicate: I searched open PRs for index_share_for_mtp_iteration, share_mtp_topk_indices, glm5_next_mtp and GLM-5.3-Flash MTP and found nothing covering this.

Claims

• With MTP on GLM-5.3-Flash, draft steps 1+ reuse step 0's top-k again, as before [Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor #57387.
• Accepted tokens per draft step go from 3.28 to 3.42 on GSM8K (MTP 5), and decode throughput is +2.7% at a fixed acceptance length.

Validation

4x B200, nightly 8cbd5d03, TP1 x DP4 with expert parallel and #60895's MegaMoE path, MTP 5, one run each.

	sharing off (main)	sharing on
GSM8K (lm-eval, 5-shot, 1,319 questions)	0.9704 ± 0.0047	0.9750 ± 0.0043
draft steps for that GSM8K run	58,471	56,072
accepted tokens per draft step	3.28	3.42
random 1024 in / 1024 out, concurrency 128, output tok/s	11,639	11,952
mean TPOT	10.13 ms	9.90 ms

The GSM8K rows use real acceptance; the throughput rows fix it with synthetic_acceptance_length 3.61.

The GPU runs switched the same proposer flag (share_mtp_topk_indices) through a local patch that read it from text_config. With this PR's code I ran:

• tests/config/test_speculative_draft_hf_overrides.py: 17 passed; the new test fails without the change. Run against nightly 8cbd5d03, whose speculative.py is identical to main's.
• SpeculativeConfig.hf_config_override on the real checkpoint config: index_share_for_mtp_iteration is True (missing before).
• pre-commit on the changed files.

Commands

vllm serve zai-org/GLM-5.3-Flash \
  --tensor-parallel-size 1 --data-parallel-size 4 --enable-expert-parallel --max-num-seqs 96 \
  --language-model-only --kv-cache-dtype fp8 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --kernel-config '{"enable_flashinfer_autotune":true,"moe_backend":"deep_gemm_mega_moe"}' \
  --max-num-batched-tokens 8192 --gpu-memory-utilization 0.97

lm_eval run --model local-chat-completions --tasks gsm8k --num_fewshot 5 \
  --apply_chat_template --fewshot_as_multiturn --gen_kwargs max_gen_toks=2048 temperature=0.0
# accepted per draft step = vllm:spec_decode_num_accepted_tokens / vllm:spec_decode_num_drafts

# throughput rows: same server with
#   "rejection_sample_method":"synthetic","synthetic_acceptance_length":3.61
vllm bench serve --backend openai --endpoint /v1/completions --model zai-org/GLM-5.3-Flash \
  --dataset-name random --random-input-len 1024 --random-output-len 1024 --random-range-ratio 0 \
  --num-prompts 512 --num-warmups 64 --max-concurrency 128 --request-rate inf --ignore-eos \
  --temperature 0 --seed 42

The remaining serve flags (CUDA graph sizes, KV offload, --prefix-match-unit 16) are the ones listed in #60895.

Details

• Root cause: the MTP proposers read the flag from the draft config (mtp/speculator.py, llm_base_proposer.py). The in-tree Glm5NextConfig forwarded text_config attributes to the top level; the upstream Transformers config that [Model] Use upstream GLM-5.3 and Qwen4-Exp configs and processor #57387 switched to does not. That PR moved num_nextn_predict_layers to get_text_config() but not this flag.
• Fix: promote the flag onto the draft config in the glm5_next override, as the Qwen4-Exp override already does. An explicit index_share_for_mtp_iteration in --speculative-config still takes precedence.
• The GSM8K difference is within one standard error; I read it as no regression, not as an accuracy gain.
• Limitations: validated on NVIDIA SM100 only, not on ROCm. Acceptance was measured on GSM8K only.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[Isotr0py]

Thanks for fixing! #57387 breaks so many things. 😅

[Isotr0py]

/ci run --allow-stale

[github-actions]

✅ Triggered Buildkite CI #94085 for commit f71577cad82e.

⚠️ This PR is 6 commits behind upstream main. Running CI at your own risk because --allow-stale was requested; outdated CI configuration may cause failures. Before merging, merge or rebase onto the latest main, then rerun /ci run on the latest PR commit.

[JaredforReal]

LGTM, thanks for fixing~ @gcanlin

[gcanlin]

/ci retry

[github-actions]

✅ Queued 3 failed job(s) for retry in Buildkite CI #94085.