Fix TP6 virtual padding for GLM MTP drafts - #96
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
05f50ae to
522c626
Compare
|
Validated after rebasing directly onto
The release uses commit |
|
/review |
|
Superseded by |
Summary
Root cause
The GLM target config was padded from 64 to 66 attention heads in
VllmConfig.__post_init__, butSpeculativeConfigvalidates the independently constructedDeepSeekMTPModelfirst. The v17 stack contained the draft-padding test but omitted the corresponding helper and validator call, so TP6 worked with MTP off and failed with MTP enabled.Validation
19 passedintests/config/test_virtual_tp.py