Skip to content

[GG] fix: pin same-repository MTP to the target revision - #118

Closed
voipmonitor wants to merge 1 commit into
local-inference-lab:dev/gilded-gnosis-old-basefrom
voipmonitor:fix/gg-mtp-serialized-detection-20260718
Closed

[GG] fix: pin same-repository MTP to the target revision#118
voipmonitor wants to merge 1 commit into
local-inference-lab:dev/gilded-gnosis-old-basefrom
voipmonitor:fix/gg-mtp-serialized-detection-20260718

Conversation

@voipmonitor

Copy link
Copy Markdown

Summary

  • make an in-checkpoint MTP draft inherit the target weight and code revisions when both use the same repository
  • resolve the serialized NextN weight-index probe against that effective revision instead of implicitly probing main
  • preserve explicit draft revisions and local model paths
  • cover inherited, explicit, and cache-probe revision selection

Root cause

The GLM-5.2 launcher pins lukealonso/GLM-5.2-NVFP4 to a concrete target revision. Its MTP configuration names the same repository but does not repeat the revision. Target weights therefore load from the requested commit while DeepSeekMTP probes the Hugging Face cache at implicit main to decide whether the checkpoint contains serialized NextN modules.

On a cold cache this selected the wrong snapshot and then the fallback loader expected a different module layout. TP6/DCP6/MTP3 failed with a missing serialized weight such as:

model.layers.78.mtp_block.mlp.experts.routed_experts.w2_input_scale

For same-repository MTP, the draft is part of the target checkpoint and must be revision-atomic with it. This change performs that inheritance before draft config loading and passes the effective revision into try_to_load_from_cache. A genuinely separate draft repository or an explicit draft revision is unchanged.

Validation

  • ruff check passed
  • ruff format --check passed
  • four focused revision tests passed
  • the complete two-file run had 10 passing tests plus one unrelated baseline failure that reproduces unchanged on dev/gilded-gnosis
  • cold-cache GLM-5.2 TP6 MTP loading proceeded from the pinned target snapshot after mounting these changes

@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 56 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1c91357a-fa6c-47ae-aca0-3cfc3ac59eaf

📥 Commits

Reviewing files that changed from the base of the PR and between 5b116d1 and c38d7fb.

📒 Files selected for processing (4)
  • tests/config/test_speculative_draft_hf_overrides.py
  • tests/models/test_deepseek_mtp_mxfp8_loader.py
  • vllm/config/speculative.py
  • vllm/model_executor/models/deepseek_mtp.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@voipmonitor

Copy link
Copy Markdown
Author

Baked-image E2E validation: all TP8/DCP1 MTP3 standard variants and TP6/DCP3+DCP6 MTP3 variants loaded from the intended target checkpoint/revision and accepted draft tokens. The TP6 local MXFP4 cases reached mean accepted lengths 2.75-2.79 with no fallback probe of an unrelated repository revision.

@voipmonitor voipmonitor changed the title Pin same-repository MTP to the target revision [GG] fix: pin same-repository MTP to the target revision Jul 18, 2026
@voipmonitor

Copy link
Copy Markdown
Author

Superseded by the canonical dev/gilded-gnosis consolidation. The corresponding implementation is present in commit(s): a888406. Closing the old-base PR so future work targets the canonical GG branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant