Skip to content

[None][fix] Correct MiniMax-M3 Eagle draft sub-page pool bound - #17656

Merged
zheyuf merged 1 commit into
NVIDIA:feat/m3_with_msafrom
zheyuf:fix/minimax-m3-eagle-draft-subpage-bound
Aug 17, 2026
Merged

[None][fix] Correct MiniMax-M3 Eagle draft sub-page pool bound#17656
zheyuf merged 1 commit into
NVIDIA:feat/m3_with_msafrom
zheyuf:fix/minimax-m3-eagle-draft-subpage-bound

Conversation

@zheyuf

@zheyuf zheyuf commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

What this fixes

This PR fix the cold start AL regression for Minimax-m3 eagle. It's a functionality fix but not a perf boost. No extra overhead is added.

MiniMax-M3 uses a shared KV cache for the target model and the Eagle3 draft model. The target uses 128-token pages, while the draft model views the same memory as 32-token pages.

During cold start, the draft view reported the wrong number of available cache pages. Some valid draft pages could therefore be treated as out of range, causing the draft model to read incorrect KV data. The target model still verified the output, so final answers remained correct, but Eagle acceptance length dropped to roughly 1.8-2.0 and speculative decoding became less effective.

What changed

The draft view now calculates its cache boundary from its own 32-token layout instead of reusing the target manager's 128-token value.

This PR only fixes that boundary calculation. The aggregated shared-KV routing and performance changes are handled separately in #17835.

Result

Three independent cold starts recovered first-round acceptance length:

cold start acceptance length
1 3.264
2 3.417
3 3.489

Mean acceptance length was 3.390. Cold generation time improved from 13.407 s to 10.946 s on average, an 18.4% reduction. Hot runs showed no regression.

Validation

  • Three independent two-node cold starts completed successfully.
  • 288/288 long-prompt requests completed.
  • No traceback, assertion, CUDA, segmentation-fault, or fatal errors were found.
  • All workers used the expected fixed source.
  • Focused tests: 13 passed.
  • Formatting, static checks, and DCO passed.

PR checklist

  • PR title follows the [JIRA/NVBUG/None][type] format
  • Commit is signed off (DCO)
  • Focused unit tests pass
  • Cold-start regression reproduced before the fix and passed after it

Signed-off-by: Zheyu Fu <zheyuf@nvidia.com>
@zheyuf zheyuf changed the title [None][fix] Correct MiniMax-M3 Eagle draft sub-page pool bound [None][perf] Fix and enable MiniMax-M3 shared draft KV in aggregated serving Aug 17, 2026
@zheyuf
zheyuf force-pushed the fix/minimax-m3-eagle-draft-subpage-bound branch from dea01d3 to b09f72c Compare August 17, 2026 18:07
@zheyuf zheyuf changed the title [None][perf] Fix and enable MiniMax-M3 shared draft KV in aggregated serving [None][fix] Correct MiniMax-M3 Eagle draft sub-page pool bound Aug 17, 2026
@zheyuf
zheyuf requested review from brb-nv, pcicotti and peihu-nv August 17, 2026 18:40
@zheyuf
zheyuf marked this pull request as ready for review August 17, 2026 18:40
@zheyuf
zheyuf requested review from a team as code owners August 17, 2026 18:40
@zheyuf
zheyuf requested review from VALLIS-NERIA, kris1025, nvpohanh and yunruis and removed request for a team, VALLIS-NERIA, kris1025, nvpohanh and yunruis August 17, 2026 18:40
@zheyuf

zheyuf commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66836 [ run ] triggered by Bot. Commit: b09f72c Link to invocation

@brb-nv brb-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #66836 [ run ] completed with state SUCCESS. Commit: b09f72c
/LLM/main/L0_MergeRequest_PR pipeline #54396 completed with status: 'SUCCESS'

CI Report

Link to invocation

@zheyuf
zheyuf merged commit fa1cbaf into NVIDIA:feat/m3_with_msa Aug 17, 2026
30 checks passed
brb-nv pushed a commit to brb-nv/TensorRT-LLM that referenced this pull request Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants