[None][fix] Allow MiniMax-M3's to use separate draft KV cache in disagg - #17341
Merged
zheyuf merged 1 commit intoAug 6, 2026
Merged
Conversation
zheyuf
force-pushed
the
fix/m3-eagle3-disagg-separate-draft-kv-cache
branch
from
August 6, 2026 08:58
65db48f to
bb6a4c7
Compare
…he in disaggregated serving The nvbug 5807902 WAR disables the separate draft KV cache manager whenever a cache transceiver is configured. MiniMax-M3 cannot take the shared-manager path that WAR forces: its cache manager declares supports_shared_draft_layers=False, and the drafter then inherits the target's tokens_per_block=128 pages, which miss the SM10x Eagle context cubins (the unfused-MHA fallback requests a 6.17 TiB workspace on a real 32K-token warmup) and hit the known tokens_per_block=128 trtllm-gen generation-kernel IMA. Both context and generation workers crashed during startup on every disaggregated Eagle3 attempt. Exempt MiniMax-M3 from the WAR so both worker roles keep the designed tokens_per_block=32 separate draft manager (symmetry is required for a consistent target pool layout across the KV transfer). Validated on Lyris GB300 (2xCTX TP2 + GEN TP4/ADP, NIXL): startup completes end to end, and the test_nvfp4_eagle3 chat-GSM8K acceptance workload measures AL 3.330 disagg vs 3.474 aggregated on the same build (drafter card reference 3.518). The remaining gap is the transceiver not transferring draft-layer KV, tracked separately. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com>
zheyuf
force-pushed
the
fix/m3-eagle3-disagg-separate-draft-kv-cache
branch
from
August 6, 2026 09:02
bb6a4c7 to
719dee1
Compare
zheyuf
changed the title
zheyuf
marked this pull request as ready for review
August 6, 2026 09:05
zheyuf
requested review from
allisonlim-nv and
mikeiovine
and removed request for
a team,
allisonlim-nv and
mikeiovine
August 6, 2026 09:05
Collaborator
Author
|
/bot run |
Collaborator
|
PR_Github #64291 [ run ] triggered by Bot. Commit: |
Collaborator
|
PR_Github #64291 [ run ] completed with state |
Collaborator
|
Let's add a disagg accuracy test with Eagle to protect the functionality on side-branch, Zheyu? |
brb-nv
reviewed
Aug 6, 2026
4 tasks
Collaborator
Author
Yes I will add the disagg accuracy test after the disagg+eagle get fixed. |
This was referenced Aug 7, 2026
brb-nv
pushed a commit
to brb-nv/TensorRT-LLM
that referenced
this pull request
Aug 25, 2026
…gg (NVIDIA#17341) Signed-off-by: Zheyu Fu <zheyuf@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pipe clean disagg + eagle. This PR makes disagg+eagle runnable. TODO in future PR: support disagg + seperate kv cache to restore AL.
PR Checklist
[JIRA/NVBUG/None][type]format/bot run)