Repository navigation

[Attention] Shard MiniMax M3.1's NVFP4 index cache with indexer CP - #64631

Draft

mdavis36 wants to merge 62 commits into

vllm-project:mainfrom

mdavis36:md/m31-icp-nvfp4

Draft

[Attention] Shard MiniMax M3.1's NVFP4 index cache with indexer CP#64631
mdavis36 wants to merge 62 commits into
vllm-project:mainfrom
mdavis36:md/m31-icp-nvfp4

Commits on Oct 29, 2026
- [Attention] Add opt-in MiniMax M3 indexer cache sharding
  
  wolfcomos
  and
  codex
  committed
  f674136
- [Attention] Support TP2 indexer cache sharding
  wolfcomos
  committed
  81fd7bb
- [Attention] Add local scoring for sharded MiniMax M3 prefill
  wolfcomos
  committed
  33f18cd
- [Docs] Include FP8 index cache in CP launch examples
  wolfcomos
  committed
  8948aa1
- [Attention] Parallelize sharded prefill scoring over KV blocks
  
  wolfcomos
  and
  codex
  committed
  bec266a
- [Test] Synchronize prefill CP mailbox snapshots between phases
  
  wolfcomos
  and
  codex
  committed
  f860faf
- [Attention] Exchange prefill index candidates once per layer
  
  wolfcomos
  and
  codex
  committed
  e4fc824
- [Attention] Pack prefill candidates in reusable exchange storage
  
  wolfcomos
  and
  codex
  committed
  a187155
- [Attention] Bound larger prefill score chunks by scratch capacity
  
  wolfcomos
  and
  codex
  committed
  210a76b
- [Docs] Describe bounded prefill candidate exchange
  
  wolfcomos
  and
  codex
  committed
  cc2f047
- Avoid unused full score storage with local indexer prefill
  
  wolfcomos
  and
  codex
  committed
  5d1f1c0
- Initialize CP score scratch in the scorer
  
  wolfcomos
  and
  codex
  committed
  9d36d06
- Evaluate native prefill scoring on interleaved TP2 index cache
  
  wolfcomos
  and
  codex
  committed
  81514af
- Check native index score validity across buffer reuse
  
  wolfcomos
  and
  codex
  committed
  10d14d5
- Cover speculative mixed indexer CP dispatch
  
  wolfcomos
  and
  codex
  committed
  488e421
- Drop unused prefill index validity scratch
  
  wolfcomos
  and
  codex
  committed
  eadd6bd
- [Attention] Fuse sharded decode Top16 and candidate publication
  
  wolfcomos
  and
  codex
  committed
  15d8890
- [Attention] Merge TP2 indexer candidates within one warp
  
  wolfcomos
  and
  codex
  committed
  ba7e7e9
- test: check stable indexer merge ties
  
  wolfcomos
  and
  codex
  committed
  6101757
- Describe bounded local prefill without scorer-specific limits
  
  wolfcomos
  and
  codex
  committed
  53c3a0e
- Overlap CP decode with PDL and historical K prefetch
  wolfcomos
  committed
  9b5e904
- Use generation-tagged candidates for multi-layer CP decode
  wolfcomos
  committed
  b0fda8e
- Fix sharded indexer allocation fault injection
  wolfcomos
  committed
  e248fc6
- Batch prefill candidate merge after NCCL completion
  
  wolfcomos
  and
  codex
  committed
  e06b950
- Emit prefill score candidates directly into the send carrier
  
  wolfcomos
  and
  codex
  committed
  cf86798
- Cover prefill candidate emission and NCCL merge reuse
  
  wolfcomos
  and
  codex
  committed
  96cc3b4
- Skip prefill score fill for valid-prefix TopK consumers
  
  wolfcomos
  and
  codex
  committed
  bc25733
- Validate admissible tied TopK candidates after scratch reuse
  
  wolfcomos
  and
  codex
  committed
  7987177
- Reuse bounded prefill candidate and score storage across steps
  wolfcomos
  committed
  91743b0
- Group tagged TP2 decode merge rows in a bounded grid
  
  wolfcomos
  and
  codex
  committed
  5501ce0
- Prepare sharded indexer prefill windows and writer metadata on device
  wolfcomos
  committed
  11ab69f
- Make unified mixed index scoring an explicit experimental mode
  wolfcomos
  committed
  c36b046
- Document mixed indexer scoring comparison mode
  
  wolfcomos
  and
  codex
  committed
  dfeaf80
- Skip empty cache requests when poisoning causal tails
  
  wolfcomos
  and
  codex
  committed
  d9ea5ed
- Bound sharded decode score storage at model initialization
  
  wolfcomos
  and
  codex
  committed
  d9e3aed
- Include score capacity in indexer communication reuse key
  
  wolfcomos
  and
  codex
  committed
  bf3dbe6
- Cover causal tails with no rank-local index K
  
  wolfcomos
  and
  codex
  committed
  8bc3816
- Mask padded indexer rows using original cache slots
  
  wolfcomos
  and
  codex
  committed
  4cc4e05
- Keep indexer test references out of production paths
  
  wolfcomos
  and
  codex
  committed
  05c62a7
- Avoid redundant polling lanes in tagged TP4 candidate merge
  
  wolfcomos
  and
  codex
  committed
  fef1dec
- Use the validated TP4 FMHA prefill scorer in the P128 parent
  
  wolfcomos
  and
  codex
  committed
  53c8528
- Remove unreachable Triton prefill fallback for supported CP sizes
  
  wolfcomos
  and
  codex
  committed
  629412d
- Drop retired CP Triton scorer and unused chunk metadata
  
  wolfcomos
  and
  codex
  committed
  e733146
- Trim indexer CP scaffolding and update TP4 planner assertion
  
  wolfcomos
  and
  codex
  committed
  85be07d
- [Attention] Reuse shared decode query width for indexer CP
  
  committed
  35f9c41
- Keep graph-padding prefill CSR nonempty
  wolfcomos
  committed
  dc68fa7
- Add P256 contiguous indexer shards and attention page reads
  
  wolfcomos
  and
  codex
  committed
  efc0ab6
- Keep P256 builder initialization aligned with its parent
  
  wolfcomos
  and
  codex
  committed
  ae6c44f
- Refresh the indexer test communication state for each page size
  
  committed
  859a380
- Compact P256 indexer score storage
  
  wolfcomos
  and
  fans-nv
  committed
  a76900d
Commits on Oct 30, 2026
- Score NVFP4 index-K shards in the sharded decode scorer
  mdavis36
  committed
  26c521f
- Fold each 16-row tile into the sharded scorer's page maxima
  mdavis36
  committed
  67448a6
- Round NVFP4 index-K like the stock decode indexer
  mdavis36
  committed
  2740926
- Shard the NVFP4 MiniMax M3 index cache with indexer CP
  mdavis36
  committed
  ac7416e
- Write NVFP4 index-K into TP-sharded index pages
  mdavis36
  committed
  d0ce3f2
- Stage sharded NVFP4 index pages to FP8 for the gather prefill
  mdavis36
  committed
  683523d
- Score P256 TP2 NVFP4 index shards with the stock Q8KV4 indexer
  mdavis36
  committed
  d5a53c3
Commits on Oct 31, 2026
- Fix CPU mask in sharded candidate merge test
  
  wolfcomos
  and
  codex
  committed
  9fcb124
Commits on Nov 2, 2026
Commits on Nov 5, 2026
- Merge branch 'main' of https://github.com/vllm-project/vllm into md/m31-icp-nvfp4
  mdavis36
  committed
  34534c5

Commits

Commits on Oct 29, 2026

Commits on Oct 30, 2026

Commits on Oct 31, 2026

Commits on Nov 2, 2026

Commits on Nov 5, 2026