[None][perf] Extend packed MiniMax-M3 producer to mixed and decode batches

[peihu-nv]

@coderabbitai summary

Prerequisite

• [None][perf] Fuse MiniMax-M3 prefill projections #16852

Description

PR #16852 uses one packed MXFP8 [Q|K|V|index-Q|index-K] projection for
MiniMax-M3, but restricts its horizontal norm/RoPE/cache-insertion producer to
pure prefill. Aggregate mixed and decode batches therefore split the packed
output and return to the separate main and index producer paths, repeating
intermediate materialization, normalization, RoPE, quantization, and cache
writes on generation steps.

This PR reuses the existing token-major horizontal producer for aggregate
mixed batches and CUDA-graph decode. Per-token positions and cache slots already
describe all three batch types, so the change removes the batch-type and
CUDA-graph restrictions while preserving the existing compile, shape, dtype,
MSA, FP8-cache, RoPE, and layout guards. The packed optimization remains opt-in
through fuse_qkv_index_projection=False by default.

An initial matched GB300 aggregate TP8/EP8 attention-DP A/B used exact 8K input,
1K output, concurrency 256, and 2,560 requests. Both variants completed every
request with identical input and output token counts:

Metric	Pure-prefill producer	All-batches producer	Change
Benchmark duration	575.10 s	509.24 s	-11.45%
Total token throughput	37,082 tok/s	41,878 tok/s	+12.93%
Median TTFT	604.44 ms	531.15 ms	-12.13%
Median TPOT	64.27 ms	56.95 ms	-11.39%
Median E2E latency	59.38 s	52.40 s	-11.76%

This aggregate measurement predates #16923, which optimizes the downstream MSA
mixed-batch execution stage and is now part of the target branch. The two
changes operate on different pipeline stages. A matched GB300 disaggregated
TP8/EP8 GEN-only A/B on the current target branch showed no measurable decode
throughput gain, while median TPOT improved by 1.3%.

Test Coverage

• Changed-file pre-commit passed for all three files in this PR.
• Focused SM100 CUDA test passed for 1-, 16-, and 129-token shapes:
  • exact main-Q and paged main-K/V parity against the separate producers;
  • FP8 index-Q and index-K parity;
  • strided HND cache layouts; and
  • CUDA-graph capture/replay with refreshed projection values, positions,
    cache destinations, and a padded negative cache slot.
• Full GB300 aggregate TP8/EP8 serving A/B captured the CUDA-graph schedule
  through batch size 256 and completed 2,560/2,560 requests in both variants
  with identical token counts.
• Matched GB300 disaggregated TP8/EP8 GEN-only serving A/B completed
  successfully and showed flat decode throughput with a 1.3% median TPOT
  improvement.
• No C++ or CUDA source changes are introduced by this follow-up; it reuses the
  producer built and validated in [None][perf] Fuse MiniMax-M3 prefill projections #16852.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[peihu-nv]

/bot run

[peihu-nv]

/bot run

[tensorrt-cicd]

PR_Github #62214 [ run ] triggered by Bot. Commit: 42a1bed Link to invocation

[tensorrt-cicd]

PR_Github #62218 [ run ] triggered by Bot. Commit: 42a1bed Link to invocation

[tensorrt-cicd]

PR_Github #62214 [ run ] completed with state ABORTED. Commit: 42a1bed

Link to invocation

[tensorrt-cicd]

PR_Github #62218 [ run ] completed with state SUCCESS. Commit: 42a1bed
/LLM/main/L0_MergeRequest_PR pipeline #50385 completed with status: 'SUCCESS'

CI Report

Link to invocation