[https://nvbugs/6018046][fix] Drop max_batch_size 32→8 for throughput_pp4_mtp (matching throughput_bs8_mtp), l

[tensorrt-cicd]

Summary

• Root cause: throughput_pp4_mtp pairs pp=4 with ep_size=1, keeping all 256 experts on every rank; the default 0.70 KV-cache fraction left too little headroom for runtime cuBLAS/NCCL workspaces under post-merge B300 load, causing intermittent CUBLAS_STATUS_EXECUTION_FAILED in the router GEMM mid-MMLU.
• Fix: Drop max_batch_size 32→8 for throughput_pp4_mtp (matching throughput_bs8_mtp), lower KV fraction to 0.5 for the pp>1 + ep_size=1 + mtp>0 path, and remove the waiver.
• Automated fix generated by repair-bot

Test plan

• Verify fix on the same GPU type as the original failure
• Check for regressions in related tests

Links

• Bug: https://nvbugs/6018046

Summary by CodeRabbit

• Tests
  • Adjusted multi-GPU test parametrization and KV-cache sizing logic for improved test reliability.
  • Removed test waiver entry to re-enable previously skipped test case.

Note: This release contains internal testing improvements with no user-facing changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d42c0160-874b-4860-99f4-74d6451ace33

📥 Commits

Reviewing files that changed from the base of the PR and between 3a354dc and c48c523.

📒 Files selected for processing (2)

• tests/integration/defs/accuracy/test_llm_api_pytorch.py
• tests/integration/test_lists/waives.txt

💤 Files with no reviewable changes (1)

• tests/integration/test_lists/waives.txt

---

📝 Walkthrough

Walkthrough

This PR adjusts the DeepSeekR1 NVFP4 multi-GPU test to handle multi-token prediction (MTP) scenarios more gracefully. The KV-cache memory fraction is now conditional based on parallelization and MTP settings, the batch size parameter is reduced for one configuration, and the test waiver is removed to allow the test to run.

Changes

DeepSeekR1 NVFP4 KV-cache and test adjustment

Layer / File(s)	Summary
KV-cache sizing logic and test parameter adjustment tests/integration/defs/accuracy/test_llm_api_pytorch.py	KV-cache free_gpu_memory_fraction is made conditional, selecting 0.5 when pipeline parallelism > 1, expert parallelism == 1, and MTP > 0, otherwise 0.70; max_batch_size is reduced from 32 to 8 for the affected parameter case.
Test waiver removal tests/integration/test_lists/waives.txt	Waiver entry for the throughput_pp4_mtp parametrization of TestDeepSeekR1::test_nvfp4_multi_gpus is removed, allowing the test to execute.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#12823: Both PRs update tests/integration/defs/accuracy/test_llm_api_pytorch.py KV-cache sizing to use a conditional free_gpu_memory_fraction/kv_fraction driven by MTP (mtp_nextn > 0) and adjust the corresponding waives.txt entry for the affected DeepSeek parametrized tests.
• NVIDIA/TensorRT-LLM#14086: Both PRs modify tests/integration/test_lists/waives.txt by adding/removing waivers for accuracy/test_llm_api_pytorch.py DeepSeekR1 test_nvfp4_multi_gpus parametrizations, so they're directly related at the test-skipping configuration level.
• NVIDIA/TensorRT-LLM#13641: Both PRs modify tests/integration/test_lists/waives.txt by removing an existing waiver line for a specific TestDeepSeekR1/test_nvfp4_multi_gpus parametrization, so they're directly related.

Suggested reviewers

• zhenhuaw-me
• ZhanruiSunCh
• StanleySun639

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly specifies the main change: reducing max_batch_size from 32 to 8 for throughput_pp4_mtp test case, with a reference to the matching variant (throughput_bs8_mtp) and includes the NVBugs ticket.
Description check	✅ Passed	The PR description provides a clear summary with root cause analysis, the applied fix, test verification, and a bug link, meeting the requirements of the template.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[trtllm-agent]

NVBug 6018046 is closed as Bug - Fixed. The linked bug appears resolved elsewhere or for a reason that does not prove this PR is redundant. This PR should be judged on its own merits; repair-bot is not auto-closing it.

[trtllm-agent]

Close this stale PR since the nvbug is closed.