Skip to content

[https://nvbugs/6633268][test] waive DeepSeekV3Lite bfloat16 mtp_nextn=2 v2_kv_cache test on DGX B200 - #17957

Merged
xxi-nv merged 1 commit into
NVIDIA:mainfrom
xxi-nv:waive-6633268-dgx-b200
Aug 19, 2026
Merged

[https://nvbugs/6633268][test] waive DeepSeekV3Lite bfloat16 mtp_nextn=2 v2_kv_cache test on DGX B200#17957
xxi-nv merged 1 commit into
NVIDIA:mainfrom
xxi-nv:waive-6633268-dgx-b200

Conversation

@xxi-nv

@xxi-nv xxi-nv commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Description

Waive accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=2-attention_dp=True-cuda_graph=True-overlap_scheduler=True-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=True] on DGX_B200, tracked by https://nvbugs/6633268.

The test failed in pre-merge LLM/main/L0_MergeRequest_PR #54717, stage DGX_B200-PyTorch-6, with Test terminated unexpectedly.

OpenSearch history over the last 30 days on DGX_B200 shows the case is flaky rather than consistently broken: 13 failures out of 1000 runs (1.3%), flip-flop rate 28.9%, longest failure streak 1.

The waive is scoped to full:DGX_B200/ so that coverage on other platforms is preserved. This case also runs from l0_h100.yml, which stays enabled. An existing waive for the same case is scoped to full:RTX_PRO_6000_Blackwell_Server_Edition/ (https://nvbugs/6451323) and does not cover DGX_B200.

Test Coverage

N/A - test list change only.

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why.
  • PR follows the PR title format.
  • Test cases are provided for new code paths (see test instructions)
  • Any new dependencies have been scanned for license and vulnerabilities
  • CODEOWNERS updated if ownership changes
  • Documentation updated as needed
  • The reviewers assigned automatically/manually are appropriate for the PR.

Dev Engineer Review

  • Added one waiver in tests/integration/test_lists/waives.txt.
  • The waiver targets the DGX B200 DeepSeekV3Lite bfloat16 mtp_nextn=2 v2_kv_cache test.
  • The waiver is tracked by NVBUG 6633268.
  • The scope remains limited to full:DGX_B200/.
  • No public API or code changes were made.
  • The change is correctly scoped and has low review risk.

QA Engineer Review

  • No test-db/ or qa/ files were modified.
  • Added one DGX B200 waiver entry.
  • Removed no entries.
  • CBTS coverage data is unavailable.
  • Verdict: needs follow-up.

…n=2 v2_kv_cache test on DGX B200

Signed-off-by: xxi <xxi@nvidia.com>
@coderabbitai

coderabbitai Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3857001b-7b94-4fde-83db-d773973c2af0

📥 Commits

Reviewing files that changed from the base of the PR and between f42674a and d9f93a6.

📒 Files selected for processing (1)
  • tests/integration/test_lists/waives.txt

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

The integration test waiver list adds a DGX_B200 waiver for the configured TestDeepSeekV3Lite::test_bfloat16 scenario.

Changes

Integration test waiver

Layer / File(s) Summary
DGX_B200 test waiver
tests/integration/test_lists/waives.txt
Adds a waiver for TestDeepSeekV3Lite::test_bfloat16 with MTP, attention-DP, CUDA-graph, overlap-scheduler, and v2 KV-cache settings.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Merge Risk: ⚪ Minimal · up to d9f93

This localized test-waiver change does not alter product or runtime behavior, and the waiver remains limited to DGX_B200 while other listed coverage stays enabled. No actionable merge-blocking risk remains beyond normal validation of the active test list.

Suggested reviewers: brnguyen2

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the NVBugs ticket, test waiver, affected configuration, and DGX B200 platform.
Description check ✅ Passed The description explains the waiver rationale, scope, evidence of flakiness, test coverage, and checklist status.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@xxi-nv
xxi-nv requested a review from ZhanruiSunCh August 19, 2026 07:26
@xxi-nv

xxi-nv commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

/bot skip --comment "waive the flacky test to unblock"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #67344 [ skip ] triggered by Bot. Commit: d9f93a6 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #67344 [ skip ] completed with state FAILURE. Commit: d9f93a6

Link to invocation

@xxi-nv
xxi-nv enabled auto-merge (squash) August 19, 2026 07:40
@xxi-nv

xxi-nv commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

/bot skip --comment "waive the flacky test to unblock"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #67393 [ skip ] triggered by Bot. Commit: d9f93a6 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #67393 [ skip ] completed with state SUCCESS. Commit: d9f93a6
Skipping testing for commit d9f93a6

Link to invocation

@xxi-nv
xxi-nv merged commit bbc40d8 into NVIDIA:main Aug 19, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants