Skip to content

[None][test] Waive 6 failed cases for main in QA CI #13457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
xinhe-nv wants to merge 1 commit into from
Closed

[None][test] Waive 6 failed cases for main in QA CI #13457

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions tests/integration/test_lists/waives.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -430,3 +430,6 @@ disaggregated/test_disaggregated.py::test_disaggregated_gpt_oss_120b_harmony[gpt
accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache-dp4-cutlass-auto] SKIP (https://nvbugs/5596343)
unittest/_torch/speculative/test_eagle3.py::test_llama_eagle3_dynamic_tree[True-False] SKIP (https://nvbugs/6113021)
perf/test_perf_sanity.py::test_e2e[aggr_upload-deepseek_r1_fp4_v2_grace_blackwell-r1_fp4_v2_tep4_mtp3_1k1k] SKIP (https://nvbugs/6114727)
accuracy/test_llm_api_pytorch.py::TestGemma3_1BInstruct::test_fp8_prequantized[torch_compile=False] SKIP (https://nvbugs/6087946)
accuracy/test_llm_api_pytorch.py::TestGemma3_1BInstruct::test_fp8_prequantized[torch_compile=True] SKIP (https://nvbugs/6087946)
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_bfloat16_4gpus[pp4-attn_backend=TRTLLM-torch_compile=False] SKIP (https://nvbugs/6112497)
Comment on lines +433 to +435

@coderabbitai coderabbitai Bot Apr 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Search for the missing waive entries in test lists

echo "=== Searching for missing waive entries ==="

# Search for the missing test cases
rg -n "nvidia_Llama-3.1-8B-Instruct-FP8-True" tests/integration/test_lists/
rg -n "TestNemotronNanoV3.*fp8-4-trtllm" tests/integration/test_lists/
rg -n "TestNemotronV2.*test_fp8.*True" tests/integration/test_lists/

echo ""
echo "=== Checking for bug references ==="
rg -n "6112502|6112503" tests/integration/test_lists/

Repository: NVIDIA/TensorRT-LLM

Length of output: 1340

Add the missing 3 waive entries to complete the PR objective of waiving 6 failed cases.

The code changes add only 3 waive entries (lines 433–435), but the PR objectives state "Waive 6 failed cases" and list 6 specific cases. The following 3 entries are missing and should be added to waives.txt:

  • accuracy/test_llm_api_autodeploy.py::TestModelRegistryAccuracy::test_autodeploy_from_registry[nvidia_Llama-3.1-8B-Instruct-FP8-True] SKIP (https://nvbugs/6112502)
  • accuracy/test_llm_api_autodeploy.py::TestNemotronNanoV3::test_accuracy[fp8-4-trtllm] SKIP (https://nvbugs/6112503)
  • accuracy/test_llm_api_autodeploy.py::TestNemotronV2::test_fp8[True] SKIP (https://nvbugs/5921674)

The 3 entries present are correctly formatted per coding guidelines.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@tests/integration/test_lists/waives.txt` around lines 433 - 435, The PR added
only 3 waive entries but the objective requires 6; add the three missing waive
lines to the same file so all six failed cases are waived: add entries for
accuracy/test_llm_api_autodeploy.py::TestModelRegistryAccuracy::test_autodeploy_from_registry[nvidia_Llama-3.1-8B-Instruct-FP8-True]
SKIP (https://nvbugs/6112502),
accuracy/test_llm_api_autodeploy.py::TestNemotronNanoV3::test_accuracy[fp8-4-trtllm]
SKIP (https://nvbugs/6112503), and
accuracy/test_llm_api_autodeploy.py::TestNemotronV2::test_fp8[True] SKIP
(https://nvbugs/5921674) using the same formatting as the existing waive entries
so the PR meets the "Waive 6 failed cases" objective.

Loading