From b418e5800c8fae82e19c6212287faa624a1cf94c Mon Sep 17 00:00:00 2001 From: Dongfeng Yu Date: Sun, 12 Jul 2026 20:25:36 +0000 Subject: [PATCH] [None][test] Waive TestGemma4MoE::test_bf16 (missing bf16 MMMU reference) accuracy/test_llm_api_autodeploy.py::TestGemma4MoE::test_bf16 evaluates MMMU with the plain bf16 spec, but references/mmmu.yaml only registers a NVFP4/FP8-KV entry for google/gemma-4-26B-A4B-it (added in #16108), so the accuracy harness raises ValueError (Not registered specs) before scoring. The failure is deterministic and PR-independent: it failed identically in every recent LLM/main/L0_Test-x86_64-Multi-GPU run (builds 2327-2336), blocking pre-merge CI for any PR that triggers the multi-GPU stage. Waive until the bf16 MMMU reference is added. Co-Authored-By: Claude Fable 5 Signed-off-by: Dongfeng Yu --- tests/integration/test_lists/waives.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/integration/test_lists/waives.txt b/tests/integration/test_lists/waives.txt index d67988ac1721..df6f71570383 100644 --- a/tests/integration/test_lists/waives.txt +++ b/tests/integration/test_lists/waives.txt @@ -225,6 +225,7 @@ full:DGX_B200/unittest/auto_deploy/multigpu/custom_ops/test_ad_dist_strategies.p full:DGX_B200/unittest/auto_deploy/multigpu/custom_ops/test_ad_dist_strategies.py::test_allreduce_strategies[ONESHOT] SKIP (https://nvbugs/6403920) full:DGX_B200/unittest/auto_deploy/multigpu/custom_ops/test_ad_dist_strategies.py::test_allreduce_strategies[TWOSHOT] SKIP (https://nvbugs/6403920) full:DGX_B300/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16_4gpus[ep4-mtp_nextn=2-attention_dp=True-cuda_graph=True-overlap_scheduler=True-torch_compile=False] SKIP (https://nvbugs/6432831) +full:DGX_H100/accuracy/test_llm_api_autodeploy.py::TestGemma4MoE::test_bf16 SKIP (temporary ToT main waive; bf16 MMMU reference missing since PR 16108, accuracy harness raises Not-registered-specs before scoring) full:DGX_H100/unittest/_torch/attention/test_attention_backends.py::test_attention_backend[qwen2_0_5b_gqa_hd64-ctx-bf16-HND-p32-v1] SKIP (https://nvbugs/6403909) full:GB200/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_eagle3[eagle3_one_model=False-overlap_scheduler=False] SKIP (https://nvbugs/6402500) full:GB200/accuracy/test_disaggregated_serving.py::TestLlama3_1_8BInstruct::test_guided_decoding_with_eagle3[llguidance-eagle3_one_model=False] SKIP (https://nvbugs/6402500)