Skip to content

[TRTLLM-13027][ci] Relocate under-using tests to right-sized stages #14684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
QiJune merged 1 commit into from
Jun 6, 2026
+38 −20
Merged

[TRTLLM-13027][ci] Relocate under-using tests to right-sized stages #14684

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions jenkins/L0_Test.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3949,6 +3949,7 @@ def launchTestJobs(pipeline, testFilter)
"DGX_B200-Triton-Post-Merge-1": ["auto:dgx-b200-flex", "l0_b200", 1, 1, 1, 1, true],
"DGX_B200-PyTorch-Post-Merge-1": ["auto:dgx-b200-flex", "l0_b200", 1, 2, 1, 1, true],
"DGX_B200-PyTorch-Post-Merge-2": ["auto:dgx-b200-flex", "l0_b200", 2, 2, 1, 1, true],
"DGX_B200-2_GPUs-PyTorch-1": ["auto:dgx-b200-flex", "l0_dgx_b200", 1, 1, 2, 1, true],
"DGX_B200-4_GPUs-PyTorch-1": ["auto:dgx-b200-flex", "l0_dgx_b200", 1, 3, 4, 1, true],
"DGX_B200-4_GPUs-PyTorch-2": ["auto:dgx-b200-flex", "l0_dgx_b200", 2, 3, 4, 1, true],
"DGX_B200-4_GPUs-PyTorch-3": ["auto:dgx-b200-flex", "l0_dgx_b200", 3, 3, 4, 1, true],
Expand Down
11 changes: 11 additions & 0 deletions tests/integration/test_lists/test-db/l0_b200.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ l0_b200:
- unittest/_torch/modules/moe/test_moe_module.py::test_configurable_moe_single_gpu -k "CUTEDSL"
- unittest/_torch/modules/moe/test_moe_module.py::test_configurable_moe_single_gpu -k "DEEPGEMM"
- unittest/_torch/modules/moe/test_moe_module.py::test_configurable_moe_single_gpu -k "DENSEGEMM"
- unittest/_torch/modules/moe/test_moe_module.py::test_configurable_moe_single_gpu -k "MEGAMOE_DEEPGEMM"
# ------------- MoE: FlashInfer & TRTLLM symbol collision tests ---------------
- unittest/_torch/flashinfer/test_trtllm_flashinfer_symbol_collision.py
# --- MoE end
Expand Down Expand Up @@ -303,6 +304,16 @@ l0_b200:
- accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_1gpu[v1_kv_cache-True-True-trtllm-auto]
- accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_1gpu[v2_kv_cache-True-True-trtllm-auto]
- accuracy/test_llm_api_pytorch_multimodal.py::TestNanoV3Omni::test_auto_dtype[bf16]
# ------------- VisualGen single-GPU tests ---------------
- examples/test_visual_gen.py::test_visual_gen_quickstart
- examples/test_visual_gen.py::test_visual_gen_api_walkthrough
- examples/test_visual_gen.py::test_flux1_lpips_against_golden
- examples/test_visual_gen.py::test_flux2_lpips_against_golden
- examples/test_visual_gen.py::test_ltx2_lpips_against_golden
- examples/test_visual_gen.py::test_wan21_t2v_lpips_against_golden
- examples/test_visual_gen.py::test_wan22_t2v_lpips_against_golden
- visual_gen/test_visual_gen_benchmark.py::test_offline_benchmark
- visual_gen/test_visual_gen_benchmark.py::test_online_benchmark[openai-videos]
# ------------- AutoDeploy Backend Stages ---------------
- condition:
ranges:
Expand Down
46 changes: 26 additions & 20 deletions tests/integration/test_lists/test-db/l0_dgx_b200.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,30 @@
version: 0.0.1
l0_dgx_b200:
- condition:
ranges:
system_gpu_count:
gte: 2
lte: 2
wildcards:
gpu:
- '*b200*'
linux_distribution_name: ubuntu*
cpu: x86_64
terms:
stage: pre_merge
backend: pytorch
orchestrator: mpi
tests:
- unittest/_torch/misc/test_autotuner.py::test_autotuner_distributed_strategy
- accuracy/test_llm_api_pytorch.py::TestQwen3_5_35B_A3B::test_bf16[tp2-CUTLASS]
- accuracy/test_llm_api_pytorch.py::TestQwen3_5_35B_A3B::test_bf16[tp2-TRTLLM]
# ------------- KV Cache V2 Scheduler IT (multi-GPU) ---------------
- kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2DSv3Lite::test_mtp_draft_tokens
- kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2DSv3Lite::test_mtp_chunked_draft_tokens
- kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2DSv3Lite::test_mtp_eviction
# ------------- VisualGen multi-GPU tests ---------------
- unittest/_torch/visual_gen/test_flux_pipeline.py::TestFluxParallelism::test_ulysses_2gpu_correctness
- unittest/_torch/visual_gen/test_flux_pipeline.py::TestFluxCombinedOptimizations::test_all_optimizations_combined
- condition:
ranges:
system_gpu_count:
Expand All @@ -15,7 +40,6 @@ l0_dgx_b200:
backend: pytorch
orchestrator: mpi
tests:
- unittest/_torch/misc/test_autotuner.py::test_autotuner_distributed_strategy
- accuracy/test_llm_api_pytorch.py::TestNemotronV3Super::test_auto_dtype_4gpus[4-4-False-True-True]
- accuracy/test_llm_api_pytorch.py::TestNemotronV3Super::test_auto_dtype_4gpus[4-4-True-True-True]
- accuracy/test_llm_api_pytorch.py::TestNemotronV3Super::test_nvfp4_4gpu_mtp_ar TIMEOUT (60)
Expand All @@ -30,8 +54,6 @@ l0_dgx_b200:
- accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B::test_nvfp4[dep4_latency_moe_trtllm-torch_compile=False]
- accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B::test_nvfp4[dep4_latency_moe_cutlass-torch_compile=False]
- accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B::test_nvfp4[dep4_latency_moe_cutlass-torch_compile=True]
- accuracy/test_llm_api_pytorch.py::TestQwen3_5_35B_A3B::test_bf16[tp2-CUTLASS]
- accuracy/test_llm_api_pytorch.py::TestQwen3_5_35B_A3B::test_bf16[tp2-TRTLLM]
- disaggregated/test_disaggregated.py::test_disaggregated_deepseek_v3_lite_fp8_ucx[DeepSeek-V3-Lite-fp8]
- disaggregated/test_disaggregated.py::test_disaggregated_deepseek_v3_lite_fp8_nixl[DeepSeek-V3-Lite-fp8]
- disaggregated/test_disaggregated.py::test_disaggregated_gpt_oss_120b_harmony[gpt_oss/gpt-oss-120b]
Expand All @@ -42,10 +64,6 @@ l0_dgx_b200:
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16_4gpus_python_scheduler[ep4-mtp_nextn=2]
- accuracy/test_llm_api_pytorch.py::TestMiniMaxM2::test_4gpus[attention_dp=False-cuda_graph=True-overlap_scheduler=True-tp_size=4-ep_size=4] TIMEOUT (60)
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16_4gpus[pp4-mtp_nextn=0-attention_dp=False-cuda_graph=False-overlap_scheduler=False-torch_compile=False] TIMEOUT (60)
# ------------- KV Cache V2 Scheduler IT (multi-GPU) ---------------
- kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2DSv3Lite::test_mtp_draft_tokens
- kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2DSv3Lite::test_mtp_chunked_draft_tokens
- kv_cache/test_kv_cache_v2_scheduler.py::TestKVCacheV2DSv3Lite::test_mtp_eviction
# ------------- NVBug 6025177: trtllm-serve cross-request KV contamination (OpenAI) ---------------
- test_e2e.py::test_openai_kv_cache_contamination TIMEOUT (120)
- condition:
Expand Down Expand Up @@ -81,7 +99,6 @@ l0_dgx_b200:
- unittest/_torch/modules/moe/test_moe_module.py::test_configurable_moe_multi_gpu -k "DEEPGEMM and not MEGAMOE_DEEPGEMM"
# --- MEGAMOE_DEEPGEMM (W4A8_MXFP4_MXFP8 only) ---
- unittest/_torch/modules/moe/test_moe_module.py::test_configurable_moe_multi_gpu -k "MEGAMOE_DEEPGEMM"
- unittest/_torch/modules/moe/test_moe_module.py::test_configurable_moe_single_gpu -k "MEGAMOE_DEEPGEMM"
# ------------- MoE: test_multi_gpu_eplb ---------------
- unittest/_torch/modules/moe/test_moe_module.py::test_configurable_moe_multi_gpu_eplb
- condition:
Expand Down Expand Up @@ -165,8 +182,6 @@ l0_dgx_b200:
- accuracy/test_disaggregated_serving.py::TestQwen3NextInstruct::test_auto_dtype[use_py_transceiver=False] TIMEOUT (60)
# ------------- VisualGen multi-GPU tests ---------------
- unittest/_torch/visual_gen/multi_gpu
- unittest/_torch/visual_gen/test_flux_pipeline.py::TestFluxParallelism::test_ulysses_2gpu_correctness
- unittest/_torch/visual_gen/test_flux_pipeline.py::TestFluxCombinedOptimizations::test_all_optimizations_combined
- condition:
ranges:
system_gpu_count:
Expand All @@ -192,7 +207,6 @@ l0_dgx_b200:
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_fp8_blockscale[baseline_fp8kv] TIMEOUT (60)
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_fp8_blockscale[latency] TIMEOUT (60)
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_fp8_blockscale[disable_skip_indexer] TIMEOUT (60)
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_attn_multi_gpus TIMEOUT (60)
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_multi_gpus[baseline_fp8kv] TIMEOUT (60)
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_multi_gpus[latency] TIMEOUT (60)
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_multi_gpus[disable_skip_indexer] TIMEOUT (60)
Expand Down Expand Up @@ -305,30 +319,22 @@ l0_dgx_b200:
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTEDSL-mtp_nextn=2-ep4-fp8kv=False-attention_dp=True-cuda_graph=False-overlap_scheduler=False-low_precision_combine=True-torch_compile=False]
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTEDSL-mtp_nextn=2-ep4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=True-torch_compile=False]
- accuracy/test_llm_api_pytorch.py::TestLlama3_3_70BInstruct::test_fp4_tp2pp2[torch_compile=False-enable_gemm_allreduce_fusion=False]
- examples/test_visual_gen.py::test_visual_gen_quickstart
- examples/test_visual_gen.py::test_visual_gen_api_walkthrough
- examples/test_visual_gen.py::test_wan_t2v_example
- examples/test_visual_gen.py::test_flux1_lpips_against_golden
- examples/test_visual_gen.py::test_flux2_lpips_against_golden
- examples/test_visual_gen.py::test_ltx2_lpips_against_golden
- examples/test_visual_gen.py::test_wan21_t2v_lpips_against_golden
- examples/test_visual_gen.py::test_wan22_t2v_lpips_against_golden
- examples/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[ulysses4]
- examples/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[cfg2_ulysses2]
- examples/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[ulysses2_ring2]
- examples/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_multi_gpu[attn2d_2x2]
- examples/test_visual_gen.py::test_vbench_dimension_score_wan
- examples/test_visual_gen.py::test_vbench_dimension_score_wan22_a14b_fp8
- examples/test_visual_gen.py::test_vbench_dimension_score_wan22_a14b_nvfp4
- visual_gen/test_visual_gen_benchmark.py::test_offline_benchmark
- visual_gen/test_visual_gen_benchmark.py::test_online_benchmark[openai-videos]
- examples/test_visual_gen.py::test_vbench_dimension_score_ltx2_bf16
- examples/test_visual_gen.py::test_vbench_dimension_score_ltx2_fp8
- accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B_Instruct_2507::test_skip_softmax_attention_4gpus[target_sparsity_0.5-fp8kv=False]
- accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B_Instruct_2507::test_skip_softmax_attention_4gpus[target_sparsity_0.5-fp8kv=True]
- accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B_Instruct_2507::test_skip_softmax_attention_4gpus[target_sparsity_0.9-fp8kv=False]
- accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B_Instruct_2507::test_skip_softmax_attention_4gpus[target_sparsity_0.9-fp8kv=True]
- disaggregated/test_disaggregated.py::test_disaggregated_mamba_conc_greater_than_mbs[NVIDIA-Nemotron-3-Super-120B-A12B-FP8]
- accuracy/test_llm_api_pytorch.py::TestDeepSeekV32::test_nvfp4_attn_multi_gpus TIMEOUT (60)
# ------------- AutoDeploy Backend Stages ---------------
- condition:
ranges:
Expand Down
Loading