Skip to content

[CI] Move tests that do not need GB300 off the per-PR GB300 suite - #43784

Merged
Fridge003 merged 2 commits into
mainfrom
mmangkad/ci-gb300-suite-rebalance
Oct 12, 2026
Merged

Fridge003 merged 2 commits into
mainfrom
mmangkad/ci-gb300-suite-rebalance

Conversation

@mmangkad

@mmangkad mmangkad commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Run tests that do not depend on GB300 hardware on other runners, and replace one end-to-end class with CPU unit tests.

Test From To
kernels/ops/attention/test_flash_attention_4_sm103.py 4-gpu-gb300 base-c 8-gpu-b300 base-c (B300 is also SM103)
kernels/ops/attention/test_kda_prefill.py 4-gpu-gb300 base-c 8-gpu-b300 base-c; the 4-gpu-b200 registration is unchanged
ep/test_flashinfer_a2a.py 4-gpu-gb300 base-c 4-gpu-b200 nightly; GB300-specific NCCL port pin removed; GLM-5.2 static-dispatch class replaced by unit tests (below)
e2e/models_large/test_deepseek_v3_cutedsl_4gpu.py 4-gpu-gb300 base-c 4-gpu-b200 nightly; GB300-specific NCCL port pin removed
e2e/models/test_deepseek_v41_flash_gb300.py est_time=1800 est_time=1100, the measured runtime

disaggregation_aarch64, cuda_vmm_transport and numa_utils stay on 4-gpu-gb300 base-c.

Why nightly is enough for the two moved end-to-end tests: the components they exercise are already checked per-PR:

Component Per-PR test
FlashInfer A2A dispatcher unit/layers/moe/test_flashinfer_dispatcher.py, unit/layers/moe/test_flashinfer_a2a_wide_ep.py
NVFP4 MoE runners (CuteDSL, TRT-LLM) unit/layers/quantization/test_nvfp4_moe_backends.py
DeepEP dispatch e2e/models/test_deepseek_v4_flash_fp4_b200.py, ep/test_deepep_small.py

These two files add end-to-end accuracy across models, which does not need per-PR cadence.

FlashInfer A2A top-k unit tests

#29929 fixed two things in FlashinferDispatcher, which feeds FlashInfer's MoeAlltoAll:

  • it casts expert IDs to int32, because static EP dispatch produces int64 and MoeAlltoAll rejects it;
  • it materializes bypassed top-k output, from runners that fuse routing, before dispatch.

The GLM-5.2 class in test_flashinfer_a2a.py covered only the first, by launching a full GLM-5.2-NVFP4 server and asserting a non-empty /generate response. It used the CuteDSL runner, which produces standard top-k, so the second fix had no test.

Two cases in TestFlashinferDispatcher (test_flashinfer_dispatcher.py) now cover both on CPU, per-PR, with a recording MoeAlltoAll:

Case Asserts With that part of the fix removed
test_int64_expert_ids_reach_moe_a2a_as_int32 int64 IDs arrive as int32 (argument and expert-ID payload), values unchanged AssertionError: torch.int64 != torch.int32
test_bypassed_topk_is_routed_before_moe_a2a a bypassed top-k output arrives as routed int32 expert IDs plus weights AttributeError: 'BypassedTopKOutput' object has no attribute 'topk_ids'

The file's existing case now lives in the same CustomTestCase class, with its assertions unchanged.

FlashInfer A2A with the CuteDSL runner stays covered end to end by moe/test_flashinfer_a2a_cutedsl_v2.py (extra-b) and e2e/models_large/test_nvidia_nemotron_3_super_nvfp4.py (nightly). The GLM class used the standard top-k format, so no bypassed-format coverage is lost. test_flashinfer_a2a.py drops from about 544s to about 360s.

Shard counts

From compute_partitions with the live timing model:

Suite main this branch
base-c-test-4-gpu-gb300 3 1
base-c-test-8-gpu-b300 1 1
nightly-test-4-gpu-b200 2 2

Test plan

  • pre-commit, including the registered-test CI registry check
  • test_flashinfer_dispatcher.py on CPU: 3 passed; each new case fails with its part of the fix removed
  • 4-gpu-b200: test_flashinfer_a2a.py, test_deepseek_v3_cutedsl_4gpu.py (run)
  • 8-gpu-b300: test_flash_attention_4_sm103.py, test_kda_prefill.py (run)

CI States

Latest PR Test (Base): ❌ Run #38213861067
Latest PR Test (Extra): ❌ Run #38213860851
Latest PR Test (AMD ROCm 10): ❌ Run #38213861027

Mohammad Angkad added 2 commits October 12, 2026 14:32
- test_flash_attention_4_sm103, test_kda_prefill: run on 8-gpu-b300
  base-c; B300 is also SM103.
- test_flashinfer_a2a, test_deepseek_v3_cutedsl_4gpu: run nightly on
  4-gpu-b200. The FlashInfer A2A dispatcher, the NVFP4 MoE runners and
  DeepEP dispatch are already checked per-PR (test_flashinfer_dispatcher,
  test_flashinfer_a2a_wide_ep, test_nvfp4_moe_backends,
  test_deepseek_v4_flash_fp4_b200); these files add end-to-end accuracy
  across models, which does not need per-PR cadence. Drop their
  GB300-specific NCCL port pins.
- test_deepseek_v41_flash_gb300: set est_time to the measured runtime.
- test_flashinfer_dispatcher: add regression cases for both parts of
  #29929. One dispatches int64 expert IDs, as static EP
  dispatch produces, and asserts MoeAlltoAll receives int32. The other
  dispatches a bypassed top-k output, as runners that fuse routing produce,
  and asserts MoeAlltoAll receives routed int32 expert IDs and weights.
  Both run per-PR on CPU. Move the file's cases into a CustomTestCase class.
- test_flashinfer_a2a: drop the GLM-5.2 static-dispatch class, which
  checked the expert-ID dtype with a full server launch. FlashInfer A2A
  with the CuteDSL runner stays covered by test_flashinfer_a2a_cutedsl_v2
  and test_nvidia_nemotron_3_super_nvfp4.
@mmangkad

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/e2e/models/test_deepseek_v41_flash_gb300.py test/registered/e2e/models_large/test_deepseek_v3_cutedsl_4gpu.py test/registered/ep/test_flashinfer_a2a.py test/registered/kernels/ops/attention/test_flash_attention_4_sm103.py test/registered/kernels/ops/attention/test_kda_prefill.py test/registered/unit/layers/moe/test_flashinfer_dispatcher.py

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

🚀 4-gpu-gb300 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300.py

🚀 4-gpu-b200 (3 tests): ✅ View workflow run

cd test/ && python3 registered/e2e/models_large/test_deepseek_v3_cutedsl_4gpu.py
cd test/ && python3 registered/ep/test_flashinfer_a2a.py
cd test/ && python3 registered/kernels/ops/attention/test_kda_prefill.py

🚀 8-gpu-b300 (2 tests): ✅ View workflow run

cd test/ && python3 registered/kernels/ops/attention/test_flash_attention_4_sm103.py
cd test/ && python3 registered/kernels/ops/attention/test_kda_prefill.py

🚀 ubuntu-latest (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/layers/moe/test_flashinfer_dispatcher.py

@Fridge003
Fridge003 merged commit 0688782 into main Oct 12, 2026
129 of 141 checks passed
@Fridge003
Fridge003 deleted the mmangkad/ci-gb300-suite-rebalance branch October 12, 2026 07:38
Liwansi added a commit to Liwansi/sglang_vans that referenced this pull request Oct 13, 2026
…n_allgather

* 'main' of https://github.com/sgl-project/sglang: (236 commits)
  [NPU] Stabilize HiCache and registered CI tests (sgl-project#43308)
  [Docs] Cookbook: refresh SemiAnalysis AgentX launches (GLM-5.2 B300, DeepSeek-V4.1 GB300) (sgl-project#43799)
  [Docs] Update GLM-5.3-Flash playground: B300 default, CP8, EP and PD (sgl-project#43802)
  [DSV4.1][PD][1/N] Support prefill node use cp parallel strategy (sgl-project#40323)
  [BCG][DSV4-Flash][CP] Support DeepSeek V4 prefill with interleave CP and breakable CUDA graphs (sgl-project#38603)
  [XPU] Support decode context parallelism (DCP) on Intel XPU (sgl-project#34355)
  Keep NIXL Mamba checkpoint keys separate by TP rank (sgl-project#43594)
  Capture KV host transfer buffers (sgl-project#43711)
  [CI] Move tests that do not need GB300 off the per-PR GB300 suite (sgl-project#43784)
  [HiCache] Preserve all MTP KV and recurrent state (sgl-project#40223)
  [HiCache] Support Mamba in buffer-only host memory mode (sgl-project#36345)
  [PD] Poison the deferred abort ack when a Mooncake send fails (sgl-project#43719)
  [Perf] Route BF16 mixed-batch decode tails through decode attention (sgl-project#43238)
  [Fix] Link cuda_ipc JIT against CUDA driver stubs dir (sgl-project#43764)
  docs: clarify prefill CP terminology and simplify wording (sgl-project#43787)
  [diffusion][xpu] FLUX.2: enable the bit-exact Triton SwiGLU on XPU (sgl-project#41940)
  [Fix] Adapt to transformers 5.18 / 5.19 changes (sgl-project#43688)
  [XPU CI] Optimize PR CI per-file timeouts and fail hanging tests fast (sgl-project#42048)
  [Deps] Bump FlashInfer to 0.7.1 (sgl-project#43494)
  docs: sync LMSYS SGLang blog cards (sgl-project#43462)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants