Skip to content

[TRTLLM-16020][test] Add Kimi K3 GSM8K accuracy tests to GB300 multi-node post-merge CI #18461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
moraxu wants to merge 5 commits into
base: main
Choose a base branch
from
+12 −4
Open

[TRTLLM-16020][test] Add Kimi K3 GSM8K accuracy tests to GB300 multi-node post-merge CI #18461

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6a09ee1
[TRTLLM-16020][test] Add Kimi K3 GSM8K accuracy tests to GB300 multi-…
moraxu Aug 31, 2026
4775875
[TRTLLM-16020][test] Document effective-timeout semantics on the K3 e…
moraxu Sep 1, 2026
7befbb2
[TRTLLM-16020][test] Reflow the K3 GSM8K docstring per review
moraxu Sep 1, 2026
d695d62
[TRTLLM-16020][test] Tighten the K3 GSM8K stage timeouts to 90 minutes
moraxu Sep 1, 2026
9fce386
[TRTLLM-16020][test] Trim the K3 entry comment to three lines per review
moraxu Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions tests/integration/defs/accuracy/test_llm_api_pytorch.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4652,16 +4652,19 @@ class TestKimiK3(LlmapiAccuracyTestHarness):
# so gate on GB300-class device memory. B300 clears this memory gate but
# pairs 8-GPU nodes over InfiniBand (same non-NVL72 topology) — do not
# schedule these tests on B300; that exclusion is enforced by QA's
# platform selection, not by this marker.
# platform selection and by the CI stage's gb300-only gpu wildcard,
Comment thread
moraxu marked this conversation as resolved.
# not by this marker.
@pytest.mark.skip_less_device_memory(200000)
@pytest.mark.parametrize("mode", ["baseline", "reuse", "sa"])
def test_w4a16_mxfp4(self, mode: str,
monkeypatch: pytest.MonkeyPatch) -> None:
"""GSM8K on the bf16 + MXFP4-routed-expert checkpoint (16 GPUs, DEP16).

No automated L0 stage schedules 16-GPU functional tests; this case is
registered in qa/llm_function_multinode.txt and run by QA's weekly
multinode pipeline (qualified on 4x4 GB300 nodes). Each mode mirrors
The baseline and sa legs run post-merge in the GB300 16-GPU 4-node CI
stage (test-db list l0_gb300_multi_nodes_node4_gpu16.yml); all three
modes of this test are also registered in qa/llm_function_multinode.txt
and run by QA's weekly multinode pipeline (qualified on 4x4 GB300
nodes; the reuse mode is QA-weekly-only). Each mode mirrors
the corresponding examples/kimi_k3/eval_extra_llm_options*.yaml config
- keep them in sync when editing either. The `sa` leg additionally
records spec-dec acceptance: AL/AR lines in the eval log (via
Expand Down
5 changes: 5 additions & 0 deletions tests/integration/test_lists/test-db/l0_gb300_multi_nodes_node4_gpu16.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,8 @@ l0_gb300_multi_nodes_node4_gpu16:
backend: pytorch
tests:
- accuracy/test_llm_api_pytorch.py::TestQwen3_8_2_4T_A95B::test_nvfp4_adp16_cutedsl TIMEOUT (90)
# Kimi K3 GSM8K accuracy guard (DEP16). The reuse and MMMU legs stay
# QA-weekly-only via qa/llm_function_multinode.txt, which also keeps
# running the two legs below.
- accuracy/test_llm_api_pytorch.py::TestKimiK3::test_w4a16_mxfp4[baseline] TIMEOUT (90)
Comment thread
moraxu marked this conversation as resolved.
- accuracy/test_llm_api_pytorch.py::TestKimiK3::test_w4a16_mxfp4[sa] TIMEOUT (90)
Loading