Skip to content

[NPU][CI] Test Qwen3-235B on CANN 9.0/9.1 with upstream PR 40814 - #1280

Open
McZyWu wants to merge 2 commits into
Ascend:testcasesfrom
McZyWu:ci/qwen3-235b-cann90-cann91-pr40814
Open

McZyWu wants to merge 2 commits into
Ascend:testcasesfrom
McZyWu:ci/qwen3-235b-cann90-cann91-pr40814

Conversation

@McZyWu

@McZyWu McZyWu commented Sep 28, 2026 •

Copy link
Copy Markdown

Motivation

Run the existing Qwen3-235B-A22B W8A8 performance case from testcases on both CANN 9.0.0 and 9.1.0 A3 images, ensuring that the runtime includes sgl-project/sglang#40814. The images may predate that fix.

Modifications

  • Update .github/workflows/single-test-npu.yml and retain its pull-request trigger against testcases.
  • Run two independent matrix jobs on linux-aarch64-a3-16, using:
    • swr.cn-southwest-2.myhuaweicloud.com/base_image/dockerhub/lmsysorg/sglang:main-cann9.0.0-a3
    • swr.cn-southwest-2.myhuaweicloud.com/base_image/dockerhub/lmsysorg/sglang:main-cann9.1.0-a3
  • Before testing, apply the change from upstream commit e98ffc4cc699d7953f17e751ce669043a7e17bcb directly to /sgl-workspace/sglang, preserving the image's CANN, PyTorch and kernel dependencies. Accept an already-applied patch only when the reverse check succeeds; fail if the patch cannot be applied or verified. Verify that Python resolves the image's SGLang package and that graph.replay() precedes update_future.result().
  • Run test/registered/npu/performance/qwen3_235b_a22b/test_npu_qwen3_235b_w8a8_8p_in3k5_out1k5_50ms.py, with the testcase and Ascend test helpers from this testcases-based checkout. Despite the 8p filename, its existing arguments are --nnodes 1 --tp 16 --dp-size 16.
  • Use the explicit --cuda-graph-bs-decode option in the testcase to avoid the ambiguous --cuda-graph-bs prefix now shared by decode and prefill options. Keep the existing batch-size values.
  • Use a 300-minute test timeout and upload separate logs and the applied patch for each image, including on failure.

Accuracy Tests

No model or accuracy-threshold changes. The testcase only changes the decode graph argument name. Hardware execution is delegated to the PR's NPU CI jobs.

Speed Tests and Profiling

The selected case retains its existing 3,500-input / 1,500-output token workload, concurrency 432 and performance thresholds. CANN 9.0.0 and 9.1.0 results are pending the two CI jobs.

Local validation:

  • Actionlint 1.7.12 with the repository's custom A3 runner label configured: passed.
  • YAML parsing, Bash syntax, embedded Python syntax and git diff --check: passed.
  • Testcase AST, isort, Ruff and registered-test validation: passed.
  • Applied the embedded patch to the actual upstream parent source and verified that the resulting file exactly matches upstream commit e98ffc4cc699d7953f17e751ce669043a7e17bcb. Verified the already-applied and incompatible-source checks.
  • Applicable pre-commit checks passed except two Windows launcher issues (application-control block on the destroyed-symlinks executable and unavailable python3 alias). Both underlying checks were then run directly with Python and passed.

CI States

Latest PR Test (Base): ❌ Run #36369201551
Latest PR Test (Extra): ✅ Run #36369209152
Latest PR Test (AMD ROCm 7.2): ❌ Run #36369201498

@ascend-robot

Copy link
Copy Markdown

CLA Signature Pass

McZyWu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

@github-actions github-actions Bot added the npu label Sep 28, 2026
@ascend-robot

Copy link
Copy Markdown

CLA Signature Pass

McZyWu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants