[AMD] Add Qwen3-Coder-Next accuracy and functionality test scripts for MI35x 8-GPU by yichiche · Pull Request #18608 · sgl-project/sglang

yichiche · 2026-02-11T11:02:57Z

Motivation

Add CI test coverage for the Qwen3-Coder-Next model on AMD MI35x (gfx950) 8-GPU systems. This model features a hybrid architecture combining full attention (GQA) with linear attention (GDN/Gated Delta Net) and MoE (512 experts), requiring dedicated test scripts to validate accuracy and performance on AMD hardware. The tests ensure the aiter attention backend, fp8 KV cache quantization, and chunked prefill work correctly with this model's unique architecture across both basic and MTP (speculative decoding) configurations.

Modifications

test/registered/amd/test_qwen3_coder_next_8gpu.py:
- Added functionality test registered under stage-c-test-large-8-gpu-amd-mi35x suite. Includes TestQwen3CoderNext class testing GSM8K few-shot accuracy (threshold >0.90) and BS=1 decode speed (>50 tok/s) with TP=8, aiter backend, and fp8 KV cache. Also includes TestQwen3CoderNextMTP class (currently skipped) for MTP/EAGLE speculative decoding — disabled because Triton extend_attention does not yet support fp8 KV cache on gfx950.
test/registered/amd/accuracy/mi35x/test_qwen3_coder_next_eval_mi35x.py:
- Added nightly accuracy evaluation test registered under nightly-amd-8-gpu-mi35x suite. Tests two model variants: basic (with --kv-cache-dtype fp8_e4m3) and MTP (without fp8 KV cache), both using 200-question GSM8K few-shot completion benchmark with accuracy threshold of 0.90.

Accuracy Tests

Benchmarking and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

Review Process

Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
Get approvals from CODEOWNERS and other reviewers.
Trigger CI tests with comments or contact authorized users to do so.
- /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
After green CI and required approvals, ask Merge Oncalls to merge.

gemini-code-assist · 2026-02-11T11:03:14Z

Summary of Changes

Hello @yichiche, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the testing infrastructure by integrating new CI tests for the Qwen3-Coder-Next model on AMD MI35x 8-GPU hardware. The added tests ensure the correct operation and performance of the model's advanced features, such as its attention backend, KV cache quantization, and prefill strategies, across different operational modes. This expansion of test coverage is crucial for maintaining stability and validating the model's capabilities on AMD platforms.

Highlights

New CI Test Coverage: Added comprehensive CI test coverage for the Qwen3-Coder-Next model specifically tailored for AMD MI35x (gfx950) 8-GPU systems.
Model Architecture Validation: Tests validate the model's unique hybrid architecture, including the aiter attention backend, fp8 KV cache quantization, and chunked prefill mechanisms.
Functionality and Accuracy Tests: Introduced functionality tests covering GSM8K few-shot accuracy and batch-size 1 decode speed, alongside dedicated nightly accuracy evaluation tests for both basic and MTP (speculative decoding) configurations.
MTP Test Status: The MTP (speculative decoding) functionality tests are currently skipped due to a known limitation where Triton's extend_attention does not yet support fp8 KV cache on gfx950.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

test/registered/amd/accuracy/mi35x/test_qwen3_coder_next_eval_mi35x.py
- Added a new nightly accuracy evaluation test for the Qwen3-Coder-Next model on MI35x.
- Configured tests for both basic and MTP variants, with specific KV cache settings.
- Implemented GSM8K few-shot completion benchmark with an accuracy threshold of 0.90.
test/registered/amd/test_qwen3_coder_next_8gpu.py
- Added a new functionality test for the Qwen3-Coder-Next model on MI35x 8-GPU systems.
- Included tests for GSM8K few-shot accuracy (threshold >0.90) and BS=1 decode speed (>50 tok/s).
- Temporarily skipped the MTP test class due to current limitations with fp8 KV cache support on gfx950.

Activity

No human activity has been recorded on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces two new test scripts to validate the functionality and accuracy of the Qwen3-Coder-Next model on AMD MI35x GPUs, which is a valuable addition for CI coverage. The code is well-structured, but I have identified a few areas for improvement concerning code duplication, maintainability, and test correctness. My main suggestions are to refactor the code to eliminate duplicated logic by reusing existing utilities, simplify model configurations, and ensure all test assertions are active to properly validate the model's performance.

test/registered/amd/accuracy/mi35x/test_qwen3_coder_next_eval_mi35x.py

test/registered/amd/test_qwen3_coder_next_8gpu.py

HaiShaw · 2026-02-26T20:42:48Z

/tag-and-rerun-ci

HaiShaw

Largely okay, please take care of a few comments

yichiche · 2026-03-02T07:05:32Z

Largely okay, please take care of a few comments

Thanks @HaiShaw, I have incorporated the feedback and completed the refactoring. Everything is now ready.

…ulative decoding cuda graph replay Guard `_use_mla_ps_kernel` checks with `self.use_mla` in `init_forward_metadata_replay_cuda_graph` for `is_target_verify()` and `is_draft_extend()` branches, matching the existing pattern in `init_forward_metadata_capture_cuda_graph`. `_use_mla_ps_kernel` is a global variable set to True when any MLA-enabled AiterAttnBackend instance is initialized. `self.max_split_per_batch` is only set inside `if self.use_mla:` during __init__. When HybridLinearAttnBackend iterates over its backend list (for hybrid attention models like Qwen3-Coder-Next), a non-MLA AiterAttnBackend instance would enter the `if _use_mla_ps_kernel:` block and crash with: AttributeError: 'AiterAttnBackend' object has no attribute 'max_split_per_batch' This was exposed by the Qwen3-Coder-Next MTP test added in #18608.

…r MI35x 8-GPU (sgl-project#18608)

[AMD] Add test script for qwen3-coder-next

524e265

gemini-code-assist bot reviewed Feb 11, 2026

View reviewed changes

yctseng0211 added amd run-ci labels Feb 11, 2026

Merge branch 'main' into test-qwen3-coder-next-hip

c9603c2

yichiche force-pushed the test-qwen3-coder-next-hip branch from c9603c2 to e6f4803 Compare February 25, 2026 09:07

yichiche mentioned this pull request Feb 26, 2026

[AMD] Support Qwen3-Coder-Next on AMD platform #18355

Merged

5 tasks

HaiShaw reviewed Mar 2, 2026

View reviewed changes

[AMD] Refactor test script for better readibility

7a246aa

yichiche force-pushed the test-qwen3-coder-next-hip branch from e6f4803 to 7a246aa Compare March 2, 2026 06:55

HaiShaw approved these changes Mar 2, 2026

View reviewed changes

HaiShaw merged commit 98f47d8 into sgl-project:main Mar 2, 2026
64 of 91 checks passed

michaelzhang-ai mentioned this pull request Mar 3, 2026

[AMD] Fix Qwen3-Coder-Next: Add missing k_scale/v_scale args to extend_attention_fwd in aiter_backend #19736

Merged

1 task

Kangyan-Zhou pushed a commit to Kangyan-Zhou/sglang that referenced this pull request Mar 4, 2026

[AMD] Add Qwen3-Coder-Next accuracy and functionality test scripts fo…

f2d272a

…r MI35x 8-GPU (sgl-project#18608)

magicYang1573 pushed a commit to magicYang1573/sglang that referenced this pull request Mar 9, 2026

[AMD] Add Qwen3-Coder-Next accuracy and functionality test scripts fo…

61382fd

…r MI35x 8-GPU (sgl-project#18608)

Wangzheee pushed a commit to Wangzheee/sglang that referenced this pull request Mar 21, 2026

[AMD] Add Qwen3-Coder-Next accuracy and functionality test scripts fo…

609bd69

…r MI35x 8-GPU (sgl-project#18608)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[AMD] Add Qwen3-Coder-Next accuracy and functionality test scripts for MI35x 8-GPU#18608

[AMD] Add Qwen3-Coder-Next accuracy and functionality test scripts for MI35x 8-GPU#18608
HaiShaw merged 3 commits intosgl-project:mainfrom
yichiche:test-qwen3-coder-next-hip

yichiche commented Feb 11, 2026

Uh oh!

gemini-code-assist bot commented Feb 11, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HaiShaw commented Feb 26, 2026

Uh oh!

HaiShaw left a comment

Uh oh!

yichiche commented Mar 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

yichiche commented Feb 11, 2026

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Review Process

Uh oh!

gemini-code-assist bot commented Feb 11, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HaiShaw commented Feb 26, 2026

Uh oh!

HaiShaw left a comment

Choose a reason for hiding this comment

Uh oh!

yichiche commented Mar 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants