[AMD] CI - Add MI35x nightly/PR tests for kv-cache-fp8 and allreduce-fusion (DeepSeek)

[yctseng0211]

Motivation

Track accuracy and performance regression for two new DeepSeek-R1-MXFP4 server configurations on MI35x:

• --kv-cache-dtype fp8_e4m3
• --enable-aiter-allreduce-fusion

Update work_flow dispatch mechanism to enable multi-job triggering.
Add PD/D test to pr-test-amd-rocm720.yml

Modifications

• Add 6 test files under test/registered/amd/
• Add corresponding nightly jobs in nightly-test-amd.yml and nightly-test-amd-rocm720.yml

Accuracy Tests

Nightly Test:

• PASSED nightly-8-gpu-mi35x-deepseek-r1-mxfp4-kv-fp8
• PASSED nightly-8-gpu-mi35x-deepseek-r1-mxfp4-ar-fusion
• PASSED nightly-8-gpu-mi35x-deepseek-r1-mxfp4-ar-fusion-rocm720
• PASSED nightly-8-gpu-mi35x-deepseek-r1-mxfp4-kv-fp8-rocm720
• PASSED nightly-8-gpu-deepseek-v3-kv-fp8
• PASSED nightly-8-gpu-deepseek-v3-kv-fp8-rocm720

PR Test:

• PASSED stage-c-test-large-8-gpu-amd

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[yctseng0211]

@bingxche @michaelzhang-ai please help review, thanks!
Nightly Test:

• PASSED nightly-8-gpu-mi35x-deepseek-r1-mxfp4-kv-fp8
• PASSED nightly-8-gpu-mi35x-deepseek-r1-mxfp4-ar-fusion
• PASSED nightly-8-gpu-mi35x-deepseek-r1-mxfp4-kv-fp8-rocm720
• PASSED nightly-8-gpu-mi35x-deepseek-r1-mxfp4-kv-fp8-rocm720

PR Test:

• PASSED stage-c-test-large-8-gpu-amd

[michaelzhang-ai]

Please double check with test args.
Inconsistent server args between accuracy and perf tests:

The accuracy tests launch the server with --attention-backend aiter and SGLANG_USE_AITER=1, but the corresponding perf tests omit both.

Accuracy test (test_deepseek_r1_mxfp4_kv_fp8_eval_mi35x.py):

other_args=[
    "--attention-backend", "aiter",
    "--chunked-prefill-size", "131072",
    "--disable-radix-cache",
    "--mem-fraction-static", "0.85",
    "--trust-remote-code",
    "--kv-cache-dtype", "fp8_e4m3",
],
env_vars={"SGLANG_USE_AITER": "1"},

Perf test (test_deepseek_r1_mxfp4_kv_fp8_perf_mi35x.py):

"other_args": [
    "--trust-remote-code",
    "--tp", "8",
    "--chunked-prefill-size", "131072",
    "--disable-radix-cache",
    "--mem-fraction-static", "0.85",
    "--kv-cache-dtype", "fp8_e4m3",
],

This means the perf tests benchmark a different server configuration than what the accuracy tests validate. If the aiter backend is needed for the MXFP4 model on MI35x, the perf numbers won't reflect production behavior. The same issue exists for the allreduce-fusion variant.

[yctseng0211]

Please double check with test args. 1. Inconsistent server args between accuracy and perf tests (potential bug)

The accuracy tests launch the server with --attention-backend aiter and SGLANG_USE_AITER=1, but the corresponding perf tests omit both.

Accuracy test (test_deepseek_r1_mxfp4_kv_fp8_eval_mi35x.py):

other_args=[
    "--attention-backend", "aiter",
    "--chunked-prefill-size", "131072",
    "--disable-radix-cache",
    "--mem-fraction-static", "0.85",
    "--trust-remote-code",
    "--kv-cache-dtype", "fp8_e4m3",
],
env_vars={"SGLANG_USE_AITER": "1"},

Perf test (test_deepseek_r1_mxfp4_kv_fp8_perf_mi35x.py):

"other_args": [
    "--trust-remote-code",
    "--tp", "8",
    "--chunked-prefill-size", "131072",
    "--disable-radix-cache",
    "--mem-fraction-static", "0.85",
    "--kv-cache-dtype", "fp8_e4m3",
],

This means the perf tests benchmark a different server configuration than what the accuracy tests validate. If the aiter backend is needed for the MXFP4 model on MI35x, the perf numbers won't reflect production behavior. The same issue exists for the allreduce-fusion variant.

@michaelzhang-ai thanks for your review, the configs are based on the existing amd nightly test

[bingxche]

LGTM