[WIP]enable mxfp8 on nvidia sm120

[wolfcomos]

Motivation

Following up to #17449 , sm120 indeed supports mxfp8 forward pass according to discussion from NVIDIA/TransformerEngine#2668.

Modifications

enabled sm120 mxfp8_block_scaled_matmul_triton kernel: added selection flag and set default sm120 machine stage size to 1 due to the shared memory constraint. Updated MoE assertions to include sm120 support

Accuracy Tests

Testing machine: RTX5070 TI

python3 benchmark/gsm8k/bench_sglang.py --num-shots 8 --num-questions 1000 --parallel 10 --platinum

python3 -m sglang.launch_server --model-path Qwen/Qwen2.5-0.5B-Instruct --port 30000

Baseline bf16:
Accuracy: 0.359
Invalid: 0.003
Latency: 57.425 s
Output throughput: 2188.128 token/s

python3 -m sglang.launch_server --model-path Qwen/Qwen2.5-0.5B-Instruct --quantization mxfp8 --port 30000

bf16 with online mxfp8 quantization
Accuracy: 0.348
Invalid: 0.003
Latency: 307.610 s
Output throughput: 414.304 token/s

unit test based on test_block_fp8.py

python3 -m pytest python/sglang/test/test_block_fp8.py::TestMXFP8DenseLinear -v


==================================== test session starts ====================================
platform linux -- Python 3.12.3, pytest-9.0.2, pluggy-1.6.0 -- /opt/venv-sglang-dev/bin/python3
cachedir: .pytest_cache
rootdir: /sglang-dev/sglang/python
configfile: pyproject.toml
plugins: anyio-4.12.1
collected 1 item                                                                            

python/sglang/test/test_block_fp8.py::TestMXFP8DenseLinear::test_mxfp8_dense_linear 
python/sglang/test/test_block_fp8.py::TestMXFP8DenseLinear::test_mxfp8_dense_linear PASSED [100%]

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[b8zhong]

/tag-and-rerun-ci