[zhyncs]

Motivation

➜  tests git:(zhyncs/math) ls | grep test_per | xargs -n 1 python3
INFO 03-13 09:03:35 __init__.py:190] Automatically detected platform cuda.
=================================================================================== test session starts ===================================================================================
platform linux -- Python 3.10.12, pytest-8.3.5, pluggy-1.5.0
rootdir: /sgl-workspace/sglang/sgl-kernel
configfile: pyproject.toml
plugins: anyio-4.8.0, typeguard-4.3.0
collected 9 items

test_per_tensor_quant_fp8.py .........                                                                                                                                              [100%]

==================================================================================== warnings summary =====================================================================================
../../../../usr/local/lib/python3.10/dist-packages/_pytest/config/__init__.py:1277
  /usr/local/lib/python3.10/dist-packages/_pytest/config/__init__.py:1277: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: anyio
    self._mark_plugins_for_rewrite(hook)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================== 9 passed, 1 warning in 0.53s ===============================================================================
=================================================================================== test session starts ===================================================================================
platform linux -- Python 3.10.12, pytest-8.3.5, pluggy-1.5.0
rootdir: /sgl-workspace/sglang/sgl-kernel
configfile: pyproject.toml
plugins: anyio-4.8.0, typeguard-4.3.0
collected 240 items

test_per_token_group_quant_fp8.py ................................................................................................................................................. [ 60%]
...............................................................................................                                                                                     [100%]

==================================================================================== warnings summary =====================================================================================
../../../../usr/local/lib/python3.10/dist-packages/_pytest/config/__init__.py:1277
  /usr/local/lib/python3.10/dist-packages/_pytest/config/__init__.py:1277: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: anyio
    self._mark_plugins_for_rewrite(hook)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================= 240 passed, 1 warning in 1.25s ==============================================================================
INFO 03-13 09:03:46 __init__.py:190] Automatically detected platform cuda.
=================================================================================== test session starts ===================================================================================
platform linux -- Python 3.10.12, pytest-8.3.5, pluggy-1.5.0
rootdir: /sgl-workspace/sglang/sgl-kernel
configfile: pyproject.toml
plugins: anyio-4.8.0, typeguard-4.3.0
collected 9 items

test_per_token_quant_fp8.py .........                                                                                                                                               [100%]

==================================================================================== warnings summary =====================================================================================
../../../../usr/local/lib/python3.10/dist-packages/_pytest/config/__init__.py:1277
  /usr/local/lib/python3.10/dist-packages/_pytest/config/__init__.py:1277: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: anyio
    self._mark_plugins_for_rewrite(hook)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================== 9 passed, 1 warning in 0.51s ===============================================================================

Modifications

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[zhyncs]

@elfiegg @hebiao064 @zcnrex @qingquansong @BBuf

[yiakwy-xpu-ml-framework-team]

感觉这个影响更大，因为 fast_math 实现是 用 多项式拟合，超越函数，倒数，基本都是按taylor展开写，应该会掉很多精度。

[hebiao064]

We need to be very cautious about it: https://simonbyrne.github.io/notes/fastmath/

At the same time we will verify the accuracy of per token quant asap in our recommendation use cases