[sgl-kernel][1/2] Fused qk_norm_rope for Qwen3-MoE

[yuan-luo]

Motivation

This is the CUDA kernel to fuse Q K normalization and apply_rope_embedding.
More details refer to #13998.

➜  sglang_dev git:(fused_qk_norm_rope) ✗ python -m pytest ./sgl-kernel/tests/test_fused_qk_norm_rope.py
=============================================================================================================== test session starts ===============================================================================================================
platform linux -- Python 3.12.12, pytest-8.4.2, pluggy-1.6.0
rootdir: /sgl-workspace/workspace/sglang_dev/sgl-kernel
configfile: pyproject.toml
plugins: anyio-4.11.0, typeguard-4.4.4
collected 80 items

sgl-kernel/tests/test_fused_qk_norm_rope.py ................................................................................                                                                                                                [100%]

================================================================================================================ warnings summary =================================================================================================================
<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================================================= 80 passed, 2 warnings in 11.55s =========================================================================================================
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.
• Work with maintainers to merge your PR. See the PR Merge Process

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[Kangyan-Zhou]

/tag-and-rerun-ci

[BBuf]

LGTM