[DSV4] hc-prenorm: fuse the combine step into a Triton kernel

[AliceChenyy]

Purpose

The torch hc-prenorm path in DeepSeek-V4 combines the sinkhorn weights with the
flattened activations as

y = (pre.squeeze(1).unsqueeze(-1) * x_flat.view(M, HC, H)).sum(dim=1).to(dtype)

which materialises an [M, HC, H] intermediate and then reduces it. hc_combine
accumulates over HC in registers in fp32 and writes [M, H] once.

Measurements

RTX 6000D, bf16, HC=4, H=7168:

M	before	after	speedup
1	11.2 us	9.7 us	1.16x
8	10.4 us	9.3 us	1.12x
64	10.7 us	9.3 us	1.15x
1024	133.8 us	12.6 us	10.60x
4096	647.7 us	231.3 us	2.80x
16384	2533.1 us	920.3 us	2.75x

Numerical agreement with the previous expression, max relative error over the
reduction:

shape	dtype	rel err
m=7 hc=4 h=128	bf16	4.61e-03
m=1024 hc=4 h=512	bf16	4.18e-03
m=33 hc=8 h=256	fp16	6.53e-04
m=2048 hc=4 h=1024	bf16	3.52e-03
m=1 hc=4 h=64	bf16	3.42e-03

Within bf16 rounding. The fused kernel accumulates in fp32; the previous expression
accumulated in the input dtype, so the fused result is the more accurate of the two.

Context

Split out of #29927. That PR is SM120 enablement; this change is arch-independent and
was flagged in review as the kind of unrelated optimization that should be its own PR.

Measured on SM120 only — the datacenter part I had access to is no longer available. The
kernel itself has no arch-specific code, and correctness is pinned by the numerical
comparison above, but I have not measured the speedup on SM90/SM100. Happy to add those
numbers if someone can run the microbenchmark, or to hold the PR until I can.

---

CI States

Latest PR Test (Base): ❌ Run #33482048236
Latest PR Test (Extra): ❌ Run #33482048167
Latest PR Test (AMD ROCm 7.2): ❌ Run #33482048264

[Fridge003]

/rerun-test test/registered/kernels/ops/layernorm/test_hc_combine.py test/registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py

[github-actions]

⚠️ Rebase Required Before Re-run

A major update has landed on main. Your PR is diverged relative to required base commit 5972211.

Re-run was not dispatched. What to do:

• Rebase your branch onto the latest main and push again
• Follow issue #21065 for context
• CI-fix PRs may request the bypass-maintenance label to skip this check

[Fridge003]

/rerun-test test/registered/kernels/ops/layernorm/test_hc_combine.py test/registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py

[github-actions]

Results for /rerun-test test/registered/kernels/ops/layernorm/test_hc_combine.py test/registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py:

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/kernels/ops/layernorm/test_hc_combine.py

🚀 4-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/models_e2e/test_deepseek_v4_flash_fp4_b200.py

[iforgetmyname]

Traceback (most recent call last):
  File "/__w/sglang/sglang/test/run_suite.py", line 532, in <module>
    main()
  File "/__w/sglang/sglang/test/run_suite.py", line 527, in main
    exit_code = run_a_suite(args)
                ^^^^^^^^^^^^^^^^^
  File "/__w/sglang/sglang/test/run_suite.py", line 370, in run_a_suite
    all_tests = collect_tests(files, sanity_check=sanity_check)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/__w/sglang/sglang/python/sglang/test/ci/ci_register.py", line 410, in collect_tests
    raise ValueError(
ValueError: /__w/sglang/sglang/test/registered/kernels/ops/layernorm/test_hc_combine.py: missing `if __name__ == "__main__":` entry. Pytest-style tests in this file will silently skip under `python3 file.py -f`. Add `unittest.main()` (for unittest.TestCase) or `sys.exit(pytest.main([__file__, "-v"]))` (for pytest-style).
[ERROR] 2026-09-01-10:21:11 (PID:10938, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception

Hi @Fridge003, looks like the newly-added test file breaks pytest suite. Could you please give it a check?

action: https://github.com/sgl-project/sglang/actions/runs/33493738565/job/99811468552?pr=37399

update: bugfix #37406