[DeepSeek-V4] Emit fp32 directly in linear_bf16_fp32 (drop .float() copy)

[AMD-yanfeiwang]

Motivation

In the DSV4 indexer kv_score path, linear_bf16_fp32 (AITER branch) computes a bf16 GEMM and then casts to fp32 with .float():

return tgemm.mm(x, y, otype=x.dtype).float()

Profiling DeepSeek-V4 (8xMI355, tp=8) shows this .float() as a separate bfloat16tofloat32_copy kernel launched once per call (per layer, per step), e.g. output shapes [256, {512,1024,2048}], launched from compute_kv_score -> linear_bf16_fp32. It is small individually but recurs every decode step, and computing in bf16 then up-casting also loses precision.

Modifications

• Request fp32 output directly from the GEMM (otype=torch.float32) so the up-cast is fused into the epilogue and the separate copy kernel is removed.

Dependency

This is the consumer-side half. To actually avoid the copy (rather than relocate it), AITER must expose a native fp32-output tuned kernel for these bf16xbf16 shapes — companion: ROCm/aiter#3836. Without it, tgemm falls back internally (correctness unaffected, but the copy/torch-fallback may remain), which is why this is kept as a draft until the AITER configs land.

Accuracy / Speed

• Functionally identical math; emitting fp32 directly is strictly more accurate than bf16-GEMM + up-cast (avoids intermediate bf16 rounding).
• Removes one bfloat16tofloat32_copy kernel per linear_bf16_fp32 call.

Checklist

• Land companion AITER fp32-output GEMM tuned configs ([DSV4] Add fp32-output untuned GEMM shapes for indexer kv_score ROCm/aiter#3836)
• Re-profile to confirm bfloat16tofloat32_copy is gone in the kv_score path
• Confirm no GEMM perf regression vs the current tuned bf16 kernel + cast

---

CI States

Latest PR Test (Base): ❌ Run #27928226891
Latest PR Test (Extra): ❌ Run #27928226844

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[github-actions]

Thanks @AMD-yanfeiwang. Closing this because it is still a draft and has not been updated in 87 days.

Reopen it if the work is still relevant.

Some directories moved recently, so an older branch may need retargeting:
sgl-kernel/ -> python/sglang/kernels/aot/, python/sglang/jit_kernel/
-> python/sglang/kernels/jit/, docs/ -> docs/docs/ (.mdx),
bench_serving.py -> benchmark/serving.py, test/srt/ -> test/registered/.