[Bugfix][Kernel] fix bias adding in triton kernel implemented fused moe

[xuebwang-amd]

Purpose

Since bias is typically not quantized, it should be added after the dequantization at last.
y = s_x * s_w * (Wq - zw) * (xq - zx) + bias
where:
s_x, s_w: scaling factors for activation and weight
Wq, xq: quantized weight and activation
zw, zx: zero points for weight and activation

Test Plan

• (end-2-end): amd/gpt-oss-20b-WFP8-AFP8-KVFP8, see results below
  • quantization scheme:
    • all weights (both attention and moe): FP8 per-tensor
    • all activation (both attention and moe): FP8 per-tensor
    • kv cache: FP8 per-tensor
• (TODO?) may need a unit test on triton implemented fused moe kernel

Test Result

Example (TP=2):

[xuebwang-amd]

Originated from PR#29008, #29008

[gemini-code-assist]

Code Review

This pull request addresses a correctness issue in the fused MoE Triton kernel by ensuring that bias addition and routed weight multiplication are performed after dequantization. The change correctly moves these operations to follow the scaling of the accumulator, which aligns with the standard mathematical formulation for quantized operations. This fix is crucial for numerical accuracy and appears to be implemented correctly.

[xuebwang-amd]

@tjtanaa @gshtras @BowenBao Please help to review this PR. Thanks.
BTW, Happy new year 2026!

[tjtanaa]

@mgoin PTAL . Thank you.

[ApostaC]

@mgoin Hey, could you take a look at this PR? Thanks!

[mgoin]

Thanks, this makes sense to me

[AndreasKaratzas]

@xuebwang-amd @tjtanaa

This PR breaks ROCm at: LoRA TP Test (Distributed). Specifically, the failure we are seeing is in test: FAILED lora/test_olmoe_tp.py::test_olmoe_lora_mixed. I am going to make a PR that fixes this failure, and I am going to CC you there. Feel free to check it out and see if it still serves the purpose you were aiming without introducing any test failures.