[Bugfix] Fix int32 overflow in DeepGEMM SiLU/mul FP8 Triton kernel

[Flink-ddd]

Purpose

Fixes #42173

_silu_mul_per_token_group_quant_fp8_colmajor computes row/column offsets using int32 arithmetic:

m_offset = pid_m * BLOCK_M
n_offset = pid_n * BLOCK_N

With large DeepGEMM MoE warmup/workspace shapes (e.g. DPEP=16, 36k max tokens per rank), the maximum element offset M * N - 1 = 18,882,756,607 far exceeds the int32 limit of 2,147,483,647, causing the Triton kernel to access illegal memory addresses.

This PR promotes m_offset and n_offset to tl.int64 before pointer arithmetic to ensure correct 64-bit memory addressing.

Test Plan

1. Verified on NVIDIA H100 PCIe (80GB) using a minimal single-GPU reproducer with the first aligned overflow shape:

• M = 524_416, N = 4096
• max_offset = M * N - 1 = 2,148,007,935 (exceeds int32 max)
• Environment: vllm 0.19.0, torch 2.10.0+cu128, triton 3.6.0, cuda 12.8

2. Reproduction script:

import torch
from vllm.model_executor.layers.quantization.utils.fp8_utils import (
    silu_mul_per_token_group_quant_fp8_colmajor,
)

# Single-card minimum overflow shape
M = 524_416
N = 4096

print(f"M={M}, N={N}")
print(f"max_offset = {M*N-1}")
print(f"int32_max  = {2**31-1}")
print(f"overflow?  = {M*N-1 > 2**31-1}")
print()

x = torch.empty((M, N), device="cuda", dtype=torch.bfloat16)
torch.cuda.synchronize()

print("Calling silu_mul_per_token_group_quant_fp8_colmajor ...")
y, scales = silu_mul_per_token_group_quant_fp8_colmajor(x, use_ue8m0=False)
torch.cuda.synchronize()

print(f"successful！output shape={y.shape}, scales shape={scales.shape}")

Test Result

Before fix:

M=524416, N=4096
max_offset = 2148007935
int32_max  = 2147483647
overflow?  = True
Calling silu_mul_per_token_group_quant_fp8_colmajor ...
File ".../fp8_utils.py", line 785, in silu_mul_per_token_group_quant_fp8_colmajor
_silu_mul_per_token_group_quant_fp8_colmajor[grid](https://github.com/vllm-project/vllm/compare/main...Flink-ddd:vllm:fix/...)
RuntimeError: Triton Error [CUDA]: an illegal memory access was encountered

After fix (testing in progress on H100 PCIe):

M=524416, N=4096
max_offset = 2148007935
int32_max  = 2147483647
overflow?  = True
Calling silu_mul_per_token_group_quant_fp8_colmajor ...
successful！output shape=torch.Size([524416, 2048]), scales shape=torch.Size([524416, 16])

[gemini-code-assist]

Code Review

This pull request updates the Triton kernels in fp8_utils.py to use int64 for offset calculations to prevent potential integer overflows. The review feedback correctly points out that casting to int64 after the multiplication is insufficient, as the intermediate 32-bit product could still overflow. The reviewer suggests casting the program IDs to int64 before the multiplication to ensure robust overflow protection, consistent with other kernels in the codebase.

[Flink-ddd]

/gemini review

[gemini-code-assist]

Code Review

This pull request updates the Triton kernels in fp8_utils.py to cast program IDs to int64 before calculating memory offsets. This change prevents potential integer overflow issues during offset computation in large-scale operations. As there were no review comments provided, I have no feedback to provide.

[Flink-ddd]

Pre-commit failures are seem like pre-existing main branch issues unrelated to this PR. all checks pass for the modified file through pre-commit run --files vllm/model_executor/layers/quantization/utils/fp8_utils.py.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

Hi @Flink-ddd, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[yewentao256]

LGTM, thanks for the work! Also CC @ivanium

[ivanium]

LGTM too. Thanks for the fix! cc @zyongye as well

[zyongye]

LGTM

[Flink-ddd]

Hi @yewentao256 @ivanium @zyongye , All 69 CI checks are passed, ready for merge, Thanks!