Fix SGL_CUDA_ARCH mismatch in NVFP4 JIT compilation

[csahithi]

Motivation

On CUDA 13+, _get_nvfp4_cuda_arch_list() appends "10.3a" to the arch list for any Blackwell device that isn't sm_103a. This causes the JIT compiler to generate code for multiple architectures (e.g. "10.0a 10.3a" on GB200).
This is causing cross-compilation, __CUDA_ARCH__ = 1030 != SGL_CUDA_ARCH = 1000, causing the assertion to fail and crashing FP4 model startup.

Observed this on GB200:

Error: -20 00:22:29 [ERROR] 
Last 50 lines of log:
Error: -20 00:22:29 [ERROR]       return forward_call(*args, **kwargs)
Error: -20 00:22:29 [ERROR]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: -20 00:22:29 [ERROR]     File "/sgl-workspace/sglang/python/sglang/srt/layers/linear.py", line 1483, in forward
Error: -20 00:22:29 [ERROR]       output_parallel = self.quant_method.apply(self, input_parallel, bias=bias_)
Error: -20 00:22:29 [ERROR]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: -20 00:22:29 [ERROR]     File "/sgl-workspace/sglang/python/sglang/srt/layers/quantization/modelopt_quant.py", line 1468, in apply
Error: -20 00:22:29 [ERROR]       x_fp4, x_scale_interleaved = fp4_quantize(x, layer.input_scale_inv)
Error: -20 00:22:29 [ERROR]                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: -20 00:22:29 [ERROR]     File "/sgl-workspace/sglang/python/sglang/jit_kernel/nvfp4.py", line 347, in scaled_fp4_quant
Error: -20 00:22:29 [ERROR]       _scaled_fp4_quant_custom_op(input, output, output_scale, input_global_scale)
Error: -20 00:22:29 [ERROR]     File "/usr/local/lib/python3.12/dist-packages/torch/_ops.py", line 1255, in __call__
Error: -20 00:22:29 [ERROR]       return self._op(*args, **kwargs)
Error: -20 00:22:29 [ERROR]              ^^^^^^^^^^^^^^^^^^^^^^^^^
Error: -20 00:22:29 [ERROR]     File "/sgl-workspace/sglang/python/sglang/jit_kernel/nvfp4.py", line 312, in _scaled_fp4_quant_custom_op
Error: -20 00:22:29 [ERROR]       module = _jit_nvfp4_quant_module()
Error: -20 00:22:29 [ERROR]                ^^^^^^^^^^^^^^^^^^^^^^^^^
Error: -20 00:22:29 [ERROR]     File "/sgl-workspace/sglang/python/sglang/jit_kernel/utils.py", line 43, in wrapper
Error: -20 00:22:29 [ERROR]       result_map[key] = fn(*args, **kwargs)
Error: -20 00:22:29 [ERROR]                         ^^^^^^^^^^^^^^^^^^^
Error: -20 00:22:29 [ERROR]     File "/sgl-workspace/sglang/python/sglang/jit_kernel/nvfp4.py", line 136, in _jit_nvfp4_quant_module
Error: -20 00:22:29 [ERROR]       return load_jit(
Error: -20 00:22:29 [ERROR]              ^^^^^^^^^
Error: -20 00:22:29 [ERROR]     File "/sgl-workspace/sglang/python/sglang/jit_kernel/utils.py", line 198, in load_jit
Error: -20 00:22:29 [ERROR]       return load_inline(
Error: -20 00:22:29 [ERROR]              ^^^^^^^^^^^^
Error: -20 00:22:29 [ERROR]     File "/usr/local/lib/python3.12/dist-packages/tvm_ffi/cpp/extension.py", line 1035, in load_inline
Error: -20 00:22:29 [ERROR]       build_inline(
Error: -20 00:22:29 [ERROR]     File "/usr/local/lib/python3.12/dist-packages/tvm_ffi/cpp/extension.py", line 877, in build_inline
Error: -20 00:22:29 [ERROR]       return _build_impl(
Error: -20 00:22:29 [ERROR]              ^^^^^^^^^^^^
Error: -20 00:22:29 [ERROR]     File "/usr/local/lib/python3.12/dist-packages/tvm_ffi/cpp/extension.py", line 672, in _build_impl
Error: -20 00:22:29 [ERROR]       build_ninja(str(build_dir))
Error: -20 00:22:29 [ERROR]     File "/usr/local/lib/python3.12/dist-packages/tvm_ffi/cpp/extension.py", line 542, in build_ninja
Error: -20 00:22:29 [ERROR]       raise RuntimeError("\n".join(msg))
Error: -20 00:22:29 [ERROR]   RuntimeError: ninja exited with status 1
Error: -20 00:22:29 [ERROR]   stdout:
Error: -20 00:22:29 [ERROR]   [1/2] /usr/local/cuda/bin/nvcc  --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_100a,code=sm_100a -gencode=arch=compute_103a,code=sm_103a -std=c++20 -O3 --expt-relaxed-constexpr -DSGL_CUDA_ARCH=1000 -DNDEBUG -DFLASHINFER_ENABLE_F16 -DCUTE_USE_PACKED_TUPLE=1 -DCUTLASS_ENABLE_TENSOR_CORE_MMA=1 -DCUTLASS_VERSIONS_GENERATED -DCUTLASS_TEST_LEVEL=0 -DCUTLASS_TEST_ENABLE_CACHED_RESULTS=1 -DCUTLASS_DEBUG_TRACE_LEVEL=0 --expt-extended-lambda -I/usr/local/lib/python3.12/dist-packages/tvm_ffi/include -I/usr/local/lib/python3.12/dist-packages/tvm_ffi/include -I/sgl-workspace/sglang/python/sglang/jit_kernel/include -I/usr/local/lib/python3.12/dist-packages/flashinfer/data/cutlass/include -I/usr/local/lib/python3.12/dist-packages/flashinfer/data/cutlass/tools/util/include -I/usr/local/lib/python3.12/dist-packages/deep_gemm/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_nvfp4_quant_d59541df8227dfb8/cuda.cu -o cuda_0.o
Error: -20 00:22:29 [ERROR]   FAILED: [code=1] cuda_0.o 
Error: -20 00:22:29 [ERROR]   /usr/local/cuda/bin/nvcc  --generate-dependencies-with-compile --dependency-output cuda_0.o.d -Xcompiler -fPIC -std=c++17 -O2 -gencode=arch=compute_100a,code=sm_100a -gencode=arch=compute_103a,code=sm_103a -std=c++20 -O3 --expt-relaxed-constexpr -DSGL_CUDA_ARCH=1000 -DNDEBUG -DFLASHINFER_ENABLE_F16 -DCUTE_USE_PACKED_TUPLE=1 -DCUTLASS_ENABLE_TENSOR_CORE_MMA=1 -DCUTLASS_VERSIONS_GENERATED -DCUTLASS_TEST_LEVEL=0 -DCUTLASS_TEST_ENABLE_CACHED_RESULTS=1 -DCUTLASS_DEBUG_TRACE_LEVEL=0 --expt-extended-lambda -I/usr/local/lib/python3.12/dist-packages/tvm_ffi/include -I/usr/local/lib/python3.12/dist-packages/tvm_ffi/include -I/sgl-workspace/sglang/python/sglang/jit_kernel/include -I/usr/local/lib/python3.12/dist-packages/flashinfer/data/cutlass/include -I/usr/local/lib/python3.12/dist-packages/flashinfer/data/cutlass/tools/util/include -I/usr/local/lib/python3.12/dist-packages/deep_gemm/include -c /root/.cache/tvm-ffi/sgl_kernel_jit_nvfp4_quant_d59541df8227dfb8/cuda.cu -o cuda_0.o
Error: -20 00:22:29 [ERROR]   nvcc warning : incompatible redefinition for option 'std', the last value of this option was used
Error: -20 00:22:29 [ERROR]   nvcc warning : incompatible redefinition for option 'optimize', the last value of this option was used
Error: -20 00:22:29 [ERROR]   /sgl-workspace/sglang/python/sglang/jit_kernel/include/sgl_kernel/utils.cuh(99): error: static assertion failed with "SGL_CUDA_ARCH mismatch: injected arch flag does not match device target"
Error: -20 00:22:29 [ERROR]     static_assert(
Error: -20 00:22:29 [ERROR]     ^
Error: -20 00:22:29 [ERROR]   
Error: -20 00:22:29 [ERROR]   1 error detected in the compilation of "/root/.cache/tvm-ffi/sgl_kernel_jit_nvfp4_quant_d59541df8227dfb8/cuda.cu".

Modifications

Remove the cross-arch addition. Each device now compiles only for its own architecture, keeping __CUDA_ARCH__ and SGL_CUDA_ARCH in sync.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[DarkSharpness]

It seems that this is a duplicate of #20874 . Thanks for your contribution. We will get it merged ASAP

[csahithi]

It seems that this is a duplicate of #20874 . Thanks for your contribution. We will get it merged ASAP

@DarkSharpness thank you for sharing the existing PR! Hoping it will be merged soon since it is a blocking bug for GB200

[csahithi]

closing this PR since the change is now merged