[DarkSharpness]

(auto generated by Claude)

Motivation

Kernels query GPU architecture in many ad-hoc ways: torch.cuda.get_device_capability(...), get_device_properties(...).multi_processor_count, gcnArchName string checks, sglang.srt.utils helpers, and the JIT compile target get_jit_cuda_arch(), which several kernels also used for runtime decisions. These are duplicated, sometimes hard-code device 0, and make sglang.kernels depend on sglang.srt.

This PR adds one self-contained entry point, sglang.kernels.arch.current_arch(), and moves kernel arch queries onto it.

Modifications

• New sglang.kernels.arch (torch only, no srt dependency):
  • current_arch() returns a process-wide cached object chosen from PlatformInfo.detect(): CudaArch on NVIDIA, HipArch on AMD, a plain DeviceArch elsewhere. All GPUs in a node are assumed to be the same model, so there is no device argument. Attributes are cached_property, read lazily on first use.
  • Shared: num_sms, warp_size, l2_cache_bytes, smem_per_block_optin, smem_per_sm, regs_per_sm, max_threads_per_sm, name, props, supports_pdl.
  • is_sm(major, minor=None) / is_sm_at_least(major, minor=0): False off NVIDIA, so call sites need no platform guard.
  • .cuda / .hip narrow to the platform object (None elsewhere). CudaArch adds major, minor, capability, sm_version, max_vec_bytes, supports_cluster; HipArch adds gcn_arch.
• PlatformInfo.detect() is now cached (_probe() is the uncached probe) and gains is_hip / is_musa; the selector drops its own lru_cache.
• JIT compile target: ArchInfo is renamed to CompileArch (a frozen msgspec.Struct). get_jit_cuda_arch() becomes the internal get_compile_arch() used only by the build pipeline, and make_jit_cuda_arch() becomes CompileArch.for_capability(). override_jit_cuda_arch() is removed (its only user compiled for the local GPU anyway). is_arch_support_pdl() reads current_arch().supports_pdl, and is_hip_runtime() / is_musa_runtime() read PlatformInfo.
• About 80 call sites in sglang.kernels (plus a few kernel tests and benchmarks) now use current_arch() instead of direct torch/srt queries. Several one-line local helpers (_num_sms, _get_num_sm, _mla_core_count, ...) are removed.

Left unchanged on purpose: vendored code (flash_attn/cute, fla, code ported from vLLM/aiter/FlashInfer/CUTLASS), import-time queries, srt predicates that also gate on the CUDA toolkit version (is_sm100_supported, ...), AMD family predicates (is_gfx95_supported, ...), and sites that rely on ROCm reporting gfx-derived capabilities. These can follow once the matching interfaces exist.

Behavior notes:

• Kernel helpers that took a device argument for an arch query now ignore it (same-model assumption). The parameters are kept to avoid touching callers.
• CUDA-only paths that used to compare ROCm's gfx-derived capability now treat ROCm as "not NVIDIA".

Accuracy Tests

No numerical change: the patch only changes how architecture facts are read.

Speed Tests and Profiling

N/A.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ❌ Run #41361360932
Latest PR Test (Extra): ❌ Run #41361360833
Latest PR Test (AMD ROCm 10): ❌ Run #41361360820

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[DarkSharpness]

ping @BBuf (or his/her agent) for review

[DarkSharpness]

/rerun-test test/registered/kernels/ops/gemm/test_cutedsl_dual_gemm.py

[github-actions]

🚀 4-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/kernels/ops/gemm/test_cutedsl_dual_gemm.py

[BBuf]

LGTM. The earlier hardware-validation request remains open.

[BBuf]

/rerun-failed-ci

[BBuf]

[BBuf Review Agent] @DarkSharpness these CI failures look related to this change, could you take a look?

• multimodal-gen-test-2-gpu-amd (rocm10, linux-mi300-2gpu-sglang, 0): E Mismatched elements: 96539 / 177099 (54.5%)" in TestDisaggFlux3Image.test_t2i_and_edit_match_monolithic. The Flux3 inference path reaches diffusion kernel dispatch and architecture detection chang

[DarkSharpness]

(auto generated by Claude)

@BBuf this one is pre-existing on main, not from this change. TestDisaggFlux3Image.test_t2i_and_edit_match_monolithic fails in multimodal-gen-test-2-gpu-amd (0) on every recent scheduled pr-test-amd.yml run on main, with the same signature:

• 3a345c0 (11-02 01:11Z): 99701 / 177099 (56.3%) mismatched, https://github.com/sgl-project/sglang/actions/runs/41371177861
• 0f93be7 (11-01 19:04Z): 100404 / 177099 (56.7%), https://github.com/sgl-project/sglang/actions/runs/41339864292
• cb0a7a9 (11-01 13:08Z): 101462 / 177099 (57.3%), https://github.com/sgl-project/sglang/actions/runs/41309823139

It has failed this way since 10-29. The open fix is #46947 (deterministic VAE convolutions on ROCm).

The CUDA side of this PR is blocked by two other main-side failures in multimodal-gen-unit-test: test_cfg_parallel_warmup.py::test_bcg_default_resolution_of_two_stage_is_the_output_resolution (#47474's test vs #47582's fallthrough for unaligned two-stage defaults) and test_fused_lora_compose.py::test_alpha_override_reloads_fused_sections (/adapter not found, added by #46255). /rerun-failed-ci reuses the old merge SHA, so it cannot pick up a main fix; I'll push a fresh merge once those land.

[BBuf]

[BBuf Review Agent] @DarkSharpness The test and VAE code match main, and the changed Flux3 PrimsTS path is still CUDA-only. I'll drop the AMD CI attribution.

[BBuf]

/rerun-failed-ci

[BBuf]

/rerun-failed-ci