fix(xpu): enable compressed-tensors FP8 W8A8 on XPU (RedHatAI FP8-dynamic models)

[vshekhawat-hlab]

Motivation

compressed-tensors FP8 W8A8 quantized models (e.g. RedHatAI's *-FP8-dynamic family — Apertus-8B-Instruct-2509-FP8-dynamic, Mistral-Small-3.1-24B-Instruct-2503-FP8-dynamic, granite-4.0-h-small-FP8-dynamic, NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamic) currently fail to serve on XPU. The very first failure happens at layer-construction time, before any kernel dispatch:

AssertionError: Torch not compiled with CUDA enabled

Fixing that surfaces two further, distinct XPU-specific issues before the model actually serves and generates correctly.

Modifications

• compressed_tensors.py (CompressedTensorsConfig): _check_scheme_supported unconditionally called torch.cuda.get_device_capability(), which is CUDA-only. Only CompressedTensorsW8A8Fp8 has a real XPU-usable kernel path today (via the existing device-agnostic _apply_fallback_scaled_mm in fp8_utils.py, not a missing native kernel); every other capability-gated scheme (NVFP4/cutlass, Marlin-based W8A16Fp8, WNA16, ...) is explicitly rejected with a clean RuntimeError on XPU instead of crashing on the CUDA-only call.
• fp8_kernel.py: the non-CUDA/non-MUSA branch never imported sgl_per_token_quant_fp8/sgl_per_tensor_quant_fp8 for XPU, even though sgl-kernel-xpu registers both natively — this caused a NameError on the first forward pass once the above crash was fixed.
• fp8_utils.py (apply_fp8_linear): the activation-quant padding decision (num_token_padding=17, meant to make torch._scaled_mm more performant on CUDA for small batches) was only skipped when cutlass_fp8_supported (CUDA-only). sgl-kernel-xpu's per-token quant kernel requires output_q to exactly match input's shape and doesn't support padded output, raising a shape-mismatch RuntimeError whenever a forward pass had fewer than 17 rows (e.g. small prefill/decode steps).
• fp8.py (Fp8MoEMethod.apply): the native sgl-kernel-xpu MoE fast path was taken unconditionally on XPU, before checking the explicitly-selected MoE runner backend.
  This made --moe-runner-backend triton a no-op on XPU for this quantization scheme — the native kernel path was still hit and raised "current MoE does not support
  use_fp8_w8a8" even when the Triton runner had been explicitly requested. Now skips the native XPU path when the runner backend is explicitly triton, falling through to
  the existing Triton dispatch below.
• internvl.py: InternVLChatConfig has no top-level num_experts — the MoE config lives on the nested llm_config (the text-backbone config). Fixes an AttributeError when
  loading MoE-based InternVL variants.
• runtime_context.py (get_stream): replaced a hardcoded torch.cuda.Stream() with torch.get_device_module(device).Stream(), so the named process-level side stream is
  created on whichever device backend is active instead of always assuming CUDA.
• topk.py (biased_grouped_topk_gpu, XPU path): the topk_sigmoid kernel requires correction_bias to be float32. The CUDA branch a few lines above already casts it; the
  XPU branch passed it through unchanged, raising RuntimeError: correction_bias must be float32 on the first generation request for some MoE models. Now cast to float32
  before the call, matching the CUDA branch.

Accuracy Tests

Verified locally end-to-end in a docker container (XPU backend, Intel Arc Pro/BMG) with the patched sglang installed in place of the image's baked-in copy:

• RedHatAI/Apertus-8B-Instruct-2509-FP8-dynamic (TP=1, --attention-backend intel_xpu) serves and evaluates gsm8k (5-shot) correctly:
  • exact_match,flexible-extract = 0.5891 ± 0.0136
  • strict-match = 0.5390 ± 0.0137
• Regression check that the shared code paths touched here don't affect other backends/schemes:
  • All three changes are gated behind if _is_xpu: / elif _is_xpu: branches that are new no-ops on every other platform; cutlass_fp8_supported-driven CUDA behavior is unchanged (verified by re-reading every other call site of the touched functions — scaled_fp8_quant's other callers in fp8_utils.py are gated by if _is_cuda:/if _is_hip:, never reaching the changed XPU branch on other devices).
  • No currently-passing XPU model can regress from these changes: any compressed-tensors quantized model on XPU previously hit the _check_scheme_supported crash unconditionally at layer-construction time (before this PR), so none could have been passing beforehand.

Speed Tests and Profiling

Not applicable — these are correctness/compatibility fixes for an existing fallback code path (_apply_fallback_scaled_mm), no new kernel or scheduling changes. The activation-quant padding skip on XPU has a minor performance implication (an extra kernel-launch on small batches that would otherwise reuse a padded buffer), but padding was never functional on XPU before this PR (it crashed), so there's no regression relative to current behavior.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): 🚫 Run #32336898587
Latest PR Test (Extra): ✅ Run #32336898486
Latest PR Test (AMD ROCm 7.2): ❌ Run #32336898668

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.