Skip to content

[diffusion][xpu] FLUX.2: enable the bit-exact Triton SwiGLU on XPU - #41940

Merged
mingfeima merged 2 commits into
mainfrom
xpu/flux2-swiglu
Oct 12, 2026
Merged

mingfeima merged 2 commits into
mainfrom
xpu/flux2-swiglu

Conversation

@AgainstEntropy

@AgainstEntropy AgainstEntropy commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

FLUX.2's packed SwiGLU fast path (fused_packed_silu_mul_bitexact) is a Triton
kernel, so there is no reason it cannot run on Intel XPU. Both the kernel's own
guard and the FLUX.2 call site required x.is_cuda, and the kernel entered
torch.cuda.device, so XPU always fell back to the two-kernel eager chain
(F.silu(x[..., :half]) * x[..., half:]).

Modifications

  • kernels/ops/diffusion/activation/silu_mul_bitexact.py: accept XPU tensors in
    fused_packed_silu_mul_bitexact, and enter the device through
    torch.get_device_module(x.device).device(...) instead of torch.cuda.device.
  • multimodal_gen/runtime/models/dits/flux_2.py (_flux2_swiglu): accept XPU in
    the can_fuse predicate, and use the device-generic
    torch.get_device_module(x.device).is_current_stream_capturing() for the
    graph-capture check.

No CUDA behaviour changes: the predicate gains an or x.is_xpu, and both
torch.get_device_module calls resolve to torch.cuda on a CUDA tensor.

Accuracy Tests

The existing BitExactFusionGate already verifies this path: on the first call
per signature the kernel output is compared with torch.equal against the live
eager chain, and the fast path is disabled permanently on mismatch. On Arc Pro
B70 (torch 2.13.0+xpu) it verifies and stays enabled — no
"not bit-exact ... falling back to eager" warning in any end-to-end run.

Existing CUDA coverage (test_activation.py::test_packed_silu_mul_is_bit_exact,
test_model_fast_paths.py::TestFlux2EagerFusions) is unchanged and still applies.

Speed Tests and Profiling

FLUX.2-dev, 1024x1024, 8 denoise steps, guidance 4.0, seed 42, bf16, eager (no
torch.compile), warm request, TP4 on 4x Arc Pro B70:

denoise (8 steps)
before 6.64 s
after 6.569 / 6.572 s

About 1%. The fused kernel replaces two eager elementwise passes over a
per-rank [4608, 4608] bf16 buffer; at this size it is a small win, and 1% is
close to run-to-run noise on this machine — the main value is that the
already-existing fast path stops being silently skipped on XPU.

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): 🚫 Run #37272593101
Latest PR Test (Extra): ❌ Run #37272592827
Latest PR Test (AMD ROCm 10): ❌ Run #37272592993

The packed SwiGLU fast path (`fused_packed_silu_mul_bitexact`) is a Triton
kernel, but both it and the FLUX.2 gate required `x.is_cuda` and entered
`torch.cuda.device`, so XPU always ran the two-kernel eager chain.

Accept XPU tensors, enter the device through `torch.get_device_module`, and use
the device-generic capture check. The first call per signature is still
verified with `torch.equal` against eager; on Arc Pro B70 it passes.

FLUX.2-dev, 1024x1024, TP4 on 4x Arc Pro B70, eager: 8-step denoise
6.64 s -> 6.57 s (~1%).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mickqian

Copy link
Copy Markdown
Collaborator

/tag-and-rerun-ci

@mingfeima
mingfeima merged commit 208df29 into main Oct 12, 2026
227 of 260 checks passed
@mingfeima
mingfeima deleted the xpu/flux2-swiglu branch October 12, 2026 07:17
Liwansi added a commit to Liwansi/sglang_vans that referenced this pull request Oct 13, 2026
…n_allgather

* 'main' of https://github.com/sgl-project/sglang: (236 commits)
  [NPU] Stabilize HiCache and registered CI tests (sgl-project#43308)
  [Docs] Cookbook: refresh SemiAnalysis AgentX launches (GLM-5.2 B300, DeepSeek-V4.1 GB300) (sgl-project#43799)
  [Docs] Update GLM-5.3-Flash playground: B300 default, CP8, EP and PD (sgl-project#43802)
  [DSV4.1][PD][1/N] Support prefill node use cp parallel strategy (sgl-project#40323)
  [BCG][DSV4-Flash][CP] Support DeepSeek V4 prefill with interleave CP and breakable CUDA graphs (sgl-project#38603)
  [XPU] Support decode context parallelism (DCP) on Intel XPU (sgl-project#34355)
  Keep NIXL Mamba checkpoint keys separate by TP rank (sgl-project#43594)
  Capture KV host transfer buffers (sgl-project#43711)
  [CI] Move tests that do not need GB300 off the per-PR GB300 suite (sgl-project#43784)
  [HiCache] Preserve all MTP KV and recurrent state (sgl-project#40223)
  [HiCache] Support Mamba in buffer-only host memory mode (sgl-project#36345)
  [PD] Poison the deferred abort ack when a Mooncake send fails (sgl-project#43719)
  [Perf] Route BF16 mixed-batch decode tails through decode attention (sgl-project#43238)
  [Fix] Link cuda_ipc JIT against CUDA driver stubs dir (sgl-project#43764)
  docs: clarify prefill CP terminology and simplify wording (sgl-project#43787)
  [diffusion][xpu] FLUX.2: enable the bit-exact Triton SwiGLU on XPU (sgl-project#41940)
  [Fix] Adapt to transformers 5.18 / 5.19 changes (sgl-project#43688)
  [XPU CI] Optimize PR CI per-file timeouts and fail hanging tests fast (sgl-project#42048)
  [Deps] Bump FlashInfer to 0.7.1 (sgl-project#43494)
  docs: sync LMSYS SGLang blog cards (sgl-project#43462)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion jit-kernel run-ci CI: run the baseline test suite on this PR xpu intel gpu with device `torch.xpu`

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants