Repository navigation
[diffusion][xpu] FLUX.2: enable the bit-exact Triton SwiGLU on XPU - #41940
Merged
Merged
Conversation
The packed SwiGLU fast path (`fused_packed_silu_mul_bitexact`) is a Triton kernel, but both it and the FLUX.2 gate required `x.is_cuda` and entered `torch.cuda.device`, so XPU always ran the two-kernel eager chain. Accept XPU tensors, enter the device through `torch.get_device_module`, and use the device-generic capture check. The first call per signature is still verified with `torch.equal` against eager; on Arc Pro B70 it passes. FLUX.2-dev, 1024x1024, TP4 on 4x Arc Pro B70, eager: 8-step denoise 6.64 s -> 6.57 s (~1%). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AgainstEntropy
requested review from
BBuf,
HaiShaw,
kevin-mii,
mickqian,
niehen6174,
ping1jing2,
yichiche and
yingluosanqian
as code owners
September 30, 2026 22:25
AgainstEntropy
added this pull request to stack #41945
September 30, 2026 22:31
Collaborator
|
/tag-and-rerun-ci |
This was referenced Sep 30, 2026
This was referenced Oct 9, 2026
mingfeima
approved these changes
Oct 12, 2026
Liwansi
added a commit
to Liwansi/sglang_vans
that referenced
this pull request
Oct 13, 2026
…n_allgather * 'main' of https://github.com/sgl-project/sglang: (236 commits) [NPU] Stabilize HiCache and registered CI tests (sgl-project#43308) [Docs] Cookbook: refresh SemiAnalysis AgentX launches (GLM-5.2 B300, DeepSeek-V4.1 GB300) (sgl-project#43799) [Docs] Update GLM-5.3-Flash playground: B300 default, CP8, EP and PD (sgl-project#43802) [DSV4.1][PD][1/N] Support prefill node use cp parallel strategy (sgl-project#40323) [BCG][DSV4-Flash][CP] Support DeepSeek V4 prefill with interleave CP and breakable CUDA graphs (sgl-project#38603) [XPU] Support decode context parallelism (DCP) on Intel XPU (sgl-project#34355) Keep NIXL Mamba checkpoint keys separate by TP rank (sgl-project#43594) Capture KV host transfer buffers (sgl-project#43711) [CI] Move tests that do not need GB300 off the per-PR GB300 suite (sgl-project#43784) [HiCache] Preserve all MTP KV and recurrent state (sgl-project#40223) [HiCache] Support Mamba in buffer-only host memory mode (sgl-project#36345) [PD] Poison the deferred abort ack when a Mooncake send fails (sgl-project#43719) [Perf] Route BF16 mixed-batch decode tails through decode attention (sgl-project#43238) [Fix] Link cuda_ipc JIT against CUDA driver stubs dir (sgl-project#43764) docs: clarify prefill CP terminology and simplify wording (sgl-project#43787) [diffusion][xpu] FLUX.2: enable the bit-exact Triton SwiGLU on XPU (sgl-project#41940) [Fix] Adapt to transformers 5.18 / 5.19 changes (sgl-project#43688) [XPU CI] Optimize PR CI per-file timeouts and fail hanging tests fast (sgl-project#42048) [Deps] Bump FlashInfer to 0.7.1 (sgl-project#43494) docs: sync LMSYS SGLang blog cards (sgl-project#43462) ...
4 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
FLUX.2's packed SwiGLU fast path (
fused_packed_silu_mul_bitexact) is a Tritonkernel, so there is no reason it cannot run on Intel XPU. Both the kernel's own
guard and the FLUX.2 call site required
x.is_cuda, and the kernel enteredtorch.cuda.device, so XPU always fell back to the two-kernel eager chain(
F.silu(x[..., :half]) * x[..., half:]).Modifications
kernels/ops/diffusion/activation/silu_mul_bitexact.py: accept XPU tensors infused_packed_silu_mul_bitexact, and enter the device throughtorch.get_device_module(x.device).device(...)instead oftorch.cuda.device.multimodal_gen/runtime/models/dits/flux_2.py(_flux2_swiglu): accept XPU inthe
can_fusepredicate, and use the device-generictorch.get_device_module(x.device).is_current_stream_capturing()for thegraph-capture check.
No CUDA behaviour changes: the predicate gains an
or x.is_xpu, and bothtorch.get_device_modulecalls resolve totorch.cudaon a CUDA tensor.Accuracy Tests
The existing
BitExactFusionGatealready verifies this path: on the first callper signature the kernel output is compared with
torch.equalagainst the liveeager chain, and the fast path is disabled permanently on mismatch. On Arc Pro
B70 (torch 2.13.0+xpu) it verifies and stays enabled — no
"not bit-exact ... falling back to eager" warning in any end-to-end run.
Existing CUDA coverage (
test_activation.py::test_packed_silu_mul_is_bit_exact,test_model_fast_paths.py::TestFlux2EagerFusions) is unchanged and still applies.Speed Tests and Profiling
FLUX.2-dev, 1024x1024, 8 denoise steps, guidance 4.0, seed 42, bf16, eager (no
torch.compile), warm request, TP4 on 4x Arc Pro B70:About 1%. The fused kernel replaces two eager elementwise passes over a
per-rank
[4608, 4608]bf16 buffer; at this size it is a small win, and 1% isclose to run-to-run noise on this machine — the main value is that the
already-existing fast path stops being silently skipped on XPU.
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): 🚫 Run #37272593101
Latest PR Test (Extra): ❌ Run #37272592827
Latest PR Test (AMD ROCm 10): ❌ Run #37272592993