[niehen6174]

Motivation

On the comfy_kitchen INT8 backend, Qwen-Image 2.1's MLP ran three GEMMs plus a standalone SiLU-mul: gate_layer, proj, silu(gate) * up written to HBM in BF16, then out. ComfyUI runs the same checkpoint as one [gate | up] GEMM and a down-projection that applies the SwiGLU inside its quantizer (int8_linear(..., input_act="swiglu")).

Modifications

• convrot_int8_comfy_kitchen.py: add comfy_kitchen_swiglu_mlp, which makes the two kitchen calls. The linear method's apply is unchanged.
• qwen_image21.py:
  • pack_gate_up_weights() runs from post_load_weights and from _apply, so packing survives .to() and CPU offload. It puts the gate_layer/proj INT8 weights and row scales into one storage each. This mirrors pack_qkv_weights: parameters keep their names and shapes, and no reference to the shared storage is kept.
  • packed_gate_up() re-derives the stacked views on every forward.
  • LoRA wrappers, layerwise-offload placeholders, TP > 1, torch.compile and the JIT backend keep the separate projections.

Accuracy Tests

• The stacked GEMM computes the same per-column arithmetic as the two separate projections.
• The fused quantizer keeps silu(gate) * up in FP32 instead of rounding it to BF16 first, so outputs are not bitwise the previous path.
  • Against an FP32 reference, relative L2 error is 1.33% for the fused path vs 1.35% for eager at (2048, 3072→1024), and 1.44% vs 1.46% at (4096, 12288→4096).
  • It is the same call ComfyUI makes for this model.
• New unit test (test_convrot_int8_backends.py) checks:
  • the fused error vs FP32 is no worse than eager's;
  • re-packing after a CPU round trip;
  • the fallback is bitwise the eager path once the shared storage is gone.

Speed Tests and Profiling

Qwen-Image 2.1 INT8 checkpoint, comfy_kitchen backend, SGLD ComfyUI workflow, 8 steps × 3 seeds. Steady model-time gain:

GPU	CFG 1	CFG 4
RTX 4090	+4.75%	+4.46%
RTX 5090	+4.44%	+4.16%
H200 (kitchen forced)	+3.17%	+2.46%

H200, B200 and RTX 5090 default to the JIT backend, where this path is inactive. RTX 4090 (sm89) always uses comfy_kitchen.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): ✅ Run #39116725304
Latest PR Test (Extra): ❌ Run #39116725117
Latest PR Test (AMD ROCm 10): ⏳ Run #39116725108

[niehen6174]

/rerun-failed-ci