[murthy-l]

Motivation

FLUX.2-klein has no XPU test coverage. This adds a manual 1-GPU test for
klein-4B and klein-9B on Intel XPU. It covers text-to-image, and
image-to-image against a stored reference, so a quality regression in
either path is caught by hand before release.

Modifications

One new file: test/registered/xpu/test_xpu_flux2_klein.py. No changes to
the shared test harness, CI workflows, or other test files.

Earlier revisions of this PR also added test/registered/xpu/conftest.py.
Upstream has since landed a byte-identical copy of that file, so the rebase
onto dc81f17b90 absorbed it and this PR is now a single file.

• t2i (4B, 9B): one fixed prompt, gated on CLIP score
  (clip-vit-base-patch32, threshold 0.25), the same approach as
  test_xpu_flux2_dev.py. The threshold was measured on this prompt:
  healthy 4B/9B output scores >= 0.4198, black/white/noise frames
  <= 0.1842. A CLIP model load or scoring failure fails the test rather
  than passing unchecked, as it would under test_xpu_flux2_dev.py.
• ti2i (4B, 9B): CUDA flux_2_ti2i's edit prompt and input image, at
  1024x1024 with seed 0. The output is compared against a stored reference
  image with the existing compare_with_gt (CLIP similarity, SSIM, PSNR,
  mean abs diff), at flux_2_ti2i's H100 thresholds.
  • There is no XPU reference set in sgl-project/ci-data-diffusion, so
    references are read from SGLANG_CONSISTENCY_GT_DIR.
  • A run without a reference fails, and saves its output under
    $SGLANG_DIFFUSION_ARTIFACT_DIR/missing_consistency_gt/ to be
    inspected and promoted.
• Every output is saved to $SGLANG_DIFFUSION_ARTIFACT_DIR before it is
  checked, so a failing image is kept.
• Config: bf16 DiT/VAE/text encoder, 4 steps (klein is
  step-distilled), 1024x1024. Residency is pinned per size: 4B uses
  dit=resident, and 9B adds text_encoder=layerwise-offload so
  1024x1024 warmup fits in 24 GiB.
• Functional only, no perf validation (collect_perf=False). The
  flux2_klein_* case ids added here have no perf_baselines/xpu_b60.json
  entry. klein-4B t2i perf on XPU is
  already guarded by the CI case flux_2_klein_image_t2i.
• Registered nightly=True, disabled=, like its nightly-xpu-1-gpu
  siblings: the Intel nightly lists it as skipped and never runs it.
• perf_results fixture: a plain pytest run of any DiffusionServerBase
  test in test/registered/xpu/ needs test/registered/xpu/conftest.py to
  re-export it, as test/registered/amd/conftest.py does ([AMD] Fix the diffusion perf fixture lookup  #38686); without it
  collection errors with fixture 'perf_results' not found. That file is now
  on main, so this PR no longer carries it.

How to run

export SGLANG_CONSISTENCY_GT_DIR=/path/to/klein_references   # ti2i only
export SGLANG_DIFFUSION_ARTIFACT_DIR=/path/to/artifacts
python3 -m pytest -v -s \
    test/registered/xpu/test_xpu_flux2_klein.py -k image_t2i    # or image_ti2i, or image for all 4

To create references: run the ti2i cases once without them, inspect the
images saved under missing_consistency_gt/, then copy them into
$SGLANG_CONSISTENCY_GT_DIR.

Accuracy Tests

Run on Intel Arc Pro B60 (24 GiB), torch 2.13.0+xpu, 1 card, on the
pre-rebase commit (872989c6d0, upstream base a749d84303), with the plain
command from "How to run" (no -p).

These results carry over to the current head 085153f80c (rebased onto
dc81f17b90): test_xpu_flux2_klein.py is byte-identical across the rebase
(blob 3551310a95), and the only other change was dropping the conftest.py
that upstream now ships.

case	result	metrics
flux2_klein_4b_image_t2i	pass	CLIP 0.4291
flux2_klein_9b_image_t2i	pass	CLIP 0.4198
flux2_klein_4b_image_ti2i	pass	clip 1.0000, ssim 0.9999, psnr 62.21, mad 0.019
flux2_klein_9b_image_ti2i	pass	clip 1.0000, ssim 0.9999, psnr 62.41, mad 0.020

4 passed in 3m 11s. The metrics and t2i images are identical to the earlier
revisions of this PR.

Checks that the gates fail when they should:

• Every case forced to fail (t2i CLIP threshold raised to 1.01; ti2i
  input image swapped): 4 failed, and all 4 failing images were still saved
  to $SGLANG_DIFFUSION_ARTIFACT_DIR.
• ti2i, swapped input image: fails all four metrics (clip <= 0.49,
  ssim <= 0.62, psnr <= 10.6, mad >= 56.5), well outside the thresholds.

On the previous revision, whose test logic for these paths is unchanged:

• ti2i, different card: a rerun on a second card gives identical
  metrics, so the reference is not card-specific.
• t2i, CLIP scorer forced to fail: both cases fail with the scorer's
  own exception, rather than passing unchecked.
• t2i, rerun: produces byte-identical images (also true of this
  commit's run).

Checklist

• Format your code according to pre-commit
• Add unit tests (manual XPU test, registered disabled)
• Provide accuracy results
• Update documentation (not needed)

---

CI States

Latest PR Test (Base): ❌ Run #40823519082
Latest PR Test (Extra): ❌ Run #40823519169
Latest PR Test (AMD ROCm 10): ❌ Run #40823519080

[msinnha1]

this seems contradicting with the finding of black frame clip_score of 17.2-25.3

[murthy-l]

Addressed it, made it 25 and test is passing for 100 images.

[airMeng]

@murthy-l-habana — CI failure from this source patch in #46623.

XPU job: https://github.com/sgl-project/sglang/actions/runs/40776996977/job/123128957120

ValueError: Tests registered to invalid suites:
  /sglang-checkout/test/registered/xpu/test_xpu_flux2_klein.py: backend=XPU, suite='nightly-xpu-1-gpu'

The same failure is present in this CPU job:
https://github.com/sgl-project/sglang/actions/runs/40776997409/job/123129155325

ValueError: Tests registered to invalid suites:
  /home/runner/work/sglang/sglang/test/registered/xpu/test_xpu_flux2_klein.py: backend=XPU, suite='nightly-xpu-1-gpu'

[murthy-l]

@airMeng thanks, fixed in f2d6b75.