[jetd1]

Fixes #33464

Motivation

JPEG images produced by phone cameras (iOS/Android) routinely carry an EXIF Orientation
tag — typically 6 (rotate 90° CW). Browsers, image viewers and OS thumbnailers apply this
tag automatically, but SGLang's image ingestion never does. The model therefore receives raw
sensor pixels while the human who uploaded the photo sees a rotated image.

The failure is silent: no error, no warning, just degraded OCR / spatial / grounding quality,
plus a human-vs-model visual mismatch.

Reproduction: a 640×480 JPEG with a red bar on the left edge and a green bar on the bottom
edge, tagged Orientation=6, is described by a served VLM as "red on the left, green on the
bottom". A human viewing the same file sees red on top and green on the left. Baking
the rotation into the pixels makes the model answer correctly.

What this changes

Three paths in python/sglang/srt/utils/common.py decode user images, and all three ignored
EXIF orientation. Fixing only one leaves a live bypass, so this PR covers all three:

1. _load_image() nvJPEG branch — the primary path, and the reason a CPU-only fix is not
enough. is_jpeg_with_cuda() routes every JPEG to torchvision.io.decode_jpeg on a CUDA
server and returns before Image.open is ever reached. nvJPEG decodes the bitstream and never
consults EXIF. Since phone photos are JPEG, this is exactly the path real rotated images take.

Fix: read the orientation from the header (Image.open is lazy, so this parses header + EXIF
only, no pixel decode) and, when it is non-trivial, fall through to the PIL path rather than
using nvJPEG. This keeps PIL's exif_transpose as the single ground-truth implementation
instead of hand-rolling tensor transposes for all 8 orientation values. Images with
orientation 1 or no EXIF — the overwhelming majority — still take the fast GPU path, so
there is no throughput regression for the common case.

2. _load_image() CPU fallback. Image.open does not auto-apply EXIF; ImageOps.exif_transpose
is the canonical normalization.

3. load_image() direct-PIL branch. isinstance(image_file, Image.Image) returned the
image untouched, bypassing _load_image() entirely.

A small helper _get_exif_orientation() reads the tag from raw bytes and returns 1 for
missing, malformed, or out-of-range EXIF; _safe_exif_transpose() wraps the transpose so that
broken metadata can never turn a decodable image into an exception, and short-circuits when no
rotation is needed so unrotated (and multi-frame) images are not eagerly copied.

Consistency with the rest of the repo

python/sglang/multimodal_gen/runtime/utils/vision.py already calls
PIL.ImageOps.exif_transpose. The diffusion side normalizes orientation; the VLM serving path
did not. This PR removes that inconsistency.

Prior art

vLLM shipped the equivalent fix in #44974 ("Fix image EXIF orientation and tRNS transparency
handling") and then had to follow up a month later with #47566 ("Normalize direct PIL image
inputs") because the direct-PIL branch was missed the first time. This PR deliberately covers
that branch in the same pass.

Scope is intentionally limited to EXIF orientation; the PNG tRNS transparency half of the
vLLM change is a separate concern and is not included here.

Tests

test/registered/unit/multimodal/test_base_processor_image_decode.py — CPU-only, no server,
no model load:

• all 8 orientation values, asserting pixel-identical output to PIL.ImageOps.exif_transpose
  (a parity assertion, so no transpose/flip sign error can survive)
• nvJPEG routing, mocked so it needs no CUDA: decode_jpeg must not be called when
  orientation is non-trivial, and must still be called when it is trivial
• the direct-PIL branch across all 8 orientations, including the returned image_size
• no-EXIF, malformed bytes, corrupt EXIF inside an otherwise valid JPEG, out-of-range
  orientation values, and non-JPEG input
• _safe_exif_transpose returning the original image on failure, and short-circuiting
  without a copy when no rotation is required

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[github-actions]

Thanks @jetd1. Closing this because it has had no updates in 91 days.

Reopen it if the work is still relevant.

Some directories moved recently, so an older branch may need retargeting:
sgl-kernel/ -> python/sglang/kernels/aot/, python/sglang/jit_kernel/
-> python/sglang/kernels/jit/, docs/ -> docs/docs/ (.mdx),
bench_serving.py -> benchmark/serving.py, test/srt/ -> test/registered/.