[dghosef]

Pending GPU testing: only the DiT's own recompile limit under a step-dtype schedule (it needs mxfp8/nvfp4 linears, so Blackwell). Everything else below ran on an H100.

What this does

• FLUX.2 klein, Z-Image and LTX-2.5 failed to build on torch 2.10 and 2.11, because torch.compile doesn't take recompile_limit= before 2.12. They build now, and the DiT still gets its own recompile limit. CPU CI fixes: pass recompile_limit only where torch.compile takes it, and skinny GEMM pass tests without AITER #1145 fixes the same crash with a version check instead.
• Qwen3-TTS Base voice cloning failed on transformers 5.21 and later ("property 'upsample' has no setter"), and the weights also failed to load under 5.21's new Mimi layout. Both now work on every transformers version, with no version cap (Qwen3-TTS Base voice cloning fails with transformers 5.21: qwen-tts's Mimi encoder raises "property 'upsample' ... has no setter" #1025).
• Request images are now turned upright by their EXIF orientation, so phone photos no longer reach vision models sideways. This covers read_image (static WebP included), FLUX.2 and LingBot-World. An image with a broken EXIF block is still read as stored ([Model][Bug fix] Request images ignore EXIF orientation, so phone photos reach the model sideways #675).
• Cosmos3: a domain_id past the embodiment table used to trip a CUDA device assert that broke the worker for every later request. It is now a 400 (Cosmos3 domain_id and V-JEPA2 mask indices aren't range-checked, so a bad request can trip a CUDA device assert #1031).
• Cosmos3: a forward-dynamics action that was narrow or 1-D used to crash the whole DiT batch with a 500, and a missing action quietly conditioned on zeros. Inverse dynamics with a clip shorter than num_frames used to fail mid-pipeline (Cosmos3 inverse dynamics fails on a clip shorter than num_frames: the clip is truncated but never padded #1053). A side under one latent pixel used to fail mid-pipeline too (Cosmos3 generation silently ignores a malformed size and accepts unaligned or tiny width/height #1054). All of these are 400s now.
• V-JEPA 2: an out-of-range or non-integer context_mask index used to device-assert the worker, which then failed every later request too. An out-of-range target_mask index didn't assert but sized a tensor by that index (10**6 → a [1, 1000001, 384] buffer) and returned a 200. A lone context_mask or target_mask was silently swapped for the defaults. All are now 400s (Cosmos3 domain_id and V-JEPA2 mask indices aren't range-checked, so a bad request can trip a CUDA device assert #1031). num_frames < 1 is left to [05/06] Sampling Parameters | Non- Sample Resource Models #413, which fixes it.
• Cross attention: a batch whose rows share context pages (the same audio sent repeatedly, forks) failed as a whole on an assert that the step's page list fits the pool. With CUDA graphs on, the same assert also kept Whisper from starting on any cross pool under 384 pages, which validation accepts (e.g. 160 pages at max_concurrent_requests: 13). The assert is gone, and the captured wrapper's page-index buffer is now sized per row, so those steps fit it (Cross-attention plan asserts when forked contexts index more pages than the context pool holds, failing the whole batch #1006).
• Smaller fixes:
  • Pi0.5: an eager action_gen batch over 4 rows no longer crashes.
  • Z-Image: a side over 8192 px is a 400, not a 500.
  • Chatterbox: a stream_first_chunk_tokens too small to fill the vocoder cache is refused at startup, instead of crashing on the second chunk (Chatterbox streaming crashes on the second chunk when the first chunk is shorter than the vocoder cache #1048).
  • FLUX.2: a BFL-named LoRA keeps its .alpha, so it merges at the right strength.
  • Qwen3.5: an image 1 or 3 px tall is no longer read as channels-first.
  • The safetensors loaders open weights on the node's own GPU, not GPU 0.

How it works

• Bad requests are refused where each model resolves the request (process_prompt / _resolve_gen_params), as a ValueError (400). The bounds come from the model's config: the embodiment table size, the VAE scale, the rotary table, the encoder's token count. Nothing new reaches the worker.
• compile_transformer_forward sets Dynamo's recompile_limit around each call. Dynamo reads the limit when a call compiles, so this works on every torch version. Before 2.12 the setting is process-wide while the forward runs, so a compile on another thread at the same time would see it too. Today only the GPU thread compiles.
• Qwen3-TTS: mstar already loads qwen-tts' tokenizer source itself. On that private copy it makes upsample / decoder_transformer plain class attributes. On 5.21+ it maps the checkpoint's flat Mimi names to the nested layout, using the same three renames as transformers' own conversion_mapping.
• EXIF: one helper, media_limits.open_upright_image, opens an image with Pillow and rotates it. read_image sends a static WebP with an orientation tag through it, because torchvision's WebP decoder ignores EXIF.
• Cross attention drops the assert. Its captured wrapper sizes the page index buffer with captured_page_capacity, the same helper the self-attention managers use, which comes to about 1 MB for Whisper.

Tests

Test	Where	Result
ruff check .	dev box	pass
CPU Core suite vs main	CPU (dev box)	11216 passed, 1155 skipped on main 9c6f1ad; 0 failures only on this branch
Changed test files repeated 20× (13 files)	CPU (dev box)	20008 passed, 0 failed
DiT recompile limit under the real torch.compile on torch 2.10 (test_flux2_klein_model.py)	CPU	pass; fails without the fix
Qwen3-TTS codec encoder loads a flat Mimi checkpoint (test_qwen3_tts_model.py)	CPU, transformers 5.20.2 and 5.22.1	pass on both; fails without the fix on 5.21+
Same seeded encoder and clip on transformers 5.20.2 vs 5.22.1	CPU	identical codes (1, 32, 38)
EXIF orientation on JPEG, static WebP and LingBot, plus a malformed-EXIF case (test_media_decode_limits.py)	CPU	pass; 6 of 16 cases fail without the fix
Cosmos3 domain_id, action shape, short clip and size cases (cosmos3/tests)	CPU	pass; fail without the fixes
V-JEPA 2 mask cases, Pi0.5 batch of 5, Z-Image 8208 px, Chatterbox first chunk, FLUX.2 LoRA alpha, Qwen3.5 1/3 px images, loader device spy	CPU	pass; fail without the fixes
Cross-attention plan with shared context pages, eager and captured (test_cross_attention_plan.py)	CPU	pass; fails without the fix
Whisper at c=128 with repeated audio on a 400-page cross pool, CUDA graphs on	mstar_sim (emulated H100, #1180)	main: 363 of 384 fail on the #1006 assert ("indexes 427 pages but the context cache holds 400"); this branch: 384 of 384 served
Whisper at c=128 with repeated audio on a 400-page cross pool, CUDA graphs on	GPU (H100)	main: 363 of 384 fail on the #1006 assert. This branch: 384 of 384 served, every transcript correct
FLUX.2 klein 4B / Z-Image-Turbo compiled builds on torch 2.10, stock configs	GPU (H100)	main: both fail at startup (compile() got an unexpected keyword argument 'recompile_limit'). This branch: both build (16 compiled shapes each) and serve a 1024² image, no recompile-limit warning
Whisper with a 160-page cross pool, CUDA graphs on	mstar_sim (emulated H100)	main: dies at startup on the assert ("indexes 384 pages but the context cache holds 160"); without the assert, the old captured buffer (pool + batch size) would overflow, 192 vs 384. This branch: starts and serves 128 of 128
Served end to end, main vs this branch: EXIF upright reference image (FLUX.2), Cosmos3 action shape / short clip / sub-latent size, V-JEPA 2 lone mask, Pi0.5 eager batch of 8 and 16, Qwen3.5 256×1 image, Chatterbox stream_first_chunk_tokens: 6, Z-Image 8208 px, FLUX.2 / Z-Image compile on torch 2.10	mstar_sim (emulated H100)	main fails each as described (500s, a dropped mask, a sideways image, chunk-2 crash, startup TypeError); this branch: 400s where refused, correct output otherwise. Cosmos3 domain_id=40 and a V-JEPA 2 context_mask index of 10**6 device-assert on main and poison the worker (the next requests fail too); this branch answers 400 and keeps serving. The loaders stage a cuda:1 node's weights on cuda:0 on main (64 MB peak on cuda:0), on cuda:1 here
Qwen3-TTS Base reference encoder build and load on transformers 5.22.1, header-only checkpoint	mstar_sim (emulated H100)	main: "no setter"; with only the setter half, 125 tensors not loaded; this branch: loads
Qwen3-TTS-12Hz-1.7B-Base voice clone on transformers 5.22.1	GPU (H100)	main: the worker dies at startup ("no setter"). This branch: clones one sentence (6.0 s of audio), which Whisper transcribes exactly

🤖 Generated with Claude Code