[Quantization] Redo Z-Image text encoder FP8 online quantization

[Isotr0py]

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

• Redo [Quantization] Enable FP8 online quantization for Z-image text encoder #1338, which is reverted because of a broken import before.

Test Plan

python examples/offline_inference/text_to_image/text_to_image.py --model /mnt/data0/LLM/Z-Image-Turbo/ --width 512 --height 512 --quantization fp8 --tensor-parallel-size 2

Test Result

INFO 04-30 18:20:24 [diffusers_loader.py:355] Loading weights took 8.20 seconds
INFO 04-30 18:20:25 [diffusion_model_runner.py:142] Model loading took 7.7360 GiB and 11.881425 seconds
INFO 04-30 18:20:25 [diffusion_model_runner.py:147] Model runner: Model loaded successfully.
INFO 04-30 18:20:25 [diffusion_model_runner.py:142] Model loading took 7.7371 GiB and 11.893948 seconds
INFO 04-30 18:20:25 [diffusion_model_runner.py:147] Model runner: Model loaded successfully.
INFO 04-30 18:20:25 [diffusion_model_runner.py:86] Model runner: transformer compiled with torch.compile.
INFO 04-30 18:20:25 [diffusion_model_runner.py:188] Model runner: Initialization complete.
INFO 04-30 18:20:25 [diffusion_model_runner.py:86] Model runner: transformer compiled with torch.compile.
INFO 04-30 18:20:25 [diffusion_model_runner.py:188] Model runner: Initialization complete.
INFO 04-30 18:20:26 [diffusion_worker.py:199] Worker 0: Process-scoped GPU memory after model loading: 8.25 GiB.
INFO 04-30 18:20:26 [manager.py:96] Initializing DiffusionLoRAManager: device=cuda:0, dtype=torch.bfloat16, max_cached_adapters=1, static_lora_path=None
INFO 04-30 18:20:26 [diffusion_worker.py:97] Worker 0: Initialization complete.
INFO 04-30 18:20:26 [diffusion_worker.py:703] Worker 0: Scheduler loop started.
INFO 04-30 18:20:26 [diffusion_worker.py:613] Worker 0 ready to receive requests via shared memory
INFO 04-30 18:20:26 [diffusion_worker.py:199] Worker 1: Process-scoped GPU memory after model loading: 8.27 GiB.
INFO 04-30 18:20:26 [manager.py:96] Initializing DiffusionLoRAManager: device=cuda:1, dtype=torch.bfloat16, max_cached_adapters=1, static_lora_path=None
INFO 04-30 18:20:26 [diffusion_worker.py:97] Worker 1: Initialization complete.
INFO 04-30 18:20:26 [diffusion_worker.py:703] Worker 1: Scheduler loop started.
INFO 04-30 18:20:26 [diffusion_worker.py:613] Worker 1 ready to receive requests via shared memory
INFO 04-30 18:20:26 [diffusion_engine.py:443] dummy run to warm up the model
INFO 04-30 18:20:26 [kv_transfer_manager.py:1268] Rank-aware KV receive: rank 1 independently receiving (from_tp=2, to_tp=2)
WARNING 04-30 18:20:26 [kv_transfer_manager.py:985] No connector available for receiving KV cache
INFO 04-30 18:20:26 [kv_transfer_manager.py:1268] Rank-aware KV receive: rank 0 independently receiving (from_tp=2, to_tp=2)
WARNING 04-30 18:20:26 [kv_transfer_manager.py:985] No connector available for receiving KV cache
WARNING 04-30 18:20:26 [pipeline_z_image.py:541] strength parameter (0.60) is only applicable for image-to-image (I2I) generation. It will be ignored for text-to-image (T2I) generation.
INFO 04-30 18:20:26 [marlin_utils.py:433] Marlin kernel can achieve better performance for small size_n with experimental use_atomic_add feature. You can consider set environment variable VLLM_MARLIN_USE_ATOMIC_ADD to 1 if possible.
INFO 04-30 18:20:31 [diffusion_model_runner.py:213] Peak GPU memory (this request): 8.78 GB reserved, 8.34 GB allocated, 0.44 GB pool overhead (5.0%)
INFO 04-30 18:20:31 [inline_stage_diffusion_client.py:63] [InlineStageDiffusionClient] Stage-0 initialized inline (batch_size=1)
INFO 04-30 18:20:31 [async_omni_engine.py:823] [AsyncOmniEngine] Stage 0 initialized (diffusion, batch_size=1)
INFO 04-30 18:20:31 [orchestrator.py:192] [Orchestrator] Starting event loop
INFO 04-30 18:20:31 [async_omni_engine.py:377] [AsyncOmniEngine] Orchestrator ready with 1 stages
INFO 04-30 18:20:31 [omni_base.py:162] [Omni] AsyncOmniEngine initialized in 28.80 seconds
INFO 04-30 18:20:31 [omni_base.py:181] [Omni] Initialized with 1 stages for model /mnt/data0/LLM/Z-Image-Turbo/

============================================================
Generation Configuration:
  Model: /mnt/data0/LLM/Z-Image-Turbo/
  Inference steps: 50
  Cache backend: None (no acceleration)
  Quantization: fp8
  Parallel configuration: tensor_parallel_size=2, ulysses_degree=1, ulysses_mode=strict, ring_degree=1, cfg_parallel_size=1, vae_patch_parallel_size=1, enable_expert_parallel=False.
  CPU offload: False; CPU Layerwise Offload: False
  Image size: 1024x1024
============================================================

INFO 04-30 18:20:31 [orchestrator.py:894] [Orchestrator] _handle_add_request: stage=0 req=0_1e0f8561-372e-466f-bbd5-d2a534740a7a prompt_type=dict original_prompt_type=dict final_stage=0 num_sampling_params=1
Processed prompts:   0%|                                                                                                                              | 0/1 [00:00<?, ?it/s]INFO 04-30 18:20:31 [kv_transfer_manager.py:1268] Rank-aware KV receive: rank 0 independently receiving (from_tp=2, to_tp=2)
WARNING 04-30 18:20:31 [kv_transfer_manager.py:985] No connector available for receiving KV cache
INFO 04-30 18:20:31 [kv_transfer_manager.py:1268] Rank-aware KV receive: rank 1 independently receiving (from_tp=2, to_tp=2)
WARNING 04-30 18:20:31 [pipeline_z_image.py:541] strength parameter (0.60) is only applicable for image-to-image (I2I) generation. It will be ignored for text-to-image (T2I) generation.
WARNING 04-30 18:20:31 [kv_transfer_manager.py:985] No connector available for receiving KV cache
INFO 04-30 18:22:03 [diffusion_model_runner.py:213] Peak GPU memory (this request): 12.17 GB reserved, 10.13 GB allocated, 2.04 GB pool overhead (16.8%)
INFO 04-30 18:22:03 [diffusion_engine.py:127] Generation completed successfully.
INFO 04-30 18:22:03 [diffusion_engine.py:174] Post-processing completed in 0.0336 seconds
INFO 04-30 18:22:03 [diffusion_engine.py:177] DiffusionEngine.step breakdown: preprocess=0.00 ms, add_req_and_wait=91458.09 ms, postprocess=33.60 ms, total=91492.05 ms
Processed prompts: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [01:31<00:00, 91.49s/it]
Total generation time: 91.4952 seconds (91495.16 ms)
INFO 04-30 18:22:03 [text_to_image.py:499] Outputs: [OmniRequestOutput(request_id='0_1e0f8561-372e-466f-bbd5-d2a534740a7a', finished=True, stage_id=0, final_output_type='image', request_output=OmniRequestOutput(request_id='0_1e0f8561-372e-466f-bbd5-d2a534740a7a', finished=True, stage_id=None, final_output_type='image', request_output=None, images=[1 PIL Images], prompt={'prompt': 'a cup of coffee on the table', 'negative_prompt': None}, latents=None, metrics={'preprocess_time_ms': 0.0, 'diffusion_engine_exec_time_ms': 91458.09239707887, 'diffusion_engine_total_time_ms': 91492.05475999042, 'image_num': 1, 'resolution': 640, 'postprocess_time_ms': 33.60368404537439}, multimodal_output={}, custom_output={}, stage_durations={'queue_wait_ms': 0.609284732490778, 'stage_0_gen_ms': 91493.41702461243}, peak_memory_mb=12462.0), images=[1 PIL Images], prompt=None, latents=None, metrics={}, multimodal_output={}, custom_output={}, stage_durations={'queue_wait_ms': 0.609284732490778, 'stage_0_gen_ms': 91493.41702461243}, peak_memory_mb=12462.0)]
Saved generated image to qwen_image_output.png

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
• The test results. Please paste the results comparison before and after, or the e2e results.
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
• (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

[chatgpt-codex-connector]

💡 Codex Review

vllm-omni/vllm_omni/diffusion/model_loader/diffusers_loader.py

Lines 167 to 170 in a8303ba

	download_safetensors_index_file_from_hf(
	model_name_or_path,
	index_file_with_subfolder,
	self.load_config.download_dir,

Skip index download when no safetensors index exists

When a component has .safetensors weights but no index file (common for single-file components like vae/diffusion_pytorch_model.safetensors), available_index_file is empty and index_file_with_subfolder becomes None; this code still calls download_safetensors_index_file_from_hf(...) with that None value. In vLLM, that helper forwards index_file to hf_hub_download(filename=...), so remote loading fails at runtime for these repos (local-path loads may not hit this path).

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Isotr0py]

@lishunyang12 CI should pass this time.

[lishunyang12]

Please check again. Seems CI failed still. Thanks.

[Isotr0py]

=================================================================== short test summary info ====================================================================
--
FAILED tests/e2e/online_serving/test_mimo_audio.py::test_audio_to_text_audio_001[omni_server0] - AssertionError: The audio content is not same as the text
=============================================== 1 failed, 4 passed, 1 skipped, 25 warnings in 711.27s (0:11:51) ================================================

I thought the failing CI is omni model instead, which should be unrelated to this PR.

[Isotr0py]

@lishunyang12 @hsliuustc0106 All merge-tests should pass this time.