[Example] Migrate HunyuanImage-3.0 to model_extras + shared task examples

[suyanli220]

[Example] Migrate HunyuanImage-3.0 to model_extras + shared task examples

Closes #4544 (sub-task of RFC #4539).

Summary

Moves HunyuanImage-3.0-Instruct's standard four-modality path off bespoke
per-model scripts onto declarative model_extras registration + the shared
task examples (text_to_image.py / image_edit.py / x_to_text.py),
following the MammothModa2 precedent from #4225.

• Adds vllm_omni/model_extras/hunyuan_image3.py: declares
  EXTRA_BODY_PARAMS (bot_task / use_system_prompt / system_prompt /
  negative_prompt), the text/image prompt builders, and an
  ar_input_builder hook.
• Adds prompt_utils.build_ar_prompt_inputs(): one seam wrapping
  build_prompt_tokens + resolve_stop_token_ids (byte-for-byte HF parity,
  with a string fallback when no tokenizer is available), reused by the
  examples and the OpenAI server.
• Wires the shared text_to_image.py / image_edit.py to invoke a model's
  declared ar_input_builder generically (guarded; models without one are
  untouched).
• Adds --trust-remote-code to the shared scripts (opt-in, off by default).
• AR-only comprehension (i2t / t2t) routes through the shared
  examples/offline_inference/x_to_text/x_to_text.py entrypoint (see
  "Rebase onto upstream/main" below).
• Removes examples/offline_inference/hunyuan_image3/ entirely (end2end.py,
  reproduce.sh, README, and its auto-generated docs page); consolidates all
  usage documentation into recipes/Tencent/HunyuanImage-3.0-Instruct.md.

RFC #4539 asks for model-specific example directories to collapse into
task-level examples. All four modalities now do: t2i/it2i via
text_to_image.py / image_edit.py, i2t/t2t via the shared x_to_text.py,
and examples/offline_inference/hunyuan_image3/ no longer has any dedicated
script or docs page. end2end.py's streaming chain-of-thought display and
diffusion KV-cache dtype / skip-steps controls have no equivalent in the
shared examples yet -- accepted as a known gap rather than a reason to keep
the directory, matching every other already-migrated model in the RFC (see
"Second rebase" below for how this played out across review).
recipes/Tencent/HunyuanImage-3.0-Instruct.md is the single documentation
home for run commands, deploy-config selection, and hardware notes.

A real bug found and fixed during GPU verification

Found by actually running the migrated path end-to-end on 4×H200 — not
visible from reading the diff alone.

registry.py originally registered HunyuanImage3's extra_body_params /
ar_input_builder under "HunyuanImage3Pipeline" — the diffusers pipeline
class name. But od_config.model_class_name at runtime is
"HunyuanImage3ForCausalMM" (the diffusion-registry arch key set by
vllm_omni/model_executor/models/hunyuan_image3/pipeline.py and consumed via
vllm_omni/config/stage_config.py's engine_args.setdefault("model_class_name", ps.model_arch)).
The old key never matched at runtime, so extra_body routing
(bot_task / use_system_prompt / negative_prompt) and the
ar_input_builder lookup were silent no-ops — the entire declarative
routing this migration adds would never have fired.

Fixed by re-keying the spec to "HunyuanImage3ForCausalMM", keeping
"HunyuanImage3Pipeline" as a compatibility alias for any path that
surfaces the pipeline class name instead. Covered by a new regression test:
test_hunyuan_image3_extra_registry_matches_runtime_model_class_name.

Also found (already fixed on main — no change carried here)

GPU verification also hit ValueError: No images found in request_output
on every think-mode t2i/it2i request, traced to
output_formatter.format_diffusion_outputs():
is_text_output = postprocess_output.primary_key == "text" or "text" in postprocess_output.metadata.
HunyuanImage-3.0's DiT post_decode() attaches the AR's chain-of-thought
text to metadata["text"] alongside the real image payload (same pattern
as Bagel / SenseNova-U1 "think" mode), so the or "text" in metadata clause
misclassified the response as text-only and discarded the image.
Checked current main and this is already fixed there in #5238
(is_text_output keyed off primary_key alone), so no fix is carried in
this PR. Verification below was run with that fix applied locally to
unblock testing.

Also removed a speculative _recover_images_from_outputs /
_tensor_to_pil deep-scan fallback that had been added to
text_to_image.py / image_edit.py while chasing the bug above before its
root cause was found — once fixed, HunyuanImage-3.0's DiT output populates
OmniRequestOutput.images through the exact same default
output_formatter.py path every other diffusers-style model uses, so the
fallback never had a live code path and only added ~150 lines of unverified
heuristics.

Response to review

• --trust-remote-code opt-in was bypassed: _apply_ar_stage_inputs()
  hardcoded AutoTokenizer.from_pretrained(model, trust_remote_code=True)
  regardless of the CLI flag. Fixed: threaded the resolved flag through in
  both text_to_image.py and image_edit.py.
• E2E didn't cover this PR's actual new path: added
  tests/model_extras/test_shared_script_ar_integration.py (CPU, calls the
  shared scripts' real _apply_ar_stage_inputs against the real
  model_extras registry — the path where the registry-key bug above would
  have gone undetected) and test_shared_script_ar_path_reaches_generation
  in the GPU e2e file (drives get_model_class_name → get_ar_input_builder
  → _apply_ar_stage_inputs → Omni.generate() through the real registry,
  reusing the en_recaption reference embedding to also check output parity
  with the hand-constructed path the existing tests use).
• stop_token_ids broadcast too broadly: was applied to every
  non-diffusion-typed stage by inference. ARStageInputs now declares
  stage_indices explicitly; the shared scripts apply stop_token_ids by
  that index instead of type-based inference.
• Special-token ids are tied to a specific tokenizer revision: added a
  registry-declared ar_tokenizer_validator hook
  (get_ar_tokenizer_validator) that checks a loaded real tokenizer's
  special-token ids against HUNYUAN_IMAGE3_SPECIAL_TOKEN_IDS and raises on
  drift.
• AR tokenizer load failure was silently swallowed: _apply_ar_stage_inputs
  now re-raises by default instead of degrading to the string-prompt form
  (which can complete "successfully" with subtly wrong stop tokens). Added
  an explicit --allow-tokenizer-fallback opt-in on both scripts for a
  deliberate offline-compat run.
• Directory convergence with RFC [RFC]: Unify example model usage — migrate all special-usage examples to standard task examples + declarative model_extras #4539 incomplete: documented above
  rather than silently diverging.

Deferred to a follow-up PR (re-assessed each individually rather than
batching them; these three specifically need either a live GPU re-run to
verify or a larger, separate design):

• Reusing the engine's already-loaded tokenizer in _apply_ar_stage_inputs
  instead of a second independent AutoTokenizer.from_pretrained call.
  Concretely: AsyncOmniEngine._bootstrap_orchestrator creates its event
  loop as a local variable, never stored on self, so nothing outside the
  orchestrator's background thread can currently reach it; exposing
  get_tokenizer() to the sync Omni wrapper needs that loop stored (e.g.
  self._loop) and a run_coroutine_threadsafe-based sync wrapper. That
  touches core engine bootstrap/threading code every model goes through,
  not just HunyuanImage3, so it needs a live run to confirm it doesn't
  introduce a deadlock or shutdown-ordering issue -- unlike the fixes above,
  which are scoped to one function with exactly one current caller.
• Pinning a verified model revision for the shared-script commands above —
  blocked on Omni() not having a revision passthrough today; that'd need
  to land first. (Previously tracked against reproduce.sh, since removed
  along with the rest of the directory -- see "Second rebase" below.)

Rebase onto upstream/main

Rebased to pick up 143 commits, including upstream's own independent
x_to_text.py shared example + build_x_to_text_prompt/
get_x_to_text_model_family seam (#5384) — landed after this PR's original
i2t/t2t design, so it made the "generic shared understanding example"
follow-up item above moot and left our dedicated
run_hunyuan_image3_understanding.py fully duplicated. Resolved by:

• Dropping run_hunyuan_image3_understanding.py; i2t/t2t now go through
  the shared x_to_text.py, matching t2i/it2i's use of the shared
  text_to_image.py / image_edit.py.
• Refactoring build_x_to_text_prompt (upstream's function, in the same
  model_extras/hunyuan_image3.py file this PR already owns) to delegate to
  build_ar_stage_inputs instead of re-deriving the AR prefill / stop-token
  logic inline — so all four modalities share one seam down to
  prompt_utils.build_ar_prompt_inputs. Verified this is behavior-preserving:
  for t2t/i2t, bot_task resolves to None whether passed explicitly
  (the old code's behavior) or omitted from extra_body (the new code's
  behavior; _normalize_task_and_bot_task's legacy-preset table maps both
  tasks' omitted-bot_task sentinel to None).
• Regenerated the example README + docs page for the new entrypoint, and
  the CPU test that pins "no duplicated tokenization logic"
  (test_example_routes_through_shared_seam) to check the seam function
  itself rather than the now-deleted script.

Second rebase — directory removal, doc consolidation

Rebased again onto a further-diverged upstream/main. This raised three
review points from @princepride: (1) fully deleting
examples/offline_inference/hunyuan_image3/, (2) folding
--trust-remote-code / --allow-tokenizer-fallback into a generic
--extra-args mechanism instead of dedicated flags, (3) a tokenizer
docstring's clarity.

While resolving the rebase conflict on end2end.py (this PR's earlier
commit had deleted it; upstream/main had kept it), I found upstream had
independently kept extending that exact file with real, unique
functionality (streaming chain-of-thought display, diffusion KV-cache
dtype / skip-steps control) that has no equivalent in the shared task
examples yet, and initially restored it from upstream/main rather than
re-deleting it to avoid silently regressing that functionality. @princepride
weighed that tradeoff explicitly and confirmed the RFC's directory-removal
goal still applies regardless -- that gap is accepted as known, not a reason
to special-case this model. Resolved by fully removing
examples/offline_inference/hunyuan_image3/ (end2end.py, reproduce.sh,
README, and its auto-generated docs page) and consolidating all usage
documentation into recipes/Tencent/HunyuanImage-3.0-Instruct.md, matching
every other already-migrated model in the RFC. Also, upstream's own
"canonical image task prompts" refactor (#6049) made per-model
text_to_image_prompt_builder hooks optional; HunyuanImage3's builder did
nothing beyond reproducing the shared script's own canonical envelope, so
it (plus its registry entry and tests) is deleted, matching the
cosmos3/LingBot precedent from that same upstream commit.

The two other review points (--extra-args folding, tokenizer docstring)
were addressed via threaded replies rather than code changes — the CLI
flags are opt-in safety switches with a small closed set of values, not
open-ended model tuning knobs, so keeping them as dedicated flags (with
clear --help text) reads better than moving them behind an opaque JSON
blob; and the docstring reply pointed at validate_ar_tokenizer's existing
wording, which already explains why it exists (catching an AR/tokenizer
revision drift that would otherwise silently produce wrong stop-token ids)
rather than just what it does.

A separate, earlier reviewer (@hsliuustc0106) had left four inline comments
that were still open with no replies: two turned out to already be fixed in
code from the review-response round above (a test-argv bug, and the
tokenizer validator's base-vs-Instruct false positive), and two were
genuinely unaddressed until this round — serving_chat.py calling
resolve_stop_token_ids with a different effective bot_task than
build_prompt_tokens/build_prompt used for the same request (fixed, with
a regression test), and test_system_prompt_scores only printing its CLIP
score instead of asserting a threshold (fixed, reusing the same
MIN_SEMANTIC_SIMILARITY floor test_shared_script_ar_path_reaches_generation
already used).

Verification

Round 1 — 4×H200, tencent/HunyuanImage-3.0-Instruct (pre-rebase)

Ran examples/offline_inference/hunyuan_image3/reproduce.sh end-to-end
(CUDA_VISIBLE_DEVICES=0,1,2,3, AR on devices 0-1 / DiT on devices 2-3, per
vllm_omni/deploy/hunyuan_image_3_moe.yaml):

Modality	Result
t2i	✅ correct, high-quality 1024×1024 output
it2i	✅ completes correctly, extra_args routed correctly, edit applied (image quality artifact noted below)
i2t	✅ accurate description of the generated image
t2t	⚠️ unexpected model behavior — see below

t2i ("A brown and white dog is running on the grass"):

it2i ("Make the dog neon pink", edited from the t2i output above):

Recolor applied correctly, but the output has visible noise/oversaturation
(see "Known limitations" below — pre-existing, not from this PR's changes).

Unit tests: 51 passed on this pass (tests/model_extras/test_model_extras.py,
tests/diffusion/test_diffusion_output_formatter.py).

Round 2 — 4×H200, post-rebase, all four modalities re-run

After the review-response round and the rebase onto upstream/main (see
below), re-ran reproduce.sh end-to-end with all four modalities,
including i2t/t2t for the first time since they were re-routed onto the
shared x_to_text.py:

Modality	Result
t2i	✅ 36s, correct 1024×1024 output, no artifacts
it2i	✅ 41s, edit correctly applied (neon-pink flowers on a different source image than round 1); same pre-existing noise/oversaturation artifact reproduces on this unrelated input too, reinforcing it's model-inherent, not input-specific
i2t	✅ accurate, coherent description of the input image ("The image features a vibrant scene with a prominent tower, likely the Tokyo Skytree, set against a clear blue sky...")
t2t	✅ reproduces the exact known hallucination (model describes an imagined image containing the question text instead of answering "Paris" directly) — confirms this is a stable, pre-existing model behavior, not something the migration changed

t2i ("A brown and white dog is running on the grass"):

it2i ("Make the flowers neon pink", edited from a different source image):

KV cache transfer (AR→DiT, Mooncake connector, TP=2 both stages) succeeded
cleanly on both TP ranks for both t2i and it2i, and the AR-only i2t/t2t path
(TP=4, hunyuan_image3_ar.yaml) initialized and generated without incident.

Environment notes specific to this round (all environment/infra, none in
this PR's code):

• Same DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA requirement as round 1 (see
  "Known limitations").
• Omni()'s default init_timeout/stage_init_timeout (600s/300s) was too
  tight for a cold start on a fresh container (no kernel cache) — confirmed
  via logs that the AR worker was still making genuine progress toward
  becoming ready when the timeout fired and tore it down. Not a hang; a
  cold-start budget issue. Not changed in this PR's shipped code — worth a
  follow-up if others hit it on fresh hardware.
• Hit and worked around a pre-existing bug in the (unrelated,
  vllm_omni/distributed/omni_connectors/) Mooncake KV-transfer connector:
  its ZMQ port allocation didn't reliably differentiate between a stage's
  TP ranks on a cold multiprocessing spawn, causing an EADDRINUSE bind
  race. Out of scope for this PR (core connector infra, not
  model_extras/HunyuanImage3 code) — filing a separate issue upstream with
  the repro.

Unit tests: the 51 from round 1 still pass. The review-response round's CPU
tests (tests/model_extras/test_shared_script_ar_integration.py, plus
additions to test_model_extras.py/test_prompt_utils.py) are
syntax-checked and manually traced against real behavior but still not
executed via pytest in this environment (no working Python env with
network access to install pytest locally). The GPU e2e pytest additions
(test_shared_script_ar_path_reaches_generation) require 8 GPUs
(pre-existing skipif gate in this test file, unrelated to this PR) and
were not run either round — however, round 2 exercised the exact same
production code path that test targets
(get_model_class_name → get_ar_input_builder → _apply_ar_stage_inputs
→ Omni.generate()) via the real reproduce.sh CLI invocation, which is
the load-bearing evidence for that code path now.

Known limitations (pre-existing, not introduced by this PR)

• it2i image quality: edited images show visible noise/oversaturation.
  Isolated to not be caused by the environment's TORCH_SDPA attention
  fallback (see below) — t2i, running through the identical DiT model and
  backend, is clean. Root cause likely sits in the pre-existing DiT
  image-conditioning / VAE path, not in code this PR touches. Round 2
  reproduced the identical artifact on a different source image and edit
  prompt, reinforcing that it's model/pipeline-inherent rather than tied to
  one specific input.

• t2t hallucination: the model answers a plain-text question by
  describing an imagined image containing that text, instead of answering
  directly. Round 2 reproduced this verbatim in kind (again describing an
  imagined image containing the literal question text) on a live rerun,
  reinforcing the code-level argument below with a second live data point.
  Confirmed not a regression: the pre-migration end2end.py
  (_MODALITY_TASK_MAP["text2text"] = ("t2t", None)) resolves to the exact
  same task="t2t", bot_task=None → en_unified system-prompt path as the
  migrated code (build_ar_stage_inputs/build_x_to_text_prompt with
  task="t2t", bot_task resolving to None). Both call the same
  build_ar_prompt_inputs() seam with identical inputs, so this is a
  pre-existing model/prompt-template behavior, not something this PR
  introduced. (A live pre-migration GPU rerun was attempted for a visual
  side-by-side but hit unrelated environment issues — legacy transformers
  trust_remote_code prompt handling on the deleted script, vllm/driver
  version skew — that aren't worth resolving for a script this PR removes;
  the code-level comparison above is the load-bearing evidence.)

• This environment's CUDA driver (575.57.08, max CUDA 12.9) required
  installing a +cu129 wheel variant instead of the PyPI default
  (round 1: vllm 0.25.0; round 2: vllm 0.26.0, matching this repo's
  docker/Dockerfile.cuda base image as of the round-2 rebase); the driver
  also predates a flash-attn build exposing the non-varlen flash_attn_func
  HunyuanImage3's piecewise-attention path calls (fa3-fwd only provides
  flash_attn_varlen_func), so both rounds ran with
  DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA. Unrelated to this PR's code.

Repro

examples/offline_inference/hunyuan_image3/reproduce.sh is removed along
with the rest of the directory (see "Second rebase" above); the four
modalities now run directly through the shared task examples --
copy-pasteable commands for all four are in
recipes/Tencent/HunyuanImage-3.0-Instruct.md. Summarized:

export MODEL=tencent/HunyuanImage-3.0-Instruct

# t2i
python examples/offline_inference/text_to_image/text_to_image.py \
  --model "$MODEL" --deploy-config vllm_omni/deploy/hunyuan_image_3_moe.yaml \
  --trust-remote-code --prompt "A brown and white dog is running on the grass" \
  --height 1024 --width 1024 --guidance-scale 5.0 --num-inference-steps 50 --seed 42 \
  --extra-body '{"bot_task": "think", "use_system_prompt": "en_recaption"}' \
  --output t2i.png

# it2i (set EDIT_IMAGE to a real reference image path)
python examples/offline_inference/image_to_image/image_edit.py \
  --model "$MODEL" --deploy-config vllm_omni/deploy/hunyuan_image_3_moe.yaml \
  --trust-remote-code --image "$EDIT_IMAGE" --prompt "Make the flowers neon pink" \
  --guidance-scale 5.0 --num-inference-steps 50 --seed 42 \
  --extra-args '{"bot_task": "think"}' --output it2i.png

# i2t / t2t (AR-only deploy config, auto-selected)
python examples/offline_inference/x_to_text/x_to_text.py \
  --model "$MODEL" --image "$EDIT_IMAGE" --prompt "Describe the content of this image."
python examples/offline_inference/x_to_text/x_to_text.py \
  --model "$MODEL" --prompt "Explain why the sky appears blue."

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[hsliuustc0106]

Inline findings from review of c5ad2e00. Separately, this PR is currently dirty/non-rebaseable against main. Main now contains the shared Hunyuan x_to_text.py path and its own model_extras/hunyuan_image3.py; please preserve build_x_to_text_prompt while resolving the add/add conflict and route comprehension through the shared x-to-text entrypoint rather than retaining the now-duplicated dedicated script.

[suyanli220]

Inline findings from review of c5ad2e00. Separately, this PR is currently dirty/non-rebaseable against main. Main now contains the shared Hunyuan x_to_text.py path and its own model_extras/hunyuan_image3.py; please preserve build_x_to_text_prompt while resolving the add/add conflict and route comprehension through the shared x-to-text entrypoint rather than retaining the now-duplicated dedicated script.

@hsliuustc0106 Thank you for the detailed review. I verified the findings.

Current progress:

The P1 issues have been addressed. trust_remote_code now correctly follows the user-provided CLI flag, and I added both CPU integration coverage and GPU E2E coverage for the previously untested get_ar_input_builder → build_ar_stage_inputs → _apply_ar_stage_inputs path.
The P2 items, including overly broad stop_token_ids propagation, token-ID drift validation, and the RFC directory-structure differences, have either been fixed or explicitly documented in the PR.

I am rebasing onto the latest main, which includes PR #5384’s shared x_to_text.py path and introduces an add/add conflict in model_extras/hunyuan_image3.py. I will preserve build_x_to_text_prompt and migrate the HunyuanImage3 I2T/T2T flows to the shared entrypoint instead of retaining the duplicated model-specific script.

After resolving the rebase and conflicts, I will rerun all affected tests in the GPU environment and update the PR with the results.

Thanks again.

[suyanli220]

@hsliuustc0106 Following up on my earlier reply — rebase is done and the GPU re-run is complete.

Rebased onto latest upstream/main (adopted the shared x_to_text.py entrypoint per your comment, preserving build_x_to_text_prompt and routing it through build_ar_stage_inputs). Re-ran all four modalities end-to-end on 4×H200 (round 2 in the PR description above):

• t2i: ✅ correct, clean 1024×1024 output
• it2i: ✅ edit applied correctly; same pre-existing noise/oversaturation artifact from round 1 reproduces on an unrelated input, confirming it's model-inherent
• i2t: ✅ accurate, coherent description of the input image
• t2t: ✅ reproduces the known hallucination behavior verbatim — confirms it's a stable pre-existing model quirk, not something the migration changed

KV cache transfer (AR→DiT, Mooncake connector) succeeded on both TP ranks for t2i/it2i, and the AR-only i2t/t2t path (TP=4) initialized and generated without incident.

Full details, environment notes, and the remaining known limitations (all pre-existing, unrelated to this PR's code) are in the "Verification" section of the PR description.

One honest caveat: the CPU tests from the review-response round are syntax-checked and manually traced but not executed via pytest in this environment (no network access to install pytest locally), and the GPU e2e pytest addition needs 8 GPUs (pre-existing skipif gate, unrelated to this PR) which I didn't have available. Round 2's reproduce.sh run does exercise the exact same production code path that GPU e2e test targets, though, which is the load-bearing evidence for it.

[princepride]

Please resolve conflicts first.😊

[princepride]

A little recommend, ptal.

[princepride]

LGTM

[vllm-omni-review-bot]

Omni ReviewBot triage note

Automated triage of commit 8929dd48602e produced:

• Priority: high. Prompt maintainer attention is suggested.

These are automated triage suggestions only — the final decision belongs to the maintainers.