Skip to content

[Bug][Hunyuanimage 3.0] fix different AR encode behavior between online and offline#3500

Merged
princepride merged 1 commit into
vllm-project:mainfrom
Bounty-hunter:fix_online
May 11, 2026
Merged

[Bug][Hunyuanimage 3.0] fix different AR encode behavior between online and offline#3500
princepride merged 1 commit into
vllm-project:mainfrom
Bounty-hunter:fix_online

Conversation

@Bounty-hunter
Copy link
Copy Markdown
Contributor

@Bounty-hunter Bounty-hunter commented May 11, 2026

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

The reason see #3499

We modify the online AR encoding behavior to be consistent with the offline AR (or DiT) path, i.e., using segment-wise encoding (build_prompt_tokenids).

Test Plan

vllm serve tencent/HunyuanImage-3.0-Instruct \
  --omni \
  --port 8091 \
  --stage-configs-path ./test_hunyuan_reuse.yaml \
  --enforce-eager
curl -X POST http://localhost:8091/v1/images/edits \
  -F "image=@./input_0_0.png" \
  -F "prompt=新年宠物海报,Q版圆润的可爱标题\"新年快乐汪\",副标题\"HAPPY NEW YEAR\"。 鱼眼镜头,背景是房间门口,近景,上传的主体歪头笑,围着红色围巾,戴着红色毛线帽,高清,绒毛细节,面部特写。 宝丽莱相纸,超现实主义,写实主义,胶片摄影,打印颗粒感肌理。肌理,超写实,复古感。" \
  -F "bot_task=it2i_think" \
  -F "n=1" \
  -F "num_inference_steps=50" \
  -F "guidance_scale=2.5" \
  -F "seed=42" \
  | jq -r '.data[0].b64_json' \
  | base64 -d > result.png

Test Result

image output_0_0
Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
  • The test results. Please paste the results comparison before and after, or the e2e results.
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
  • (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

Signed-off-by: dengyunyang <584797741@qq.com>
@Bounty-hunter Bounty-hunter requested a review from tzhouam as a code owner May 11, 2026 09:38
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c2846f032a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread vllm_omni/entrypoints/openai/serving_chat.py
@Bounty-hunter Bounty-hunter changed the title fix online kv reuse [Bug][Hunyuanimage 3.0] fix different behavir between ar and dit May 11, 2026
@Bounty-hunter Bounty-hunter changed the title [Bug][Hunyuanimage 3.0] fix different behavir between ar and dit [Bug][Hunyuanimage 3.0] fix different ar encode behavior between online and offline May 11, 2026
@Bounty-hunter Bounty-hunter changed the title [Bug][Hunyuanimage 3.0] fix different ar encode behavior between online and offline [Bug][Hunyuanimage 3.0] fix different AR encode behavior between online and offline May 11, 2026
Copy link
Copy Markdown
Collaborator

@princepride princepride left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@princepride princepride added the ready label to trigger buildkite CI label May 11, 2026
@princepride princepride enabled auto-merge (squash) May 11, 2026 10:56
@princepride princepride merged commit 3bf4f28 into vllm-project:main May 11, 2026
8 checks passed
MaciejBalaNV pushed a commit to MaciejBalaNV/vllm-omni that referenced this pull request May 11, 2026
…ne and offline (vllm-project#3500)

Signed-off-by: dengyunyang <584797741@qq.com>
clodaghwalsh17 pushed a commit to clodaghwalsh17/nm-vllm-omni-ent that referenced this pull request May 12, 2026
…ne and offline (vllm-project#3500)

Signed-off-by: dengyunyang <584797741@qq.com>
Galleons2029 pushed a commit to Galleons2029/vllm-omni-ljl that referenced this pull request May 18, 2026
…ne and offline (vllm-project#3500)

Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: Jialong Liu <88185941+Galleons2029@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants