Skip to content

[Model]: add FLUX.2-dev model #1630

Closed
nuclearwu wants to merge 9 commits intovllm-project:mainfrom
nuclearwu:flux2-dev
Closed

[Model]: add FLUX.2-dev model #1630
nuclearwu wants to merge 9 commits intovllm-project:mainfrom
nuclearwu:flux2-dev

Conversation

@nuclearwu
Copy link
Copy Markdown
Contributor

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

support https://huggingface.co/black-forest-labs/FLUX.2-dev

Test Plan

vLLM-Omni:

python examples/offline_inference/text_to_image/text_to_image.py \
  --model /workspace/cache/ymttest/johnjan/models/black-forest-labs/FLUX___2-dev/ \
  --prompt "a lovely bunny holding a sign that says 'vllm-omni'" \
  --seed 42 \
  --tensor-parallel-size 2 \
  --num-images-per-prompt 1 \
  --num-inference-steps 50 \
  --guidance-scale 4.0 \
  --height 1024 \
  --width 1024 \
  --output outputs/flux2-dev.png

Test Result

vLLM-Omni:
Reproduced with 4xA800.

Model/TP diffusers TP=1 TP=2 TP=4
Flux.2-dev flux2-dev OOM flux2-dev flux2-dev
Time 104.9411s/img OOM 39.1087s/img 29.0770s/img

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
  • The test results. Please paste the results comparison before and after, or the e2e results.
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
  • (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
# Conflicts:
#	docs/user_guide/diffusion_acceleration.md
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
@nuclearwu nuclearwu requested a review from hsliuustc0106 as a code owner March 3, 2026 07:07
@nuclearwu nuclearwu closed this Mar 3, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a53145a246

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

"""Taking only the first image for now.""",
)
first_prompt = req.prompts[0]
prompt = first_prompt if isinstance(first_prompt, str) else (first_prompt.get("prompt") or "")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Allow embedding-only prompts by preserving None prompt

When a request supplies prompt_embeds without a text prompt, this line coerces missing text to "" instead of keeping it None. That makes check_inputs think both prompt and prompt_embeds were provided and raises a ValueError, so embedding-only inference for Flux2Pipeline is effectively broken.

Useful? React with 👍 / 👎.

Comment on lines +1054 to +1055
# compute the previous noisy sample x_t -> x_t-1
latents = self.scheduler.step(noise_pred, t, latents, return_dict=False)[0]
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Invoke step-end callback during denoising iterations

The pipeline accepts callback_on_step_end and validates callback_on_step_end_tensor_inputs, but the denoising loop never invokes the callback. Any caller relying on per-step hooks (progress reporting, custom stopping, or latent inspection) will silently lose that behavior in Flux2Pipeline compared with other diffusion pipelines in this repo.

Useful? React with 👍 / 👎.

@mergify
Copy link
Copy Markdown

mergify Bot commented Mar 3, 2026

⚠️ The sha of the head commit of this PR conflicts with #1629. Mergify cannot evaluate rules on this PR. Once #1629 is merged or closed, Mergify will resume processing this PR. ⚠️

@nuclearwu nuclearwu deleted the flux2-dev branch March 11, 2026 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant