Skip to content

[Bugfix] Update Flux2-dev & Dynin_omni L4 e2e test#2723

Merged
gcanlin merged 3 commits into
vllm-project:mainfrom
wtomin:fix-flux2-test
Apr 13, 2026
Merged

[Bugfix] Update Flux2-dev & Dynin_omni L4 e2e test#2723
gcanlin merged 3 commits into
vllm-project:mainfrom
wtomin:fix-flux2-test

Conversation

@wtomin
Copy link
Copy Markdown
Collaborator

@wtomin wtomin commented Apr 13, 2026

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

  1. Solving [CI Failure]: Diffusion · Other · Function Test with L4, test_flux_2_dev_expansion.py, Model loading out of memory #2722. The main reason is that Flux.2-dev model parameters are too large to fit in L4 (24GB).

This PR:

  • changes the hardware from L4 to H100;
  • remove cache-dit argument, because Flux.2-dev does not support cache-dit yet.
  1. Solving [CI Failure]: Function Test with L4, test_dynin_omni_expansion.py, Model loading out of memory #2721. The major reason is the Dynin_omni model weights are too large to fit in L4 card.

This PR:

  • changes the hardware from L4 to H100;
  1. Additional edit: enlarge diffusion perf test baseline threshold

In buildkite test , it indicates the threshold for test_qwen_image_ulysses2_cfg2_vae_patch4 is a little too tight, which can cause failure occasionally.

This PR enlarges the baseline threshold from 2.34 to 2.7.

Test Plan

Run it in nightly CI.

pytest -s -v tests/e2e/online_serving/test_*_expansion.py -m "advanced_model and diffusion and L4" --run-level "advanced_model"

Test Result

Nightly tests were all passed. See results in this url.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
  • The test results. Please paste the results comparison before and after, or the e2e results.
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
  • (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
@wtomin wtomin changed the title [Bugfix] Update Flux2-dev L4 e2e test [Bugfix] Update Flux2-dev & Dynin_omni L4 e2e test Apr 13, 2026
@wtomin wtomin added the nightly-test label to trigger buildkite nightly test CI label Apr 13, 2026
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
@pytest.mark.advanced_model
@pytest.mark.omni
@hardware_test(res={"cuda": "L4", "rocm": "MI325"})
@hardware_test(res={"cuda": "H100", "rocm": "MI325"})
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this requires H100, I think tests/e2e/offline_inference/test_dynin_omni.py needs to be modified accordingly.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update it in a new PR.

@hsliuustc0106
Copy link
Copy Markdown
Collaborator

Blocker scan:

Category Result
Correctness PASS
Reliability/Safety PASS
Breaking Changes PASS
Test Coverage PASS
Documentation PASS
Security PASS

OVERALL: NO BLOCKERS

VERDICT: COMMENT

Test-only PR: hardware changes (L4→H100) for Flux2-dev & Dynin_omni due to OOM, removed cache-dit arg, perf baseline relaxation 2.34→2.7. Gates pass. Buildkite PENDING (expected - this PR changes test hardware).

Non-blocking: Please paste nightly CI results in Test Result section once buildkite completes.

@yenuo26 yenuo26 added ready label to trigger buildkite CI and removed nightly-test label to trigger buildkite nightly test CI labels Apr 13, 2026
@wtomin
Copy link
Copy Markdown
Collaborator Author

wtomin commented Apr 13, 2026

Nightly tests were all passed. See results in url.

Copy link
Copy Markdown
Collaborator

@gcanlin gcanlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gcanlin gcanlin merged commit 6b5a52a into vllm-project:main Apr 13, 2026
7 of 8 checks passed
Celeste-jq pushed a commit to IsleOfDawnlight/vllm-omni-voxcpm that referenced this pull request Apr 14, 2026
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
lengrongfu pushed a commit to lengrongfu/vllm-omni that referenced this pull request May 1, 2026
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
clodaghwalsh17 pushed a commit to clodaghwalsh17/nm-vllm-omni-ent that referenced this pull request May 12, 2026
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants