Skip to content

Bump Transformers version to 5.14.1 - #49223

Merged
vllm-bot merged 2 commits into
vllm-project:mainfrom
hmellor:bump-transformers-5-14
Jul 24, 2026
Merged

Bump Transformers version to 5.14.1#49223
vllm-bot merged 2 commits into
vllm-project:mainfrom
hmellor:bump-transformers-5-14

Conversation

@hmellor

@hmellor hmellor commented Jul 20, 2026

Copy link
Copy Markdown
Member

No fixes needed for this one, it just worked!

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the cpu Related to CPU backends label Jul 20, 2026
@hmellor hmellor added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 20, 2026
@github-project-automation github-project-automation Bot moved this to Ready in NVIDIA Jul 20, 2026
@hmellor

hmellor commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

The only failure was a known failure for the main commit this CI was run against. Triggering all CI

@hmellor hmellor added ready-run-all-tests Trigger CI with all tests for wide-ranging PRs and removed ready ONLY add when PR is ready to merge/full CI is needed labels Jul 21, 2026
@hmellor

hmellor commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

CI failures in build 79145 — evidence they are not caused by the Transformers 5.14.1 bump

All three hard-failed steps reproduce on main (transformers 5.13.1) with identical failure signatures.

1. plugin-tests-2-gpus — failing on main until #49351, which this branch predates

PR failure: test_oot_registration_offline.py::test_oot_registration_embedding

ValueError: To serve at least one request with the model's max seq len (2048),
(0.66 GiB KV cache is needed, which is larger than the available KV cache memory (0.64 GiB)

Same failure on main (identical to the hundredth of a GiB), e.g. build 79066 (616c9bd0f, Jul 21):

ValueError: To serve at least one request with the model's max seq len (2048),
(0.66 GiB KV cache is needed, which is larger than the available KV cache memory (0.64 GiB)
FAILED plugins_tests/test_oot_registration_offline.py::test_oot_registration_embedding

Also failing on main builds 79024, 79022, 78971, 78935, 78813 (Jul 20–21).

Fixed on main by fca252d59e[CI][Bugfix] Reduce max_model_len in OOT embedding test to fix KV-cache OOM on small GPUs (#49351, merged Jul 21 19:00 UTC), which reduces max_model_len from 2048 to 512. This branch's last main merge predates it. Merging main fixes this step.

2. lm-eval-humming-act-fp8-slash-int8-h100-temporary — red on 5/5 recent main builds

PR failure: test_gsm8k_correctness[Qwen3-4B-mixed-quant-RTN-humming]0.7202 < 0.7700

Same test failing on main:

Main build Commit Date Result
79241 b8fb56d97 Jul 21 failure
79322 387189c42 Jul 22 failure
79460 b44311b6e Jul 22 GSM8K metric too low: 0.6452 < 0.7700
79511 910cc8543 Jul 22 GSM8K metric too low: 0.7286 < 0.7700
79617 239fc7355 Jul 23 0.7096 < 0.7700, plus Qwen3-30B-A3B-MXFP4A16-humming-act-fp8: 0.0000 < 0.7800

The PR's 0.7202 sits inside main's 0.6452–0.7286 range: the model hovers just below its threshold regardless of transformers version.

3. lm-eval-pcp-4xb200 — same garbage-output signature on main

PR failure: both GLM-5.2-NVFP4 PCP configs produce garbage:

FAILED ...[GLM-5.2-NVFP4-TP2-PCP2-EP] - GSM8K metric too low: 0.0986 < 0.8200
FAILED ...[GLM-5.2-NVFP4-TP1-PCP4-EP] - GSM8K metric too low: 0.0205 < 0.8200
  Invalid rate: 0.976

Identical signature on main build 79241 (b8fb56d97, Jul 21):

FAILED ...[GLM-5.2-NVFP4-TP2-PCP2-EP] - GSM8K metric too low: 0.0212 < 0.8200
FAILED ...[GLM-5.2-NVFP4-TP1-PCP4-EP] - GSM8K metric too low: 0.0235 < 0.8200
  Invalid rate: 0.975

Also failed on main builds 79460 (different mode: TP1-PCP4 worker CUDA OOM allocating 24.43 GiB at startup) and 79511 (job marked failed without starting); passed on 79322 and 79617 — i.e. flaky on main, and GLM is a native (non-Transformers-backend) model.

Conclusion

None of the remaining failures are caused by bumping Transformers to 5.14.1.

@vllm-bot
vllm-bot merged commit 0d77325 into vllm-project:main Jul 24, 2026
309 of 313 checks passed
@github-project-automation github-project-automation Bot moved this from Ready to Done in NVIDIA Jul 24, 2026
@hmellor
hmellor deleted the bump-transformers-5-14 branch July 24, 2026 04:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build cpu Related to CPU backends nvidia ready-run-all-tests Trigger CI with all tests for wide-ranging PRs

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants