Bump Transformers version to 5.14.1 - #49223
Conversation
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
|
The only failure was a known failure for the main commit this CI was run against. Triggering all CI |
CI failures in build 79145 — evidence they are not caused by the Transformers 5.14.1 bumpAll three hard-failed steps reproduce on 1.
|
| Main build | Commit | Date | Result |
|---|---|---|---|
| 79241 | b8fb56d97 |
Jul 21 | failure |
| 79322 | 387189c42 |
Jul 22 | failure |
| 79460 | b44311b6e |
Jul 22 | GSM8K metric too low: 0.6452 < 0.7700 |
| 79511 | 910cc8543 |
Jul 22 | GSM8K metric too low: 0.7286 < 0.7700 |
| 79617 | 239fc7355 |
Jul 23 | 0.7096 < 0.7700, plus Qwen3-30B-A3B-MXFP4A16-humming-act-fp8: 0.0000 < 0.7800 |
The PR's 0.7202 sits inside main's 0.6452–0.7286 range: the model hovers just below its threshold regardless of transformers version.
3. lm-eval-pcp-4xb200 — same garbage-output signature on main
PR failure: both GLM-5.2-NVFP4 PCP configs produce garbage:
FAILED ...[GLM-5.2-NVFP4-TP2-PCP2-EP] - GSM8K metric too low: 0.0986 < 0.8200
FAILED ...[GLM-5.2-NVFP4-TP1-PCP4-EP] - GSM8K metric too low: 0.0205 < 0.8200
Invalid rate: 0.976
Identical signature on main build 79241 (b8fb56d97, Jul 21):
FAILED ...[GLM-5.2-NVFP4-TP2-PCP2-EP] - GSM8K metric too low: 0.0212 < 0.8200
FAILED ...[GLM-5.2-NVFP4-TP1-PCP4-EP] - GSM8K metric too low: 0.0235 < 0.8200
Invalid rate: 0.975
Also failed on main builds 79460 (different mode: TP1-PCP4 worker CUDA OOM allocating 24.43 GiB at startup) and 79511 (job marked failed without starting); passed on 79322 and 79617 — i.e. flaky on main, and GLM is a native (non-Transformers-backend) model.
Conclusion
- plugin-tests-2-gpus: pre-existing main failure, already fixed by [CI][Bugfix] Reduce max_model_len in OOT embedding test to fix KV-cache OOM on small GPUs #49351 — resolved by merging main.
- lm-eval-humming-act-fp8-slash-int8-h100: solidly red on main.
- lm-eval-pcp-4xb200: flaky on main with the identical failure signature reproduced on a 5.13.1 main commit.
None of the remaining failures are caused by bumping Transformers to 5.14.1.
No fixes needed for this one, it just worked!