Skip to content

build(deps): bound the cua extra to vllm 0.23 and transformers v4 - #177

Merged
ashtonchew merged 1 commit into
mainfrom
deps/pin-cua-vllm-transformers
Aug 18, 2026
Merged

ashtonchew merged 1 commit into
mainfrom
deps/pin-cua-vllm-transformers

Conversation

@ashtonchew

@ashtonchew ashtonchew commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Applies the pin recommended by the vLLM numerics review merged in #174, and corrects that review's argument for it.

Base commit: 8804de42 ("docs(research): review the unreviewed vLLM 0.16 -> 0.23 upgrade for numerics", #174).

Why

pyproject.toml declared vllm>=0.6, which bounds nothing. That is how an unreviewed seven-minor-version move — 0.16.0 → 0.23.0 — rode into the lockfile on the fork collapse in #172.

#174 argued the risk was forward drift: that the next uv lock would cross the Transformers v4 removal boundary at 0.24.0. Measured, it drifts backwards instead. On 8804de42 with the unbounded spec:

$ uv lock --upgrade
  vllm           0.20.2
  transformers   4.57.6
  torch          2.11.0

0.20.2 sits inside the Qwen3-VL deepstack regression window that #174 bisected: introduced at 0.20.0 (vllm-project/vllm#40145), fixed at 0.23.0 (vllm-project/vllm#43617), with 0.20.0–0.22.1 serving GUI-Owl at roughly 30% relative accuracy loss on visual grounding.

So the floor holds the resolver out of a known-bad range. It does not hold back an upgrade — the resolver was never going to take one.

Why it cannot go forward

From package metadata retrieved 2026-08-17 (pypi.org/pypi/vllm/<version>/json, requires_dist):

vllm transformers torch
0.23.0 !=5.0.*,…,!=5.5.0,>=4.56.0 ==2.11.0
0.24.0 >=5.5.3 ==2.11.0
0.27.1 (latest, 2026-08-11) >=5.5.3 ==2.13.0

0.24.0 requires Transformers v5 outright, against a lock pinned at transformers 4.57.5. Its release notes file this under "Deprecated" (#45161), but the metadata makes it a hard floor — worth stating, because reading the changelog alone gives the wrong answer. 0.27.1 additionally pins a torch this tree does not carry.

0.23.0 is the single release satisfying both the deepstack fix and Transformers v4.

Why transformers<5 is separate from the vllm bound

vllm 0.16.0 carried its own transformers<5 ceiling. 0.23.0 dropped it, replacing it with an exclusion list that admits >=5.5.1. So the vllm pin alone leaves a later resolve free to raise transformers to v5 underneath it.

Transformers v5 stops OpenCUA at startup:

'OpenCUAProcessor' object has no attribute 'image_processor'

The upstream fix, vllm-project/vllm#47438, has been a draft since 2026-07-02 — verified open and unmerged at time of writing. Both bounds lift together once it lands.

This ceiling is scoped to the cua extra because that is where it was measured. Whether the base transformers>=4.38 also breaks on v5 is untested and deliberately unclaimed.

The bound is load-bearing, demonstrated

Same command, before and after:

uv lock --upgrade result
vllm>=0.6 (before) vllm 0.20.2 — inside the regression window
vllm>=0.23,<0.24 (after) vllm 0.23.0 — holds

No package versions moved in the committed lock. uv.lock changes only the declared requirements; the resolution is byte-identical at vllm 0.23.0 / transformers 4.57.5 / torch 2.11.0, still 360 packages.

Verification

$ uv run ruff check .        All checks passed!
$ uv run black --check .     96 files would be left unchanged.
$ uv run mypy                Success: no issues found in 96 source files
$ uv lock --check            Resolved 360 packages in 2ms

CI was not waited on, by instruction.

Scope and gaps


Correction, 2026-08-18 (see #183). This text states that vLLM 0.20.0-0.22.1 serve GUI-Owl with roughly 30% relative accuracy loss on visual grounding. That is wrong on two points.

Upstream (vllm#43617) measured Geo3K, a visual-math set, not visual grounding. It tested Qwen3-VL-2B and Qwen3-VL-8B, not a 32B model. The 2B shows 29.71% relative on the compiled-vs-eager contrast; the 8B shows 4.85% on the same contrast, so the effect shrinks as the model grows. Upstream never measured GUI-Owl-1.5-32B-Think.

The affected-tag list also omits v0.20.1, which does carry the fault.

The bound vllm>=0.23,<0.24 is unchanged and still correct. Only its stated justification was overstated.

`vllm>=0.6` bounded nothing, which is how an unreviewed 0.16.0 -> 0.23.0
move rode in on the lockfile fork collapse in #172.

Measured on 8804de4, `uv lock --upgrade` with the unbounded spec resolves
*backwards* to vllm 0.20.2 -- inside the Qwen3-VL deepstack regression
window (introduced #40145 at 0.20.0, fixed #43617 at 0.23.0). It cannot go
forward: 0.24.0 requires transformers>=5.5.3 against a lock at 4.57.5, and
0.27.1 pins torch==2.13.0 against this tree's 2.11.0.

transformers<5 is separate: 0.16.0 carried that ceiling, 0.23.0 dropped it
and admits >=5.5.1, and v5 stops OpenCUA at startup. Upstream fix #47438
is still a draft.

No package versions move; the lock records only the declared requirements.
@ashtonchew
ashtonchew merged commit 3116b67 into main Aug 18, 2026
5 checks passed
@ashtonchew
ashtonchew deleted the deps/pin-cua-vllm-transformers branch August 18, 2026 06:22
ashtonchew added a commit that referenced this pull request Aug 18, 2026
#183)

pyproject.toml said 0.20.0-0.22.1 serve GUI-Owl with roughly 30% relative
accuracy loss on visual grounding. Two parts of that are wrong, and I
wrote it in #177.

Upstream (vllm#43617) measured Geo3K, a visual-math set, on Qwen3-VL-2B
and Qwen3-VL-8B. The 2B shows 29.71% relative and the 8B shows 4.85% on
the same compiled-vs-eager contrast, so the effect shrinks as the model
grows. GUI-Owl-1.5-32B-Think is larger again and upstream never measured
it. The comment now gives both numbers, names the benchmark, and says
upstream did not measure this model.

Also adds a dated errata to the research doc: its affected-tag table
omits v0.20.1, which does carry the fault, and which was the current
release on the day the [cua] extras landed. The table stays as measured.

The bound does not move. Only its stated reason changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant