Skip to content

[CI] Restore MiniCPMV transformers cap, scoped to HF runner only - #45042

Merged
DarkLight1337 merged 1 commit into
vllm-project:mainfrom
hmellor:enable-minicpmv-ci
Aug 11, 2026
Merged

DarkLight1337 merged 1 commit into
vllm-project:mainfrom
hmellor:enable-minicpmv-ci

Conversation

@hmellor

@hmellor hmellor commented Jun 9, 2026

Copy link
Copy Markdown
Member

Purpose

Restores the max_transformers_version cap on the MiniCPMV test registry entry, scoped to the HF runner only (transformers_version_reason={"hf": ...}).

This PR previously flipped an existing "vllm" reason to "hf". Since then #48413 removed the cap outright, so it has been rewritten: the cap is re-added, with the correct reason.

Why the cap is needed again

Nightly build 83094 (Multi-Modal Models (Extended Generation 3), an optional: true step that only runs in the scheduled full CI run) fails 9 tests, all with:

AttributeError: 'MiniCPMV' object has no attribute 'all_tied_weights_keys'
  • test_single_image_models[minicpmv_25-test_case100..102]
  • test_single_image_models[minicpmv_26-test_case103..105]
  • test_multi_image_models[minicpmv_26-test_case77..79]

The failure is entirely on the HF reference side. The traceback ends in transformers/modeling_utils.py:

for key in missing_keys - self.all_tied_weights_keys.keys()

all_tied_weights_keys is set by PreTrainedModel.post_init(), and MiniCPMV's remote code never calls it: post_init appears zero times in modeling_minicpmv.py for MiniCPM-Llama3-V-2_5, MiniCPM-V-4 and MiniCPM-V-4_5 (2_6 is gated on the Hub now, but fails identically). Same failure mode as SkyworkR1VChatModel (#42104) and the sibling MiniCPMO entry, both already capped.

Why it surfaced now

The MiniCPMV entry used to carry max_transformers_version="4.57" with a "vllm" reason. Because vlm_utils/core.py::run_test calls check_transformers_version(on_fail="skip") with the default check_max_version=True, that cap gated the HF-runner tests too. #48413 fixed the vLLM-side MiniCPMVBatchFeature problem and removed the cap entirely, which correctly re-enabled vLLM-only coverage but also un-gated the HF comparisons and exposed this separate, still-unfixed HF-side break.

Using an "hf"-scoped reason keeps both halves right: HF-runner comparisons skip, vLLM-only tests (including the ones #48413 added in tests/models/multimodal/processing/test_minicpmv.py) keep running, because those callers pass check_max_version=False, check_version_reason="vllm".

Tracks under #38379.

Not a duplicate

This is the pre-existing PR for this fix, rewritten rather than superseded by a new one. Duplicate checks run:

gh pr list --repo vllm-project/vllm --state open --search "minicpmv"
gh pr list --repo vllm-project/vllm --state open --search "all_tied_weights_keys in:body"
gh pr list --repo vllm-project/vllm --state open --search "48413 in:body"

No other open PR touches the MiniCPMV registry entry. #43760 and #42785 are runtime/CUDA-graph changes to the model itself.

Testing

pytest tests/models/test_registry.py -q
# 368 passed, 2 failed, 16 skipped

The two failures (test_registry_imports[HCXVisionForCausalLM], test_registry_imports[KananaVForConditionalGeneration]) are pre-existing and unrelated: confirmed by re-running them on a stashed tree.

Behaviour of the gate itself, against transformers==5.16.0.dev0:

for m in ["openbmb/MiniCPM-Llama3-V-2_5", "openbmb/MiniCPM-V-2_6",
          "openbmb/MiniCPM-V-4", "openbmb/MiniCPM-V-4_5"]:
    info = HF_EXAMPLE_MODELS.find_hf_info(m)
    info.check_transformers_version(on_fail="return")                    # -> skip message
    info.check_transformers_version(on_fail="return",
                                   check_max_version=False,
                                   check_version_reason="vllm")          # -> None

All four variants skip HF-runner tests and none skip vLLM-only tests, which is the intended split.

No model evaluation results: this is a test-gating change only, with no effect on model output, accuracy or serving.

AI assistance

AI assistance was used for this change (investigating the Buildkite failure, bisecting the cause to #48413, and drafting the patch and this description). I have reviewed every changed line and run the tests reported above.

@DarkLight1337
DarkLight1337 enabled auto-merge (squash) June 9, 2026 16:34
@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 9, 2026
@mergify

mergify Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Aug 6, 2026
vllm-project#48413 fixed the vLLM-side `MiniCPMVBatchFeature` incompatibility and
dropped the `max_transformers_version` cap on `MiniCPMV` entirely. That
also un-gated the tests that build an HF reference model, exposing a
separate HF-side break: MiniCPMV's remote code never calls
`self.post_init()`, so `all_tied_weights_keys` is never set and
Transformers v5 raises in `_move_missing_keys_from_meta_to_device`.

Restore the cap with an `hf`-scoped reason so HF-runner comparisons skip
while vLLM-only coverage keeps running.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@hmellor
hmellor force-pushed the enable-minicpmv-ci branch from 7cb6fe1 to 7f92af5 Compare August 11, 2026 15:06
@hmellor hmellor changed the title Enable MiniCPMV for vLLM in CI [CI] Restore MiniCPMV transformers cap, scoped to HF runner only Aug 11, 2026
@mergify mergify Bot removed the needs-rebase label Aug 11, 2026
@hmellor

hmellor commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

/ci run

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #83373 for commit 7f92af551114.

@AndreasKaratzas AndreasKaratzas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DarkLight1337
DarkLight1337 merged commit 9cc347a into vllm-project:main Aug 11, 2026
30 checks passed
@hmellor
hmellor deleted the enable-minicpmv-ci branch August 11, 2026 17:15
zyp2014 pushed a commit to zyp2014/vllm that referenced this pull request Aug 21, 2026
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants