Skip to content

[CI] Fix PaddleOCR-VL HF test failure due to create_causal_mask API rename#37328

Merged
hmellor merged 1 commit intovllm-project:mainfrom
ROCm:akaratza_fix_multimod
Mar 18, 2026
Merged

[CI] Fix PaddleOCR-VL HF test failure due to create_causal_mask API rename#37328
hmellor merged 1 commit intovllm-project:mainfrom
ROCm:akaratza_fix_multimod

Conversation

@AndreasKaratzas
Copy link
Copy Markdown
Collaborator

@AndreasKaratzas AndreasKaratzas commented Mar 17, 2026

  • The PaddleOCR-VL HF model calls create_causal_mask(inputs_embeds=...) but transformers renamed the parameter to input_embeds (singular), causing a TypeError in the HF runner
  • Adds paddleocr_vl_patch_hf_runner that monkey-patches create_causal_mask in the model's module to accept the old parameter name

Test plan

  • pytest -s -v tests/models/multimodal/generation/test_common.py::test_single_image_models[paddleocr_vl-test_case34]

cc @kenroche

…ename

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@AndreasKaratzas AndreasKaratzas marked this pull request as ready for review March 17, 2026 17:35
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request fixes a TypeError in the PaddleOCR-VL HF test by monkey-patching the create_causal_mask function to support a renamed parameter. The fix is correct, but the implementation of the monkey-patch has a potential issue with test isolation as it modifies a global module without reverting the change. I've added a comment with a suggestion for a more robust implementation to prevent side effects on other tests.

Comment on lines +1152 to +1174
def paddleocr_vl_patch_hf_runner(hf_model: HfRunner) -> HfRunner:
"""Patches the HfRunner to fix create_causal_mask API mismatch.

The PaddleOCR-VL HF model passes `inputs_embeds` to create_causal_mask,
but transformers renamed this parameter to `input_embeds`.
"""
import sys

model_module = sys.modules.get(type(hf_model.model.model).__module__)
if model_module is None:
return hf_model

original_create_causal_mask = getattr(model_module, "create_causal_mask", None)
if original_create_causal_mask is None:
return hf_model

def patched_create_causal_mask(*args, **kwargs):
if "inputs_embeds" in kwargs:
kwargs["input_embeds"] = kwargs.pop("inputs_embeds")
return original_create_causal_mask(*args, **kwargs)

model_module.create_causal_mask = patched_create_causal_mask # type: ignore[attr-defined]
return hf_model
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This monkey-patch modifies a module in sys.modules, which is a global state. This can break test isolation and cause side effects in other tests running in the same process. For instance, a different test might rely on the original behavior of create_causal_mask and would fail in an unexpected way.

To ensure test reliability, this patch should be reverted after the test finishes. The standard way to achieve this is by using a context manager or a pytest fixture that handles the setup and teardown of the patch.

A possible approach would be to refactor this into a context manager that can be used within the test execution logic:

from contextlib import contextmanager

@contextmanager
def patched_create_causal_mask_for_paddleocr_vl(hf_model: HfRunner):
    import sys
    model_module = sys.modules.get(type(hf_model.model.model).__module__)
    
    original_create_causal_mask = getattr(model_module, "create_causal_mask", None)
    if original_create_causal_mask is None:
        yield
        return

    def patched_create_causal_mask(*args, **kwargs):
        if "inputs_embeds" in kwargs:
            kwargs["input_embeds"] = kwargs.pop("inputs_embeds")
        return original_create_causal_mask(*args, **kwargs)

    model_module.create_causal_mask = patched_create_causal_mask
    try:
        yield
    finally:
        model_module.create_causal_mask = original_create_causal_mask

This would require changes in the test runner to use the context manager, but it would make the test suite more robust.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Global state is technically valid but not practically relevant here. The module itself is test-specific and only renames inputs_embeds.

@DarkLight1337 DarkLight1337 requested a review from hmellor March 17, 2026 17:37
@mergify mergify bot added the multi-modality Related to multi-modality (#4194) label Mar 17, 2026
@hmellor hmellor enabled auto-merge (squash) March 18, 2026 08:27
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 18, 2026
@hmellor hmellor merged commit eaf7c9b into vllm-project:main Mar 18, 2026
23 checks passed
wendyliu235 pushed a commit to wendyliu235/vllm-public that referenced this pull request Mar 18, 2026
…ename (vllm-project#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@AndreasKaratzas AndreasKaratzas deleted the akaratza_fix_multimod branch March 18, 2026 17:49
khluu pushed a commit that referenced this pull request Mar 19, 2026
…ename (#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
(cherry picked from commit eaf7c9b)
fxdawnn pushed a commit to fxdawnn/vllm that referenced this pull request Mar 19, 2026
…ename (vllm-project#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
maoxx241 pushed a commit to maoxx241/vllm that referenced this pull request Mar 24, 2026
…ename (vllm-project#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
(cherry picked from commit eaf7c9b)
SouthWest7 pushed a commit to SouthWest7/vllm that referenced this pull request Mar 27, 2026
…ename (vllm-project#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
khairulkabir1661 pushed a commit to khairulkabir1661/vllm that referenced this pull request Mar 27, 2026
…ename (vllm-project#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Monishver11 pushed a commit to Monishver11/vllm that referenced this pull request Mar 27, 2026
…ename (vllm-project#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
JiantaoXu pushed a commit to JiantaoXu/vllm that referenced this pull request Mar 28, 2026
…ename (vllm-project#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
vrdn-23 pushed a commit to vrdn-23/vllm that referenced this pull request Mar 30, 2026
…ename (vllm-project#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
EricccYang pushed a commit to EricccYang/vllm that referenced this pull request Apr 1, 2026
…ename (vllm-project#37328)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: EricccYang <yangyang4991@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

multi-modality Related to multi-modality (#4194) ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants