[nrenvo]

Fixes #64075.

_encode_local_features in deepseek_ocr.py and deepseek_ocr2.py used torch.sum(patches).item() == 0 to detect "no crop patches", so a tiled image whose normalized pixels cancel to zero (e.g. half-white/half-black 1536x768) wrongly returned None, dropped local features, and crashed EngineCore with a placeholder/embedding shape mismatch. Replace the guard with patches.numel() == 0; #59417 fixed the equivalent check in _parse_and_validate_image_input but left this one as "not engine-fatal", which #64075 disproves.

New test class TestDeepseekOCREncodeLocalFeaturesGuard fails on the base commit (2 failed) and passes with the fix (13 passed in tests/models/multimodal/processing/test_deepseek_ocr.py); ruff check and ruff format are clean.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[noone-zhhh]

Thanks for working on this!
I had already investigated the root cause in #64075 and mentioned that I was planning to submit a PR to address it.
Since I've been working on a fix as well, would you be willing to close this PR and let me submit mine?
Appreciate for your understanding!

[nrenvo]

Hey, thanks for the note. I already have a fix up with regression tests for both DeepSeek-OCR and OCR-2, validated against the base commit, so I'd prefer to keep this PR open and let the maintainers decide. Happy to compare approaches if that helps.

[noone-zhhh]

Thanks, I understand you’d like to keep this PR open. The current unit tests cover the zero-sum guard, but they don’t exercise the end-to-end inference path reported in #64075. My commit adds a vllm_runner regression test for that path.

I ran the tests on an RTX 5070 Ti and observed:

Image	Before fix	After fix
Normal OCR photo	Passed	Passed
Constructed black-and-white image	EngineCore crashed	Passed

I’d be happy to contribute the end-to-end test if it would help strengthen this PR. As a minor style suggestion, the commit message is too long maybe.

[nrenvo]

Thanks, fair point — the unit tests cover the guard, but you're right that they don't exercise the full inference path. Happy to include your e2e test. Could you put the test on top of 91c7398 and open a cross-fork PR against nrenvo:fix/deepseek-ocr-zero-sum-patches with just the test file? I'll merge it and keep your authorship. If that's a hassle, paste the test file here and I'll apply it with a Co-authored-by. And thanks for the commit message note.

[noone-zhhh]

FYI @nrenvo

Details

import os

import pytest
import torch
from PIL import Image

from vllm.transformers_utils.processors.deepseek_ocr import (
    ImageTransform,
    dynamic_preprocess,
)


@pytest.mark.core_model
@pytest.mark.parametrize(
    ("model_id", "image_size", "model_env", "load_format"),
    [
        pytest.param(
            "deepseek-ai/DeepSeek-OCR",
            640,
            "DEEPSEEK_OCR_TEST_MODEL",
            "dummy",
            id="ocr1",
        ),
        pytest.param(
            "deepseek-ai/DeepSeek-OCR-2",
            768,
            "DEEPSEEK_OCR2_TEST_MODEL",
            "auto",
            id="ocr2",
        ),
    ],
)
def test_deepseek_ocr_balanced_tiled_image(
    vllm_runner, model_id, image_size, model_env, load_format
):
    image = Image.new("RGB", (1536, 768), "white")
    image.paste("black", (768, 0, 1536, 768))

    crops, crop_ratio = dynamic_preprocess(image, image_size=image_size)
    patches = torch.stack([ImageTransform()(crop) for crop in crops])
    assert crop_ratio == (2, 1)
    assert patches.numel() > 0
    assert patches.sum().item() == 0

    model = os.environ.get(model_env, model_id)
    with vllm_runner(
        model,
        max_model_len=2048,
        gpu_memory_utilization=0.8,
        load_format=load_format,
    ) as vllm_model:
        outputs = vllm_model.generate_greedy(
            ["<image>\nFree OCR."], max_tokens=8, images=[[image]]
        )

    assert len(outputs) == 1
    assert len(outputs[0][0]) > 0

[nrenvo]

Added your test as a separate commit with your authorship — thanks for putting it together.

[DarkLight1337]

Let's move this to generation directory

[nrenvo]

Done, moved TestDeepseekOCREncodeLocalFeaturesGuard to generation/test_deepseek_ocr_guard.py!

[DarkLight1337]

Let's save CI cost and only include the processing tests

[nrenvo]

Yep, dropped the vllm_runner e2e test — the guard tests cover the fix without the extra CI cost.

[DarkLight1337]

Thanks!

[DarkLight1337]

/ci run

[github-actions]

✅ Triggered Buildkite CI #97898 for commit eaad20df1ff9.

ℹ️ This PR is 78 commits behind upstream main. Merged upstream main into this branch as eaad20df1ff9 before starting CI. Pull the branch before pushing more changes.

[mergify]

Hi @nrenvo, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[DarkLight1337]

/ci run

[github-actions]

✅ Triggered Buildkite CI #97902 for commit 7a6f22c3e7f6.

ℹ️ This PR is 2 commits behind upstream main. Merged upstream main into this branch as 7a6f22c3e7f6 before starting CI. Pull the branch before pushing more changes.