mtmd: fix LFM2 image tiling threshold

[BlackFoil]

Overview

This PR fixes a discrepancy in the LFM2-VL tiling logic by aligning its tiling predicate with the Hugging Face Transformers reference implementation, addressing significant OCR degradation observed at specific image resolutions.

Details of Changes
Fixed Tiling Condition: The previous tiling logic for LFM2-VL was based on edge length (width/height), which differed from the Transformers reference implementation. This has been updated to use a pixel-area-based condition to align with the reference tiling criterion.

Mitigated Tiling-Induced OCR Degradation: The previous edge-based logic caused the processing path to diverge incorrectly for certain image resolutions (e.g., 1024x977). Ensuring the correct tiling path mitigates the severe degradation caused by the wrong path, and we observed substantial OCR performance recovery specifically for the tested 1024x977 case.

Added Regression Test: Added a regression test to verify the tiling decision behavior and prevent future regressions.

Additional information

Empirical Encoding Observations:
Under our specific validation setup (testing OCR on Japanese forms with the same model), we observed the following behavior:

For 1024x977: The old logic resulted in 1 encoding, whereas the new logic produces 5 encodings, leading to the substantial OCR recovery mentioned above.

For 1025x978: The behavior remained consistent at 5 encodings in both implementations.

Relation to Issue #17290:
This PR shares highly similar symptoms with Issue #17290 (severe output degradation around 1024px) where tiling is also discussed. However, the investigations in #17290 encompass multiple preprocessing differences (such as resize/padding, position embeddings, and antialiasing). Therefore, while related, this PR specifically corrects the area-based tiling predicate and may not encompass all the regressions discussed in #17290.

Testing Details:

Added a regression test (test-mtmd-image.cpp) that explicitly verifies the tiling decision across specific resolutions (e.g., 704x704, 736x736, 1024x977, 1056x384).

Confirmed that this test FAILS when applying the old edge-based logic to the predicate, and PASSES with the new area-based logic.

Executed the full local CI suite (ci/run.sh) and confirmed all tests passed.

Ran llama-perplexity and llama-bench (tested on Qwen3 0.6B Q4_0). Observed no regressions in this specific setup (PPL matched exactly at 12.1282, and benchmark throughput differences in tokens/sec were within measurement noise).

Requirements

I have read and agree with the contributing guidelines

AI usage disclosure: YES - I identified the bug and used an AI assistant as a copilot to investigate the root cause, analyze the logic, and help draft the code fix and regression test. I have reviewed and tested the changes myself, and I take responsibility for all submitted code.

[ngxson]

CC @tdakhran , seems like there was a fix similar to this that was merged recently. Could you confirm if this is still valid?

[tdakhran]

Thanks for the fix. We will evaluate it internally, and I'll report back.

[tdakhran]

tested, looks good

[tdakhran]

@ngxson, does this look good to merge, or do we need anything else?

[ngxson]

this PR adds a new test infra structure, I need to clean it up first