fix(sglang): support tensor image sizes for multimodal P/D

[Chokoyo]

Overview:

Fix Qwen3-VL multimodal requests falling back to decode and retokenize in SGLang prefill/decode deployments.

SGLang PR #26555 added an exact-token path for pre-tokenized multimodal requests. Its image-token resolver assumes every decoded image exposes PIL-style height and width attributes, while the nvJPEG path returns a CHW torch.Tensor.

This causes both workers to log:

'Tensor' object has no attribute 'height'
falling back to decode+retokenize

This PR adds a temporary Dynamo compatibility hook so the exact-token path handles Tensor images without falling back.

Details:

• Add the idempotent ensure_sglang_tensor_image_size() compatibility hook.
• Derive Tensor image dimensions from shape[-2:].
• Delegate to SGLang's original implementation when all images are non-Tensor inputs.
• Install the hook before creating the SGLang engine.
• Add regression coverage that directly exercises process_and_combine_mm_data.
• Verify that original non-media token IDs are preserved.
• Verify that no decode-and-retokenize fallback warning is emitted.

The compatibility hook can be removed once the minimum supported SGLang release handles Tensor image dimensions upstream.

Validation completed:

• SGLang compatibility unit tests passed.
• Dynamo SGLang same-GPU P/D E2E passed with Qwen3-VL-2B, NIXL, DYN_CHAT_PROCESSOR=sglang, and the reported JPEG input.
• E2E returned HTTP 200 with prompt_tokens=276.
• No .height, decode+retokenize, or token drift warnings were logged.

Where should the reviewer start?

Start with components/src/dynamo/sglang/_compat.py, which contains the compatibility hook and Tensor dimension handling.

Then review:

• components/src/dynamo/sglang/args.py for hook installation before engine creation.
• components/src/dynamo/sglang/tests/test_sglang_unit.py for direct no-fallback regression coverage.

Related Issues

• Fixes DYN-3390

Summary by CodeRabbit

• New Features

  • Improved multimodal image handling so tensor-based image inputs are supported more reliably.
  • Added automatic compatibility updates during startup to keep image token counting working across supported setups.

• Bug Fixes

  • Fixed issues where decoded image tensors could be misread during processing.
  • Reduced unnecessary fallback behavior and warning messages in certain image-processing flows.

• Tests

  • Added coverage for tensor image size support and repeat-safe compatibility updates.

[coderabbitai]

Walkthrough

Adds a new SGLang compatibility function, ensure_sglang_tensor_image_size, that monkey-patches BaseMultimodalProcessor.resolve_image_token_counts to support tensor-based image inputs. Wires this call into parse_args and adds a corresponding unit test verifying idempotent patching and correct token resolution.

Changes

Tensor Image Size Compatibility Patch

Layer / File(s)	Summary
Compat patch implementation and export components/src/dynamo/sglang/_compat.py	Adds wraps import and implements ensure_sglang_tensor_image_size, which conditionally wraps resolve_image_token_counts to derive (height, width) from tensor inputs and compute image token counts, tagging the wrapper to avoid re-wrapping; exports the function via __all__.
Wiring into args parsing components/src/dynamo/sglang/args.py	Expands _compat import to include the new function and invokes it in parse_args right after ServerArgs.from_cli_args, before later validation and streaming setup.
Unit test for tensor image size support components/src/dynamo/sglang/tests/test_sglang_unit.py	Adds torch and _compat import updates, and a new test confirming the patch updates resolve_image_token_counts, is idempotent across repeated calls, and produces expected token sequences without retokenize fallback warnings when SGLANG_MM_AVOID_RETOKENIZE=1.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Related PRs: None identified.

Suggested labels: sglang, compatibility, multimodal

Suggested reviewers: None identified.

Poem:
A rabbit peeked at tensors tall,
"Height and width," it heard them call.
It wrapped the old resolver tight,
So image tokens count just right.
Idempotent hops, no double patch—
This bunny's code is quite a catch! 🐇

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: adding tensor image size support for SGLang multimodal P/D.
Description check	✅ Passed	The description matches the template well and includes overview, details, reviewer guidance, and a linked issue.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 18.45% (-26.94%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 3214e96 | Docs | Give us feedback!}

[devin-ai-integration]

Devin Review found 1 potential issue.