feat(multimodal): ThreadedMicroBatcher + batcher-backed AsyncVisionEncoder (cross-request batching)

[furionw]

Why

The merged custom encoder path runs each request's image forward independently, underfilling the encoder GPU under concurrent load. Cross-request batching must preserve one actor thread for CUDA-affine work, request ordering, and bounded variable-resolution sizing. Cancellation must retire unclaimed items so abandoned input cannot poison a live request sharing a later batch. Backend authors also need discoverable setup and sizing guidance.

What Change

• Add supervised, thread-affine eager and cost-bounded micro-batching.
• Tombstone canceled requests before dispatch while preserving in-flight best-effort semantics.
• Document custom encoder setup, sizing, lifecycle, and limitations.

Test Plan

• 114 multimodal-utils tests and pre-commit; semantic agg_custom e2e returned 42.
• Stress: 1,000 cancellation races, 1,000 start races, and 12,800 concurrent submissions.

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 99.15%
• Overall Coverage: 38.30% (-9.81%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 0392783 | Docs | Give us feedback!}

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Walkthrough

Changes

The PR adds a dedicated ThreadedMicroBatcher, integrates AsyncVisionEncoder with cost-aware batching and actor-thread lifecycle callbacks, and expands tests for batching, failures, shutdown, startup races, and encoder behavior.

Async vision batching

Layer / File(s)	Summary
Batcher contract and lifecycle components/src/dynamo/vllm/multimodal_utils/threaded_micro_batcher.py	Adds lifecycle states, request/work tracking, validated submission, startup, and idempotent shutdown.
Batch collection and failure handling components/src/dynamo/vllm/multimodal_utils/threaded_micro_batcher.py	Adds eager queue draining, cost-bounded dispatch, result validation, request completion, and worker-crash handling.
AsyncVisionEncoder integration components/src/dynamo/vllm/multimodal_utils/async_vision_encoder.py, components/src/dynamo/vllm/handlers.py	Routes preprocessing and backend execution through the batcher while running build and close on its worker thread.
Batcher and encoder validation components/src/dynamo/vllm/tests/multimodal_utils/*	Adds batching and lifecycle tests and updates encoder tests for cost metadata, atomic preprocessing failures, and batcher teardown.

Estimated code review effort: 4 (Complex) | ~60 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR uses a custom Why/What/Test format and omits required Overview, Details, reviewer start, and Related Issues sections.	Rewrite the description to match the template with Overview, Details, Where should reviewer start?, and a completed Related Issues section.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly names the new micro-batcher and AsyncVisionEncoder routing change, matching the main PR change.

---

_{Comment @coderabbitai help to get the list of available commands.}

[indrajit96]

LGTM!
Non Blocking :
IIUC this PR improves GPU utilization for custom multimodal vision encoders by batching images across concurrent requests.
Can we have a readme or any form of docs to explain how to enable this or use it effectively under what conditions?

[devin-ai-integration]

Devin Review found 1 new potential issue.

[furionw]

addressed comments by changing code and added docs

[furionw]

/ok to test 0392783

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-019f644f-dfb3-7269-8df2-cd44001cf91e.docs.buildwithfern.com/dynamo/dev