Skip to content

fix(image_routing): check main model vision support before aux override - #44305

Closed
blut-agent wants to merge 1 commit into
NousResearch:mainfrom
blut-agent:fix-image-routing-20260611
Closed

fix(image_routing): check main model vision support before aux override#44305
blut-agent wants to merge 1 commit into
NousResearch:mainfrom
blut-agent:fix-image-routing-20260611

Conversation

@blut-agent

Copy link
Copy Markdown
Contributor

Bug

When auxiliary.vision is explicitly configured, images were forced through the text pipeline even when the main model supports vision (e.g. xiaomi/mimo-v2.5).

Root Cause

In agent/image_routing.py, decide_image_input_mode() checked _explicit_aux_vision_override() before checking main model vision support, unconditionally blocking the native path.

Fix

Reorder checks: main model vision support is checked first, aux override becomes a fallback for non-vision models only.

Changes

  • agent/image_routing.py: reordered checks in decide_image_input_mode(), updated module docstring
  • tests/agent/test_image_routing.py: split test into two — verifies native when main model supports vision (with aux configured), and text when main model is non-vision (with aux configured)

Tests

  • tests/agent/test_image_routing.py: 76 passed
  • tests/agent/test_custom_providers_vision.py: 13 passed

When the main model supports vision (supports_vision=True), images now
go through the native path even if auxiliary.vision is explicitly
configured. The aux override is now a fallback for non-vision models
only, not a blanket override that blocks native vision.

This fixes the bug where providers like Xiaomi MiMo (which has
supports_vision=True) were incorrectly routed to text-only mode when
auxiliary.vision was configured.

Also updates the module docstring to reflect the new priority order.
@liuhao1024

Copy link
Copy Markdown
Contributor

✅ Verified — vision routing priority reorder

Reviewed the diff for decide_image_input_mode priority logic in agent/image_routing.py.

  • Priority inversion is correct: The main model's supports_vision check now runs before _explicit_aux_vision_override. Previously, an explicit aux backend would override even a vision-capable main model — forcing unnecessary round-trips through the text pipeline. The new order (native if main sees → text with aux if configured → text fallback) matches the intended "main model first, aux as fallback" semantics.
  • Test coverage is thorough: test_explicit_aux_vision_override_only_applies_when_main_model_not_vision verifies the new priority (native wins over aux), and test_aux_vision_override_wins_when_main_model_not_vision confirms aux still activates for non-vision models. Both paths are covered.
  • Docstring updated: The module docstring correctly reflects the new priority order.

No issues found. The reorder is a clean semantic improvement.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/vision Vision analysis and image generation provider/xiaomi Xiaomi MiLM duplicate This issue or pull request already exists labels Jun 11, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #32519 (and the earlier #29138 lineage) — same fix in agent/image_routing.py: check main-model vision support before the auxiliary.vision override in decide_image_input_mode() so aux acts as a true fallback only for non-vision models. Also implements the bug half of #44299.

@blut-agent

Copy link
Copy Markdown
Contributor Author

Closing — duplicate of earlier PR #45651 or #33699/#32519. See the canonical PR for tracking.

@blut-agent blut-agent closed this Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists provider/xiaomi Xiaomi MiLM tool/vision Vision analysis and image generation type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants