Skip to content

fix(vision): route image analysis through active thread model - #27015

Closed
chromalinx wants to merge 2 commits into
NousResearch:mainfrom
chromalinx:fix/vision-thread-model-routing
Closed

fix(vision): route image analysis through active thread model#27015
chromalinx wants to merge 2 commits into
NousResearch:mainfrom
chromalinx:fix/vision-thread-model-routing

Conversation

@chromalinx

Copy link
Copy Markdown
Contributor

Pass the active provider/model through tool dispatch so vision_analyze can follow the current conversation backend instead of forcing the auxiliary vision task defaults. This keeps gateway thread model switches consistent for image analysis while preserving explicit model/provider overrides.

What does this PR do?

Related Issue

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

How to Test

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform:

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

  • This skill is broadly useful to most users (if bundled) — see Contributing Guide
  • SKILL.md follows the standard format (frontmatter, trigger conditions, steps, pitfalls)
  • No external dependencies that aren't already available (prefer stdlib, curl, existing Hermes tools)
  • I've tested the skill end-to-end: hermes --toolsets skills -q "Use the X skill to do Y"

Screenshots / Logs

Pass the active provider/model through tool dispatch so vision_analyze can follow the current conversation backend instead of forcing the auxiliary vision task defaults. This keeps gateway thread model switches consistent for image analysis while preserving explicit model/provider overrides.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/tools Tool registry, model_tools, toolsets tool/vision Vision analysis and image generation labels May 16, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related: #24070 (computer_use aux vision routing fix), #24875 (respect auxiliary vision routing), #25900 (vision tool analysis not used in model responses). This PR takes the approach of threading provider/model through tool dispatch via contextvars rather than fixing per-tool routing individually.

@cardtest15-coder

This comment was marked as spam.

- Remove dead contextvars code (_thread_provider, _thread_model)
  The variables were declared but never .set() by any caller, making the
  entire mechanism a no-op.  Clean up the import, declarations, and the
  dead fallback branch inside async_call_llm.

- Restore task='vision' for per-task config resolution
  Without it _resolve_task_provider_model() and _get_task_extra_body()
  miss vision-specific overrides (temperature, extra_body) from config.yaml.

- Extract _tool_routing_kwargs() helper in AIAgent
  Replaces three identical provider=self.provider or None,
  model=self.model or None blocks in run_agent.py with a single source
  of truth.

- Add tests for provider/model pass-through and fallback
  tests/tools/test_vision_thread_routing.py covers:
  • handle_function_call forwards provider/model
  • _handle_vision_analyze extracts them from **kw
  • vision_analyze_tool includes task='vision' in call_kwargs
  • Fallback via AUXILIARY_VISION_MODEL env var
  • _resolve_task_provider_model prefers explicit args over task config
@chromalinx

Copy link
Copy Markdown
Contributor Author

Closing in favor of a cleaner, atomic PR. The new PR will contain only the vision thread-model routing fix.

@chromalinx chromalinx closed this Jun 1, 2026
@chromalinx
chromalinx deleted the fix/vision-thread-model-routing branch July 19, 2026 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/tools Tool registry, model_tools, toolsets P2 Medium — degraded but workaround exists tool/vision Vision analysis and image generation type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants