Skip to content

fix(auxiliary): bridge runtime credentials into vision auto-detect (sibling gap of #35259) - #43254

Closed
arxeme wants to merge 1 commit into
NousResearch:mainfrom
arxeme:fix/vision-aux-runtime-credentials
Closed

arxeme wants to merge 1 commit into
NousResearch:mainfrom
arxeme:fix/vision-aux-runtime-credentials

Conversation

@arxeme

@arxeme arxeme commented Jun 10, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes vision_analyze failing with No LLM provider configured for task=vision provider=auto for users whose main provider is a named custom provider (custom:<name>), even though normal chat and every other auxiliary task work.

This is the sibling gap of #35259. That PR taught set_runtime_main() to record base_url/api_key/api_mode and bridged them in _resolve_auto() — but the parallel vision path in resolve_vision_provider_client() was never updated, so vision alone still loses the credentials.

Related Issue

Fixes #43251

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • agent/auxiliary_client.py — in the resolve_vision_provider_client() auto branch, forward _RUNTIME_MAIN_BASE_URL / _RUNTIME_MAIN_API_KEY / _RUNTIME_MAIN_API_MODE into the resolve_provider_client() call as explicit_base_url / explicit_api_key / api_mode, mirroring the existing _resolve_auto() bridge.
  • tests/agent/test_set_runtime_main_custom_provider.py — add TestResolveVisionUsesRuntimeGlobals::test_vision_auto_forwards_runtime_credentials asserting the vision auto path forwards the runtime credentials.

Root Cause

For a custom:<name> provider, the agent resolver flattens agent.provider to bare "custom" and carries the endpoint credentials separately. Each turn turn_context.py calls set_runtime_main("custom", model, base_url=..., api_key=..., api_mode=...), so _RUNTIME_MAIN_PROVIDER == "custom" with the credentials in _RUNTIME_MAIN_BASE_URL / _RUNTIME_MAIN_API_KEY.

  • Non-vision aux → _resolve_auto() bridges those globals (added by fix(auxiliary): route custom-provider aux tasks to the user endpoint (salvage #34783) #35259) → works.
  • Vision → resolve_vision_provider_client(provider="auto") reads main_provider = "custom" and called resolve_provider_client("custom", ...) without explicit_base_url/explicit_api_key. The anonymous-custom branch then finds no credentials, logs custom/main requested but no endpoint credentials found, and the chain returns None.

How to Test

Reproduced on v0.16.0 with a custom:garena-gateway-anthropic main provider (api_mode: anthropic_messages, key via key_env):

from agent.auxiliary_client import resolve_vision_provider_client, set_runtime_main
set_runtime_main("custom", "claude-opus-4-7",
                 base_url="https://<gateway>/...",
                 api_key="<key>", api_mode="anthropic_messages")
prov, client, model = resolve_vision_provider_client(provider="auto")
# before: client is None  →  "No LLM provider configured for task=vision"
# after:  client is AnthropicAuxiliaryClient

Tests:

pytest tests/agent/test_set_runtime_main_custom_provider.py -q -o 'addopts='   # 8 passed
pytest tests/agent/test_auxiliary_client.py tests/agent/test_auxiliary_main_first.py \
       tests/agent/test_auxiliary_named_custom_providers.py tests/tools/test_vision_tools.py -q -o 'addopts='   # 335 passed

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(auxiliary): …)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix
  • I've run the relevant tests and they pass
  • I've added tests for my changes
  • I've tested on my platform: Linux (Python 3.11), v0.16.0

Documentation & Housekeeping

  • Documentation update — N/A (internal routing fix)
  • cli-config.yaml.example — N/A (no config keys changed)
  • CONTRIBUTING.md / AGENTS.md — N/A (no architecture change)
  • Cross-platform impact — N/A (pure Python, no platform primitives)
  • Tool descriptions/schemas — N/A (no tool behavior change)

- Forward the _RUNTIME_MAIN_BASE_URL/_RUNTIME_MAIN_API_KEY/_RUNTIME_MAIN_API_MODE runtime globals into resolve_provider_client from the vision auto branch, mirroring the _resolve_auto bridge from #35259, so a custom:<name> main provider flattened to bare "custom" still builds a vision client instead of returning "No LLM provider configured for task=vision"
- Add a regression test asserting the vision auto path forwards the runtime credentials as explicit_base_url/explicit_api_key/api_mode

Fixes #43251

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/vision Vision analysis and image generation P2 Medium — degraded but workaround exists labels Jun 10, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

✅ Looks Good

  • Regression fix with test coverage: The fix bridges runtime credentials (_RUNTIME_MAIN_BASE_URL, _RUNTIME_MAIN_API_KEY, _RUNTIME_MAIN_API_MODE) into the vision auto-detect path, mirroring _resolve_auto() behavior from PR #35259.
  • Clean test: TestResolveVisionUsesRuntimeGlobals covers the key assertion — that explicit_base_url, explicit_api_key, and api_mode are forwarded to resolve_provider_client when provider="auto" and the runtime main is a custom provider.
  • No security issues: Test uses "***" as a placeholder API key — not a real credential.
  • Well-scoped: Only 2 files changed (+50/-1), single concern.

ℹ️ Minor

  • The api_mode=resolved_api_mode or (_RUNTIME_MAIN_API_MODE or None) fallback chains two falsy-check operators — this is intentional and correct (preserves explicit None from the caller while falling back to runtime global).

Reviewed by Hermes Agent

arxeme added a commit to arxeme/hermes-seatalk that referenced this pull request Jun 11, 2026
…tials

- Package the vision auto-detect runtime-credentials bridge as a standalone patch under deploy/patches for manual application until upstream PR NousResearch/hermes-agent#43254 is merged
- Provide an idempotent apply.py with one-time backup and rollback path, plus the equivalent unified diff for git-apply workflows
- Document symptom, root cause, verification steps, and the removal condition in the patch README

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@teknium1

Copy link
Copy Markdown
Collaborator

Implemented on main; thank you for the precise diagnosis and focused regression test.

Automated hermes-sweeper review found:

  • 25aa626cb42cfec620083ea69fe12e8afe171ff7 (fix(vision): forward custom-endpoint credentials in vision auto-detect) implements this runtime-credential bridge.
  • agent/auxiliary_client.py:5373-5402 forwards _RUNTIME_MAIN_BASE_URL, _RUNTIME_MAIN_API_KEY, and _RUNTIME_MAIN_API_MODE for custom and custom:<name> vision auto-detect paths, and also preserves a configured-endpoint fallback.
  • tests/agent/test_auxiliary_main_first.py:577-690 covers bare custom, named custom, and no-runtime configured-endpoint cases.
  • The fixing commit shipped in v2026.7.7.

The linked issue #43251 was also closed with this same implemented-on-main finding.

@teknium1 teknium1 closed this Jul 14, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:implemented-on-main Sweeper: behavior already present on current main tool/vision Vision analysis and image generation type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: auxiliary vision auto-detect doesn't bridge _RUNTIME_MAIN_* credentials for custom: providers (sibling gap of #35259)

4 participants