Skip to content

fix: preserve provider identity when auxiliary task has base_url - #49545

Closed
zh-xl-kang wants to merge 1 commit into
NousResearch:mainfrom
zh-xl-kang:fix/vision-provider-base-url
Closed

fix: preserve provider identity when auxiliary task has base_url#49545
zh-xl-kang wants to merge 1 commit into
NousResearch:mainfrom
zh-xl-kang:fix/vision-provider-base-url

Conversation

@zh-xl-kang

@zh-xl-kang zh-xl-kang commented Jun 20, 2026

Copy link
Copy Markdown

What does this PR do?

_resolve_task_provider_model() forces provider to "custom" whenever base_url is set as a direct argument, even when the caller specified a known provider (e.g. alibaba, openrouter). This causes vision and other auxiliary calls to resolve credentials from the wrong env var (OPENAI_API_KEY instead of DASHSCOPE_API_KEY, etc.) and fail with 401 Unauthorized.

The fix preserves the caller's provider when it is a non-empty, non-auto value, so each provider resolves credentials from its own env vars.

Related Issue

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • agent/auxiliary_client.py_resolve_task_provider_model(): preserve a known (non-empty, non-auto) provider when base_url is set instead of collapsing it to "custom":

    if base_url:
        if provider and provider not in {"auto", ""}:
            return provider, resolved_model, base_url, api_key, resolved_api_mode
        return "custom", resolved_model, base_url, api_key, resolved_api_mode

    auto / empty / None / missing provider still collapse to "custom" (unchanged behavior).

  • tests/agent/test_vision_provider_base_url.py — 8 regression tests (see below).

How to Test

Reproduction via config.yaml:

auxiliary:
  vision:
    provider: alibaba
    base_url: https://dashscope.aliyuncs.com/compatible-mode/v1
  1. Before: vision calls resolve to the custom provider → read OPENAI_API_KEY401.
  2. After: vision calls resolve to the alibaba provider → read DASHSCOPE_API_KEY → works.

Automated: 8 regression tests cover:

  • Known provider preserved with base_url (alibaba, openrouter), via both direct arguments and config-driven (task="vision") resolution.
  • auto / empty / None / missing provider still collapses to "custom" (unchanged behavior); base_url alone (no provider arg) still returns "custom".
  • End-to-end credential resolution: the preserved provider string flows into resolve_api_key_provider_credentials, proving alibaba reads DASHSCOPE_API_KEY (not OPENAI_API_KEY).

All existing vision routing tests (12) continue to pass — 20 passed total.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform:

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/vision Vision analysis and image generation P2 Medium — degraded but workaround exists labels Jun 20, 2026
@zh-xl-kang
zh-xl-kang force-pushed the fix/vision-provider-base-url branch 3 times, most recently from 251bdee to 1e46652 Compare June 25, 2026 12:45
@zh-xl-kang
zh-xl-kang force-pushed the fix/vision-provider-base-url branch from 1e46652 to eac0b9f Compare July 1, 2026 07:47
When an auxiliary task (e.g. vision) has a base_url configured,
_resolve_task_provider_model was collapsing known providers to 'custom'
because the base_url triggered the 'unknown endpoint' heuristic. This
broke credential resolution — provider 'custom' looks for OPENAI_API_KEY
instead of the provider-specific key (e.g. alibaba → DASHSCOPE_API_KEY).

Fix: when the caller explicitly passes a known provider name, preserve
it regardless of base_url. The base_url only indicates a proxy/mirror,
not an unknown vendor.

- agent/auxiliary_client.py: keep provider string when it matches a
  known provider, even with custom base_url
- tests/agent/test_vision_provider_base_url.py: 135 + 86 lines of
  regression tests covering direct-argument, config-driven, and
  end-to-end credential resolution paths
@zh-xl-kang
zh-xl-kang force-pushed the fix/vision-provider-base-url branch from eac0b9f to 759b2a9 Compare July 2, 2026 07:00
@teknium1

teknium1 commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Closing as implemented on main: _preserve_provider_with_base_url() landed in commit a8841e2 (PR #55605, merged Jun 30) and covers this scenario — a first-class provider paired with a base_url keeps its identity instead of being flattened to custom. Verified E2E on current main with provider="alibaba" + base_url. Note @tino-chen's #39732 reported the same bug earliest. The merged version also gates on the provider catalog so unknown names still route to custom. Thanks for the contribution!

@teknium1 teknium1 closed this Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists tool/vision Vision analysis and image generation type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants