Skip to content

Fix: Preserve anthropic_messages api_mode during fallback activation - #79787

Closed
sjungwon03 wants to merge 2 commits into
NousResearch:mainfrom
sjungwon03:fix/fallback-anthropic-url-rewrite
Closed

Fix: Preserve anthropic_messages api_mode during fallback activation#79787
sjungwon03 wants to merge 2 commits into
NousResearch:mainfrom
sjungwon03:fix/fallback-anthropic-url-rewrite

Conversation

@sjungwon03

Copy link
Copy Markdown
Contributor

Problem

When activating a fallback provider configured with api_mode='anthropic_messages', the URL was being incorrectly rewritten from /apps/anthropic to /apps/v1, causing 404 errors.

This occurred because:

  1. try_activate_fallback() called resolve_provider_client() without passing the api_mode parameter
  2. resolve_provider_client() called _to_openai_base_url() which rewrites /anthropic to /v1 for OpenAI wire compatibility
  3. The api_mode detection happened AFTER URL transformation, so the anthropic endpoint signal was lost

Example Scenario

A fallback provider configured with:

fallback_providers:
  - provider: custom
    model: qwen3.7-plus
    base_url: https://token-plan.ap-southeast-1.maas.aliyuncs.com/apps/anthropic
    api_mode: anthropic_messages
    api_key: ***

Would incorrectly send requests to /apps/v1 instead of /apps/anthropic, resulting in 404 errors.

Solution

  • In try_activate_fallback(): pre-compute api_mode from the ORIGINAL fallback entry's base_url before calling resolve_provider_client()
  • Pass api_mode to resolve_provider_client() so it can skip URL transformation when api_mode='anthropic_messages'
  • In resolve_provider_client(): check api_mode before calling _to_openai_base_url() for custom providers
  • Skip redundant api_mode detection after URL transformation since it's already determined

Testing

Verified that Anthropic-compatible endpoints (like Alibaba's token-plan with /anthropic path suffix) now work correctly in fallback chains.

Impact

This fix ensures that custom providers using Anthropic Messages API format work properly when configured as fallback providers, which is essential for multi-provider resilience setups.

Problem:
When activating a fallback provider with api_mode='anthropic_messages',
the URL was being rewritten from /apps/anthropic to /apps/v1, causing
404 errors. This happened because:

1. try_activate_fallback() called resolve_provider_client() without
   passing the api_mode parameter
2. resolve_provider_client() called _to_openai_base_url() which
   rewrites /anthropic to /v1 for OpenAI wire compatibility
3. The api_mode detection happened AFTER URL transformation, so the
   anthropic endpoint signal was lost

Solution:
- In try_activate_fallback(): pre-compute api_mode from the ORIGINAL
  fallback entry's base_url before calling resolve_provider_client()
- Pass api_mode to resolve_provider_client() so it can skip URL
  transformation when api_mode='anthropic_messages'
- In resolve_provider_client(): check api_mode before calling
  _to_openai_base_url() for custom providers
- Skip redundant api_mode detection after URL transformation since
  it's already determined

This ensures Anthropic-compatible endpoints (like Alibaba's token-plan)
that use /anthropic path suffix work correctly in fallback chains.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/anthropic Anthropic native Messages API area/config Config system, migrations, profiles labels Aug 6, 2026
@spfcraze

spfcraze commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Summary:
A fallback entry with provider: anthropic and no explicit base_url now falls through to chat_completions, because the only anthropic detection in the new code runs behind elif fb_base_url_hint:.

Problems:

  • The pre-computed fb_api_mode block (agent/chat_completion_helpers.py:1847-1858 at the PR head) runs the anthropic check only under elif fb_base_url_hint:, so an entry that names provider: anthropic without a base_url (the provider's default endpoint applies) keeps fb_api_mode = "chat_completions".
  • The branch this replaces matched anthropic by provider name and by the resolved client URL (fb_provider == "anthropic", a resolved URL ending /anthropic, or resolved host api.anthropic.com — origin/main agent/chat_completion_helpers.py:1885-1896); the new post-detection chain (PR head :1895) re-detects openai-codex, nous, azure, direct-openai, responses-api and bedrock modes.
  • For such an entry, resolve_provider_client's provider == "anthropic" branch (agent/auxiliary_client.py:6211) still returns a client for the default endpoint, and with fb_api_mode = "chat_completions" the fallback takes the OpenAI-swap path (agent.client = fb_client, origin/main agent/chat_completion_helpers.py:1992) instead of the native-Anthropic branch at :1985.

Solution:
Evaluate the provider-name and resolved-host checks outside the elif fb_base_url_hint: gate (or keep an anthropic arm in the post-detection chain), so entries that rely on the provider's default endpoint still resolve to anthropic_messages.


Checked against b5e2496 — the tip of fix/fallback-anthropic-url-rewrite when this was written — and 01a1037, main at the same moment.

Address triage review feedback: an entry with provider: anthropic and
no explicit base_url uses the provider's default endpoint and must
still resolve to anthropic_messages. The provider-name check was
nested inside elif fb_base_url_hint:, so such entries fell through to
chat_completions and took the OpenAI-swap path instead of the native
Anthropic branch.

Move the fb_provider == 'anthropic' check to its own elif branch,
ahead of the base_url-suffix check, so it is evaluated regardless
of whether base_url is present.
@sjungwon03

Copy link
Copy Markdown
Contributor Author

Thanks for the triage review! I've pushed a fix in commit 3830334 that moves the fb_provider == "anthropic" check to its own elif branch, ahead of the fb_base_url_hint gate. This ensures entries with provider: anthropic and no explicit base_url still resolve to anthropic_messages instead of falling through to chat_completions.

teknium1 pushed a commit that referenced this pull request Aug 13, 2026
Fallback activation determined api_mode from the POST-rewrite client
base_url, losing the Anthropic wire signal for /anthropic endpoints
routed through provider 'custom', and never honored an explicit
fb.api_mode config field. Pre-compute fb_api_mode from the ORIGINAL
fallback base_url hint (before _to_openai_base_url rewriting), honor
the explicit api_mode config field, check provider name before the
base_url gate, and pass api_mode into resolve_provider_client at the
fallback call site.

Salvaged from PR #79787 (chat_completion_helpers.py hunks; the
auxiliary_client.py hunk is redundant with #85466's wrap_base fix).
teknium1 added a commit that referenced this pull request Aug 13, 2026
Maintainer fixup on the #79787 salvage:

- An explicit fb.api_mode of "chat_completions" was silently overridden
  by the codex_responses / bedrock re-detection pass (which only skipped
  re-detection when the pre-computed mode was non-default). Track
  explicitness in fb_api_mode_explicit and gate the whole re-detection
  block on it.
- Replace the locals().get('fb_api_mode') dead-code hack with clean code
  (fb_api_mode is always bound at that point).
- Restore the post-resolve /anthropic + api.anthropic.com host check for
  named custom providers whose base_url comes from config rather than
  the fallback entry (#32243, #49247), which the PR's restructure dropped.
- Add regression tests: explicit api_mode honored (incl. explicit
  chat_completions not overridden), /anthropic-hint fallback detected
  pre-rewrite, api_mode forwarded to resolve_provider_client, plain
  fallback unchanged.
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #85576 with your commits cherry-picked onto current main — authorship preserved in git log, including your follow-up that fixed the provider-name gating. Two adjustments on top: an explicit fb.api_mode now always wins (your version let codex_responses/bedrock re-detection override an explicit chat_completions), and the auxiliary_client hunk was dropped as redundant with the #85466 fix that landed earlier. All three behavioral pieces — original-hint detection, explicit config honor, and api_mode pass-through to the resolver — are yours and are on main. Thanks for the honest sibling-PR enumeration in your PR body; it made cluster triage much easier.

@teknium1 teknium1 closed this Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/anthropic Anthropic native Messages API type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants