Fix: Preserve anthropic_messages api_mode during fallback activation - #79787
Fix: Preserve anthropic_messages api_mode during fallback activation#79787sjungwon03 wants to merge 2 commits into
Conversation
Problem: When activating a fallback provider with api_mode='anthropic_messages', the URL was being rewritten from /apps/anthropic to /apps/v1, causing 404 errors. This happened because: 1. try_activate_fallback() called resolve_provider_client() without passing the api_mode parameter 2. resolve_provider_client() called _to_openai_base_url() which rewrites /anthropic to /v1 for OpenAI wire compatibility 3. The api_mode detection happened AFTER URL transformation, so the anthropic endpoint signal was lost Solution: - In try_activate_fallback(): pre-compute api_mode from the ORIGINAL fallback entry's base_url before calling resolve_provider_client() - Pass api_mode to resolve_provider_client() so it can skip URL transformation when api_mode='anthropic_messages' - In resolve_provider_client(): check api_mode before calling _to_openai_base_url() for custom providers - Skip redundant api_mode detection after URL transformation since it's already determined This ensures Anthropic-compatible endpoints (like Alibaba's token-plan) that use /anthropic path suffix work correctly in fallback chains.
|
This was generated by AI during triage. Summary: Problems:
Solution: Checked against |
Address triage review feedback: an entry with provider: anthropic and no explicit base_url uses the provider's default endpoint and must still resolve to anthropic_messages. The provider-name check was nested inside elif fb_base_url_hint:, so such entries fell through to chat_completions and took the OpenAI-swap path instead of the native Anthropic branch. Move the fb_provider == 'anthropic' check to its own elif branch, ahead of the base_url-suffix check, so it is evaluated regardless of whether base_url is present.
|
Thanks for the triage review! I've pushed a fix in commit 3830334 that moves the |
Fallback activation determined api_mode from the POST-rewrite client base_url, losing the Anthropic wire signal for /anthropic endpoints routed through provider 'custom', and never honored an explicit fb.api_mode config field. Pre-compute fb_api_mode from the ORIGINAL fallback base_url hint (before _to_openai_base_url rewriting), honor the explicit api_mode config field, check provider name before the base_url gate, and pass api_mode into resolve_provider_client at the fallback call site. Salvaged from PR #79787 (chat_completion_helpers.py hunks; the auxiliary_client.py hunk is redundant with #85466's wrap_base fix).
Maintainer fixup on the #79787 salvage: - An explicit fb.api_mode of "chat_completions" was silently overridden by the codex_responses / bedrock re-detection pass (which only skipped re-detection when the pre-computed mode was non-default). Track explicitness in fb_api_mode_explicit and gate the whole re-detection block on it. - Replace the locals().get('fb_api_mode') dead-code hack with clean code (fb_api_mode is always bound at that point). - Restore the post-resolve /anthropic + api.anthropic.com host check for named custom providers whose base_url comes from config rather than the fallback entry (#32243, #49247), which the PR's restructure dropped. - Add regression tests: explicit api_mode honored (incl. explicit chat_completions not overridden), /anthropic-hint fallback detected pre-rewrite, api_mode forwarded to resolve_provider_client, plain fallback unchanged.
|
Merged via PR #85576 with your commits cherry-picked onto current main — authorship preserved in git log, including your follow-up that fixed the provider-name gating. Two adjustments on top: an explicit |
Problem
When activating a fallback provider configured with
api_mode='anthropic_messages', the URL was being incorrectly rewritten from/apps/anthropicto/apps/v1, causing 404 errors.This occurred because:
try_activate_fallback()calledresolve_provider_client()without passing theapi_modeparameterresolve_provider_client()called_to_openai_base_url()which rewrites/anthropicto/v1for OpenAI wire compatibilityapi_modedetection happened AFTER URL transformation, so the anthropic endpoint signal was lostExample Scenario
A fallback provider configured with:
Would incorrectly send requests to
/apps/v1instead of/apps/anthropic, resulting in 404 errors.Solution
try_activate_fallback(): pre-computeapi_modefrom the ORIGINAL fallback entry'sbase_urlbefore callingresolve_provider_client()api_modetoresolve_provider_client()so it can skip URL transformation whenapi_mode='anthropic_messages'resolve_provider_client(): checkapi_modebefore calling_to_openai_base_url()for custom providersapi_modedetection after URL transformation since it's already determinedTesting
Verified that Anthropic-compatible endpoints (like Alibaba's token-plan with
/anthropicpath suffix) now work correctly in fallback chains.Impact
This fix ensures that custom providers using Anthropic Messages API format work properly when configured as fallback providers, which is essential for multi-provider resilience setups.