feat: added /v1/models/{model}/next-fallback endpoint

[murad-khafizov]

Type

🆕 New Feature

Changes

Summary

This PR implements a new REST API endpoint /v1/models/{model}/next-fallback that provides progressive fallback
discovery for LiteLLM models. Instead of returning all fallbacks at once, this endpoint returns only the immediate next fallback model in the chain, enabling clients to discover fallbacks step-by-step.

Key Features

• Progressive Discovery: Returns only the next immediate fallback model
• Fallback Type Support: Supports general, context_window, and content_policy fallback types
• Authentication & Authorization: Integrates with existing LiteLLM auth system
• Access Control: Respects user/team model permissions and filters inaccessible fallbacks
• Provider Prefix Handling: Works with model names like openai/gpt-4
  and bedrock/claude-3
• OpenAI-Compatible Response Format: Consistent with existing LiteLLM API patterns

API Behavior Example

For fallback configuration:

fallbacks:
  - claude-4-sonnet: [bedrock-claude-sonnet-4, google-claude-sonnet-4]

API calls return:
- GET /v1/models/claude-4-sonnet/next-fallback → bedrock-claude-sonnet-4
- GET /v1/models/bedrock-claude-sonnet-4/next-fallback →  google-claude-sonnet-4
- GET /v1/models/google-claude-sonnet-4/next-fallback → 404 Not Found

Response Format

Success (200 OK):
{
  "current_model": "claude-4-sonnet",
  "next_fallback": "bedrock-claude-sonnet-4",
  "fallback_type": "general",
  "object": "next_fallback"
}

No Fallback (404 Not Found):
{
  "error": {
    "message": "No fallback available for model: google-claude-sonnet-4",
    "type": "not_found",
    "code": "no_fallback_available"
  }
}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 21, 2025 10:04am

[CLAassistant]

All committers have signed the CLA.

[murad-khafizov]

Closing it in favor of #12811