fix(router): wildcard order fallback to higher-order deployments

[Sameerlite]

Summary

• fix order-based fallback to use wildcard-aware deployment lookup, so wildcard model groups (for example openai/*) can advance from order=1 to higher order levels
• preserve existing order fallback behavior for non-wildcard model groups
• add a regression test that reproduces wildcard order fallback and validates routing to order=2
  Fixes LIT-2344

Test plan

• poetry run pytest tests/test_litellm/test_router_order_fallback.py::test_router_order_fallback_with_wildcard_model_group -v
• poetry run pytest tests/test_litellm/test_router_order_fallback.py -v

model_list:
  - model_name: "openai/*"
    litellm_params:
      model: "openai/*"
      api_base: "https://primary-gateway/v1"
      api_key: os.environ/OPENAI_API_KEY
      order: 1
  - model_name: "openai/*"
    litellm_params:
      model: "openai/*"
      api_base: "https://api.openai.com/v1"
      api_key: os.environ/OPENAI_API_KEY
      order: 2

router_settings:
  num_retries: 0

Order 1: Fallback(order 2):

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 15, 2026 0:20am

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing litellm_wildcard_order_fallback (9ed90d5) with main (72a461b)}

[greptile-apps]

Greptile Summary

This PR fixes order-based fallback routing for wildcard model groups (e.g. openai/*) by replacing _get_all_deployments with get_model_list in the fallback handler. _get_all_deployments performs an exact index lookup and returns nothing for a specific model name like openai/gpt-4.1-mini when only openai/* is registered, causing the _order_set to be empty and order-level advancement to be silently skipped. get_model_list adds a pattern-router fallback that resolves the wildcard deployments and preserves their litellm_params.order values, enabling the existing order-fallback logic to build order=2 entries as expected. A regression test with mock responses validates the fix.

Confidence Score: 5/5

Safe to merge — targeted one-line fix with a clear regression test and no backward-incompatible changes.

The change is minimal and surgical: replacing _get_all_deployments with get_model_list (which itself calls _get_all_deployments first, then falls through to pattern routing). Exact-match model groups see identical behavior; wildcard groups gain the previously missing order-level advancement. The new test uses only mock responses and validates the end-to-end fix with a specific assertion. No P0/P1 findings.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/router.py	Replaces _get_all_deployments with get_model_list (wildcard-aware) in the order-based fallback path; existing behavior for exact model groups is preserved since get_model_list calls _get_all_deployments first.
tests/test_litellm/test_router_order_fallback.py	Adds a new asyncio test using mock_response (no real network calls) that registers two openai/* deployments at order=1 and order=2, triggers a failure on order=1, and asserts the response comes from the order=2 deployment.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[acompletion: model=openai/gpt-4.1-mini] --> B[Route to order=1 deployment]
    B --> C{Deployment fails?}
    C -- No --> D[Return response]
    C -- Yes --> E[Order-based fallback handler]
    E --> F["get_model_list(openai/gpt-4.1-mini)"]
    F --> G["_get_all_deployments → empty (no exact match)"]
    G --> H["pattern_router.route → finds openai/* deployments"]
    H --> I["_order_set = {1, 2}, order_values = [1, 2]"]
    I --> J["Build fallback: {model: openai/gpt-4.1-mini, _target_order: 2}"]
    J --> K["run_async_fallback with _target_order=2"]
    K --> L["Route to order=2 deployment"]
    L --> M[Return response]

    style G fill:#ffcccc
    style H fill:#ccffcc

Loading

_{Reviews (4): Last reviewed commit: "Merge branch 'litellm_internal_staging' ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!