fix(proxy): surface DB deployments dropped at load instead of a WARNING log

[devin-ai-integration]

TLDR

Problem this solves:

• DB-stored auto-routers over the license cap vanish on load with only a WARNING log
• Callers get the same "Invalid model name" 400 a typo gets
• Which router wins the single slot varied per worker (unordered DB read)

How it solves it:

• Router records why a deployment was dropped (dropped_deployments)
• /model/info shows them to proxy admins with the reason
• Callers of a dropped model get "configured but failed to load" instead
• Drop is logged at ERROR; DB rows load in created_at, model_id order

User Flow

Before: an admin upgrades a proxy with two DB-stored complexity routers using custom tier_definitions, no license, and one of them silently disappears

1. They restart the proxy on the new version with STORE_MODEL_IN_DB=True and no config change
2. A developer sends POST https://litellm-domain/v1/chat/completions with "model": "complexity-two"
3. They get 400 Invalid model name passed in model=complexity-two. Call /v1/models to view available models for your key.
4. The admin calls GET https://litellm-domain/model/info and the router is just missing from data, with no hint it was ever configured
5. The only signal is a WARNING line in container logs saying the deployment was dropped

After: the same restart still enforces the cap, but the admin and the caller can both see why the model is unavailable

1. They restart the proxy on the new version with STORE_MODEL_IN_DB=True and no config change
2. A developer sends POST https://litellm-domain/v1/chat/completions with "model": "complexity-two"
3. They get 400 model=complexity-two is configured on the proxy but failed to load. Ask your proxy admin to check /model/info for the reason.
4. The admin calls GET https://litellm-domain/model/info and sees a dropped_deployments list with the model name, id and the full cap error text
5. Container logs carry the same message at ERROR level

Relevant issues

Affected release

regression in v1.101.0 (the cap landed in #39468 and #39674; existing DB rows over it were dropped silently)

Linear ticket

Resolves LIT-7812

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Setup: proxy started with store_model_in_db: true, master_key: sk-1234, no LITELLM_LICENSE. complexity-one (complexity router with custom tier_definitions) was created through /model/new. complexity-two (same shape) was inserted straight into LiteLLM_ProxyModelTable to simulate a row that existed before the upgrade.

Before (d392928)

Admin model info

1. curl -s http://localhost:4000/model/info -H 'Authorization: Bearer sk-1234' | jq '{models: [.data[].model_name], dropped_deployments}'
2. Output:

   {"models": ["repro-base", "complexity-one"], "dropped_deployments": null}

Request to the dropped model

1. curl -s -i http://localhost:4000/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"complexity-two","messages":[{"role":"user","content":"hi"}]}'
2. Output:

   HTTP/1.1 400 Bad Request
   {"error":{"message":"/chat/completions: Invalid model name passed in model=complexity-two. Call `/v1/models` to view available models for your key.","type":"invalid_request_error","code":"400"}}
   

Proxy log

1. grep -m1 "Error upserting deployment complexity-two" repro_proxy.log
2. Output:

   18:49:17 - LiteLLM Router:WARNING: router.py:9890 - Error upserting deployment complexity-two (id=complexity-two-id): At most 1 auto-router(s) with operator-defined tier_definitions or an operator-written classifier prompt can be registered but this would make 2. Use the shipped tiers and classifier prompt for this router or remove an existing router with tier_definitions or its own classifier prompt.. Dropping it and continuing with other deployments.
   

After (4a6b7d0)

Admin model info

1. curl -s http://localhost:4000/model/info -H 'Authorization: Bearer sk-1234' | jq '{models: [.data[].model_name], dropped_deployments}'
2. Output:

   {
     "models": ["repro-base", "complexity-one"],
     "dropped_deployments": [
       {
         "model_name": "complexity-two",
         "model_id": "complexity-two-id",
         "error": "At most 1 auto-router(s) with operator-defined tier_definitions or an operator-written classifier prompt can be registered but this would make 2. Use the shipped tiers and classifier prompt for this router or remove an existing router with tier_definitions or its own classifier prompt."
       }
     ]
   }

Request to the dropped model

1. curl -s -i http://localhost:4000/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"complexity-two","messages":[{"role":"user","content":"hi"}]}'
2. Output:

   HTTP/1.1 400 Bad Request
   {"error":{"message":"/chat/completions: model=complexity-two is configured on the proxy but failed to load. Ask your proxy admin to check `/model/info` for the reason.","type":"invalid_request_error","param":null,"code":"400"}}
   

Proxy log

1. grep -m1 "Error upserting deployment complexity-two" repro_proxy_fixed2.log
2. Output:

   20:18:46 - LiteLLM Router:ERROR: router.py:9910 - Error upserting deployment complexity-two (id=complexity-two-id): At most 1 auto-router(s) with operator-defined tier_definitions or an operator-written classifier prompt can be registered but this would make 2. Use the shipped tiers and classifier prompt for this router or remove an existing router with tier_definitions or its own classifier prompt.. Dropping it and continuing with other deployments.
   

Type

🐛 Bug Fix

Caveats (if any)

Medium

• The cap is still enforced; this PR only makes the drop visible. Operators over the cap still need the auto_router license feature or to remove a router
• dropped_deployments lives in memory per worker, so it reflects that worker's last DB reload

Low

• Sorting DB rows by created_at changes which router wins the slot on proxies that happened to load the newer one first
• /model/info by litellm_model_id for a dropped id still returns the plain not-found response

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/acb50415376344159a4ad4d8f56deda2
Open in Devin Desktop: https://app.devin.ai/desktop/session/acb50415376344159a4ad4d8f56deda2?variant=devin

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_surface_dropped_db_deployments (4a6b7d0) with main (d07b2e8)}

[codecov]

Codecov Report

❌ Patch coverage is 97.43590% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/router.py	95.65%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR records deployments rejected during router loading, exposes those failures to proxy administrators, improves missing-model errors, and makes DB loading order deterministic

• Adds immutable dropped-deployment state to the router
• Includes dropped deployment reasons in administrator /model/info responses
• Distinguishes dropped models from unknown model names on request paths
• Sorts database rows by creation time and model ID before loading
• Adds focused routing and proxy response tests

Confidence Score: 3/5

The PR is not safe to merge until DB loading remains compatible with existing callers and removed dropped deployments are reconciled

The new sort raises before loading rows that omit created_at, while dropped-only records survive DB deletion and can indefinitely suppress registry recovery

Files Needing Attention: litellm/proxy/proxy_server.py, litellm/router.py, litellm/proxy/route_llm_request.py

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	Sorts DB deployments and exposes dropped records, but breaks row-like callers lacking created_at and returns unmasked exception text
litellm/proxy/route_llm_request.py	Adds dropped-model errors and suppresses read-through, which makes stale dropped state affect recovery
litellm/router.py	Tracks dropped deployments but does not reconcile dropped-only IDs when their persisted rows disappear
litellm/types/router.py	Adds a small immutable value type for dropped deployment details
tests/test_litellm/proxy/test_proxy_server.py	Covers administrator visibility and deterministic ordering but does not cover existing row-like fixtures or stale cleanup
tests/test_litellm/proxy/test_route_llm_request.py	Covers dropped versus unknown request errors but not stale-state read-through recovery
tests/test_litellm/router_strategy/test_complexity_router.py	Verifies failed complexity-router registration and successful same-ID cleanup

_{Reviews (1): Last reviewed commit: "fix(proxy): keep live model groups routa..." | Re-trigger Greptile}

[greptile-apps]

Sorting breaks row-like callers

Sorting reads created_at before processing, so existing row-like callers without that attribute raise immediately and no deployments are loaded

Knowledge Base Used: Proxy configuration and deployment

[greptile-apps]

Dropped records outlive deletion

DB reconciliation only deletes live router IDs, so dropped-only records persist and make removed names appear configured while disabling registry read-through

Knowledge Base Used:

• Routing and resilience
• Proxy gateway request plane

[greptile-apps]

Exception text bypasses masking

Raw exception strings bypass deployment credential masking and reach view-only administrators, risking exposure of sensitive invalid configuration

Knowledge Base Used:

• Proxy gateway request plane
• Proxy configuration and deployment