fix(proxy): list only routable ids for partial wildcards on /v1/models and /model/info

[devin-ai-integration]

TLDR

Problem this solves:

• Partial wildcards like cloudflare/@cf/* list no ids, and ids that an exact or narrower route also serves are listed twice
• gemini/gemini-2.5* lists 4 of its 10 models
• Routes like team/* over anthropic/claude-*, a team's own routes, and bare models with custom_llm_provider list ids that fail and leave out ids that work

How it solves it:

• List each known model a wildcard reaches, under the id that routes to it, once, on both endpoints
• /model/info rows name the serving deployment, the routed model, and its price
• Match patterns in bounded time, and probe health inside the route

Decisions and the alternatives they beat

• A listed id is the request the route sends to a known model
  • With one * on each side, the id is the route with its * filled by the part of the model id the deployment's * covers
  • team/* over anthropic/claude-* lists team/haiku-4-5, which routes to anthropic/claude-haiku-4-5
  • Rejected: main's namespace plus model id, which lists team/claude-haiku-4-5, sent as claude-claude-haiku-4-5 and answered 404
  • Rejected: listing nothing for a route that rewrites part of the name, which hides models it serves (round 2 of this PR)
• Some ids look odd but are exactly what the route takes
  • custom/gemini-* over gemini/* lists custom/gemini-gemini-2.5-flash, and openai/gpt-4* over openai/* lists openai/gpt-4gpt-4o
  • Rejected: listing nothing for them, which hides models the route serves
• With more than one *, a route lists ids only when it ends with the deployment's pattern under a plain namespace
  • Rejected: guessing how the router splits several captures, which can list ids that route elsewhere
• A route over one fixed model, like fixed/* over openai/gpt-4o-mini, lists one id, fixed/gpt-4o-mini
  • Main listed 208 ids for it, each answered by gpt-4o-mini whatever it names
• Route-type infixes are stripped before matching, which covers responses/ for every provider, every Bedrock route prefix that targets a model (like converse/, invoke/, nova/, claude_platform/, mantle/), and Azure's o_series/ and gpt5_series/
  • So openai/* over openai/responses/* keeps listing openai/gpt-4o-mini
  • Bedrock's agent/ and agentcore/ are left out, since they target agents, not cost-map models
• /v1/models skips an id only when a more specific route serves it and sends it to a different model
  • * over groq/* next to openai/* does not list openai/gpt-oss-120b, since the router sends that id to openai/*
  • A key scoped to openai/* still lists openai/gpt-4.1-mini next to an openai/gpt-4.1* route, since both send it to the same model
  • Only routes the router registers count, so a team-only route never hides an id
  • Two routes send an id to the same model when the names match after dropping the provider, a route-type infix, and a provider-family prefix
  • So wide/gpt-4o* over openai/responses/gpt-4o*, cust/gpt-4.1* over a bare gpt-4.1* with custom_llm_provider: openai, and az/gpt-4o* over azure/gpt-4o* keep the wider route's ids
  • A route through an aggregator's vendor namespace, like openrouter/openai/gpt-4.1*, counts as the same model as openai/gpt-4.1*
  • Rejected: skipping every id a narrower route matches, which hid callable ids from a key scoped to the wider route (audit round 1)
  • Rejected: comparing the raw model strings, which hid those ids from every key and broke a team's Azure file upload (live risk check at 668fa29)
• A key scoped to a wide route lists the ids narrower routes under its pattern serve, the same ids an admin sees
  • Rejected: listing only the wide route's own ids, which left that key's list shorter than the admin's
• A team's own route lists an id only when no proxy-wide route matches it, since routing tries every proxy-wide route first
  • A team's wide/claude-* beside the proxy-wide wide/* lists nothing, since wide/* sends those ids to OpenAI, which answers 404
  • The team's tw/claude-*, which no proxy-wide route matches, lists its 22 ids for a team key on tw/*
  • Rejected: keeping ids whose names match after dropping the provider, which listed wide/claude-haiku-4-5 for a team key on wide/* (live risk check at ef70f84)
• A deployment with a bare model and custom_llm_provider, like gpt-4* with custom_llm_provider: openai, lists the same ids as openai/gpt-4*
  • So g4/* over it lists g4/o-mini, and openai/* over a bare * lists OpenAI's 208 ids
  • Main listed no g4/ id on /v1/models, and 1442 ids under openai/*, 75 of them openai/claude... or openai/gemini... ids
• /model/info lists each id once, under the deployment that serves it, an exact deployment included
  • When the serving route cannot list the id itself, like an Azure route, the row stays under the wider route
  • Rejected: also a row under the wider route, which names a deployment the id never reaches (audit round 1)
• * over a provider lists bare ids, like gpt-4o-mini for * over openai/*
  • Main listed openai/gpt-4o-mini, which that route sends as openai/openai/gpt-4o-mini and OpenAI answers 400
• Strip only a provider-family prefix from cost-map ids
  • ollama/ under ollama_chat and azure/ under azure_text are stripped
  • Vendor namespaces stay, like openai/ under wandb and meta/ under replicate
• De-dupe keeps the first-seen order, so the list order matches main's
• Match with fnmatchcase, keeping [ and ? literal, since only * is a wildcard in a route name
  • The check for which route serves a listed id uses the same matching, so a route with many * cannot stall the listing
  • That route's captures come from placing each literal at its rightmost fit, right to left, which gives the router's greedy re.match groups in linear time
  • Rejected: main's escaped regex with .*, which backtracks exponentially on a pattern with many *
  • Rejected: re.match for the captures only, which took 44 s on /v1/models beside ns/********g (audit round 2)
• Health check probes up to three models the pattern matches, then main's three cheapest provider models
  • Rejected: only the pattern's models, which flagged wide/gpt-4o* unhealthy after its audio and search preview models answered 404 and 400 (live risk check at 668fa29)
  • Rejected: reporting the route unhealthy when no known model matches, which flags routes like databricks/system.ai.* that serve fine

Listings change on purpose, as #46384 asks. Calls work as before

User Flow

Before: a proxy admin with a cloudflare/@cf/* route finds no Cloudflare id to call

1. They add a cloudflare/@cf/* route and start the proxy
2. GET https://litellm-domain/v1/models lists no Cloudflare ids
3. GET https://litellm-domain/model/info shows only the raw cloudflare/@cf/* row

After: the same route lists every Cloudflare model once, and the listed ids work

1. They add the same route and start the proxy
2. GET https://litellm-domain/v1/models lists 34 ids like cloudflare/@cf/meta/llama-3.2-1b-instruct
3. GET https://litellm-domain/model/info lists the same ids, none duplicated
4. POST https://litellm-domain/v1/chat/completions with a listed id returns 200

Relevant issues

Fixes #46384

Linear ticket

Resolves LIT-8910

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• Every automated review on my PR's latest commit has finished, and I have addressed or replied to each of its findings before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review)

Screenshots / Proof of Fix

Two live proxies, one per tree, 2 uvicorn workers each, real provider keys, and curl. Raw output is in this gist

Route configs

Every proxy runs with LITELLM_LOCAL_MODEL_COST_MAP=True. The scoped-key, infix, shadow, narrower-route, cross-region, and team cases use Postgres

# static.yaml
model_list:
  - {model_name: cloudflare/@cf/*, litellm_params: {model: cloudflare/@cf/*, api_key: os.environ/CLOUDFLARE_API_KEY}}
  - {model_name: vertex_ai/gemini-*, litellm_params: {model: vertex_ai/gemini-*, vertex_project: os.environ/VERTEXAI_PROJECT, vertex_location: us-central1}}
  - {model_name: gemini/gemini-2.5*, litellm_params: {model: gemini/gemini-2.5*, api_key: os.environ/GEMINI_API_KEY}}
  - {model_name: fireworks_ai/accounts/fireworks/models/llama*, litellm_params: {model: fireworks_ai/accounts/fireworks/models/llama*, api_key: os.environ/FIREWORKS_AI_API_KEY}}
  - {model_name: openai/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: team/claude-*, litellm_params: {model: anthropic/claude-*, api_key: os.environ/ANTHROPIC_API_KEY}}
  - {model_name: databricks/system.ai.*, litellm_params: {model: databricks/system.ai.*, api_base: https://example.invalid/serving-endpoints}}
  - {model_name: custom/gemini-*, litellm_params: {model: gemini/*, api_key: os.environ/GEMINI_API_KEY}}
# alias.yaml
model_list:
  - {model_name: "*", litellm_params: {model: cloudflare/@cf/*, api_key: os.environ/CLOUDFLARE_API_KEY}}
  - {model_name: team/*, litellm_params: {model: anthropic/claude-*, api_key: os.environ/ANTHROPIC_API_KEY}}
  - {model_name: genai/test/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: genai/test/gpt-4o-mini, litellm_params: {model: openai/gpt-4o-mini, api_key: os.environ/OPENAI_API_KEY}}
# routes.yaml
model_list:
  - {model_name: openai/*, litellm_params: {model: openai/responses/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: team/oai-*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: fixed/*, litellm_params: {model: openai/gpt-4o-mini, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: bedrock/*, litellm_params: {model: bedrock/converse/*, aws_region_name: us-east-1}}
# star_gemini.yaml and star_openai.yaml
model_list:
  - {model_name: "*", litellm_params: {model: gemini/gemini-2.5*, api_key: os.environ/GEMINI_API_KEY}}
  - {model_name: "*", litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}  # its own proxy
# infix.yaml
model_list:
  - {model_name: bnova/*, litellm_params: {model: bedrock/nova/*, aws_region_name: us-east-1}}
  - {model_name: bcp/*, litellm_params: {model: bedrock/claude_platform/*, aws_region_name: us-east-1}}
  - {model_name: bmantle/*, litellm_params: {model: bedrock/mantle/*, aws_region_name: us-east-1}}
# collide.yaml
model_list:
  - {model_name: "*", litellm_params: {model: groq/*, api_key: placeholder-not-called}}
  - {model_name: openai/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
# narrow.yaml, on Postgres
model_list:
  - {model_name: openai/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: openai/gpt-4.1*, litellm_params: {model: openai/gpt-4.1*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: openai/gpt-4o-mini, litellm_params: {model: openai/gpt-4o-mini, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: stars/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: "stars/*******!", litellm_params: {model: "openai/*******!", api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: gstars/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: "gstars/********g", litellm_params: {model: "openai/********g", api_key: os.environ/OPENAI_API_KEY}}
# same.yaml, on Postgres
model_list:
  - {model_name: wide/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: wide/gpt-4o*, litellm_params: {model: openai/responses/gpt-4o*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: cust/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: cust/gpt-4.1*, litellm_params: {model: gpt-4.1*, custom_llm_provider: openai, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: az/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: az/gpt-4o*, litellm_params: {model: azure/gpt-4o*, api_key: os.environ/AZURE_SWEDEN_API_KEY, api_base: os.environ/AZURE_SWEDEN_API_BASE, api_version: "2024-10-21"}}
files_settings:
  - {custom_llm_provider: openai, api_key: os.environ/OPENAI_API_KEY}
# xr.yaml, on Postgres
model_list:
  - {model_name: orr/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: orr/gpt-4o*, litellm_params: {model: openrouter/openai/gpt-4o*, api_key: os.environ/OPENROUTER_API_KEY}}
  - {model_name: brk/*, litellm_params: {model: bedrock/*, aws_region_name: us-east-1}}
  - {model_name: brk/anthropic.*, litellm_params: {model: bedrock/us.anthropic.*, aws_region_name: us-east-1}}
# bare.yaml, no database
model_list:
  - {model_name: openai/*, litellm_params: {model: "*", custom_llm_provider: openai, api_key: os.environ/OPENAI_API_KEY}}
  - {model_name: g4/*, litellm_params: {model: gpt-4*, custom_llm_provider: openai, api_key: os.environ/OPENAI_API_KEY}}
# team.yaml, on Postgres, plus the team routes wide/claude-* and tw/claude-* over anthropic/claude-*, added through POST /model/new with model_info.team_id
model_list:
  - {model_name: wide/*, litellm_params: {model: openai/*, api_key: os.environ/OPENAI_API_KEY}}
# db.yaml: static.yaml plus general_settings.master_key, on Postgres
# live.yaml: gemini/gemini-2.5* and team/gemini-2.5* (both over gemini/gemini-2.5*), with litellm_settings.check_provider_endpoint: true

Case	Before (b6ca57b)	After (bb2b5b1)
cloudflare/@cf/* ids	0	34
gemini/gemini-2.5* ids	4	10
fireworks_ai/.../llama* ids	0	29
Listed custom/gemini-* id	404	200
Listed team/* over claude-* id	404	200
* over cloudflare/@cf/* ids	0	34
Listed * over gemini-2.5* id	404	200
Listed * over openai/* id	400	200
openai/* over bare * ids	1442	208
openai/claude and openai/gemini ids	75	0
g4/* over bare gpt-4* ids	0	33
team/oai-gpt-4o-mini price	null	1.5E-7
fixed/* over openai/gpt-4o-mini ids	208	1
Scoped key, cloudflare/@cf/ ids	0	34
Explicit id a wildcard covers, times listed	2	1
/model/info rows, id an exact or narrower route serves	2 and 2	1 and 1
gstars/gpt-4o-mini listings	2	0
Ids a narrower route sends to the same model	44, 7, and 22	22, 7, and 22
Admin brk/anthropic. ids on /v1/models	66	25
Key on orr/* and brk/*, brk/anthropic. ids on /v1/models and /model/info	33 and 66	25 and 25
Team key, no model list, wide/claude- ids	22	0
Team key on tw/*, tw/claude- ids	0	22
Live list, both routes	0 and 0	7 and 7

Rows with the same result on both legs

Case	Before (b6ca57b)	After (bb2b5b1)
openai/* over openai/responses/* ids	208	208
bedrock/* and bnova/* infix route ids	361 and 361	361 and 361
Duplicate ids	0	0
Key scoped to openai/*, gpt-4.1 ids	7	7
Listing beside many-* routes	200 in 0.283457s	200 in 0.146890s
Same ids, scoped key	22 and 7	22 and 7
az/* team, Azure file upload	HTTP 200	HTTP 200
Key lists admin's orr/gpt-4o and brk/anthropic. ids	identical	identical
Team key, no model list, wide/claude-haiku-4-5 call	404	404
Team key on tw/*, tw/claude-haiku-4-5 call	200	200
wide/gpt-4o* and gemini/gemini-2.5* health	healthy	healthy

Both legs report every health check healthy at this base, so unit tests, not this proof, cover probing inside the route

Before (b6ca57b)

All cases

Cloudflare partial wildcard

1. /v1/models lists no Cloudflare id, and /model/info only the raw route row
   Output in before_static.txt

Gemini 2.5 partial wildcard

1. /v1/models lists 4 audio or TTS ids, missing gemini-2.5-flash and gemini-2.5-pro
   Output in before_static.txt
2. gemini/gemini-2.5-flash-lite is not listed, though calling it works
   Output in before_static.txt
3. The health check reports the route healthy
   Output in before_static.txt

Fireworks partial wildcard

1. /v1/models lists no Fireworks id
   Output in before_static.txt

Alias that rewrites part of the model name

1. custom/gemini-* over gemini/* lists 12 ids, and a listed one answers 404
   Output in before_static.txt
2. team/* over anthropic/claude-* lists 22 ids, and a listed one is sent as claude-claude-... and answers 404
   Output in before_alias.txt

Alias that renames only the namespace

1. genai/test/* over openai/* lists 212 ids, 4 of them keeping the openai/ prefix, like genai/test/openai/sora-2, and a listed one works
   Output in before_alias.txt
2. team/claude-* over anthropic/claude-* lists 22 ids, and a listed one works
   Output in before_static.txt

Route-type infix, prefix alias, and fixed target

1. Counts, where fixed/* lists 208 ids, and /model/info gives the alias row no price
   Output in before_routes.txt
2. Listed ids work
   Output in before_routes.txt
3. Bedrock nova/, claude_platform/, and mantle/ routes list ids on both endpoints, a listed one works, and the health check reports the route healthy
   Output in before_infix.txt

Route shadowed by a more specific one

1. * over groq/* lists no openai/gpt-oss id, and calling openai/gpt-oss-120b reaches openai/*, which answers 404
   Output in before_collide.txt

Narrower, exact, and many-* routes

1. /model/info lists openai/gpt-4o-mini twice and openai/gpt-4.1-mini twice
   Output in before_narrow.txt
2. A key scoped to openai/* lists the gpt-4.1 ids an openai/gpt-4.1* route serves, and one works
   Output in before_narrow.txt
3. /v1/models answers promptly next to stars/*******! and gstars/********g routes
   Output in before_narrow.txt
4. gstars/* lists gstars/gpt-4o-mini twice, though gstars/********g serves it and sends it as openai/g, so the call fails
   Output in before_narrow.txt
5. wide/, cust/, and az/ list the ids that a narrower route sends to the same model by another path, on both endpoints
   Output in before_same.txt
6. A key scoped to wide/* and cust/* lists those ids, and they work
   Output in before_same.txt
7. A team limited to az/* uploads a batch file for Azure, and the wide/gpt-4o* health check reports healthy
   Output in before_same.txt

Key scoped to wide routes beside OpenRouter and cross-region routes

1. Admin lists 22 orr/gpt-4o ids and 66 brk/anthropic. ids on /v1/models
   Output in before_xr.txt
2. A key scoped to orr/* and brk/* lists 33 brk/anthropic. ids on /v1/models and 66 on /model/info, and the set of orr/gpt-4o and brk/anthropic. ids it lists matches admin's
   Output in before_xr.txt
3. The key lists orr/gpt-4o-mini and brk/anthropic.claude-haiku-4-5-20251001-v1:0, and both calls return 200
   Output in before_xr.txt

Key scoped to the partial routes

1. A non-admin key sees no Cloudflare id on /v1/models, /model/info, or /model_group/info, and 4 Gemini 2.5 ids
   Output in before_db.txt
2. Calls with the key work
   Output in before_db.txt

Team routes beside a proxy-wide route

1. A team adds its own wide/claude-* and tw/claude-* routes over anthropic/claude-* through /model/new, beside the proxy-wide wide/*
   Output in before_team.txt
2. A team key on wide/* lists no wide/claude- id on /v1/models and /model_group/info
   Output in before_team.txt
3. A team key with no model list lists 22 wide/claude- and 22 tw/claude- ids on both endpoints, and calling wide/claude-haiku-4-5 reaches OpenAI, which answers 404
   Output in before_team.txt
4. A team key on tw/* lists no tw/claude- id, though calling tw/claude-haiku-4-5 works
   Output in before_team.txt

Star routes

1. * over gemini/gemini-2.5* lists 4 gemini/ ids on both endpoints, and a listed one answers 404
   Output in before_star_gemini.txt
2. * over cloudflare/@cf/* lists no id on either endpoint
   Output in before_alias.txt
3. * over openai/* lists openai/gpt-4o-mini, which answers 400, while the unlisted gpt-4o-mini works
   Output in before_star_openai.txt

Bare model with custom_llm_provider

1. openai/* over a bare * with custom_llm_provider: openai lists 1442 ids on both endpoints, 75 of them openai/claude... or openai/gemini... ids, and calling openai/claude-fable-5 answers 404
   Output in before_bare.txt
2. g4/* over a bare gpt-4* lists no g4/ id on /v1/models, though calling g4/o-mini works, as does openai/gpt-4o-mini
   Output in before_bare.txt

Duplicates

1. No id is listed twice on either endpoint, same as After
   Output in before_static.txt

Explicit id that a wildcard also covers

1. /v1/models lists genai/test/gpt-4o-mini twice
   Output in before_alias.txt

Live listing

1. With check_provider_endpoint: true, neither route lists an id
   Output in before_live.txt

Unchanged routes

1. vertex_ai/gemini-* lists 31 ids, and databricks/system.ai.* lists none
   Output in before_static.txt

After (bb2b5b1)

Cloudflare partial wildcard

1. /v1/models lists 34 Cloudflare ids

$ curl -s localhost:$PORT/v1/models -H "Authorization: Bearer $KEY" | jq -r ".data[].id" | grep -c '^cloudflare/@cf/'
34

2. Calling a listed id works

$ curl -s -w '\nHTTP %{http_code}\n' localhost:$PORT/v1/chat/completions -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model": "cloudflare/@cf/meta/llama-3.2-1b-instruct", "messages": [{"role": "user", "content": "Say hi in two words"}], "max_tokens": 16}'
{"id":"id-1792203806443","created":1792203806,"model":"cloudflare/@cf/meta/llama-3.2-1b-instruct","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi","role":"assistant","refusal":null},"provider_specific_fields":{},"logprobs":null}],"usage":{"completion_tokens":2,"prompt_tokens":16,"total_tokens":18,"prompt_tokens_details":{"cached_tokens":0},"neurons":0.07581928744912148}}
HTTP 200

Other cases

Gemini 2.5 partial wildcard

1. /v1/models lists all 10 ids
   Output in after_static.txt
2. Calling a listed id works
   Output in after_static.txt
3. The health check reports the route healthy, same as Before
   Output in after_static.txt

Fireworks partial wildcard

1. /v1/models lists 29 ids
   Output in after_static.txt
2. A listed id routes to the exact model it names, which Fireworks no longer serves on this account, same as Before
   Output in after_static.txt

Alias that rewrites part of the model name

1. custom/gemini-* over gemini/* lists 73 ids, and a listed one works
   Output in after_static.txt
2. team/* over anthropic/claude-* lists 22 ids like team/fable-5, and a listed one works
   Output in after_alias.txt

Alias that renames only the namespace

1. genai/test/* over openai/* lists 208 ids, each once and none keeping the openai/ prefix, and a listed one works
   Output in after_alias.txt
2. team/claude-* over anthropic/claude-* lists the same 22 ids, and a listed one works
   Output in after_static.txt

Route-type infix, prefix alias, and fixed target

1. The infix route lists the same 208 ids as Before, the prefix route lists 208, fixed/* lists 1 id, and the alias row carries the routed model and its price
   Output in after_routes.txt
2. Listed ids work
   Output in after_routes.txt
3. Bedrock nova/, claude_platform/, and mantle/ routes keep their ids, a listed one works, and the health check reports the route healthy
   Output in after_infix.txt

Route shadowed by a more specific one

1. * over groq/* still lists no openai/gpt-oss id and no /model/info row for it, since openai/* serves that name
   Output in after_collide.txt

Narrower, exact, and many-* routes

1. /model/info lists openai/gpt-4o-mini once and openai/gpt-4.1-mini once
   Output in after_narrow.txt
2. A key scoped to openai/* lists the gpt-4.1 ids an openai/gpt-4.1* route serves, and one works
   Output in after_narrow.txt
3. /v1/models answers promptly next to stars/*******! and gstars/********g routes
   Output in after_narrow.txt
4. gstars/* no longer lists gstars/gpt-4o-mini, since gstars/********g sends it as openai/g
   Output in after_narrow.txt
5. wide/, cust/, and az/ list the ids that a narrower route sends to the same model by another path, on both endpoints
   Output in after_same.txt
6. A key scoped to wide/* and cust/* lists those ids, and they work
   Output in after_same.txt
7. A team limited to az/* uploads a batch file for Azure, and the wide/gpt-4o* health check reports healthy
   Output in after_same.txt

Key scoped to wide routes beside OpenRouter and cross-region routes

1. Admin lists 22 orr/gpt-4o ids and 25 brk/anthropic. ids on /v1/models
   Output in after_xr.txt
2. A key scoped to orr/* and brk/* lists 25 brk/anthropic. ids on /v1/models and 25 on /model/info, and the set of orr/gpt-4o and brk/anthropic. ids it lists matches admin's
   Output in after_xr.txt
3. The key lists orr/gpt-4o-mini and brk/anthropic.claude-haiku-4-5-20251001-v1:0, and both calls return 200
   Output in after_xr.txt

Key scoped to the partial routes

1. A non-admin key sees the Cloudflare and Gemini 2.5 ids on all three endpoints, and nothing outside its routes
   Output in after_db.txt
2. Calls with the key work
   Output in after_db.txt

Team routes beside a proxy-wide route

1. The same team adds the same two routes through /model/new
   Output in after_team.txt
2. A team key on wide/* still lists no wide/claude- id on /v1/models and /model_group/info
   Output in after_team.txt
3. A team key with no model list lists no wide/claude- id, since wide/* sends those ids to OpenAI, where wide/claude-haiku-4-5 still answers 404, and keeps its 22 tw/claude- ids
   Output in after_team.txt
4. A team key on tw/* lists 22 tw/claude- ids, and calling tw/claude-haiku-4-5 works
   Output in after_team.txt

Star routes

1. * over gemini/gemini-2.5* lists the bare ids the route takes, and a listed one works
   Output in after_star_gemini.txt
2. * over cloudflare/@cf/* lists 34 bare ids, and a listed one works
   Output in after_alias.txt
3. * over openai/* lists gpt-4o-mini, which works
   Output in after_star_openai.txt

Bare model with custom_llm_provider

1. openai/* over a bare * lists 208 ids on both endpoints, none of them openai/claude... or openai/gemini... ids, and the listed openai/gpt-4o-mini works
   Output in after_bare.txt
2. g4/* over a bare gpt-4* lists 33 ids on both endpoints, g4/o-mini among them, and calling it works
   Output in after_bare.txt

Duplicates

1. No id is listed twice on either endpoint
   Output in after_static.txt

Explicit id that a wildcard also covers

1. /v1/models lists genai/test/gpt-4o-mini once
   Output in after_alias.txt

Live listing

1. With check_provider_endpoint: true, both routes list ids from Gemini's live list
   Output in after_live.txt
2. Calling a listed alias id answers 200
   Output in after_live.txt

Unchanged routes

1. vertex_ai/gemini-* lists the same 31 ids, and databricks/system.ai.* lists none
   Output in after_static.txt
2. Calling a listed Vertex AI id works
   Output in after_static.txt

Audit round 3 of 3, PASS at ef70f84 (row 24 fixed past the cap)

Audit round 3 of 3 (row 24 fixed past the cap in 743790b)

Round 3 failed on row 24 at fa613b1. The fix went in past the cap in 743790b, since the failing row broke the PR's own same-model rule, and main was merged in at ef70f84. Every head cell ran again twice at ef70f84, rows 14 to 17 once more each with one worker. The base column is round 3's run at f9d45f8 with the head's test files, and the merge base is now b6ca57b. T is tests/integration/routing/test_wildcard_partial_prefix_expansion.py and A is tests/integration/authorization/test_wildcard_model_access.py. Each leg had its own two-worker proxy, Postgres database, Redis and scripted upstream, and nothing inside the proxy was mocked. Both head runs used seed 43643 and order seed 997

| # | Contract | Cell | Base | Head 1 | Head 2 | Result |
| 1 | Partial wildcard lists ids once | T::test_partial_wildcard_lists_provider_ids_once_on_both_endpoints | red | green | green | PASS |
| 2 | No-match partial lists nothing | T::test_partial_wildcard_matching_no_known_id_lists_no_guessed_ids | green | green | green | PASS |
| 3 | Provider wildcard still expands | T::test_provider_wildcard_still_expands_to_cost_map_names | green | green | green | PASS |
| 4 | Alias over a partial target | T::test_listed_wildcard_ids_reach_the_upstream_as_the_priced_model_their_row_names[alias_over_partial_model] | red | green | green | PASS |
| 5 | Alias over one fixed model | T::...[alias_over_one_fixed_model] | red | green | green | PASS |
| 6 | * over openai/* | T::...[catch_all_over_provider] | red | green | green | approved |
| 7 | Narrower route owns its rows | T::test_each_wildcard_row_names_the_deployment_its_id_routes_to[narrower_wildcard] | red | green | green | PASS |
| 8 | Exact deployment owns its row | T::...[exact_name] | red | green | green | PASS |
| 9 | Same model, same path | T::test_key_scoped_to_a_wildcard_lists_an_id_a_narrower_route_serves_only_if_it_reaches_the_named_model[same_model] | green | green | green | PASS |
| 10 | Narrower route, other model | T::...[different_model] | red | green | green | approved |
| 11 | Overlapping deployments list once | T::test_overlapping_deployments_of_one_wildcard_list_each_id_once | red | green | green | PASS |
| 12 | Health probe inside the pattern | T::test_wildcard_health_check_probes_a_model_its_pattern_names[gpt-4o*-True] | red | green | green | PASS |
| 13 | Health probe with no match | T::...[no-such-model-*-False] | green | green | green | PASS |
| 14 | /v1/models time, ! tail | T::test_wildcard_listing_answers_promptly_beside_a_route_with_many_stars[tail_in_no_listed_id-v1_models] | green | green | green | PASS |
| 15 | /model/info time, ! tail | T::...[tail_in_no_listed_id-model_info] | green | green | green | PASS |
| 16 | /v1/models time, g tail | T::...[tail_early_in_listed_ids-v1_models] | green | green | green | PASS |
| 17 | /model/info time, g tail | T::...[tail_early_in_listed_ids-model_info] | green | green | green | PASS |
| 18 | Same model via responses/ | T::...[same_model_through_responses_route] | green | green | green | PASS |
| 19 | Same model, bare plus provider | T::...[same_model_as_bare_model_with_provider] | green | green | green | PASS |
| 20 | Same model, other provider | T::...[same_model_on_another_provider] | green | green | green | PASS |
| 21 | Health falls back to provider | T::test_wildcard_health_check_stays_healthy_when_every_model_its_pattern_names_fails_but_the_provider_answers | green | green | green | PASS |
| 22 | Team files use narrow credentials | T::test_team_file_listing_takes_the_credentials_of_a_narrower_wildcard_route_on_the_requested_provider | green | green | green | PASS |
| 23 | openai/* key beside * | A::test_a_key_scoped_to_a_provider_wildcard_on_the_catch_all_lists_only_ids_that_reach_the_upstream_as_named | red | green | green | approved |
| 24 | Same model under a vendor namespace | T::...[same_model_under_a_vendor_namespace] | green | green | green | PASS |

Controls ran once at ef70f84 and all four passed, tests/integration/providers/test_coralbricks_provider_wire.py::test_wildcard_deployment_lists_every_coralbricks_catalog_row and ::test_wildcard_coralbricks_deployment_routes_the_requested_model, plus A::test_an_all_models_key_reaches_a_model_served_only_by_the_catch_all_deployment and A::test_a_key_without_models_inherits_the_users_exact_and_wildcard_grants

Row 24 now passes on both head runs. A key scoped to {ns}/* lists {ns}/gpt-4.1 beside {ns}/gpt-4.1* over openrouter/openai/gpt-4.1*, /model/info has its row, and the call reaches the upstream as openai/gpt-4.1. The same commit lets a key scoped to a wide wildcard list the ids a narrower route under its pattern serves, which the separate live risk check had flagged for brk/* beside brk/anthropic.*

Row 23 lists no id for a key scoped to openai/* beside * over openai/*, which is correct, since that route sends openai/<id> upstream unchanged. The base lists those ids and fails

Rows 6, 10 and 23 change what callers see, and each one is the listing change #46384 asks for. The health check on a route where every probe fails makes up to six probe calls where the base made three, which the Caveats section lists

• bb2b5b1 passes /audit (cap: 3 rounds used and round 3 passed at ef70f84 with no open rows, so the listing fix in bb2b5b1 gets no 4th round)

Type

Bug Fix

Caveats (if any)

Medium

• Partial routes list cost-map ids the provider no longer serves, like Fireworks llamas and retired Gemini models
  • Same on main. check_provider_endpoint: true lists from the provider's live model list instead

Low

• A key scoped to bare * lists every route's ids plus the bare ids main already listed for it, so it sees more ids than admin
  • Main lists those bare ids too, and dropping them is a separate change to what * keys see
• Listing for a key scoped to a wildcard now takes about as long as admin's, up to about 2 s on /model/info with 2600 ids
  • A cache would need clearing on every model change, which is more risk than the delay
• A bedrock/* key beside a bedrock/converse/* route lists each model under both routes, as admin does
  • Both spellings call fine, and hiding one would make the key's list differ from admin's
• The dashboard's autorouter preset matcher says it mirrors the old listing function
  • Unchanged here, and a UI edit would pull dashboard lint into a backend PR
• Health check keeps main's fallback probe when no model matches, which can sit outside the route
  • Failing instead would mark serving routes like databricks/system.ai.* unhealthy
• Health check on a partial route that is down sends up to 6 probes where main sent 3, its own pattern's models before the provider's cheapest
  • A cap of 3 would drop either the pattern probe or the provider fallback that keeps the route healthy
• /model/info keeps one row per deployment for an explicit model a wildcard also covers, like main
• /model/info names the wider route's deployment for an id a narrower route on another provider serves but cannot list, like main
• Overlaps fix(proxy): match partial wildcards against prefix-stripped model ids #46390, with the same de-dupe, and fix(router): preserve vendor namespaces and catalog paths in wildcard discovery #45531, which also fixes OpenRouter's catalog URL

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Touches core proxy model discovery, API key scoping, and pattern routing with intentional listing behavior changes; mistakes could hide callable models or expose wrong ids, though coverage is heavy on unit and integration tests.

Overview
Fixes wildcard expansion on /v1/models and /model/info so each listed id is the name the route actually accepts and maps to a known cost-map model once, instead of guessing prefixes or duplicating rows when narrower routes exist.

Listing logic moves into shared helpers (model_matches_wildcard, split_route_type_prefix, strip_provider_family_prefix, model_name_at_provider) and a _WildcardListing pipeline in model_checks.py. Expansion now strips Azure/Bedrock route-type infixes, respects bare deployments with custom_llm_provider, and skips ids when PatternMatchRouter.routes_elsewhere shows a more specific route would send a different upstream model (while keeping ids when wide and narrow routes reach the same model). Team-only routes and overlapping wildcard keys get the same rules; /model/info expansion receives pattern_router for shadowing.

Pattern router gains linear-time matches_from_start / greedy_captures / fill_wildcards plus shadowing_regex, serves, registers, and routes_elsewhere so listing stays fast beside many-* patterns and aligns with how wildcards are filled at request time.

Wildcard health checks probe up to three models that match the deployment pattern (then fall back to cheapest provider models), so partial routes like gpt-4o* are not marked unhealthy when only non-matching catalog entries fail.

Listing output changes by design (#46384); routing behavior for successful calls is unchanged.

^{Reviewed by Cursor Bugbot for commit bb2b5b1. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ gabriele-berri
✅ mateo-berri
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Confidence Score: 4/5

[Medium impact] Refactors wildcard model matching and listing logic.

Fix the missing exact-model check before merging so scoped keys keep discovering working models

Findings

1.  Working exact models disappear ▶

Summary

The PR makes wildcard listings use callable IDs, removes duplicates, and improves health probes. The latest change also checks registered routes when a key's granted pattern is not itself registered

• Wildcard model lists now show the names their routes actually accept.
• Wildcard health checks probe models their route can name.

_{Reviews (15) · Last reviewed commit: "fix(proxy): skip ids a narrower route se..." · Reviewed by Greptile}

[codspeed]

Merging this PR will not alter performance

✅ 32 untouched benchmarks

---

_{Comparing litellm_partial_wildcard_prefix_expansion (67488f8) with main (9f9a02b)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

@mateo-berri rebased onto main with the #46390 de-dupe folded in. The description compares both PRs live, your call on which to merge

[mateo-berri]

@litereview code-review

[litereview]

✅ code-review

5/5 · Ready for human review

Scope: full · Full PR review.

No correctness findings. I recheck the earlier many-* stall finding and it is still fixed: with ns/* next to ns/********g, /v1/models now returns 33 ids in 0.007s, down from about 21s.

Checks and execution details

• Passed · Real testing evidence (required): Not applicable: no LLM translation changes in the scoped paths. The PR changes no file under litellm/llms/** or litellm/litellm_core_utils/prompt_templates/**. It changes get_llm_provider_logic.py, health_check_helpers.py, proxy/auth/model_checks.py, proxy_server.py and router_utils/pattern_match_deployments.py. Those files cover model listing, wildcard matching and health-probe selection. None of them translates requests or responses between providers.

• Passed · Compatibility requirements (required): I ran the same configs through base b6ca57b and head with LITELLM_LOCAL_MODEL_COST_MAP=True.

• Unchanged: openai/*, anthropic/claude-*, ollama_server1/*→ollama/*, bedrock/*, vertex_ai/gemini-*, azure/*, openrouter/*, myco/*→openai/* and openai/gpt-4* list the same ids on both endpoints.

• Changed: *→openai/* now lists gpt-4o-mini where it used to list openai/gpt-4o-mini. Under that route the old id becomes openai/openai/..., and the PR's live gist shows it answering 400.

• Added ids: gemini/gemini-2.5*, xai/grok-*, groq/llama* and a bare * with custom_llm_provider now list ids that base did not.

Call routing has not changed: set_deployment_model_name delegates to fill_wildcards, which keeps the same logic. I found no id that worked before and is now missing from the listings, so the PR needs no breaking-change marker.

• Passed · Changed behavior and result consistency (required): Standards: I read tests/integration/AGENTS.md and its README.

/v1/models path: get_complete_model_list → _get_wildcard_models → _ids_listed_by_route and _listings_served_by_route → PatternMatchRouter.routes_elsewhere. Team-only filtering matches Router.wildcard_route_deployments, which tries proxy-wide routes first.

/model/info path: model_info_v1 → expand_wildcard_deployments_for_model_info(pattern_router=...).

Health check path: litellm.main → ahealth_check_wildcard_models → _wildcard_probe_models. pick_cheapest_chat_models_from_llm_provider already looped over every known model before, so n=sys.maxsize adds no cost.

Boundary checks:

• A fuzz run of 200k random ab*/ patterns found 0 mismatches between greedy_captures/matches_from_start and re.match.
• A 36-route config lists 3687 ids in 0.14s on /v1/models and 0.31s on /model/info.

Tests:

• Unit: test_pattern_match_deployments.py, test_model_checks.py, test_health_check_helpers.py and test_routes_model_info.py gave 316 passed.
• Integration, run against local Postgres, Redis and a scripted upstream: routing/test_wildcard_partial_prefix_expansion.py gave 23 passed; authorization/test_wildcard_model_access.py gave 3 passed.

No defects remain open.

• Passed · Prior many-star stall reproducer (optional): Router with ns/*→openai/* and ns/********g→openai/********g. get_complete_model_list returned 33 ids in 0.0068s.

---

View review · Reviewed 67488f8b0d09

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@litereview code-review

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@litereview code-review

[mateo-berri]

@litereview code-review

[litereview]

Review already requested

code-review is already reviewing bb2b5b101325. View existing review.

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit bb2b5b1. Configure here.}

[cursor]

Unregistered key grants skip shadowing

Medium Severity

shadowing_regex returns none when the listed wildcard is not a registered route, so routes_elsewhere never drops ids. A key scoped to a pattern that is only a grant, not a route, still expands through the parent wildcard and can list an id that a narrower route will serve as a different, uncallable model.

Additional Locations (2)

• litellm/proxy/auth/model_checks.py#L351-L362
• litellm/proxy/auth/model_checks.py#L444-L459

 

^{Reviewed by Cursor Bugbot for commit bb2b5b1. Configure here.}

[mateo-berri]

@litereview code-review

[mateo-berri]

@greptileai

[greptile-apps]

Working exact models disappear

Removing the registration guard can hide an exact model from a key whose granted pattern is not registered. For example, configure openai/* over openai/*, openai/gpt-4o* over anthropic/claude-sonnet-4*, and exact openai/gpt-4o-mini over itself. A key granted openai/gpt-4* now loses openai/gpt-4o-mini from both listing endpoints, although calls still use the exact deployment

This lookup checks only wildcard routes, and the other-route listing cannot restore exact names. Include exact deployments when deciding which route serves an ID

Knowledge Base Used: Routing and resilience