fix(proxy): stop /{provider}/v1/files and /v1/batches from shadowing custom pass_through_endpoints by IdoPort · Pull Request #38017 · BerriAI/litellm

IdoPort · 2026-08-23T10:03:47Z

TLDR

Problem this solves:

A custom pass_through_endpoints prefix is always shadowed by /{provider}/v1/files and /v1/batches
Those generic routes misread the custom prefix as a provider name and 422/500/400 instead of forwarding

How it solves it:

SafeRouteAdder moves a newly-registered custom route to sit before the first generic /{provider}/... route
Reuses the existing append-then-reposition point shared by both exact and subpath route registration

User Flow

Before: an operator running a pass_through_endpoints entry under a custom prefix (e.g. a self-hosted Anthropic-compatible endpoint reached via /claude-aws) cannot upload documents or create batches through the gateway at all.

The operator adds a pass_through_endpoints entry for path: /claude-aws/v1/files in config.yaml, targeting their own host
A client sends POST https://litellm-domain/claude-aws/v1/files with a file, no purpose field (the Anthropic SDK never sends one)
They get 422 {"detail":[{"type":"missing","loc":["body","purpose"],"msg":"Field required"}]}
They add purpose=user_data and retry the same request
They get 500 {"error":{"message":"files_settings is not set, set it on your config.yaml file."}}
They send POST https://litellm-domain/claude-aws/v1/batches with a batch payload
They get 400 {"error":{"message":"/batches: Missing required parameter: 'input_file_id'."}} -- the request never reached their configured target at all

After: the same requests reach the operator's configured target instead of being intercepted locally.

Same config.yaml entry
Same POST https://litellm-domain/claude-aws/v1/files with a file, no extra fields
The request reaches the configured target and the client gets back that target's real response
Same POST https://litellm-domain/claude-aws/v1/batches with a batch payload
The request reaches the configured target and the client gets back that target's real response

Relevant issues

Fixes #37925

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Config used for both runs (config.yaml), against a self-hosted echo target so the proof needs no LLM provider credentials or real spend -- the bug is pure route-matching, upstream of any provider call:

model_list:
  - model_name: openai/gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: sk-fake-key-not-used-in-this-repro

general_settings:
  master_key: sk-local-repro-master
  pass_through_endpoints:
    - path: /claude-aws/v1/files
      target: https://postman-echo.com/post
      headers:
        x-repro-header: "37925-proof-of-fix"
    - path: /claude-aws/v1/batches
      target: https://postman-echo.com/post
      headers:
        x-repro-header: "37925-proof-of-fix"

Run: litellm --config config.yaml --port 4010

Before (`f005afa`)

files

curl -X POST http://127.0.0.1:4010/claude-aws/v1/files -H "Authorization: Bearer sk-local-repro-master" -F "file=@probe.txt;type=text/plain"
```
{"detail":[{"type":"missing","loc":["body","purpose"],"msg":"Field required","input":null}]}
HTTP 422
```

Same request with -F "purpose=user_data" added:

{"error":{"message":"files_settings is not set, set it on your config.yaml file.","type":"None","param":"None","code":"500"}}
HTTP 500

batches

curl -X POST http://127.0.0.1:4010/claude-aws/v1/batches -H "Authorization: Bearer sk-local-repro-master" -H "Content-Type: application/json" -d '{"probe":"37925"}'
```
{"error":{"message":"/batches: Missing required parameter: 'input_file_id'.","type":"invalid_request_error","param":"input_file_id","code":"400"}}
HTTP 400
```

After (`26bcafc`)

files

curl -X POST http://127.0.0.1:4010/claude-aws/v1/files -H "Authorization: Bearer sk-local-repro-master" -F "file=@probe.txt;type=text/plain"

{"args":{},"data":{},"files":{"probe.txt":"data:application/octet-stream;base64,cHJvYmUgZmlsZSBmb3IgbGl0ZWxsbSBwYXNzdGhyb3VnaCByZXBybwo="},"form":{},"headers":{"host":"postman-echo.com","accept":"*/*","accept-encoding":"gzip, br","content-type":"multipart/form-data; boundary=8ee9491ba0d6e0263a6728f11a0514f4","x-repro-header":"37925-proof-of-fix","x-forwarded-proto":"https","user-agent":"litellm/1.99.0","content-length":"212"},"json":null,"url":"https://postman-echo.com/post"}
HTTP 200

The request reached the configured target -- note x-repro-header and the base64-decoded file content (probe file for litellm passthrough repro) both round-tripped correctly.

batches

curl -X POST http://127.0.0.1:4010/claude-aws/v1/batches -H "Authorization: Bearer sk-local-repro-master" -H "Content-Type: application/json" -d '{"probe":"37925"}'

{"args":{},"data":{"probe":"37925"},"files":{},"form":{},"headers":{"host":"postman-echo.com","accept":"*/*","accept-encoding":"gzip, br","content-type":"application/json","x-repro-header":"37925-proof-of-fix","x-forwarded-proto":"https","user-agent":"litellm/1.99.0","content-length":"17"},"json":{"probe":"37925"},"url":"https://postman-echo.com/post"}
HTTP 200

The request reached the configured target -- note x-repro-header and the echoed body both round-tripped correctly.

Type

🐛 Bug Fix
✅ Test

Caveats (if any)

Repositioning routes in app.routes may shift their relative order in /docs (OpenAPI UI); purely cosmetic, no functional impact
The batches shadowing symptom differs from files: it 400s on a missing OpenAI-specific input_file_id rather than 500ing on files_settings, since batch creation has no equivalent required-field gate before the provider mismatch would otherwise surface -- same root cause, different downstream error shape
Related to [Bug]: /{provider}/v1/files shadows the /anthropic passthrough → 422 "purpose Field required" #37289 / PR fix(proxy): stop /{provider}/v1/files from capturing /anthropic passthrough #37412, which fixes the same symptom only for the literal built-in /anthropic prefix; this PR generalizes to any custom pass_through_endpoints prefix and is orthogonal to that fix (neither conflicts with the other regardless of merge order)

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

…custom pass_through_endpoints Custom pass_through_endpoints entries from config.yaml are registered during proxy startup, strictly after every built-in router (including the generic /{provider}/v1/files and /v1/batches routes) is mounted at module-import time. Since they're always appended to app.routes, the generic native-provider routes always match first regardless of the configured prefix, misinterpreting it as a provider name. SafeRouteAdder now repositions a newly-added route immediately before the first route whose path template contains "{provider}" -- the shared marker for every such generic route, present and future -- so a custom pass-through path always wins the match instead. Fixes BerriAI#37925

CLAassistant · 2026-08-23T10:03:54Z

All committers have signed the CLA.

greptile-apps · 2026-08-23T10:09:23Z

Greptile Summary

The PR reorders newly registered custom pass-through routes so they take precedence over generic provider file and batch routes.

Adds a SafeRouteAdder helper that places custom routes before the first generic /{provider}/... route.
Adds route-resolution coverage for custom file and batch endpoints while checking representative native routes.

Confidence Score: 4/5

The PR appears safe to merge after addressing the non-blocking shared-route-list mutation concern.

The intended custom-route precedence is covered without an established runtime regression, but the implementation directly mutates FastAPI’s shared route collection contrary to the repository’s source-code convention.

Files Needing Attention: litellm/proxy/pass_through_endpoints/pass_through_endpoints.py

Important Files Changed

Filename	Overview
litellm/proxy/pass_through_endpoints/pass_through_endpoints.py	Adds custom-route precedence handling; the behavior is focused, but it directly mutates the shared FastAPI route list contrary to repository guidance.
tests/test_litellm/proxy/pass_through_endpoints/test_llm_pass_through_endpoints.py	Adds isolated route-resolution regression coverage without weakening existing assertions or making network calls.

_{Reviews (1): Last reviewed commit: "fix(proxy): stop /{provider}/v1/files an..." | Re-trigger Greptile}

greptile-apps · 2026-08-23T10:09:26Z

+                routes.pop()
+                routes.insert(index, new_route)


Shared route-list mutation

The helper directly mutates the shared app.routes collection with pop() and insert(), making route ownership and future registration changes harder to reason about and maintain.

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fixed — reassign app.router.routes wholesale (built via one unpacking expression) instead of pop()/insert() on the existing list, per your CLAUDE.md's no-in-place-mutation convention. See commit a099e80.

codecov · 2026-08-23T10:11:57Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…)/insert() Addresses Greptile review feedback on BerriAI#38017: avoid in-place mutation of the shared FastAPI route list. Builds the reordered route list via unpacking and reassigns app.router.routes wholesale, rather than popping the appended route and inserting it back in place.

…ric_provider_routes Addresses Codecov patch-coverage gap on BerriAI#38017: the documented safe no-op when no "/{provider}/..." route exists (e.g. a minimal deployment) was previously untested, since the real production app always has one registered.

IdoPort · 2026-08-23T10:16:24Z

Added a focused unit test for the no-generic-route fallback branch (commit 6f2b7bd) — that path was previously untested since the real production app always has a generic /{provider}/... route registered.

… fix - routes/new_route: Final, closing the LIT010 rebind-openness gap - # mutable-ok / # rebind-ok on the app.router.routes reassignment: it necessarily constructs a list literal and mutates state reachable from the app parameter, since Starlette's own Router.routes must stay a real, appendable list for the framework's own route registration to keep working -- an immutable rewrite is not possible here, per CLAUDE.md's own last-resort carve-out for this case CI failure: LIT002 total exceeded budget by the 1 new violation this PR added (https://github.com/BerriAI/litellm/pull/38017/checks).

codspeed-hq · 2026-08-23T11:15:09Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing IdoPort:fix/pass-through-endpoints-shadowed-by-provider-routes (726a343) with litellm_internal_staging (d447be1)¹}

No successful run was found on litellm_internal_staging (f005afa) during the generation of this report, so d447be1 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

IdoPort mentioned this pull request Aug 23, 2026

[Bug]: Custom pass_through_endpoints prefixes for a non-default Anthropic-compatible host can never work for file uploads — no prefix satisfies both route precedence and custom_llm_provider resolution #37925

Open

1 task

greptile-apps Bot reviewed Aug 23, 2026

View reviewed changes

IdoPort added 2 commits August 23, 2026 13:12

Navigation Menu

Uh oh!

Uh oh!

fix(proxy): stop /{provider}/v1/files and /v1/batches from shadowing custom pass_through_endpoints - #38017

fix(proxy): stop /{provider}/v1/files and /v1/batches from shadowing custom pass_through_endpoints#38017
IdoPort wants to merge 4 commits into
BerriAI:litellm_internal_stagingfrom
IdoPort:fix/pass-through-endpoints-shadowed-by-provider-routes

IdoPort commented Aug 23, 2026

Uh oh!

CLAassistant commented Aug 23, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Aug 23, 2026

Important Files Changed

Uh oh!

greptile-apps Bot Aug 23, 2026

Uh oh!

IdoPort Aug 23, 2026

Uh oh!

codecov Bot commented Aug 23, 2026 •

edited

Loading

Uh oh!

IdoPort commented Aug 23, 2026

Uh oh!

codspeed-hq Bot commented Aug 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

IdoPort commented Aug 23, 2026

TLDR

User Flow

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Before (f005afa)

files

batches

After (26bcafc)

files

batches

Type

Caveats (if any)

Final Attestation

Uh oh!

CLAassistant commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Aug 23, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

greptile-apps Bot Aug 23, 2026

Choose a reason for hiding this comment

Uh oh!

IdoPort Aug 23, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

IdoPort commented Aug 23, 2026

Uh oh!

codspeed-hq Bot commented Aug 23, 2026

Merging this PR will not alter performance

Footnotes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Before (`f005afa`)

After (`26bcafc`)

CLAassistant commented Aug 23, 2026 •

edited

Loading

codecov Bot commented Aug 23, 2026 •

edited

Loading