fix(proxy): restore pre-config-wins handling of pass-through endpoints

[yuneng-berri]

TLDR

Problem this solves:

• Config pass-throughs with forward_headers: true stopped forwarding Authorization
• Only with STORE_MODEL_IN_DB=true, starting in v1.103.0
• UI create, update and delete of pass-throughs was rejected once the config declared any
• os.environ/ pass-through targets were sent upstream unresolved and returned 500

How it solves it:

• Puts pass-throughs back on their pre-refactor(proxy): make the config file win over the database #41779 handling, a targeted revert of that one key
• The config file no longer owns general_settings.pass_through_endpoints, so the DB reader sees DB rows only
• Each DB sync and every config reload again serve DB entries next to config entries on other paths
• A UI pass-through write re-applies that merge right away, so config routes keep serving until the next sync

User Flow

Before: a team whose backend validates the caller's JWT gets "token not found" for every request through a config pass-through

1. The admin declares a pass-through with forward_headers: true and auth: false in the config, with STORE_MODEL_IN_DB=true
2. A client sends POST https://litellm-domain/copilot with Authorization: Bearer <jwt>
3. The backend receives the request with no Authorization header and rejects it
4. The admin tries to add another pass-through on the Admin UI and gets a 400 saying the config owns the key

After: the same request reaches the backend with the caller's JWT, and UI-created pass-throughs work next to the config ones

1. Same config and env as before
2. The client sends the same POST https://litellm-domain/copilot with Authorization: Bearer <jwt>
3. The backend receives Authorization: Bearer <jwt> unchanged, right after boot and after every DB sync
4. A pass-through created on the Admin UI is saved, serves traffic, and is listed next to the config ones

Affected release

Regression in v1.103.0-rc.1, still present on v1.103.0 and main

Linear ticket

Resolves LIT-9020

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live proxy on Postgres with STORE_MODEL_IN_DB=True, PT_ENV_TARGET=http://127.0.0.1:9081/api/envtarget, and a local echo server on :9081 that returns the path and Authorization header it received. The same scenario script ran against both commits on a fresh database each time. Config:

model_list: []
general_settings:
  master_key: <master key>
  pass_through_endpoints:
    - path: "/copilot"
      target: "http://127.0.0.1:9081/api/copilot"
      forward_headers: true
      include_subpath: true
      auth: false
    - path: "/envtarget"
      target: "os.environ/PT_ENV_TARGET"
      auth: false
    - path: "/keyed"
      target: "http://127.0.0.1:9081/api/keyed"
      auth: true
      headers:
        litellm_user_api_key: "x-cfg-key"
    - path: "/defaultauth"
      target: "http://127.0.0.1:9081/api/defaultauth"
    - path: "/shared"
      target: "http://127.0.0.1:9081/api/shared-config"
      auth: false

Before (38b0762)

Config pass-through with forward_headers: true and auth: false

1. Send the caller's JWT right after boot and the first DB sync; the echo upstream reports which Authorization it received

   curl -s -X POST "http://localhost:4091/copilot" -H "Authorization: Bearer caller-jwt" -H "content-type: application/json" -d '{"q":1}'

   {"path": "/api/copilot", "authorization": null} [HTTP 200]
   

2. Same request after further DB sync cycles

   curl -s -X POST "http://localhost:4091/copilot" -H "Authorization: Bearer caller-jwt" -H "content-type: application/json" -d '{"q":1}'

   {"path": "/api/copilot", "authorization": null} [HTTP 200]
   

Config pass-through with an os.environ/ target

1. Call the pass-through whose target is os.environ/PT_ENV_TARGET

   curl -s -X POST "http://localhost:4091/envtarget" -H "content-type: application/json" -d '{}'

   {"error":{"message":"/os.environ/PT_ENV_TARGET","type":"internal_server_error","param":null,"code":"500"}} [HTTP 500]
   

UI-created pass-through next to the config ones

1. Create a pass-through through the Admin UI API

   curl -s -X POST "http://localhost:4091/config/pass_through_endpoint" -H "Authorization: Bearer $MASTER_KEY" -H "content-type: application/json" -d '{"path":"/snowflake","target":"http://127.0.0.1:9081/api/snowflake","auth":false}'

   {"detail":{"error":"general_settings key 'pass_through_endpoints' is set in the config file and cannot be changed here.", ...}} [HTTP 400]
   

2. Call it right away

   curl -s -X POST "http://localhost:4091/snowflake" -H "content-type: application/json" -d '{}'

   {"detail":"Not Found"} [HTTP 404]
   

3. List pass-throughs after a DB sync (summarized as path, is_from_config, auth)

   curl -s "http://localhost:4091/config/pass_through_endpoint" -H "Authorization: Bearer $MASTER_KEY"

   /copilot      is_from_config=false auth=false
   /defaultauth  is_from_config=false auth=true
   /envtarget    is_from_config=false auth=false target=os.environ/PT_ENV_TARGET
   /keyed        is_from_config=false auth=true
   /shared       is_from_config=false auth=false
   /shared       is_from_config=false auth=true
   /snowflake    is_from_config=false auth=false
   

After (b7fbc2f)

Config pass-through with forward_headers: true and auth: false

1. Send the caller's JWT right after boot and the first DB sync; the echo upstream reports which Authorization it received

   curl -s -X POST "http://localhost:4092/copilot" -H "Authorization: Bearer caller-jwt" -H "content-type: application/json" -d '{"q":1}'

   {"path": "/api/copilot", "authorization": "Bearer caller-jwt"} [HTTP 200]
   

2. Same request after further DB sync cycles

   curl -s -X POST "http://localhost:4092/copilot" -H "Authorization: Bearer caller-jwt" -H "content-type: application/json" -d '{"q":1}'

   {"path": "/api/copilot", "authorization": "Bearer caller-jwt"} [HTTP 200]
   

Config pass-through with an os.environ/ target

1. Call the pass-through whose target is os.environ/PT_ENV_TARGET

   curl -s -X POST "http://localhost:4092/envtarget" -H "content-type: application/json" -d '{}'

   {"path": "/api/envtarget", "authorization": null} [HTTP 200]
   

UI-created pass-through next to the config ones

1. Create a pass-through through the Admin UI API

   curl -s -X POST "http://localhost:4092/config/pass_through_endpoint" -H "Authorization: Bearer $MASTER_KEY" -H "content-type: application/json" -d '{"path":"/snowflake","target":"http://127.0.0.1:9081/api/snowflake","auth":false}'

   {"endpoints":[{"id":"<id>","path":"/snowflake","target":"http://127.0.0.1:9081/api/snowflake", ..., "auth":false, "is_from_config":false, ...}]} [HTTP 200]
   

2. Call it right away

   curl -s -X POST "http://localhost:4092/snowflake" -H "content-type: application/json" -d '{}'

   {"path": "/api/snowflake", "authorization": null} [HTTP 200]
   

3. List pass-throughs after a DB sync (summarized as path, is_from_config, auth)

   curl -s "http://localhost:4092/config/pass_through_endpoint" -H "Authorization: Bearer $MASTER_KEY"

   /copilot      is_from_config=true  auth=false
   /defaultauth  is_from_config=true  auth=true
   /envtarget    is_from_config=true  auth=false
   /keyed        is_from_config=true  auth=true
   /shared       is_from_config=false auth=true
   /snowflake    is_from_config=false auth=false
   

The same scenario, extended with UI update and delete, an unrelated /config/field/update write, /config/field/info, and a restart, matched pre-#41779 main call for call except one: a UI-created auth: false pass-through now serves right away instead of answering 401 until the next DB sync. It matched v1.102.2 for every config-only call

Type

🐛 Bug Fix

Caveats (if any)

Medium

• Restores pre-refactor(proxy): make the config file win over the database #41779 behavior for a path declared in both the config and the DB
  • The config entry serves the route, the DB entry's auth applies
  • Same as v1.102 and pre-refactor(proxy): make the config file win over the database #41779 main; a proper config vs DB resource model is a follow-up
• GET /config/pass_through_endpoint lists config headers with os.environ/ values resolved, same as v1.102

Low

• A UI-created auth: false pass-through now serves immediately; before it answered 401 until the next DB sync
• Not verified live with enable_jwt_auth plus SERVER_ROOT_PATH, the customer's exact auth setup
  • Covered without JWT auth: forwarding, env targets, UI CRUD, restarts, field info

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Changes proxy config resolution, pass-through route registration, and auth behavior when YAML and DB declare the same path; regression fix but affects request routing and admin settings APIs.

Overview
pass_through_endpoints is no longer config-owned when declared in YAML. The proxy treats it as a resource list (DB-primary): YAML no longer blocks Admin UI writes, and config reloads keep serving merged endpoint state instead of wiping runtime routes.

Serving model: DB-stored endpoints are registered from the database; config-file endpoints on other paths are merged in via _pass_through_endpoints_beside_db. Resolved config exposed to loaders includes that union. On the same path, the DB entry wins for what gets served (restores pre–#41779 behavior).

Lifecycle changes: _apply_pass_through_settings only re-initializes routes when the DB payload includes a list; deletes publish an empty DB slice while keeping config paths. General-settings create/update/delete for this field calls _serve_pass_through_endpoints immediately. GET /config/field/info uses _declared_general_setting so resource-list fields read from the DB row when appropriate.

Tests add broad integration coverage (forward_headers, env targets, UI CRUD, reload/sync) and update expectations where YAML+DB overlap now locks auth from the DB side.

^{Reviewed by Cursor Bugbot for commit e5ef842. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Confidence Score: 5/5

[High risk] Changes how pass-through endpoints are stored and resolved.

The latest changes appear safe to merge; no new actionable finding remains.

Summary

The PR separates config-declared pass-through endpoints from DB-owned settings, merges both sources for serving, and preserves active pass-through settings while a DB sync reads configuration. The latest commit adds the preservation step and a test for requests made during that read.

_{Reviews (6) · Last reviewed commit: "fix(proxy): keep pass-throughs served wh..."}

[veria-ai]

High: Pass-through overrides authorize the wrong upstream

For a YAML POST /shared targeting a private upstream (auth: true) and a DB replacement POST /shared targeting a public upstream (auth: false), the merged auth list permits anonymous requests, but initialize_pass_through_endpoints() still registers both entries and the first-match registry selects the YAML entry registered at startup. An unauthenticated attacker can now reach the private upstream with its configured credentials; apply the same DB-wins filtering to route registration and remove shadowed YAML registry entries before publishing the replacement authentication policy.

[yuneng-berri]

Pre-existing: pre-#41779 and v1.102 behave the same way. This PR reverts to that handling to fix a regression, so fixing route registration is a follow-up

[veria-ai]

PR overview

The PR restores pre-configuration precedence handling for pass-through endpoints in the proxy, updating how YAML and database endpoint definitions are combined.

One issue has been addressed, but an authorization mismatch remains between merged endpoint settings and route registration. When a public database endpoint overrides a private YAML endpoint at the same path, anonymous requests can still be routed to the private upstream using its configured credentials. This leaves a concrete authentication bypass open until registration follows the same override rules and removes shadowed entries.

Open issues (1)

• High: Pass-through overrides authorize the wrong upstream — litellm/proxy/proxy_server.py:7840

Fixed/addressed: 1 · PR risk: 8/10

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_/jwt-forwarding-auth-false-ed0a9a (2144aef) with main (ef6aa4a)¹}

Footnotes

1. No successful run was found on main (2b19ddb) during the generation of this report, so ef6aa4a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yuneng-berri]

@greptileai please re-review the latest commit

[yuneng-berri]

@greptileai please re-review the latest commits

[yuneng-berri]

@greptileai please re-review

[yuneng-berri]

@greptileai please re-review the latest commit

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[yuneng-berri]

@greptileai please re-review the latest commit

[yuneng-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit e5ef842. Configure here.}

[mateo-berri]

Lgtm. Thanks!