Skip to content

fix(proxy): reject attaching strategy router config to non-router models (#39838) - #39896

Closed
amasen02 wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
amasen02:fix/reject-strategy-router-config-on-non-router-39838
Closed

amasen02 wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
amasen02:fix/reject-strategy-router-config-on-non-router-39838

Conversation

@amasen02

@amasen02 amasen02 commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Model-less PATCH requests could attach complexity router config to non-router deployments
  • Corrupted deployment rows and exhausted capability router slots

How it solves it:

  • Resolves effective model using incoming model falling back to existing params model
  • Validates strategy router parameters against the effective model and rejects non-router attachments
  • Requires model to be set when attaching strategy router settings

User Flow

Before: an operator sending a partial PATCH update to an existing deployment could attach router configuration to a non-router model

  1. Operator has an existing deployment with model "gpt-4o"
  2. Operator sends PATCH /model/update or /model/patch with litellm_params containing complexity_router_config and no model field
  3. Proxy accepts the update and attaches router configuration to a standard deployment, corrupting the router state

After: the proxy rejects attaching strategy router settings to non-router models during partial updates

  1. Operator sends the same partial PATCH with complexity_router_config to the existing "gpt-4o" deployment
  2. Proxy validates against the existing model and returns HTTP 400 error indicating complexity_router_config cannot be attached to a non-router deployment

Relevant issues

Fixes #39838

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Type

Bug Fix

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@amasen02
amasen02 requested a review from a team September 5, 2026 10:37
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR strengthens strategy-router validation for partial deployment updates by decrypting the stored model before deriving the effective model.

  • Rejects strategy-router settings attached to non-router deployments.
  • Allows configuration-only updates to existing encrypted auto-router deployments.
  • Adds regression coverage for non-router, missing-model, plaintext auto-router, and encrypted auto-router cases.
  • Adds the required Final annotations identified in the previous review.

Confidence Score: 5/5

The PR appears safe to merge; the prior findings are resolved and no new actionable failures remain.

Stored encrypted models are now decrypted before effective-model validation, while decryption failures safely retain the original value. The added tests cover the corrected encrypted-model path, and both previous threads were manually resolved after their underlying concerns were addressed.

Important Files Changed

Filename Overview
litellm/proxy/management_endpoints/model_management_endpoints.py Decrypts the stored deployment model before validating strategy-router settings, with graceful fallback when decryption fails.
tests/test_litellm/proxy/management_endpoints/test_model_management_endpoints.py Adds focused regression tests for partial strategy-router updates and fixes the previously missing Final annotations.

Reviews (2): Last reviewed commit: "fix(proxy): decrypt stored model when va..." | Re-trigger Greptile

Comment thread litellm/proxy/management_endpoints/model_management_endpoints.py Outdated
Comment thread tests/test_litellm/proxy/management_endpoints/test_model_management_endpoints.py Outdated
@codspeed

codspeed Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing amasen02:fix/reject-strategy-router-config-on-non-router-39838 (7e13fcb) with litellm_internal_staging (c52b537)

Open in CodSpeed

@codecov

codecov Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@amasen02

amasen02 commented Sep 5, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai

@amasen02

amasen02 commented Sep 5, 2026

Copy link
Copy Markdown
Contributor Author

Resolved upstream in #39674 via _effective_complexity_router_params() which decrypts the stored deployment model during partial updates and validates against �alidate_strategy_router_model_write(). Superseded by #39674; closing cleanly.

@amasen02 amasen02 closed this Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A model-less PATCH can attach complexity_router_config to a non-router deployment

2 participants