fix(router): treat malformed configured token limits as absent on /v1/models

[yuneng-berri]

Relevant issues

Hardening follow-up to #33721. That change sources /v1/models token limits from the cost map with a per-deployment configured-limit override, but the override path ran a bare int() on the admin-configured model_info values. A single deployment configured with a non-numeric limit (for example "128,000", an empty string, "unlimited", or a YAML list) made the conversion raise inside the per-model listing loop, so the entire GET /v1/models response returned 500, taking well-formed sibling models down with it. Before #33721 the same configuration degraded to a response without limits, so this restores that contract while keeping the O(1) lookup and the configured-limit override

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Live proxy with this model_list entry in the config:

  - model_name: bad-limit-model
    litellm_params:
      model: openai/gpt-4o
      api_key: sk-fake
    model_info:
      max_input_tokens: "128,000"

Before, at d495da4ce4 (staging tip without this fix):

$ curl -s -o /dev/null -w 'HTTP %{http_code}\n' http://localhost:4001/v1/models -H 'Authorization: Bearer sk-1234'
HTTP 500
$ curl -s http://localhost:4001/v1/models -H 'Authorization: Bearer sk-1234'
{"error":{"message":"Internal server error","type":"internal_server_error"}}

After, same request with the fix applied (captured at the PR commit):

$ curl -s -o /dev/null -w 'HTTP %{http_code}\n' http://localhost:4001/v1/models -H 'Authorization: Bearer sk-1234'
HTTP 200
# per-model limits from the same response:
gpt-4o          in: 128000  out: 16384   (cost map)
custom-good     in: 32000   out: 8000    (configured model_info, override intact)
bad-limit-model in: None    out: None    (malformed value treated as absent)

Both new regression tests fail on the pre-fix code with the exact defect (ValueError: invalid literal for int() with base 10: '128,000' at litellm/router.py) and pass with it

Type

🐛 Bug Fix

Changes

Router.get_configured_token_limits now runs each configured limit through a safe int coercion: None, bools, and values that raise TypeError/ValueError on int() are treated as absent instead of propagating out of the /v1/models listing loop. Numeric strings (a YAML-quoted "32000") still coerce, and well-formed integer configs are unchanged. The docstring now states the malformed-value contract

Tests: test_get_configured_token_limits_treats_malformed_values_as_absent pins (None, None) for the malformed shapes at the Router level, test_get_configured_token_limits_coerces_numeric_strings pins the tolerant path, and test_create_model_info_response_survives_malformed_configured_limits exercises the full /v1/models enrichment with a real Router and asserts the base response comes back instead of an exception

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR hardens Router.get_configured_token_limits to tolerate malformed model_info token limit values (e.g. "128,000", "unlimited", lists, bools) that previously caused a bare int() to raise, crashing the entire /v1/models listing and returning HTTP 500. The fix wraps the coercion in a safe helper that returns None for any value that is not convertible to a sensible integer, restoring the pre-#33721 contract where bad configs degrade gracefully to "no limit" rather than taking all sibling models offline.

• Adds _as_int inside get_configured_token_limits to safely coerce model_info limit values, treating None, bool, and any TypeError/ValueError-raising values as absent while still coercing valid numeric strings like "32000".
• Adds three new targeted tests covering malformed value rejection, numeric-string coercion, and an end-to-end integration test through create_model_info_response with a real Router.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to a safe-coercion wrapper in a single method, restoring graceful degradation for misconfigured token limits without altering any other behavior.

The fix is minimal and correct: the bool guard is placed before the int() call (preventing True→1), numeric strings still coerce as intended, and all pre-existing code paths are unaffected. Tests are thorough and use only local Router initialization with no real network calls. No pre-existing tests were weakened.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/router.py	Adds safe _as_int coercion inside get_configured_token_limits; logic is correct, bool guard is properly placed before int(), and malformed values correctly return (None, None).
tests/test_litellm/test_router.py	Two new unit tests cover all targeted malformed-value shapes and the numeric-string coercion path; no real network calls.
tests/test_litellm/proxy/test_proxy_utils.py	New integration test exercises the full create_model_info_response path with a real Router and malformed limit, confirming the response returns without exception and omits the bad limit fields.

_{Reviews (1): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_tokenlimits_guard (f74adf5) with litellm_internal_staging (e238e89)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (3f9b71c) during the generation of this report, so e238e89 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩