Fix early proxy request size enforcement

[oss-agent-shin]

Summary

general_settings.max_request_size_mb was only enforced after auth had already read and parsed the request body, so oversized inline base64 payloads could be buffered before rejection. This adds an outer ASGI middleware that rejects oversized Content-Length requests before downstream code runs and counts streamed request bytes when Content-Length is absent, while preserving the existing auth-level guard as a second check.

Repro

On the starting ref, the focused repro test failed during collection because there was no early ASGI request-size guard to reject before body parsing:

$ python3 -m pytest tests/proxy_unit_tests/test_request_size_limit_middleware.py -q
E   ModuleNotFoundError: No module named 'litellm.proxy.middleware.request_size_limit_middleware'

The new test covers the issue by asserting an oversized Content-Length request returns 413 without calling the downstream app, and a streamed request without Content-Length is cut off when the configured byte limit is exceeded.

Evidence

Terminal transcript fallback (no UI involved):

$ python3 -m pytest tests/proxy_unit_tests/test_request_size_limit_middleware.py tests/test_litellm/proxy/common_utils/test_http_parsing_utils.py -q
................................                                         [100%]
32 passed in 1.68s

$ python3 -m black --check litellm/proxy/middleware/request_size_limit_middleware.py tests/proxy_unit_tests/test_request_size_limit_middleware.py litellm/proxy/proxy_server.py
All done!
3 files would be left unchanged.

$ python3 -m ruff check litellm/proxy/middleware/request_size_limit_middleware.py tests/proxy_unit_tests/test_request_size_limit_middleware.py litellm/proxy/proxy_server.py
All checks passed!

Tests

• Added tests/proxy_unit_tests/test_request_size_limit_middleware.py for early Content-Length rejection and streamed-body byte counting without Content-Length.
• Ran python3 -m pytest tests/proxy_unit_tests/test_request_size_limit_middleware.py tests/test_litellm/proxy/common_utils/test_http_parsing_utils.py -q (32 passed).
• Ran python3 -m black --check litellm/proxy/middleware/request_size_limit_middleware.py tests/proxy_unit_tests/test_request_size_limit_middleware.py litellm/proxy/proxy_server.py.
• Ran python3 -m ruff check litellm/proxy/middleware/request_size_limit_middleware.py tests/proxy_unit_tests/test_request_size_limit_middleware.py litellm/proxy/proxy_server.py.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 40.62500% with 38 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
.../proxy/middleware/request_size_limit_middleware.py	40.62%	38 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR adds an outer ASGI middleware (RequestSizeLimitMiddleware) that enforces general_settings.max_request_size_mb before downstream auth code reads the request body, fixing a gap where large inline base64 payloads could be fully buffered before rejection.

• For requests with a Content-Length header, the middleware rejects immediately (no body bytes read) by comparing the declared size to the configured limit.
• For requests without Content-Length, a wrapped receive callable counts bytes as the ASGI stream is consumed, raising RequestEntityTooLarge on overflow and sending a 413 before any response has started.
• _mb_to_bytes correctly treats values <= 0 as None (disabled), so a misconfigured max_request_size_mb: 0 does not block all traffic. The existing auth-layer guard in auth_utils.py is preserved as a second line of defence.

Confidence Score: 5/5

The change is safe to merge; it adds an opt-in, premium-gated middleware that is inactive unless max_request_size_mb is explicitly configured.

The middleware correctly sits inside ServerErrorMiddleware in Starlette's stack, so RequestEntityTooLarge propagates up to the middleware's own except block rather than being swallowed into a 500 by Starlette. Both the Content-Length fast-path and the streamed byte-counting path are covered by the new tests. The _mb_to_bytes guard correctly handles zero and negative config values. No existing tests were modified and no real network calls appear in the new test file.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/middleware/request_size_limit_middleware.py	New ASGI middleware that correctly rejects oversized requests by Content-Length header before body buffering, and counts streamed bytes via a wrapped receive callable; _mb_to_bytes properly treats <= 0 as disabled (no-limit).
litellm/proxy/proxy_server.py	Registers RequestSizeLimitMiddleware at the end of the setup block; lambdas capture general_settings and premium_user by reference so runtime changes are respected.
tests/proxy_unit_tests/test_request_size_limit_middleware.py	Adds three unit tests covering Content-Length rejection (no body read, downstream not called), zero-limit disabling the guard, and streamed body byte-counting; all tests are mock-only with no real network calls.

_{Reviews (2): Last reviewed commit: "Address request size review feedback" | Re-trigger Greptile}

[ishaan-berri]

@greptile review this again