feat(router): reject with 429 when a deployment's max_parallel_requests slots are all in use - #41555

feat(router): reject with 429 when a deployment's max_parallel_requests slots are all in use#41555

yassin-berriai merged 6 commits into

mainBerriAI/litellm:mainfrom

litellm_max_parallel_requests_queue_sizeBerriAI/litellm:litellm_max_parallel_requests_queue_sizeCopy head branch name to clipboard