docs(proxy): document per-worker admission control settings and metrics

[devin-ai-integration]

Summary

Documents the per-worker admission control shipped in BerriAI/litellm#39352 (LIT-6561). Adds a "Per-worker admission control" section to docs/proxy/server_tuning.md covering the three general_settings keys (max_in_flight_requests_per_worker, max_queued_requests_per_worker, admission_queue_timeout_seconds), the 503 overloaded_error response with retry-after: 1, the probe paths that bypass the gate, how the cap multiplies with --num_workers, replicas and an HPA, and how it complements the Redis-backed global_max_parallel_requests. Also adds the three settings to the general_settings table in config_settings.md and the litellm_admission_* gauges and counter to the pod health metrics table in prometheus.md

Merge after the code PR lands

Link to Devin session: https://app.devin.ai/sessions/3ad09a09495b482781727a767412f13c
Open in Devin Desktop: https://app.devin.ai/desktop/session/3ad09a09495b482781727a767412f13c?variant=devin
Requested by: @yassin-berriai

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
litellm	Ready	Preview	Sep 2, 2026 4:36pm UTC