feat(proxy): fairness under load controls with fair queueing and admin UI

[devin-ai-integration]

Description

TLDR

Problem this solves:

• Batch jobs and dev experiments starve production traffic on a shared model
• Priority reservation exists but is config-only, invisible, and rejects instead of queueing
• Token usage is only counted after the response, so bursts overshoot

How it solves it:

• New Fairness under Load page: workload classes, reserved shares, saturation threshold, queue deadlines
• Settings saved to the DB via PUT /fairness/settings, applied live on every worker
• Estimated tokens reserved before dispatch, reconciled to actual usage after
• Over-limit requests wait in a per-model, per-class fair queue instead of getting a 429
• Deadline per class, lowered per request with x-litellm-max-queue-wait, cancelled on disconnect
• GET /fairness/status and the page show saturation, per-class usage, queue depth, waits, rejections
• Workload class dropdown on key and team create and edit forms

User Flow

Before: a proxy admin has no way to keep a support app responsive while a nightly batch job shares gpt-4o-mini, and there is no page or endpoint for it

1. They open http://localhost:4000/ui/ and expand Settings in the sidebar: Router Settings, Logging & Alerts, Admin Settings, Cost Tracking, UI Theme. Nothing about fairness or priorities
2. They send GET http://localhost:4000/fairness/status with the master key and get 404 {"detail":"Not Found"}
3. Six batch requests and two production requests hit POST http://localhost:4000/v1/chat/completions at once on a model with rpm: 4. All eight go straight to the provider; nothing protects the production share or tells the batch caller to slow down

After: the admin sets reserved shares and deadlines on a page, production keeps its share, batch waits its deadline and then gets a clear 429

1. They open http://localhost:4000/ui/ and expand Settings: a new Fairness under Load entry appears at http://localhost:4000/ui/?page=fairness
2. On that page they flip Enable fairness under load on, add class production (50%, max wait 30s) and class batch (25%, max wait 6s), leave the default pool at 25% with max wait 0s, threshold 50%, and click Save settings. A toast confirms and the same values come back from GET http://localhost:4000/fairness/settings
3. On Virtual Keys they create a key and pick production in the new Workload class dropdown, and another with batch. The Teams create and edit forms have the same dropdown
4. Six batch requests and two production requests hit POST http://localhost:4000/v1/chat/completions at once. Both production requests return 200 in under a second. Two batch requests return 200, the other four wait 6s and return 429 with Request for gpt-4o-mini could not be admitted within 6.0s (queue_deadline_exceeded). Priority: batch, waited 6.0s and fairness_reason plus queue_wait_seconds in the error body
5. A key with no class sends the same request while the model is saturated and gets 429 immediately with fairness_reason: capacity_exhausted
6. A batch caller adds x-litellm-max-queue-wait: 2 and gets its 429 after 2s instead of 6s
7. A batch caller that hangs up after 1.5s leaves no queued request behind; the status shows disconnected_total: 1
8. GET http://localhost:4000/fairness/status and the Live status table on the page show gpt-4o-mini at 100% saturation with reservations enforced, and per class the reserved rpm, in-window usage, queue depth, queued and admitted counts, average wait, and rejections split by capacity, queue full, deadline and disconnect

Linear ticket

Resolves LIT-8648

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Setup shared by both runs: Postgres, real OpenAI calls, this config, master key sk-1234

model_list:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY
      rpm: 4
general_settings:
  master_key: sk-1234
  database_url: os.environ/DATABASE_URL
  store_model_in_db: true

Keys used by the load cases (created through POST /key/generate, metadata.priority set to the class name; the unclassified key has no metadata):

for cls in production batch; do
  curl -s -X POST localhost:4000/key/generate -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
    -d "{\"key_alias\": \"fairness-$cls-2\", \"models\": [\"gpt-4o-mini\"], \"metadata\": {\"priority\": \"$cls\"}}"
done
curl -s -X POST localhost:4000/key/generate -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"key_alias": "fairness-unclassified-2", "models": ["gpt-4o-mini"]}'

Load script used in both runs ($BATCH, $PROD hold the keys above):

call() {  # $1=class $2=key $3=n
  start=$(date +%s.%N)
  out=$(curl -s -o /tmp/body_$1_$3 -w '%{http_code}' localhost:4000/v1/chat/completions \
    -H "Authorization: Bearer $2" -H 'Content-Type: application/json' \
    -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Reply with one word: ok"}],"max_tokens":5}')
  end=$(date +%s.%N)
  printf '%-10s #%s HTTP %s after %5.2fs  %s\n' "$1" "$3" "$out" "$(echo "$end - $start" | bc)" \
    "$(python3 -c "import json,sys; d=json.load(open('/tmp/body_$1_$3')); print(d.get('choices',[{}])[0].get('message',{}).get('content') or d['error']['message'][:160])")"
}
for i in 1 2 3 4 5 6; do call batch "$BATCH" $i & done
sleep 1
for i in 1 2; do call production "$PROD" $i & done
wait

Before (974e4f1)

Fairness endpoints

1. curl -s -w '\nHTTP %{http_code}\n' localhost:4000/fairness/status -H 'Authorization: Bearer sk-1234'
2. {"detail":"Not Found"} / HTTP 404
3. curl -s -w '\nHTTP %{http_code}\n' -X PUT localhost:4000/fairness/settings -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"enabled": true}'
4. {"detail":"Not Found"} / HTTP 404

Six batch plus two production requests on a 4 rpm model

1. Run the load script above
2. Every request is dispatched, nothing reserves a share for production or slows batch down:

batch      #1 HTTP 200 after  1.28s  Okay.
batch      #2 HTTP 200 after  1.48s  Okay.
batch      #3 HTTP 200 after  1.46s  Okay.
batch      #4 HTTP 200 after  3.10s  Okay.
batch      #5 HTTP 200 after  1.50s  Okay.
batch      #6 HTTP 200 after  1.79s  Sure!
production #1 HTTP 200 after  1.30s  Sure!
production #2 HTTP 200 after  1.90s  Understood.

Admin UI

1. Open http://localhost:4000/ui/ and expand Settings in the sidebar
2. No fairness entry exists

After (207989e)

The curl cases below were rerun against a proxy started at 207989e. The only commit after it, 31119bd, changes the disable-mid-wait and paused-admission paths in the limiter and queue, which the curl legs above do not exercise; their behavior is covered by the regression tests named under CI state. The UI screenshots were taken at a0d2e7f; the commits after it touch only the limiter registration, config-load order, the queue store, the one-token reservation floor and the disable lifecycle, not the dashboard

Fairness endpoints

1. Enable the feature and define two classes:

curl -s -X PUT localhost:4000/fairness/settings -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{
  "enabled": true,
  "saturation_threshold": 0.5,
  "default_max_queue_wait_seconds": 0,
  "workload_classes": [
    {"name": "production", "reserved_share": 0.5, "max_queue_wait_seconds": 30},
    {"name": "batch", "reserved_share": 0.25, "max_queue_wait_seconds": 6}
  ]
}'

2. Response (persisted: true means the row landed in the DB):

{
    "settings": {
        "enabled": true,
        "workload_classes": [
            {"name": "production", "reserved_share": 0.5, "max_queue_wait_seconds": 30.0, "description": null},
            {"name": "batch", "reserved_share": 0.25, "max_queue_wait_seconds": 6.0, "description": null}
        ],
        "default_reserved_share": 0.25,
        "default_max_queue_wait_seconds": 0.0,
        "saturation_threshold": 0.5,
        "saturation_check_cache_ttl": 60,
        "max_queue_depth_per_class": 100,
        "queue_poll_interval_seconds": 0.1
    },
    "persisted": true
}

Six batch plus two production requests on a 4 rpm model

1. Run the load script above
2. Production keeps its 2 rpm share and answers in about half a second. Batch gets its 1 rpm plus what it could borrow before the model saturated, and the other four wait their 6s deadline and get a 429 that says why:

batch      #1 HTTP 200 after  1.67s  Okay.
batch      #2 HTTP 200 after  1.68s  Okay.
production #1 HTTP 200 after  0.78s  OK
production #2 HTTP 200 after  1.31s  Okay.
batch      #6 HTTP 429 after  6.11s  Request for gpt-4o-mini could not be admitted within 6.0s (queue_deadline_exceeded). Priority: batch, waited 6.0s
batch      #5 HTTP 429 after  6.25s  Request for gpt-4o-mini could not be admitted within 6.0s (queue_deadline_exceeded). Priority: batch, waited 6.0s
batch      #4 HTTP 429 after  6.25s  Request for gpt-4o-mini could not be admitted within 6.0s (queue_deadline_exceeded). Priority: batch, waited 6.0s
batch      #3 HTTP 429 after  6.25s  Request for gpt-4o-mini could not be admitted within 6.0s (queue_deadline_exceeded). Priority: batch, waited 6.0s

3. curl -s localhost:4000/fairness/status -H 'Authorization: Bearer sk-1234' right after the run:

{
    "enabled": true,
    "limiter_active": true,
    "saturation_threshold": 0.5,
    "window_size_seconds": 60,
    "stats_window_seconds": 300,
    "models": [
        {
            "model_group": "gpt-4o-mini",
            "rpm": 4,
            "tpm": null,
            "saturation": 1.0,
            "enforcing_reservations": true,
            "current_requests": 4,
            "current_tokens": 59,
            "classes": [
                {"name": "production", "reserved_share": 0.5, "reserved_rpm": 2, "reserved_tpm": null, "current_requests": 2, "current_tokens": 29, "queue_depth": 0, "max_queue_wait_seconds": 30.0, "queued_total": 0, "admitted_after_wait_total": 0, "rejected_capacity_total": 0, "rejected_queue_full_total": 0, "rejected_deadline_total": 0, "disconnected_total": 0, "avg_queue_wait_seconds": 0.0},
                {"name": "batch", "reserved_share": 0.25, "reserved_rpm": 1, "reserved_tpm": null, "current_requests": 2, "current_tokens": 30, "queue_depth": 0, "max_queue_wait_seconds": 6.0, "queued_total": 4, "admitted_after_wait_total": 0, "rejected_capacity_total": 0, "rejected_queue_full_total": 0, "rejected_deadline_total": 4, "disconnected_total": 0, "avg_queue_wait_seconds": 6.001119019749694},
                {"name": "default", "reserved_share": 0.25, "reserved_rpm": 1, "reserved_tpm": null, "current_requests": 0, "current_tokens": 0, "queue_depth": 0, "max_queue_wait_seconds": 0.0, "queued_total": 0, "admitted_after_wait_total": 0, "rejected_capacity_total": 0, "rejected_queue_full_total": 0, "rejected_deadline_total": 0, "disconnected_total": 0, "avg_queue_wait_seconds": 0.0}
            ]
        }
    ]
}

Immediate rejection, per-request deadline override, client disconnect

1. Unclassified key while saturated (default pool has max wait 0):

curl -s -w '\nHTTP %{http_code} after %{time_total}s\n' localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $DEF" -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":5}'

{"error":{"message":"Model capacity reached for gpt-4o-mini. Priority: None, Rate limit type: requests, Model TPM: not configured, Model RPM: 4, Remaining: 0","type":"throttling_error","param":null,"code":"429","provider_specific_fields":{"error":"...","fairness_reason":"capacity_exhausted","queue_wait_seconds":0.0}}}
HTTP 429 after 0.009556s

2. Batch key asking to wait at most 2s instead of its class default of 6s:

curl -s -w '\nHTTP %{http_code} after %{time_total}s\n' localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $BATCH" -H 'x-litellm-max-queue-wait: 2' -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":5}'

{"error":{"message":"Request for gpt-4o-mini could not be admitted within 2.0s (queue_deadline_exceeded). Priority: batch, waited 2.0s","type":"throttling_error","param":null,"code":"429","provider_specific_fields":{"error":"...","fairness_reason":"queue_deadline_exceeded","queue_wait_seconds":2.0005337910006347}}}
HTTP 429 after 2.008724s

3. Batch client hangs up while queued, then read the status:

curl -s --max-time 1.5 localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $BATCH" -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":5}'; echo "curl exit $? (28 = timed out)"
sleep 1
curl -s localhost:4000/fairness/status -H 'Authorization: Bearer sk-1234' | python3 -c "
import json,sys
for c in json.load(sys.stdin)['models'][0]['classes']:
    print(c['name'], {k: c[k] for k in ('queue_depth','queued_total','rejected_capacity_total','rejected_deadline_total','disconnected_total')})"

curl exit 28 (28 = timed out)
production {'queue_depth': 0, 'queued_total': 0, 'rejected_capacity_total': 0, 'rejected_deadline_total': 0, 'disconnected_total': 0}
batch {'queue_depth': 0, 'queued_total': 6, 'rejected_capacity_total': 0, 'rejected_deadline_total': 5, 'disconnected_total': 1}
default {'queue_depth': 0, 'queued_total': 0, 'rejected_capacity_total': 1, 'rejected_deadline_total': 0, 'disconnected_total': 0}

Turning it off live

1. With fairness on, six unclassified calls at once on the 4 rpm model: the default pool has no queue, so overflow gets 429. Then PUT /fairness/settings with {"enabled": false} and repeat the six calls:

bash 06_disable.sh   # same hit() loop as above with the unclassified key, run before and after the PUT

# 1. enable fairness (production 50%, batch 25%, threshold 50%)
{'enabled': True, 'saturation_threshold': 0.5}
# 2. six unclassified calls at once on rpm: 4; the default pool has no queue so overflow is 429
#1 HTTP 429 after 0.12s  Priority-based rate limit exceeded. Model: gpt-4o-mini, Priority: None, Rate limit type: requests, Model TPM: 
#2 HTTP 429 after 0.12s  Priority-based rate limit exceeded. Model: gpt-4o-mini, Priority: None, Rate limit type: requests, Model TPM: 
#3 HTTP 200 after 0.77s  Okay!
#4 HTTP 429 after 0.12s  Priority-based rate limit exceeded. Model: gpt-4o-mini, Priority: None, Rate limit type: requests, Model TPM: 
#5 HTTP 429 after 0.12s  Priority-based rate limit exceeded. Model: gpt-4o-mini, Priority: None, Rate limit type: requests, Model TPM: 
#6 HTTP 200 after 0.80s  Okay.
# 3. disable fairness live through the same endpoint
{'enabled': False}
status {'enabled': False, 'limiter_active': False}
# 4. same six calls right away: with the limiter paused every call reaches the provider
#1 HTTP 200 after 0.63s  Okay.
#2 HTTP 200 after 0.63s  Okay.
#3 HTTP 200 after 0.52s  Okay
#4 HTTP 200 after 0.85s  Okay.
#5 HTTP 200 after 1.79s  Okay.
#6 HTTP 200 after 1.46s  Sure!

Admin UI

1. Open http://localhost:3000/ (dashboard dev server against the proxy above), expand Settings in the sidebar: Fairness under Load is the new entry

2. Click it. The Live status table shows gpt-4o-mini with its saturation badge and one row per class with reserved rpm, in-window usage, queue depth, waits and the rejection breakdown from the run above

3. Turn on Enable fairness under load, set production 50% with max wait 30s and batch 25% with max wait 6s, click Save settings. A toast confirms, and after a reload the same values are still there

4. Push the shares past 100% and Save is refused with a validation message

5. On Virtual Keys, Create Key and Edit Key show a Workload class dropdown with the classes from the page. Pick batch, save, reopen the key: batch is still selected. Teams create and edit have the same dropdown

Type

🆕 New Feature

Caveats (if any)

Severe

• Turning the feature on changes admission for every model with rpm or tpm set: over-limit requests wait up to the class deadline instead of failing fast
• Reserved shares only apply while LITELLM_LICENSE is set (existing priority reservation behavior); without it every class gets the default weight

Medium

• Queues, wait stats and rejection counters live in each worker's memory; with several workers each queue is per worker while the rpm/tpm counters stay shared through Redis
• The saturation reading is cached (default 60s), so a burst landing on an idle model can borrow past its share before enforcement starts, as the batch Enable model / call timeouts #2 and Add support for replicate [code in ticket] #4 successes above show
• x-litellm-max-queue-wait can only shorten the wait; a longer value is clamped to the class deadline
• Queued requests re-check admission on a poll interval (default 100ms) rather than being woken by a release

Low

• Token reservation uses the request's estimated prompt plus max_tokens, so a class with large max_tokens values reserves more than it will usually spend until the response reconciles it
• The default pool cannot be given its own name; it is always default, and both default and default_pool are rejected as workload class names because the default pool's counter key uses the latter

CI state

• Bugbot round on cbc82e1 raised two findings, both fixed in 207989e: contentless requests now reserve at least one token like the v3 limiter does, and turning fairness off pauses the limiter it registered instead of removing it, so requests admitted before the switch still settle their reservation. Both have regression tests in the mapped test files
• Greptile and Bugbot rounds on 207989e raised one finding each, both fixed in 31119bd: a request still waiting in the fair queue when fairness is switched off is released as QueueReleased without taking a reservation instead of sitting until its deadline, and a request admitted while the limiter was paused is marked dynamic_admission_bypassed on its request stash so the success callback never bills it even if fairness is re-enabled before it finishes. Both have regression tests, and reverting either fix makes its test fail
• mcp-integration / Run tests failed on 207989e on test_call_tool and test_sse_mcp_handler_mock, which also fail on main at 636eb4c (run 36181414493) and on the merge base; it passes on the current head and this branch does not touch MCP code or tests
• 1f08c3b merges main into the branch (clean merge, no conflicts); the mapped tests and make lint pass on it
• Bugbot round on 1f08c3b raised three findings, all fixed in f19f5a1: a request admitted and then rejected by a later proxy hook now refunds its reservation through async_post_call_failure_hook, a request that held a reservation never falls through to the legacy token increment after settlement, and default_pool is rejected as a workload class name alongside default. Each has a regression test in the mapped test file that fails with the fix reverted
• Greptile round on f19f5a1 raised one finding, fixed in 93e7c6e: when a post-call guardrail rejects a response after the provider succeeded, the failure hook has already refunded the reservation, so the success callback now bills the actual usage as a plain increment instead of skipping it. Regression test fails with the helper stubbed out
• Greptile round on 93e7c6e raised one finding, fixed in 478f7c8: the request stash now records how many tokens a settlement already charged, so a success that lands after a partial-usage settlement bills only the uncounted remainder instead of the full response usage again. Regression test fails when the subtraction is removed
• Greptile round on 478f7c8 raised one finding, fixed in a0e7aba: the settled token count is now recorded together with the settled flag before the counter write, so a success callback racing an in-flight settlement bills only the remainder. Regression test drives the race with a blocked write and fails when the assignment moves back after the write
• Greptile scored a0e7aba 5/5. Bugbot's round on it raised one finding, fixed in 6aff9db: a failed reservation write now rolls back only the settle's own claim, so a billing record left by a concurrent success survives. Regression test fails when the rollback is unconditional
• Greptile round on 6aff9db raised one finding, fixed in 511e689: settlement and post-settlement billing now take a per-request lock, so a settle whose write fails rolls back and the success callback waiting on the lock settles the reservation at actual usage itself instead of billing on top of a stale claim. This replaces the compare-and-restore guards. Regression test fails when the lock is removed from the settle path
• Analyze (python) (CodeQL) stops at its 2 GiB result-set limit in CWE-117/LogInjection.ql with zero alerts, as on recent main pushes; it is not a required check
• oauth (MCP OAuth happy path) stays in the waiting state; it does the same on recently merged main PRs and is not a required check

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/594d6f8da0b942c6a1a05db3a3974388
Open in Devin Desktop: https://app.devin.ai/desktop/session/594d6f8da0b942c6a1a05db3a3974388?variant=devin
Requested by: @yassin-berriai

---

Note

High Risk
Changes core request admission, token reservation, and rate-limit behavior for every model with RPM/TPM when fairness is enabled; misconfiguration or queue bugs could delay or reject production traffic cluster-wide.

Overview
Adds fairness under load for the proxy: admins configure workload classes (reserved RPM/TPM shares, queue deadlines, saturation threshold) via GET/PUT /fairness/settings and GET /fairness/status, persisted as fairness_settings and synced across workers like other litellm_settings.

When enabled, apply_fairness_settings mirrors classes into priority_reservation, registers or pauses dynamic_rate_limiter_v3, and the limiter reserves estimated tokens at admission, settles/refunds on success or failure, and on capacity pressure queues over-limit traffic in a weighted per-model FairQueue instead of an immediate 429. Rejections expose fairness_reason, queue wait, and optional x-litellm-max-queue-wait; ActiveRequestMiddleware lets waiters abort on client disconnect.

The dashboard gains a Fairness settings form and live status table; the v3 parallel limiter stash and several helpers are extended/refactored to support dynamic reservations and shared token accounting.

^{Reviewed by Cursor Bugbot for commit 511e689. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fairness_under_load (511e689) with main (1e6c983)¹}

Footnotes

1. No successful run was found on main (115668f) during the generation of this report, so 1e6c983 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[greptile-apps]

Confidence Score: 5/5

[High risk] Adds fairness queueing and rate limiting to the proxy request path.

The PR appears safe to merge based on the reviewed changes and resolved prior threads.

Summary

This PR adds configurable workload-class reservations, fair queueing, token reservation and settlement, status endpoints, and an admin UI. Since the previous review, it serializes per-request settlement and billing and strengthens the associated race tests.

_{Reviews (18) · Last reviewed commit: "fix(proxy): serialize reservation settle..."}

[codecov]

Codecov Report

❌ Patch coverage is 95.97855% with 30 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/hooks/dynamic_rate_limiter_v3.py	92.21%	20 Missing ⚠️
litellm/proxy/hooks/fairness_stats.py	93.65%	4 Missing ⚠️
litellm/proxy/common_utils/active_request.py	83.33%	2 Missing ⚠️
litellm/proxy/hooks/fairness_queue.py	98.40%	2 Missing ⚠️
litellm/proxy/hooks/fairness_settings.py	98.73%	1 Missing ⚠️
litellm/proxy/proxy_server.py	96.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

@greptileai review

[yassin-berriai]

@greptileai review

[yassin-berriai]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[greptile-apps]

Comments Outside Diff

These findings sit on lines the diff does not cover, so they could not be posted inline. Each one leaves this list once its file changes.

•  Reserved name passes form validation ui/litellm-dashboard/src/app/(dashboard)/fairness/_components/schema.ts:5529 ▶

  The backend now reserves default_pool, but this form rejects only default. An admin can submit default_pool as a workload-class name and receive a server-side 422 instead of an inline validation message, costing them an avoidable failed save.

[mateo-berri]

@greptileai review

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai review

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai review

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai review

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai review

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 511e689. Configure here.}

[devin-ai-integration]

Runtime verification on 511e689

Passed persistence, validation, key/team assignment, live counters, and one-second deadlines with real completions: two 200s, two 429s