Skip to content

feat(agents): enforce aggregate agent budgets - #43229

Draft
joshua-berri wants to merge 9 commits into
litellm_agent_entra_identityfrom
litellm_agent_budgets
Draft

joshua-berri wants to merge 9 commits into
litellm_agent_entra_identityfrom
litellm_agent_budgets

Conversation

@joshua-berri

@joshua-berri joshua-berri commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Agent budgets needs an integrated gateway workflow

How it solves it:

  • Configures an aggregate budget on each agent
  • Enforces budgets during admission and deferred accounting
  • Preserves budget windows across resets and concurrent settlement

Depends on #43149; followed by #43230. Reconciles existing implementation and review fixes; no additional feature scope

Diff sizes are being refreshed after the reviewed repairs

User Flow

Before: the complete workflow below is unavailable at this PR's parent

  1. Configure an agent budget through the existing agent API
  2. Call the agent and inspect its accumulated spend
  3. Exhaust its budget and observe denied requests
  4. Reset the budget window and verify new requests and delayed charges

After: this PR provides the workflow through the existing gateway surfaces

  1. Configure an agent budget through the existing agent API
  2. Call the agent and inspect its accumulated spend
  3. Exhaust its budget and observe denied requests
  4. Reset the budget window and verify new requests and delayed charges

Relevant issues

Original complete implementation: #43149

Affected release

Linear ticket

Pre-Submission checklist

  • Existing implementation and review fixes are preserved
  • Meaningful tests remain with their owning behavior
  • Devin verification at the reconciled head is complete
  • Required CI and completed Codecov pass
  • Greptile, Veria and Bugbot reviews have no actionable findings
  • User has accepted the hosted UI workflow

Screenshots / Proof of Fix

Matched live gateway checks used a controlled A2A upstream. With a zero-budget path target and a different target in the JSON model field, the earlier edb6e0f gateway allowed three requests and contacted the upstream

At c49737d, all four conflicting-target cases returned 422 before contacting the upstream. Reversing the targets allowed the request and billed the path-selected agent. Eight focused target-resolution tests passed; the same tests exposed two behavioral failures on the parent

The current restack preserves this PR's own patch exactly. Previous-tip Codecov covered 161/161 reported changed executable lines, Greptile scored 5/5 and Veria reported no open concerns. Current-tip hosted results remain pending

Type

New Feature

Caveats

  • Part of the four-PR identity, budgets, SCIM and delegated-team stack; do not release the complete feature without fix(agents): preserve delegated team authorization context #43459
  • Current-tip verification and reviews remain pending; fresh Bugbot review is waived due to team quota
  • Optional selector cleanup is excluded and saved as an unpublished follow-up

Final Attestation

  • Current-tip CI, coverage and review gates are complete

@codecov

codecov Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@joshua-berri

Copy link
Copy Markdown
Contributor Author

@greptileai Please review 39ddd00: the split preserves identity boundaries, isolates feature dependencies, and adds regression coverage for confirmed authorization gaps.

@joshua-berri

Copy link
Copy Markdown
Contributor Author

bugbot run

Please review 39ddd00, including authorization boundaries, schema compatibility, failure handling, and regression tests for this split PR.

@greptile-apps

greptile-apps Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Critical risk] Adds database schema and budget enforcement for agents.

The PR appears safe to merge based on this review; no new actionable finding or outstanding previous finding remains.

Summary

This PR adds aggregate agent budgets across configuration, admission, spend settlement, window resets, and the dashboard. The only change since the previous review reformats agent rate-limit descriptor construction without changing its behavior. The previously raised cold-counter database-read finding was explicitly withdrawn and is not an open concern.

Reviews (13) · Last reviewed commit: "fix(agents): bill the path target on exp..."

Comment thread litellm/proxy/hooks/proxy_track_cost_callback.py Outdated
Comment thread litellm/proxy/agent_endpoints/managed_identity.py
Comment thread litellm/proxy/common_utils/reset_budget_job.py Outdated
Comment thread litellm/proxy/common_utils/reset_budget_job.py Outdated

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread litellm/proxy/auth/auth_checks.py
@joshua-berri

Copy link
Copy Markdown
Contributor Author

@greptileai Please review a67e88a: trusted billing, duration validation, budget-skip handling, and reset repairs. Shared reset concurrency limitations remain explicitly documented.

@joshua-berri

Copy link
Copy Markdown
Contributor Author

bugbot run

Please review a67e88a for verified billing, budget-skip behavior, validated durations, and reset repairs; documented concurrency caveats remain open.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread litellm/proxy/common_utils/reset_budget_job.py Outdated
@joshua-berri

Copy link
Copy Markdown
Contributor Author

Budget counters now follow separate windows. Two-worker Redis testing preserves concurrent spend and late settlement. @greptileai and @cursor review please recheck 1d67236

Comment thread litellm/proxy/db/spend_counter_reseed.py Outdated
@joshua-berri

Copy link
Copy Markdown
Contributor Author

@greptileai @veria-ai Please review 5b64f51. This reconciles the approved three-PR stack and prior fixes. Verify this complete scope against its immediate parent.

@joshua-berri

Copy link
Copy Markdown
Contributor Author

bugbot run

Please review 5b64f51, including prior findings and cross-feature interactions. The reconciled stack remains draft pending verification and user acceptance.

Comment thread litellm/proxy/agent_endpoints/auth/managed_authorization.py Outdated
@veria-ai

veria-ai Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 4 · PR risk: 0/10

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5b64f51. Configure here.

or existing.litellm_budget_table is None
or existing.litellm_budget_table.budget_reset_at != fields["budget_reset_at"]
)
else {}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lifetime budgets inherit window spend

Medium Severity

_budget_write only clears spend when a new budget_reset_at is stamped. Disconnecting a budget or switching it to a lifetime cap (no duration) sets spend_window to None but leaves the current window balance in place. That leftover window spend is then treated as lifetime usage, so a later lifetime cap can deny immediately even though prior reset windows were already zeroed away.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 5b64f51. Configure here.

@joshua-berri

Copy link
Copy Markdown
Contributor Author

@greptileai @veria-ai Please review the current tip and confirm resolution of the focused findings. Actor/target token settlement now reuses recorded reservation scopes; delegated tests use module-boundary isolation. No unrelated features or selector cleanup were added. Fresh verification runs in parallel with CI

@joshua-berri

Copy link
Copy Markdown
Contributor Author

@greptileai @veria-ai Current tip c49737d applies the exact Ruff formatting correction only; behavioral patches are unchanged. Focused delegated tests168pass, rate settlement2pass, and actual Entra before/after confirm grant-team model, route and budget denial. Please confirm current-tip review outcomes; remaining live controls and CI are pending

@joshua-berri

Copy link
Copy Markdown
Contributor Author

@greptileai @veria-ai Please review 91bc86a: own patch unchanged after restack. Live target-budget denial and focused regression tests pass.

Comment thread litellm/proxy/auth/auth_checks.py
@joshua-berri

Copy link
Copy Markdown
Contributor Author

@greptileai Please refresh the summary after your explicit withdrawal of the cold-counter database-read finding. Recovery behavior and the reviewed commit remain unchanged.

This branch was successfully deployed

1 active deployment
e2e-changed — 91bc86af Deployed Sep 27, 2026 by joshua-berri via oauth #1587
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant