refactor(e2e): bucket rate limits, budgets, and spend tracking under quota_management

[mateo-berri]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

This PR only changes the e2e coverage registry (the denominator YAMLs, their schema, and covers markers on existing tests); it adds no proxy behavior, so the proof is the registry tooling itself running against the tree at commit d37daf0. --strict performs a full pytest collect-only pass over tests/e2e, so it proves every touched suite still imports and every marker (including the 27 newly wired ones) resolves to a registry cell:

$ cd tests/e2e && uv run python -m coverage_registry.collector --strict
MODULE                             COVERED    COVERAGE
Core LLMs                             6/53       11.3%
Non-Core LLMs                         0/44        0.0%
MCPs                                  0/14        0.0%
Management/UI                        11/67       16.4%
Reliability & Performance             0/25        0.0%
Quota Management                     27/33       81.8%
Logging & Guardrails                  4/51        7.8%
Other                                 1/26        3.8%
------------------------------------------------------
ALL                                 49/313       15.7%

$ uv run pytest coverage_registry/test_collector.py -q
10 passed in 0.07s

$ uv run --no-sync basedpyright tests/e2e
0 errors, 0 warnings, 0 notes

Type

🧹 Refactoring
✅ Test

Changes

This introduces a quota_management module in the e2e coverage registry, bucketing rate limits, budgets, and spend tracking into one dashboard category (agreed with the team). Rate limiting previously sat under reliability.* while budgets and spend tracking had no behavior rows at all; their suites predate the registry and carried no covers markers, so their coverage was invisible to the dashboard

schema.py gains a QuotaCell type (behavior: ratelimit | budget | spend_tracking) in the discriminated union, plus the Quota Management dashboard rollup and its quota_management Loki label. The six ratelimit rows move from reliability.yaml into the new quota_management.yaml with ids renamed from reliability.ratelimit.* to quota_management.ratelimit.*; the source on the two enforcement rows is corrected from dynamic_rate_limiter_v3.py to parallel_request_limiter_v3.py, where key-scope rpm/tpm enforcement actually lives, and two ratelimit cells are added for window reset and pacing headers (their covering tests land in the stacked PR #32914)

The rest of the bucket is seeded from the existing suites: 15 budget rows (entity enforcement for key, internal user, end user, organization, and team member; tag budgets; per-model model_max_budget isolation; soft budget; key and team multi-window blocks-then-resets; key and team-member window resets; budget fallback rerouting; the spend-counter reseed guard) and 12 spend_tracking rows (nonzero cost rows for chat, streaming, and embeddings; zero-cost cache hits; key rollup vs sum of logs; the concurrent-burst lost-spend guard; tag and end-user attribution; per-model rows; failure rows; /spend/calculate; spend-logs v2 pagination). Every seeded row is claimed by an existing test via @pytest.mark.covers, and the two budget CRUD tests also claim their existing mgmt.budget.* cells. Following review feedback, the spend_calculate row's exercised_on names the management route its covering test actually drives (spend_calculate) rather than chat_completions

An earlier revision carried a drive-by fix for tests/e2e/logging/logging_client.py, which was red on the zero-error e2e basedpyright gate at the then-staging tip; latest litellm_internal_staging ships its own fix for that file, so the rebase drops the commit and this PR is back to registry changes only

The suite folders follow the registry: tests/e2e/budgets/ and tests/e2e/spend_tracking/ move under tests/e2e/quota_management/ as one subfolder per behavior (their files are otherwise untouched apart from the covers markers; the session-finish spend-log cleanup path in tests/e2e/conftest.py follows the move), the naming grammar in tests/e2e/CLAUDE.md and the registry README document the new module, and the router/ suite-folder description drops "rate limits" since those tests now belong to quota management (PR #32914 adds them in quota_management/ratelimit/). That stacked PR covers the remaining ratelimit cells

[greptile-apps]

Greptile Summary

This PR refactors the e2e coverage registry by introducing a quota_management module that buckets rate limits, budgets, and spend tracking under one dashboard category, and wires 27 existing tests to their registry cells via @pytest.mark.covers markers. It also carries a drive-by fix for five basedpyright gate errors in logging_client.py that were introduced on the staging tip independently of this branch.

• New quota_management.yaml adds 33 registry cells (6 ratelimit, 15 budget, 12 spend_tracking); the 6 ratelimit rows migrate from reliability.yaml with corrected source paths (parallel_request_limiter_v3.py for key-scope RPM/TPM, dynamic_rate_limiter_v3.py retained for priority rows).
• schema.py adds QuotaCell to the discriminated union with a typed behavior literal; PREFIX_ROLLUP, MODULE_ORDER, and LOKI_MODULE_LABELS are updated consistently.
• logging_client.py switches LangfuseListParams from alias + populate_by_name to serialization_alias (correct for a write-only request-param model), and replaces the raw json.loads / isinstance chain in completion_response_id with a typed pydantic model — both are straightforward correctness fixes.

Confidence Score: 5/5

Safe to merge — all changes are confined to the e2e test tree (registry metadata, test markers, and a test-helper lint fix); no proxy behaviour is touched.

The diff is entirely within tests/e2e/: YAML registry definitions, additive @pytest.mark.covers markers on existing tests, a new QuotaCell schema type, and two small correctness fixes in a test helper. None of these paths affect runtime proxy logic.

No files require special attention; the most substantive changes are schema.py and quota_management.yaml, both validated by the collector tests and basedpyright gate reported in the PR description.

Important Files Changed

Filename	Overview
tests/e2e/coverage_registry/schema.py	Adds QuotaCell to the discriminated union, registers the quota_management prefix rollup and Loki label, and updates MODULE_ORDER; all additions are consistent with existing patterns.
tests/e2e/coverage_registry/quota_management.yaml	New file with 33 registry cells across ratelimit, budget, and spend_tracking behaviors; source corrections applied (parallel_request_limiter_v3.py for key-scope RPM/TPM enforcement), and spend_calculate exercised_on fixed to use management route name per prior review feedback.
tests/e2e/coverage_registry/reliability.yaml	Removes four ratelimit rows that have moved to quota_management.yaml; remaining rows are unaffected.
tests/e2e/logging/logging_client.py	Fixes two basedpyright gate errors: switches LangfuseListParams to serialization_alias (correct for a request-param model), and replaces raw json.loads + dict check in completion_response_id with a typed pydantic model that handles both parse errors and structural mismatches via a single ValidationError catch.
tests/e2e/budgets/test_budget_enforcement_e2e.py	Wraps each parametrized budget-enforcement case in pytest.param with a covers marker; no assertion changes.
tests/e2e/spend_tracking/test_spend_tracking_e2e.py	Adds @pytest.mark.covers markers to all existing spend-tracking tests; no assertion or logic changes.
tests/e2e/CLAUDE.md	Documents the new quota_management module with its naming grammar; removes ratelimits from the router/ description since those tests now belong to quota_management.
tests/e2e/budgets/test_budget_reset_advances_e2e.py	Adds covers markers to four test functions covering key spend blocking, window resets, multi-window independence, and team-member reset; no assertion changes.

_{Reviews (2): Last reviewed commit: "fix(e2e): keep logging_client within the..." | Re-trigger Greptile}

[greptile-apps]

exercised_on metadata inaccurate for management-API cells

Both spend_calculate.returns_cost and pagination.keeps_total list exercised_on: [chat_completions], but neither test exercises a chat-completions LLM endpoint — test_spend_calculate_returns_nonzero_cost calls /spend/calculate directly and test_spend_logs_v2_pagination_caps_pages_and_keeps_total calls /spend/logs. If the collector or a future dashboard slice filters cells by exercised_on, these rows will appear in the wrong bucket. The field should reflect the actual proxy endpoint exercised (e.g. a dedicated literal, or the management surface pattern used by mgmt.* cells).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_quota_management_registry (d37daf0) with litellm_internal_staging (f2fb6b8)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (0710cf2) during the generation of this report, so f2fb6b8 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai The spend_calculate row's exercised_on now names the management route its covering test actually drives (spend_calculate, following the non-LLM vocabulary the mcp_operations row in guardrail.yaml already uses); fixed in bff4ad2. The pagination row keeps chat_completions on purpose: that test drives two real chat calls to seed the spend rows it paginates, so the LLM endpoint genuinely is exercised

[greptile-apps]

Both calls look right. The spend_calculate row in quota_management.yaml now correctly reads exercised_on: [spend_calculate], consistent with the non-LLM vocabulary used by the mcp_operations row in guardrail.yaml. And keeping exercised_on: [chat_completions] on the pagination row is accurate — the test genuinely exercises the chat endpoint to seed the rows it then paginates, so that endpoint belongs in the field.

The source corrections on the two enforcement rows (dynamic_rate_limiter_v3.py → parallel_request_limiter_v3.py) are also correct; the v3 limiter's per-key/team/model RPM and TPM enforcement lives in parallel_request_limiter_v3.py, not dynamic_rate_limiter_v3.py.

[mateo-berri]

@greptileai rebased onto latest litellm_internal_staging (which brings the new zero-error e2e basedpyright gate) and fixed the 5 gate errors in logging_client.py that the staging tip itself carries, so lint is green here

[mateo-berri]

@greptileai budgets/ and spend_tracking/ suites now live under tests/e2e/quota_management/ (one subfolder per registry behavior); the conftest spend-cleanup path follows the move

[greptile-apps]

The restructuring looks clean. The conftest.py path update is correct:

- spend_dir = str(Path(__file__).parent / "spend_tracking")
+ spend_dir = str(Path(__file__).parent / "quota_management" / "spend_tracking")

One thing worth noting: test_spend_routes.py was moved with similarity index 100% — no @pytest.mark.covers markers were added. If there are registry rows it should claim, they'd remain uncovered. Is that intentional (i.e., those rows land in a stacked PR), or does test_spend_routes.py not map to any current registry cells?