fix: roll up the open deflake fixes for the MCP logging queue, PTU rollup, license gate, and pricing test isolation

[devin-ai-integration]

TLDR

Problem this solves:

• Four CI flakes, none caused by the commits they reddened
• Only one deflake PR may be open at a time
• Each hid a real ordering, network, or state bug

How it solves it:

• Rolls all four fixes onto this branch
• PTU rollup takes its router as an argument
• License lookups retry transient PyPI failures
• Pricing memo is invalidated around every unit test
• MCP tests drain queued logging before the next test

User Flow

Before: a contributor pushes a commit that touches none of this, and Code Quality, the MCP folder job, or the unit shards go red on someone else's leftover state

1. They push a commit to their branch and open a PR
2. Code Quality reaches the check_licenses step, PyPI resets one connection, and the step reports botocore-stubs (1.43.14) - Unknown license, tornado (6.5.7) - Unknown license
3. The MCP folder job reports an MCP cost assertion failure showing a chat-completion price, assert 1.35e-05 == 1.2, on a commit that never touched MCP
4. The unit shards report PTU rollup and pricing failures in files their commit never touched
5. They re-run the same commit with no change and everything goes green, so the red run told them nothing about their change

After: the same push only goes red on something the commit actually did

1. They push a commit to their branch and open a PR
2. The license step waits briefly and asks PyPI again, then reports All dependencies have acceptable licenses
3. The MCP folder job passes whatever order the MCP files run in, and a wrong MCP price still fails it
4. The PTU rollup and pricing tests pass whatever else ran ahead of them in the same worker
5. A red check now points at their change

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI evidence, mechanism, and fix per flake

2026-08-27: the license gate flaked again on a PyPI connection reset, same mechanism this PR already fixes

Flaky gate: the check_licenses step of Code Quality Checks

Evidence: https://github.com/BerriAI/litellm/actions/runs/33026421099 on devin_ai_fix_gemini_stream_billing_36042 at e8ec34c4, attempt 1 (job 98368723006) failed with Failed to fetch license for keyring 25.6.0: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) then keyring (25.6.0) - Unknown license; attempt 2 on the identical commit passed with no change in between

Mechanism and fix: identical to the 2026-08-20 entry below (single unretried requests.get to PyPI). The retry fix already on this branch covers it; that branch ran without this PR's fix. No new code was needed, this is additional evidence the flake is live until this PR merges

2026-08-24 and 2026-08-25: MCP cost assertions read another test's logging payload

Flaky tests: tests/mcp_tests/test_mcp_logging.py::test_mcp_cost_tracking (8 job failures in the window), ::test_mcp_cost_tracking_per_tool (6)

Evidence, LiteLLM MCP Tests (folder - tests/mcp_tests), same commit red on attempt 1 and green on the re-run: https://github.com/BerriAI/litellm/actions/runs/32813031488 at 312abf26a0 (attempt 1 job 97695992376 failed, attempt 2 passed), https://github.com/BerriAI/litellm/actions/runs/32799057588 at 735195ae66 (attempt 1 job 97656213844 failed, attempt 2 passed), https://github.com/BerriAI/litellm/actions/runs/32795604643 at a5c81e525b (attempts 1 and 2 failed, jobs 97646133301 and 97648109127, attempt 3 passed). Same failure across other branches in the window: jobs 97642202286, 97629426228, 97628673139, 97629416408, 97625705387, 97622992754, 97620683608, 97614424367, 97615597485, and https://github.com/BerriAI/litellm/actions/runs/32829113666 attempt 1

The attempt-1 log of run 32813031488 names the mechanism outright: the payload the MCP logger asserted on has 'call_type': 'acompletion', 'model': 'gpt-4o-mini' and 'response_cost': 1.35e-05, a chat-completion payload from test_mcp_chat_completions.py, while the MCP payload with 'response_cost': 1.2 is logged in the same run

Mechanism, async teardown race across event loops. Callbacks are handed to the process-wide logging worker, whose queue is bound to the loop that created it, and the MCP conftest gives every test a fresh loop. Anything a test leaves queued is carried onto the next test's loop and dispatched there, so it reaches the callbacks that test registered. tests/mcp_tests runs on 4 xdist workers, so whether the earlier chat-completion callback landed inside a cost test depended on the split

Fix: the MCP conftest drains the worker's queue when each test ends, so a test's own callbacks run in its own loop and nothing is left to carry. No assertion changed, and no marker, rerun, skip, or sleep was added

2026-08-21: PTU rollup read the proxy's router off a module global

Flaky tests: tests/test_litellm/proxy/spend_tracking/test_ptu_flat_cost_rollup.py::test_a_database_only_run_sweeps_exactly_as_it_did_before, ::test_every_deployment_that_prices_is_inside_the_set_that_bounds_the_prune, ::test_rollup_prunes_stale_row_when_config_is_gone

Evidence, same branch and adjacent commits with opposite results: https://github.com/BerriAI/litellm/actions/runs/32352053117 on litellm_internal_staging at 6fcdea03b0, attempt 1, proxy-infra, all three failed, and that merge touched only the pricing JSON and an anthropic passthrough handler. The staging pushes either side of it passed: https://github.com/BerriAI/litellm/actions/runs/32338671277 at 5290150a05 and https://github.com/BerriAI/litellm/actions/runs/32357044632 at 952c6d5675. The same three also failed on unrelated branches, https://github.com/BerriAI/litellm/actions/runs/32352326757 on litellm_cli_refresh_tokens at b6fef179ff and https://github.com/BerriAI/litellm/actions/runs/32451279739 on litellm_fix_raw_key_log_persistence at fb417a5563 attempts 1 and 2, where the output shows a scanned deployment id no test in that file creates, 8792e271b8b561315c5c5471556a09088dde51606187906acdb3c3f8be10e275, a router-generated hash left on the proxy module by something else in the worker

Mechanism, shared module-level state between tests. _running_router() read litellm.proxy.proxy_server.llm_router out of sys.modules, so a rollup priced and swept every deployment any earlier test in the same xdist worker had left there. proxy-infra runs --dist loadscope, which assigns modules to workers dynamically, so the same commit passed or failed depending on what ran in front of the rollup tests

Fix: callers hand the rollup its router, and the proxy's nightly job passes its own. Nothing reads the module global

2026-08-20: license gate treated a PyPI connection reset as a missing license

Flaky gate: the check_licenses step of Code Quality Checks

Evidence: https://github.com/BerriAI/litellm/actions/runs/32353318790 on litellm_vertex_live_db_credentials, attempt 1 (job 96377074542) failed the step, attempt 2 on the identical commit passed with no change in between. Attempt 1's log gives the cause, Failed to fetch license for tornado 6.5.7: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')), then Found packages that need verification: botocore-stubs (1.43.14) - Unknown license, tornado (6.5.7) - Unknown license

Mechanism, real network dependence with no retry. get_package_license_from_pypi did a single requests.get and caught every exception the same way, so a transient TCP reset was reclassified as "no license metadata" and failed a compliance gate meant to catch real license problems

Fix: classify the failure before deciding. Connection errors, timeouts, 429 and 5xx get up to 3 attempts with a short backoff, and every other outcome, including a 404 for a version that is not on PyPI, still resolves on the first answer. Exhausting the retries keeps today's behaviour, so a real outage or an unlicensed package still fails the gate. The HTTP getter and the sleep are injected, and the tests pin attempt counts rather than only return values

2026-08-19: pricing memo survived a cost-map swap between tests

Flaky test: tests/test_litellm/batches/test_batch_utils.py::test_handle_completed_vertex_batch_computes_cost_usage_and_models

Evidence: failed with assert 3.9375e-05 == 7.875e-05 ± 7.9e-11 on https://github.com/BerriAI/litellm/actions/runs/32213874586 (attempt 2, SHA 2e1d4077) and again on https://github.com/BerriAI/litellm/actions/runs/32235776332, while the same test at the same and adjacent SHAs passed elsewhere in the window and passes locally with no edits

Mechanism, cache state leaking across tests. The misc unit workflow does not set LITELLM_LOCAL_MODEL_COST_MAP, so litellm.model_cost is fetched over the network at import. litellm.get_model_info memoizes ModelInfo keyed only on model, provider and api_base, and its only invalidation hook was never called when a test rebound litellm.model_cost itself. About a dozen tests in those paths do exactly that, so they primed the memo from the packaged map and left it pointing at that map after the fixture restored the fetched one. The 2x gap is the Vertex batch price halving that had landed on the branch but not on the map served from main

Fix: isolate_litellm_state invalidates the memo on setup and on teardown, so every test starts and ends with memoized model info consistent with the current litellm.model_cost. No assertion was relaxed, and no rerun, marker, skip, or sleep was added

Roll-up notes

Absorbed and closed as superseded earlier: #37614 (license gate) and #37461 (pricing memo). Nothing else from those two branches was dropped, and no other litellm_deflake_ PR was open for this run

2026-08-27 run: no new fixable test flakes in the 24h window, so nothing new was added. The window's only red runs besides the license gate above were infrastructure, detailed in the automation report: a staging Unit Tests job whose runner lost communication with the server (run 33003396465), a zizmor job that could not reach the GitHub advisories API (run 33014750380 attempt 1), and a CodSpeed run that failed on a 500 from its upload endpoint (run 33026389842). No other litellm_deflake_ PR was open, so nothing was absorbed or closed

Dropped from this branch's own original diff: the config_sourced flag on _LoadedDeployments and the unbounded prune path it gated. Staging #37793 landed a superseding change that always bounds the prune by the ids the run scanned, so the flag had no remaining reader and its assertion in test_a_router_left_on_the_proxy_module_is_not_scanned was removed. The router injection this PR is about is unaffected

Screenshots / Proof of Fix

None of the four has a proxy route or UI surface: three are test-isolation fixes and one is a CI gate script, so the honest proof is the CI evidence above plus mutation checks that the coverage still bites. Verification on this tip, not offered as proof of the flakes:

Before

Every fix has a mutation check

1. Halving the MCP tool-call price fails tests/mcp_tests/test_mcp_logging.py::test_mcp_cost_tracking and ::test_mcp_cost_tracking_per_tool
2. Making the PTU loader ignore the router it is handed fails test_a_router_left_on_the_proxy_module_is_not_scanned with the ambient deployment priced
3. Reducing the license lookup to one attempt fails test_get_license_retries_connection_error_then_resolves_license and test_get_license_returns_none_after_connection_retry_limit
4. The pricing memo fix has no single owning test by design, since what it fixes is ordering across files. Its reproduction is the library-level one in test: invalidate memoized model-cost lookups between unit tests #37461: prime the memo from a doubled cost map, restore the original map, and get_model_info still returns the doubled batch rate

After (09ff5bf, staging merged in on 2026-08-27)

1. tests/mcp_tests/test_mcp_logging.py with the two sibling files that leak into it, 20 consecutive runs: 0 failures, 20 passed each. The whole tests/mcp_tests folder on 4 workers: 147 passed, 2 skipped, 1 xpassed. Before this commit the same sibling combination failed on every attempt
2. tests/test_litellm/proxy/spend_tracking/test_ptu_flat_cost_rollup.py and tests/test_litellm/test_check_licenses.py, 20 consecutive runs together on this tip: 0 failures, 163 passed each
3. The PTU rollup wiring tests in tests/test_litellm/proxy/test_proxy_server.py (-k ptu), 20 consecutive runs on this tip: 0 failures, 3 passed each
4. make lint and make check on this tip: clean (earlier fuller runs at 54ea379: tests/mcp_tests folder on 4 workers 147 passed, test_proxy_server.py 328 passed, spend_tracking 602 passed)

Type

🐛 Bug Fix
✅ Test
🚄 Infrastructure

Caveats (if any)

• Scope is deliberately four fixes, one open deflake PR at a time
• The MCP drain runs per test, adding a little teardown time
• The test that leaks a router onto the proxy module stays unidentified

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/f9aaff213ca44ace81d0b20716c154ee
Open in Devin Desktop: https://app.devin.ai/desktop/session/f9aaff213ca44ace81d0b20716c154ee?variant=devin

---

Note

Medium Risk
PTU rollup still writes team spend rows and now depends on the proxy passing its router for config-sourced deployments; mis-wiring would under-attribute PTU, though tests lock the cron call site.

Overview
This PR bundles four CI/test-isolation fixes plus the small production wiring needed for PTU rollup to stay correct after the router change.

PTU flat-cost rollup no longer reads llm_router off litellm.proxy.proxy_server via sys.modules. Callers pass an optional router through _load_ptu_models, daily rollup, backfill, and the scheduled job; the proxy cron now passes llm_router. That stops xdist workers from pricing/pruning against another test’s leftover router while keeping config.yaml PTU deployments in scope.

License checking retries transient PyPI failures (connection/timeout, 429, 5xx) up to three times with backoff, with injectable HTTP/sleep for tests. 404s still fail fast.

MCP tests add an autouse fixture that drains GLOBAL_LOGGING_WORKER after each test so queued callbacks don’t run on the next test’s event loop (fixing wrong cost assertions).

Unit test isolation clears the get_model_info / model_cost memo via _invalidate_model_cost_lowercase_map() at the start and end of isolate_litellm_state, so rebinding litellm.model_cost in one test doesn’t skew pricing in later tests.

^{Reviewed by Cursor Bugbot for commit 09ff5bf. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR consolidates four CI deflake fixes while preserving the affected production behavior.

• Passes the active proxy router explicitly through PTU rollup and backfill processing.
• Retries transient PyPI license lookup failures while retaining fail-fast behavior for non-retryable responses.
• Drains queued MCP logging callbacks between tests.
• Invalidates model-cost memoization at unit-test boundaries.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	Passes the current proxy router into the scheduled PTU rollup job.
litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py	Replaces ambient module-global router discovery with explicit router propagation across rollup and backfill paths.
tests/code_coverage_tests/check_licenses.py	Adds bounded retries for transient PyPI connection, timeout, throttling, and server failures.
tests/mcp_tests/conftest.py	Adds per-test teardown that drains queued logging callbacks before the next MCP test.
tests/test_litellm/conftest.py	Invalidates cached model information before and after each isolated unit test.
tests/test_litellm/proxy/spend_tracking/test_ptu_flat_cost_rollup.py	Updates PTU tests to inject routers explicitly and verifies ambient proxy state is ignored.
tests/test_litellm/proxy/test_proxy_server.py	Verifies the scheduled rollup receives the proxy’s active router.
tests/test_litellm/test_check_licenses.py	Covers successful transient retries, retry exhaustion, and non-retryable 404 behavior.

_{Reviews (3): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 90.90909% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/proxy_server.py	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_deflake_20260821 (09ff5bf) with litellm_internal_staging (8ebcb3e)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (cd63c7e) during the generation of this report, so 8ebcb3e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 09ff5bf. Configure here.}

[mateo-berri]

@greptileai

[mateo-berri]

LGTM