[Infra] Promote interal staging to main

[yuneng-berri]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Type

🚄 Infrastructure

Changes

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
6 out of 9 committers have signed the CLA.

✅ Sameerlite
✅ shivamrawat1
✅ yuneng-berri
✅ ryan-crabbe-berri
✅ michelligabriele
✅ milan-berri
❌ ishaan-berri
❌ krrish-berri-2
❌ shin-berri
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing litellm_internal_staging (6a25866) with main (09cd7e3)}

[codecov]

Codecov Report

❌ Patch coverage is 92.82971% with 56 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/router_strategy/adaptive_router/hooks.py	84.67%	19 Missing ⚠️
...router_strategy/adaptive_router/adaptive_router.py	96.22%	6 Missing ⚠️
litellm/llms/bedrock/chat/mantle/transformation.py	75.00%	5 Missing ⚠️
...llm/llms/bedrock/messages/mantle_transformation.py	71.42%	4 Missing ⚠️
litellm/proxy/hooks/max_budget_limiter.py	42.85%	4 Missing ⚠️
litellm/router_strategy/adaptive_router/signals.py	97.24%	4 Missing ⚠️
...lm/router_strategy/adaptive_router/update_queue.py	93.54%	4 Missing ⚠️
litellm/proxy/proxy_cli.py	66.66%	3 Missing ⚠️
litellm/litellm_core_utils/core_helpers.py	92.85%	2 Missing ⚠️
litellm/router_strategy/adaptive_router/bandit.py	96.61%	2 Missing ⚠️
... and 3 more

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR promotes the internal staging branch to main, landing a substantial set of changes: a new Adaptive Router strategy (Thompson-sampling bandit with per-session signal detection, DB persistence, and an admin introspection endpoint), a total_spend column on LiteLLM_TeamMembership, Bedrock guardrail fixes for correct pre/during/post-call spend attribution, Redis spend-tracking retry-on-failure recovery, and a number of infrastructure/CI improvements.

The new feature is well-tested with comprehensive mocked unit and e2e tests, and the DB migrations are additive. All remaining findings are P2 quality/correctness suggestions.

Confidence Score: 5/5

Safe to merge; all findings are P2 quality/hardening suggestions that do not block correctness on the happy path.

No P0/P1 issues found. The adaptive router, bandit logic, DB migrations, and guardrail refactoring are well-scoped. The three P2 findings (overly broad redaction in base guardrail class, _state_loaded prematurely set on DB unavailability at startup, unreferenced asyncio task) are edge-case degradations rather than bugs on the primary request path.

litellm/integrations/custom_guardrail.py (broad redact scope), litellm/proxy/proxy_server.py (_state_loaded + task reference), litellm/router_strategy/adaptive_router/adaptive_router.py (min_quality_tier ValueError)

Important Files Changed

Filename	Overview
litellm/router_strategy/adaptive_router/adaptive_router.py	New AdaptiveRouter class: Thompson-sampling bandit with owner-cache and session-state management. Well-structured, asyncio-safe (no yields between cell read+write), comprehensive tests. Minor: no graceful fallback when min_quality_tier eliminates all models.
litellm/router_strategy/adaptive_router/bandit.py	New Thompson-sampling bandit utilities. pick_best correctly builds costs from the same eligible set as cells. apply_delta respects SAMPLE_CAP. Normalization handles zero-spread edge case.
litellm/router_strategy/adaptive_router/hooks.py	New post-call hook. Session key derivation, tool-result extraction, and logical-model lookup from metadata are clean. Exception swallowing prevents signal failures from breaking requests.
litellm/router_strategy/adaptive_router/update_queue.py	In-memory aggregator queue for state and session upserts. Lock usage is correct; last-write-wins session semantics properly handle concurrent flushers via atomic upsert.
litellm/proxy/proxy_server.py	Adds adaptive-router flusher loop and /adaptive_router/state admin endpoint. Two issues: _state_loaded set True even when prisma_client is None, and asyncio.create_task result not stored.
litellm/integrations/custom_guardrail.py	Adds use_native_during_call_hook ClassVar and applies redact_nested_match_and_regex_keys globally to all guardrail log payloads. Overly broad redaction could affect non-Bedrock guardrails.
litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py	Refactors _redact_pii_matches to delegate to shared utility, adds logging_event_type parameter to fix pre/during/post_call attribution in spend logs, and sets use_native_during_call_hook=True on BedrockGuardrail.
litellm/proxy/db/db_transaction_queue/redis_update_buffer.py	Adds Redis rpush failure recovery by restoring drained transactions back to in-memory queues. Correctly handles both DBSpendUpdateTransactions entities and daily spend transaction pairs.
litellm-proxy-extras/litellm_proxy_extras/migrations/20260418000000_add_adaptive_router_tables/migration.sql	New migration adds LiteLLM_AdaptiveRouterState and LiteLLM_AdaptiveRouterSession tables with appropriate indexes. Schema is additive, correctly documented.
litellm/router.py	Integrates adaptive router into Router lifecycle: deferred finalization after all deployments are registered, hook cleanup on reload, and pre-routing dispatch. Wildcard-aware deployment lookup fix included.

Sequence Diagram

sequenceDiagram
    participant C as Client
    participant PS as ProxyServer
    participant R as Router
    participant AR as AdaptiveRouter
    participant BN as Bandit
    participant H as PostCallHook
    participant Q as UpdateQueue
    participant FL as FlusherLoop
    participant DB as Postgres

    PS->>PS: startup: load_state_from_db
    DB-->>AR: persisted BanditCells
    PS->>PS: create_task(flusher_loop)

    C->>PS: POST /chat/completions
    PS->>R: async_pre_routing_hook(model)
    R->>AR: async_pre_routing_hook
    AR->>BN: pick_best(cells, costs)
    BN-->>AR: chosen_model
    AR-->>R: PreRoutingHookResponse(model=chosen)
    R-->>PS: route to chosen_model

    PS-->>C: response + x-litellm-adaptive-router-model header

    note over H: async_log_success_event
    H->>H: _resolve_session_key
    H->>AR: claim_or_check_owner
    AR-->>H: True/False
    H->>AR: record_turn(session_id, model, rt, turn)
    AR->>AR: apply_turn(session_state)
    AR->>Q: add_session_state(snapshot)
    AR->>AR: update _cells[cell_key]
    AR->>Q: add_state_delta(delta)

    loop every 10s
        FL->>Q: flush_state_to_db(prisma)
        Q->>DB: upsert LiteLLM_AdaptiveRouterState
        FL->>Q: flush_session_to_db(prisma)
        Q->>DB: upsert LiteLLM_AdaptiveRouterSession
    end

Loading

_{Reviews (1): Last reviewed commit: "Merge pull request #26295 from BerriAI/y..." | Re-trigger Greptile}

[greptile-apps]

Overly broad redaction may affect non-Bedrock guardrail logs

redact_nested_match_and_regex_keys is now applied to clean_guardrail_response in the shared base class path, meaning it silently replaces any key named match or regex (at any nesting depth) across ALL guardrail provider responses — not just Bedrock PII assessments. If a custom guardrail returns a response dict where match or regex is a semantically meaningful, non-PII field (e.g., a regex pattern for an allow-list, a match count from a semantic similarity check), that data is permanently lost from the standard logging payload without warning. The original _redact_pii_matches was intentionally scoped to the specific Bedrock sensitiveInformationPolicy / wordPolicy paths for this reason.

[greptile-apps]

_state_loaded set to True even when prisma_client is None

load_state_from_db silently returns early when prisma_client is None, but _ar._state_loaded = True is unconditionally set immediately after. If the DB connection is not yet established at startup time, the adaptive router starts with cold-start priors and the flusher's lazy-load path will never retry:

if not getattr(ar, "_state_loaded", False):   # already True → skipped
    await ar.load_state_from_db(prisma_client)

Any persisted bandit priors from a previous deployment are permanently ignored. Consider only setting _state_loaded = True when prisma_client is not None, so the flusher's lazy-load path can still hydrate state once the DB becomes available.

[greptile-apps]

Background task reference not retained

asyncio.create_task(...) returns a Task object that should be stored to prevent accidental garbage collection. While CPython's event loop holds a strong reference to all running tasks, the reference should still be stored (PEP guidance + avoids subtle bugs on other implementations and during shutdown):

_adaptive_router_flusher_task = asyncio.create_task(_adaptive_router_flusher_loop())

The other background health-check uses asyncio.ensure_future(...) without storing the result as well, but adding a reference here avoids the antipattern for the new code.

[greptile-apps]

pick_model raises unguarded ValueError on misconfigured min_quality_tier

If a caller passes a min_quality_tier value that is higher than every model's tier (e.g., min_quality_tier=3 when all models are tier 1/2), _eligible_models returns [] and pick_model raises:

ValueError: AdaptiveRouter[…]: no models meet min_quality_tier=…

This exception would propagate up through async_pre_routing_hook and surface as a 500 error to the end user. Consider falling back to the highest-tier eligible model (or the full model set) with a warning log rather than raising, so a misconfigured header doesn't take down traffic.