feat(proxy): enforce key/team guardrails on bedrock passthrough routes - #30194
Merged
mateo-berri merged 26 commits intoJun 12, 2026
Merged
Conversation
/bedrock/... passthrough routes silently skipped all guardrail hooks because CallTypes.allm_passthrough_route had no entry in guardrail_translation_mappings. Add a dispatcher (LlmPassthroughRouteHandler) registered for that call type that routes to BedrockPassthroughGuardrailHandler for Bedrock Converse endpoints; wire post_call_success_hook into both the JSON and AWS event-stream response paths in common_request_processing, including full de-anonymization for streaming responses with proportional text distribution across original event-stream delta frames.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Contributor
Greptile SummaryThis PR adds guardrail enforcement (pre-call blocking and Presidio de-anonymization) to
Confidence Score: 5/5Safe to merge; the changes add guardrail enforcement for previously unguarded passthrough routes without altering any existing behavior for non-passthrough routes. The core logic — per-request guardrail scoping via should_run_guardrail, correct AWS EventStream CRC32 chaining, content-length stripping before body rewrite, and early-return guarding to avoid double invocation of post_call_success_hook — is all correct. The issues flagged in the previous review thread are addressed. Tests cover extraction, write-back, blocking, event-stream re-encoding, and header forwarding. No files require special attention; the complex binary re-encoding path in litellm/llms/bedrock/passthrough/guardrail_translation/handler.py is well-exercised by the new test suite.
|
| Filename | Overview |
|---|---|
| litellm/llms/bedrock/passthrough/guardrail_translation/handler.py | New handler: correctly extracts/writes back Converse text, delegates non-Converse endpoints to the generic handler, and re-encodes AWS EventStream frames with correct CRC32 chaining (prelude CRC then message CRC seeded from prelude CRC). |
| litellm/llms/pass_through/guardrail_translation/handler.py | New LlmPassthroughRouteHandler dispatcher; provider handlers registered lazily, unknown providers log and skip cleanly, static helpers correctly proxy to per-provider implementations. |
| litellm/proxy/common_request_processing.py | Adds streaming-buffer and non-streaming guardrail paths for allm_passthrough_route; _has_post_call_guardrails_for_passthrough() is correctly scoped to per-request should_run_guardrail checks; CRC re-encoding and upstream header forwarding look correct. |
| litellm/proxy/guardrails/guardrail_hooks/unified_guardrail/unified_guardrail.py | Correctly extends the call_type fallback resolution to include allm_passthrough_route alongside pass_through, so the right guardrail_translation_mappings entry is selected. |
| litellm/types/utils.py | Adds 'allm_passthrough_route' to CallTypesLiteral; consistent with existing CallTypes enum entry and router wiring. |
| tests/test_litellm/proxy/test_common_request_processing.py | Adds four new unit tests covering the guardrailed passthrough response paths; tests verify upstream header forwarding, response_headers_hook integration, and FastAPI Response passthrough. |
| tests/test_litellm/llms/bedrock/passthrough/guardrail_translation/test_handler.py | New test file with 1163 lines covering text extraction, write-back, blocking, non-Converse skip, and event-stream CRC re-encoding; uses mocks only. |
| tests/local_testing/test_azure_anthropic_sync_post.py | Adds pytest.skip for MaskedHTTPStatusError when httpbin.org is unavailable; correct CI resilience fix, does not weaken the test's assertion logic. |
Reviews (26): Last reviewed commit: "test(ui-e2e): re-issue deep-link navigat..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
Contributor
PR overviewThis pull request adds key- and team-level guardrail enforcement for Bedrock passthrough routes in the proxy, including post-call processing for Bedrock responses and streams. It updates common proxy request handling and Bedrock passthrough guardrail translation logic to apply those controls across supported endpoints. The PR has made progress, with 8 issues already addressed, but two meaningful gaps remain open. The current implementation can buffer entire Bedrock streams in memory when post-call passthrough guardrails are enabled, allowing a key holder to create avoidable proxy memory pressure. It also does not apply guardrail rewrite results to some non-Converse Bedrock responses, so certain passthrough routes can still return output that should have been masked or transformed. Open issues (2)
Fixed/addressed: 8 · PR risk: 7/10 |
…rdrails Move botocore event-stream logic from proxy/ to BedrockPassthroughGuardrailHandler.de_anonymize_converse_stream; _handle_event_stream_allm_passthrough_route becomes a thin provider dispatcher. Pass custom_headers through _handle_non_streaming_allm_passthrough_route so early-return guardrail responses include x-litellm-call-id and related headers.
Contributor
Author
Contributor
Replace the inline JSON/eventstream block in the streaming-request branch with a call to _handle_non_streaming_allm_passthrough_route so both paths share one implementation and cannot diverge.
Contributor
The event-stream re-encoder only emitted parsed frames, so any trailing bytes left after the parse loop (truncated/corrupt final frame, or fewer than 16 bytes after the last complete frame) were silently dropped from the de-anonymized output. Capture and re-append them so the transformer never truncates the stream.
Contributor
Contributor
Contributor
…ode post-call guardrails on bedrock passthrough Pre-call extraction only read top-level Converse text blocks, so blocked content placed under toolResult.content[].text was forwarded to Bedrock without the key/team guardrail seeing it. Extraction now walks nested tool result text and write-back mutates the owning block in place. Post-call buffering for passthrough used _has_post_call_guardrails, which excludes event_hook=None guardrails. Those guardrails run at post_call, so their output processing was skipped and the raw upstream body was returned. Add a passthrough-specific predicate that counts them.
Contributor
…llm passthrough dispatcher Remove the hardcoded bedrock provider guard from common_request_processing by delegating event-stream de-anonymization to LlmPassthroughRouteHandler, which resolves the provider from the existing handler registry. Keeps proxy/ provider-agnostic and reuses the same dispatch path as the input and output guardrail handlers. Also log instead of silently dropping the result when post_call_success_hook returns a non-dict on the JSON and event-stream passthrough paths.
Contributor
Pre-call extraction and post-call output processing only handled Converse
shapes, so /bedrock/model/{modelId}/invoke and invoke-with-response-stream
returned unguarded. An authenticated caller could move blocked content into
an InvokeModel payload and skip the key/team guardrail entirely.
Non-Converse Bedrock routes now fall back to the generic passthrough handler,
which scans the full request and response payloads so blocking guardrails
still run, matching how other passthrough providers are guarded.
Contributor
… passthrough guardrails Converse passthrough guardrails only scanned system and message content, so a key holder could route blocked or PII text through toolConfig tool names, descriptions and input schemas or through additionalModelRequestFields, all of which are still forwarded to Bedrock. Collect strings from those fields too so key/team guardrails inspect and rewrite them, matching how the chat-completions path forwards tool definitions to guardrails.
Contributor
Contributor
local_testing_part1 was failing on test_post_delay_exceeds_per_request_timeout_raises because httpbin.org/delay/10 intermittently answers 503 instead of delaying, so HTTPHandler.post raised MaskedHTTPStatusError rather than the expected Timeout. The test already means to skip when httpbin is unavailable, but its guard only probed GET /get and ignored a server error on the delay endpoint. Treat a 5xx from httpbin as 'service unavailable' and skip, which is outside this repo's control, while still asserting Timeout when httpbin genuinely delays.
Contributor
Contributor
1 similar comment
Contributor
Contributor
Contributor
…able endpoints Only buffer a passthrough event stream into a non-streaming response when the resolved provider and endpoint actually have an event-stream guardrail handler that can rewrite frames (Bedrock converse-stream). Other Bedrock event-stream endpoints such as invoke-with-response-stream keep streaming, since the Converse handler leaves their frames untouched and buffering would silently break the streaming contract for no content change.
Contributor
| ) -> Any: | ||
| endpoint = (request_data or {}).get("endpoint", "") | ||
| if endpoint and not _is_converse_endpoint(endpoint): | ||
| return await _generic_passthrough_handler().process_output_response( |
Contributor
There was a problem hiding this comment.
Medium: Output guardrail bypass
PassThroughEndpointHandler.process_output_response() only sends the serialized response to apply_guardrail and ignores any returned texts, so post-call masking/de-anonymization guardrails do not rewrite non-Converse Bedrock responses. A key holder can call /bedrock/model/.../invoke or /invoke-with-response-stream and receive output that would be rewritten on /converse; add a Bedrock response extractor/write-back path for invoke responses, or reject unsupported Bedrock passthrough routes when a rewriting post-call guardrail is configured.
Contributor
There was a problem hiding this comment.
This is the intended block-only contract of the generic passthrough handler, not a Bedrock-specific bypass. The non-Converse fallback runs through PassThroughEndpointHandler, which has always scanned and blocked rather than rewritten: process_output_response calls apply_guardrail, so a blocking post-call guardrail (Bedrock Guardrails, content moderation, etc.) still rejects a violating invoke response. Its process_input_messages is symmetric and also returns data untouched, so an anonymize/de-anonymize guardrail never masks the invoke request in the first place, which leaves nothing on the response to de-anonymize; the two halves stay consistent. Converse is the only shape this PR understands well enough to extract and write back specific fields, so it gets full masking, while invoke keeps the provider-agnostic block-only behavior shared with the OpenAI, Vertex and other passthrough routes. Before this PR every /bedrock/... route skipped guardrails entirely, so invoke gains blocking where it previously had none. Adding InvokeModel response extractors would be a separate feature with no universal schema across Titan, Llama, Nova and the other foundation models, and rejecting the route whenever a post-call guardrail is configured would break the block-only guardrails that work today, so neither suggested change is a net improvement
… the page param navigateToPage deep-links to /ui?page=<page> then proceeds once the network settles, but a fresh load can race the auth bootstrap: the app momentarily treats the session as anonymous, bounces through /ui/login, and returns to the default Virtual Keys page with the ?page= query param dropped. The helper never checked where it actually landed, so any single bounce left callers asserting against the wrong page and timing out (mcpServers, modelHub, addModel). Confirm the requested page is what rendered and re-issue the navigation when it was clobbered; auth is warm by the second load so the param sticks. Migrated path routes are left alone since they intentionally leave the legacy root.
Contributor
michaelxer
pushed a commit
to michaelxer/litellm
that referenced
this pull request
Jun 17, 2026
BerriAI#30194) * feat(proxy): enforce key/team guardrails on bedrock passthrough routes /bedrock/... passthrough routes silently skipped all guardrail hooks because CallTypes.allm_passthrough_route had no entry in guardrail_translation_mappings. Add a dispatcher (LlmPassthroughRouteHandler) registered for that call type that routes to BedrockPassthroughGuardrailHandler for Bedrock Converse endpoints; wire post_call_success_hook into both the JSON and AWS event-stream response paths in common_request_processing, including full de-anonymization for streaming responses with proportional text distribution across original event-stream delta frames. * refactor(proxy): address greptile feedback on bedrock passthrough guardrails Move botocore event-stream logic from proxy/ to BedrockPassthroughGuardrailHandler.de_anonymize_converse_stream; _handle_event_stream_allm_passthrough_route becomes a thin provider dispatcher. Pass custom_headers through _handle_non_streaming_allm_passthrough_route so early-return guardrail responses include x-litellm-call-id and related headers. * style: run black on handler and common_request_processing * refactor(proxy): dedupe non-streaming passthrough guardrail handling Replace the inline JSON/eventstream block in the streaming-request branch with a call to _handle_non_streaming_allm_passthrough_route so both paths share one implementation and cannot diverge. * fix(bedrock): preserve trailing bytes when re-encoding converse stream The event-stream re-encoder only emitted parsed frames, so any trailing bytes left after the parse loop (truncated/corrupt final frame, or fewer than 16 bytes after the last complete frame) were silently dropped from the de-anonymized output. Capture and re-append them so the transformer never truncates the stream. * fix(proxy): guard non-dict post-call hook return on bedrock passthrough JSON path * fix(proxy): guard malformed JSON body on bedrock passthrough guardrail path * fix(proxy): close guardrail bypass via tool result text and default-mode post-call guardrails on bedrock passthrough Pre-call extraction only read top-level Converse text blocks, so blocked content placed under toolResult.content[].text was forwarded to Bedrock without the key/team guardrail seeing it. Extraction now walks nested tool result text and write-back mutates the owning block in place. Post-call buffering for passthrough used _has_post_call_guardrails, which excludes event_hook=None guardrails. Those guardrails run at post_call, so their output processing was skipped and the raw upstream body was returned. Add a passthrough-specific predicate that counts them. * refactor(proxy): route bedrock event-stream de-anonymization through llm passthrough dispatcher Remove the hardcoded bedrock provider guard from common_request_processing by delegating event-stream de-anonymization to LlmPassthroughRouteHandler, which resolves the provider from the existing handler registry. Keeps proxy/ provider-agnostic and reuses the same dispatch path as the input and output guardrail handlers. Also log instead of silently dropping the result when post_call_success_hook returns a non-dict on the JSON and event-stream passthrough paths. * fix(proxy): close guardrail bypass on bedrock invoke passthrough routes Pre-call extraction and post-call output processing only handled Converse shapes, so /bedrock/model/{modelId}/invoke and invoke-with-response-stream returned unguarded. An authenticated caller could move blocked content into an InvokeModel payload and skip the key/team guardrail entirely. Non-Converse Bedrock routes now fall back to the generic passthrough handler, which scans the full request and response payloads so blocking guardrails still run, matching how other passthrough providers are guarded. * fix(proxy): keep non-bedrock passthrough streams streaming under post-call guardrails * fix(bedrock): scan non-text converse blocks for passthrough guardrails Key/team guardrails on bedrock converse passthrough only saw top-level text blocks, so a caller could hide prompt content in toolUse.input or toolResult.content[].json and have it forwarded to Bedrock without the configured guardrail inspecting it, bypassing blocking guardrails by default. Walk those arbitrary-JSON subtrees and write masked values back in place. Extend the non-streaming converse response path to the equivalent model-output fields (toolUse.input, reasoningContent text and citationsContent text) while leaving structural values such as reasoning signatures and citation sources untouched. * fix(bedrock): make passthrough guardrail string collection iterative and type-safe Rewrite _collect_strings with an explicit stack so it no longer recurses, satisfying the recursive-function CI guard, and widen the holder container type so mypy accepts indexing JSON nodes by str or int keys. * fix(proxy): scope passthrough post-call guardrail buffering to the request Buffering the Bedrock event stream into a single non-streaming response was gated on whether any post_call guardrail existed globally, so every converse-stream request lost streaming once any post_call guardrail was registered, even for keys that did not reference it. Mirror the gate used by post_call_success_hook (should_run_guardrail against the request's merged guardrails) so only requests whose key/team actually trigger a post_call guardrail are buffered. * fix(bedrock): guardrail non-text converse stream deltas on passthrough de_anonymize_event_stream only routed delta.text through the post-call guardrail, so model output streamed in reasoningContent.text, toolUse.input or citationsContent.content[].text was forwarded raw and skipped masking/blocking. Collect every user-visible text field per contentBlockDelta, concatenate per logical stream so split mask tokens still reassemble, run them through the hook, then redistribute the guardrailed text back into the matching delta fields. This brings streaming coverage in line with the non-streaming Converse output handler. * refactor(proxy): keep bedrock event-stream content-type detection in llms Move the vnd.amazon.eventstream content-type check out of the proxy passthrough path into BedrockPassthroughGuardrailHandler via the LlmPassthroughRouteHandler dispatcher, so proxy code stays provider-agnostic. Also patch the actually-called _has_post_call_guardrails_for_passthrough in the malformed-body regression test instead of the unused _has_post_call_guardrails. * fix(proxy): forward upstream headers on bedrock guardrail passthrough responses Mirror the non-guardrail passthrough path by merging the upstream response headers (via get_response_headers) into the guardrailed non-streaming and event-stream responses, so headers like x-amzn-requestid survive when a post-call guardrail rewrites the body. Drop the stray fastapi HTTPException import from the SDK-tree handler test in favor of a local sentinel exception. * fix(bedrock): scan tool definitions and additional request fields for passthrough guardrails Converse passthrough guardrails only scanned system and message content, so a key holder could route blocked or PII text through toolConfig tool names, descriptions and input schemas or through additionalModelRequestFields, all of which are still forwarded to Bedrock. Collect strings from those fields too so key/team guardrails inspect and rewrite them, matching how the chat-completions path forwards tool definitions to guardrails. * fix(bedrock): log instead of silently dropping passthrough guardrail edge cases * test(local): skip httpbin timeout probe when the service returns 5xx local_testing_part1 was failing on test_post_delay_exceeds_per_request_timeout_raises because httpbin.org/delay/10 intermittently answers 503 instead of delaying, so HTTPHandler.post raised MaskedHTTPStatusError rather than the expected Timeout. The test already means to skip when httpbin is unavailable, but its guard only probed GET /get and ignored a server error on the delay endpoint. Treat a 5xx from httpbin as 'service unavailable' and skip, which is outside this repo's control, while still asserting Timeout when httpbin genuinely delays. * fix(proxy): set content-type on buffered bedrock passthrough event-stream responses * fix(bedrock): skip passthrough output write-back when guardrail returns no texts * fix(proxy): apply response-headers hook on guardrailed bedrock passthrough responses * refactor(bedrock): import event-stream crc32 from binascii not botocore internals * fix(proxy): scope bedrock passthrough stream buffering to de-anonymizable endpoints Only buffer a passthrough event stream into a non-streaming response when the resolved provider and endpoint actually have an event-stream guardrail handler that can rewrite frames (Bedrock converse-stream). Other Bedrock event-stream endpoints such as invoke-with-response-stream keep streaming, since the Converse handler leaves their frames untouched and buffering would silently break the streaming contract for no content change. * test(ui-e2e): re-issue deep-link navigation when auth bootstrap drops the page param navigateToPage deep-links to /ui?page=<page> then proceeds once the network settles, but a fresh load can race the auth bootstrap: the app momentarily treats the session as anonymous, bounces through /ui/login, and returns to the default Virtual Keys page with the ?page= query param dropped. The helper never checked where it actually landed, so any single bounce left callers asserting against the wrong page and timing out (mcpServers, modelHub, addModel). Confirm the requested page is what rendered and re-issue the navigation when it was clobbered; auth is warm by the second load so the param sticks. Migrated path routes are left alone since they intentionally leave the legacy root. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
michaelxer
pushed a commit
to michaelxer/litellm
that referenced
this pull request
Jun 17, 2026
BerriAI#30194) * feat(proxy): enforce key/team guardrails on bedrock passthrough routes /bedrock/... passthrough routes silently skipped all guardrail hooks because CallTypes.allm_passthrough_route had no entry in guardrail_translation_mappings. Add a dispatcher (LlmPassthroughRouteHandler) registered for that call type that routes to BedrockPassthroughGuardrailHandler for Bedrock Converse endpoints; wire post_call_success_hook into both the JSON and AWS event-stream response paths in common_request_processing, including full de-anonymization for streaming responses with proportional text distribution across original event-stream delta frames. * refactor(proxy): address greptile feedback on bedrock passthrough guardrails Move botocore event-stream logic from proxy/ to BedrockPassthroughGuardrailHandler.de_anonymize_converse_stream; _handle_event_stream_allm_passthrough_route becomes a thin provider dispatcher. Pass custom_headers through _handle_non_streaming_allm_passthrough_route so early-return guardrail responses include x-litellm-call-id and related headers. * style: run black on handler and common_request_processing * refactor(proxy): dedupe non-streaming passthrough guardrail handling Replace the inline JSON/eventstream block in the streaming-request branch with a call to _handle_non_streaming_allm_passthrough_route so both paths share one implementation and cannot diverge. * fix(bedrock): preserve trailing bytes when re-encoding converse stream The event-stream re-encoder only emitted parsed frames, so any trailing bytes left after the parse loop (truncated/corrupt final frame, or fewer than 16 bytes after the last complete frame) were silently dropped from the de-anonymized output. Capture and re-append them so the transformer never truncates the stream. * fix(proxy): guard non-dict post-call hook return on bedrock passthrough JSON path * fix(proxy): guard malformed JSON body on bedrock passthrough guardrail path * fix(proxy): close guardrail bypass via tool result text and default-mode post-call guardrails on bedrock passthrough Pre-call extraction only read top-level Converse text blocks, so blocked content placed under toolResult.content[].text was forwarded to Bedrock without the key/team guardrail seeing it. Extraction now walks nested tool result text and write-back mutates the owning block in place. Post-call buffering for passthrough used _has_post_call_guardrails, which excludes event_hook=None guardrails. Those guardrails run at post_call, so their output processing was skipped and the raw upstream body was returned. Add a passthrough-specific predicate that counts them. * refactor(proxy): route bedrock event-stream de-anonymization through llm passthrough dispatcher Remove the hardcoded bedrock provider guard from common_request_processing by delegating event-stream de-anonymization to LlmPassthroughRouteHandler, which resolves the provider from the existing handler registry. Keeps proxy/ provider-agnostic and reuses the same dispatch path as the input and output guardrail handlers. Also log instead of silently dropping the result when post_call_success_hook returns a non-dict on the JSON and event-stream passthrough paths. * fix(proxy): close guardrail bypass on bedrock invoke passthrough routes Pre-call extraction and post-call output processing only handled Converse shapes, so /bedrock/model/{modelId}/invoke and invoke-with-response-stream returned unguarded. An authenticated caller could move blocked content into an InvokeModel payload and skip the key/team guardrail entirely. Non-Converse Bedrock routes now fall back to the generic passthrough handler, which scans the full request and response payloads so blocking guardrails still run, matching how other passthrough providers are guarded. * fix(proxy): keep non-bedrock passthrough streams streaming under post-call guardrails * fix(bedrock): scan non-text converse blocks for passthrough guardrails Key/team guardrails on bedrock converse passthrough only saw top-level text blocks, so a caller could hide prompt content in toolUse.input or toolResult.content[].json and have it forwarded to Bedrock without the configured guardrail inspecting it, bypassing blocking guardrails by default. Walk those arbitrary-JSON subtrees and write masked values back in place. Extend the non-streaming converse response path to the equivalent model-output fields (toolUse.input, reasoningContent text and citationsContent text) while leaving structural values such as reasoning signatures and citation sources untouched. * fix(bedrock): make passthrough guardrail string collection iterative and type-safe Rewrite _collect_strings with an explicit stack so it no longer recurses, satisfying the recursive-function CI guard, and widen the holder container type so mypy accepts indexing JSON nodes by str or int keys. * fix(proxy): scope passthrough post-call guardrail buffering to the request Buffering the Bedrock event stream into a single non-streaming response was gated on whether any post_call guardrail existed globally, so every converse-stream request lost streaming once any post_call guardrail was registered, even for keys that did not reference it. Mirror the gate used by post_call_success_hook (should_run_guardrail against the request's merged guardrails) so only requests whose key/team actually trigger a post_call guardrail are buffered. * fix(bedrock): guardrail non-text converse stream deltas on passthrough de_anonymize_event_stream only routed delta.text through the post-call guardrail, so model output streamed in reasoningContent.text, toolUse.input or citationsContent.content[].text was forwarded raw and skipped masking/blocking. Collect every user-visible text field per contentBlockDelta, concatenate per logical stream so split mask tokens still reassemble, run them through the hook, then redistribute the guardrailed text back into the matching delta fields. This brings streaming coverage in line with the non-streaming Converse output handler. * refactor(proxy): keep bedrock event-stream content-type detection in llms Move the vnd.amazon.eventstream content-type check out of the proxy passthrough path into BedrockPassthroughGuardrailHandler via the LlmPassthroughRouteHandler dispatcher, so proxy code stays provider-agnostic. Also patch the actually-called _has_post_call_guardrails_for_passthrough in the malformed-body regression test instead of the unused _has_post_call_guardrails. * fix(proxy): forward upstream headers on bedrock guardrail passthrough responses Mirror the non-guardrail passthrough path by merging the upstream response headers (via get_response_headers) into the guardrailed non-streaming and event-stream responses, so headers like x-amzn-requestid survive when a post-call guardrail rewrites the body. Drop the stray fastapi HTTPException import from the SDK-tree handler test in favor of a local sentinel exception. * fix(bedrock): scan tool definitions and additional request fields for passthrough guardrails Converse passthrough guardrails only scanned system and message content, so a key holder could route blocked or PII text through toolConfig tool names, descriptions and input schemas or through additionalModelRequestFields, all of which are still forwarded to Bedrock. Collect strings from those fields too so key/team guardrails inspect and rewrite them, matching how the chat-completions path forwards tool definitions to guardrails. * fix(bedrock): log instead of silently dropping passthrough guardrail edge cases * test(local): skip httpbin timeout probe when the service returns 5xx local_testing_part1 was failing on test_post_delay_exceeds_per_request_timeout_raises because httpbin.org/delay/10 intermittently answers 503 instead of delaying, so HTTPHandler.post raised MaskedHTTPStatusError rather than the expected Timeout. The test already means to skip when httpbin is unavailable, but its guard only probed GET /get and ignored a server error on the delay endpoint. Treat a 5xx from httpbin as 'service unavailable' and skip, which is outside this repo's control, while still asserting Timeout when httpbin genuinely delays. * fix(proxy): set content-type on buffered bedrock passthrough event-stream responses * fix(bedrock): skip passthrough output write-back when guardrail returns no texts * fix(proxy): apply response-headers hook on guardrailed bedrock passthrough responses * refactor(bedrock): import event-stream crc32 from binascii not botocore internals * fix(proxy): scope bedrock passthrough stream buffering to de-anonymizable endpoints Only buffer a passthrough event stream into a non-streaming response when the resolved provider and endpoint actually have an event-stream guardrail handler that can rewrite frames (Bedrock converse-stream). Other Bedrock event-stream endpoints such as invoke-with-response-stream keep streaming, since the Converse handler leaves their frames untouched and buffering would silently break the streaming contract for no content change. * test(ui-e2e): re-issue deep-link navigation when auth bootstrap drops the page param navigateToPage deep-links to /ui?page=<page> then proceeds once the network settles, but a fresh load can race the auth bootstrap: the app momentarily treats the session as anonymous, bounces through /ui/login, and returns to the default Virtual Keys page with the ?page= query param dropped. The helper never checked where it actually landed, so any single bounce left callers asserting against the wrong page and timing out (mcpServers, modelHub, addModel). Confirm the requested page is what rendered and re-issue the navigation when it was clobbered; auth is warm by the second load so the param sticks. Migrated path routes are left alone since they intentionally leave the legacy root. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
koladefaj
pushed a commit
to koladefaj/litellm
that referenced
this pull request
Jun 17, 2026
BerriAI#30194) * feat(proxy): enforce key/team guardrails on bedrock passthrough routes /bedrock/... passthrough routes silently skipped all guardrail hooks because CallTypes.allm_passthrough_route had no entry in guardrail_translation_mappings. Add a dispatcher (LlmPassthroughRouteHandler) registered for that call type that routes to BedrockPassthroughGuardrailHandler for Bedrock Converse endpoints; wire post_call_success_hook into both the JSON and AWS event-stream response paths in common_request_processing, including full de-anonymization for streaming responses with proportional text distribution across original event-stream delta frames. * refactor(proxy): address greptile feedback on bedrock passthrough guardrails Move botocore event-stream logic from proxy/ to BedrockPassthroughGuardrailHandler.de_anonymize_converse_stream; _handle_event_stream_allm_passthrough_route becomes a thin provider dispatcher. Pass custom_headers through _handle_non_streaming_allm_passthrough_route so early-return guardrail responses include x-litellm-call-id and related headers. * style: run black on handler and common_request_processing * refactor(proxy): dedupe non-streaming passthrough guardrail handling Replace the inline JSON/eventstream block in the streaming-request branch with a call to _handle_non_streaming_allm_passthrough_route so both paths share one implementation and cannot diverge. * fix(bedrock): preserve trailing bytes when re-encoding converse stream The event-stream re-encoder only emitted parsed frames, so any trailing bytes left after the parse loop (truncated/corrupt final frame, or fewer than 16 bytes after the last complete frame) were silently dropped from the de-anonymized output. Capture and re-append them so the transformer never truncates the stream. * fix(proxy): guard non-dict post-call hook return on bedrock passthrough JSON path * fix(proxy): guard malformed JSON body on bedrock passthrough guardrail path * fix(proxy): close guardrail bypass via tool result text and default-mode post-call guardrails on bedrock passthrough Pre-call extraction only read top-level Converse text blocks, so blocked content placed under toolResult.content[].text was forwarded to Bedrock without the key/team guardrail seeing it. Extraction now walks nested tool result text and write-back mutates the owning block in place. Post-call buffering for passthrough used _has_post_call_guardrails, which excludes event_hook=None guardrails. Those guardrails run at post_call, so their output processing was skipped and the raw upstream body was returned. Add a passthrough-specific predicate that counts them. * refactor(proxy): route bedrock event-stream de-anonymization through llm passthrough dispatcher Remove the hardcoded bedrock provider guard from common_request_processing by delegating event-stream de-anonymization to LlmPassthroughRouteHandler, which resolves the provider from the existing handler registry. Keeps proxy/ provider-agnostic and reuses the same dispatch path as the input and output guardrail handlers. Also log instead of silently dropping the result when post_call_success_hook returns a non-dict on the JSON and event-stream passthrough paths. * fix(proxy): close guardrail bypass on bedrock invoke passthrough routes Pre-call extraction and post-call output processing only handled Converse shapes, so /bedrock/model/{modelId}/invoke and invoke-with-response-stream returned unguarded. An authenticated caller could move blocked content into an InvokeModel payload and skip the key/team guardrail entirely. Non-Converse Bedrock routes now fall back to the generic passthrough handler, which scans the full request and response payloads so blocking guardrails still run, matching how other passthrough providers are guarded. * fix(proxy): keep non-bedrock passthrough streams streaming under post-call guardrails * fix(bedrock): scan non-text converse blocks for passthrough guardrails Key/team guardrails on bedrock converse passthrough only saw top-level text blocks, so a caller could hide prompt content in toolUse.input or toolResult.content[].json and have it forwarded to Bedrock without the configured guardrail inspecting it, bypassing blocking guardrails by default. Walk those arbitrary-JSON subtrees and write masked values back in place. Extend the non-streaming converse response path to the equivalent model-output fields (toolUse.input, reasoningContent text and citationsContent text) while leaving structural values such as reasoning signatures and citation sources untouched. * fix(bedrock): make passthrough guardrail string collection iterative and type-safe Rewrite _collect_strings with an explicit stack so it no longer recurses, satisfying the recursive-function CI guard, and widen the holder container type so mypy accepts indexing JSON nodes by str or int keys. * fix(proxy): scope passthrough post-call guardrail buffering to the request Buffering the Bedrock event stream into a single non-streaming response was gated on whether any post_call guardrail existed globally, so every converse-stream request lost streaming once any post_call guardrail was registered, even for keys that did not reference it. Mirror the gate used by post_call_success_hook (should_run_guardrail against the request's merged guardrails) so only requests whose key/team actually trigger a post_call guardrail are buffered. * fix(bedrock): guardrail non-text converse stream deltas on passthrough de_anonymize_event_stream only routed delta.text through the post-call guardrail, so model output streamed in reasoningContent.text, toolUse.input or citationsContent.content[].text was forwarded raw and skipped masking/blocking. Collect every user-visible text field per contentBlockDelta, concatenate per logical stream so split mask tokens still reassemble, run them through the hook, then redistribute the guardrailed text back into the matching delta fields. This brings streaming coverage in line with the non-streaming Converse output handler. * refactor(proxy): keep bedrock event-stream content-type detection in llms Move the vnd.amazon.eventstream content-type check out of the proxy passthrough path into BedrockPassthroughGuardrailHandler via the LlmPassthroughRouteHandler dispatcher, so proxy code stays provider-agnostic. Also patch the actually-called _has_post_call_guardrails_for_passthrough in the malformed-body regression test instead of the unused _has_post_call_guardrails. * fix(proxy): forward upstream headers on bedrock guardrail passthrough responses Mirror the non-guardrail passthrough path by merging the upstream response headers (via get_response_headers) into the guardrailed non-streaming and event-stream responses, so headers like x-amzn-requestid survive when a post-call guardrail rewrites the body. Drop the stray fastapi HTTPException import from the SDK-tree handler test in favor of a local sentinel exception. * fix(bedrock): scan tool definitions and additional request fields for passthrough guardrails Converse passthrough guardrails only scanned system and message content, so a key holder could route blocked or PII text through toolConfig tool names, descriptions and input schemas or through additionalModelRequestFields, all of which are still forwarded to Bedrock. Collect strings from those fields too so key/team guardrails inspect and rewrite them, matching how the chat-completions path forwards tool definitions to guardrails. * fix(bedrock): log instead of silently dropping passthrough guardrail edge cases * test(local): skip httpbin timeout probe when the service returns 5xx local_testing_part1 was failing on test_post_delay_exceeds_per_request_timeout_raises because httpbin.org/delay/10 intermittently answers 503 instead of delaying, so HTTPHandler.post raised MaskedHTTPStatusError rather than the expected Timeout. The test already means to skip when httpbin is unavailable, but its guard only probed GET /get and ignored a server error on the delay endpoint. Treat a 5xx from httpbin as 'service unavailable' and skip, which is outside this repo's control, while still asserting Timeout when httpbin genuinely delays. * fix(proxy): set content-type on buffered bedrock passthrough event-stream responses * fix(bedrock): skip passthrough output write-back when guardrail returns no texts * fix(proxy): apply response-headers hook on guardrailed bedrock passthrough responses * refactor(bedrock): import event-stream crc32 from binascii not botocore internals * fix(proxy): scope bedrock passthrough stream buffering to de-anonymizable endpoints Only buffer a passthrough event stream into a non-streaming response when the resolved provider and endpoint actually have an event-stream guardrail handler that can rewrite frames (Bedrock converse-stream). Other Bedrock event-stream endpoints such as invoke-with-response-stream keep streaming, since the Converse handler leaves their frames untouched and buffering would silently break the streaming contract for no content change. * test(ui-e2e): re-issue deep-link navigation when auth bootstrap drops the page param navigateToPage deep-links to /ui?page=<page> then proceeds once the network settles, but a fresh load can race the auth bootstrap: the app momentarily treats the session as anonymous, bounces through /ui/login, and returns to the default Virtual Keys page with the ?page= query param dropped. The helper never checked where it actually landed, so any single bounce left callers asserting against the wrong page and timing out (mcpServers, modelHub, addModel). Confirm the requested page is what rendered and re-issue the navigation when it was clobbered; auth is warm by the second load so the param sticks. Migrated path routes are left alone since they intentionally leave the legacy root. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
factnn
pushed a commit
to factnn/litellm
that referenced
this pull request
Jun 18, 2026
BerriAI#30194) * feat(proxy): enforce key/team guardrails on bedrock passthrough routes /bedrock/... passthrough routes silently skipped all guardrail hooks because CallTypes.allm_passthrough_route had no entry in guardrail_translation_mappings. Add a dispatcher (LlmPassthroughRouteHandler) registered for that call type that routes to BedrockPassthroughGuardrailHandler for Bedrock Converse endpoints; wire post_call_success_hook into both the JSON and AWS event-stream response paths in common_request_processing, including full de-anonymization for streaming responses with proportional text distribution across original event-stream delta frames. * refactor(proxy): address greptile feedback on bedrock passthrough guardrails Move botocore event-stream logic from proxy/ to BedrockPassthroughGuardrailHandler.de_anonymize_converse_stream; _handle_event_stream_allm_passthrough_route becomes a thin provider dispatcher. Pass custom_headers through _handle_non_streaming_allm_passthrough_route so early-return guardrail responses include x-litellm-call-id and related headers. * style: run black on handler and common_request_processing * refactor(proxy): dedupe non-streaming passthrough guardrail handling Replace the inline JSON/eventstream block in the streaming-request branch with a call to _handle_non_streaming_allm_passthrough_route so both paths share one implementation and cannot diverge. * fix(bedrock): preserve trailing bytes when re-encoding converse stream The event-stream re-encoder only emitted parsed frames, so any trailing bytes left after the parse loop (truncated/corrupt final frame, or fewer than 16 bytes after the last complete frame) were silently dropped from the de-anonymized output. Capture and re-append them so the transformer never truncates the stream. * fix(proxy): guard non-dict post-call hook return on bedrock passthrough JSON path * fix(proxy): guard malformed JSON body on bedrock passthrough guardrail path * fix(proxy): close guardrail bypass via tool result text and default-mode post-call guardrails on bedrock passthrough Pre-call extraction only read top-level Converse text blocks, so blocked content placed under toolResult.content[].text was forwarded to Bedrock without the key/team guardrail seeing it. Extraction now walks nested tool result text and write-back mutates the owning block in place. Post-call buffering for passthrough used _has_post_call_guardrails, which excludes event_hook=None guardrails. Those guardrails run at post_call, so their output processing was skipped and the raw upstream body was returned. Add a passthrough-specific predicate that counts them. * refactor(proxy): route bedrock event-stream de-anonymization through llm passthrough dispatcher Remove the hardcoded bedrock provider guard from common_request_processing by delegating event-stream de-anonymization to LlmPassthroughRouteHandler, which resolves the provider from the existing handler registry. Keeps proxy/ provider-agnostic and reuses the same dispatch path as the input and output guardrail handlers. Also log instead of silently dropping the result when post_call_success_hook returns a non-dict on the JSON and event-stream passthrough paths. * fix(proxy): close guardrail bypass on bedrock invoke passthrough routes Pre-call extraction and post-call output processing only handled Converse shapes, so /bedrock/model/{modelId}/invoke and invoke-with-response-stream returned unguarded. An authenticated caller could move blocked content into an InvokeModel payload and skip the key/team guardrail entirely. Non-Converse Bedrock routes now fall back to the generic passthrough handler, which scans the full request and response payloads so blocking guardrails still run, matching how other passthrough providers are guarded. * fix(proxy): keep non-bedrock passthrough streams streaming under post-call guardrails * fix(bedrock): scan non-text converse blocks for passthrough guardrails Key/team guardrails on bedrock converse passthrough only saw top-level text blocks, so a caller could hide prompt content in toolUse.input or toolResult.content[].json and have it forwarded to Bedrock without the configured guardrail inspecting it, bypassing blocking guardrails by default. Walk those arbitrary-JSON subtrees and write masked values back in place. Extend the non-streaming converse response path to the equivalent model-output fields (toolUse.input, reasoningContent text and citationsContent text) while leaving structural values such as reasoning signatures and citation sources untouched. * fix(bedrock): make passthrough guardrail string collection iterative and type-safe Rewrite _collect_strings with an explicit stack so it no longer recurses, satisfying the recursive-function CI guard, and widen the holder container type so mypy accepts indexing JSON nodes by str or int keys. * fix(proxy): scope passthrough post-call guardrail buffering to the request Buffering the Bedrock event stream into a single non-streaming response was gated on whether any post_call guardrail existed globally, so every converse-stream request lost streaming once any post_call guardrail was registered, even for keys that did not reference it. Mirror the gate used by post_call_success_hook (should_run_guardrail against the request's merged guardrails) so only requests whose key/team actually trigger a post_call guardrail are buffered. * fix(bedrock): guardrail non-text converse stream deltas on passthrough de_anonymize_event_stream only routed delta.text through the post-call guardrail, so model output streamed in reasoningContent.text, toolUse.input or citationsContent.content[].text was forwarded raw and skipped masking/blocking. Collect every user-visible text field per contentBlockDelta, concatenate per logical stream so split mask tokens still reassemble, run them through the hook, then redistribute the guardrailed text back into the matching delta fields. This brings streaming coverage in line with the non-streaming Converse output handler. * refactor(proxy): keep bedrock event-stream content-type detection in llms Move the vnd.amazon.eventstream content-type check out of the proxy passthrough path into BedrockPassthroughGuardrailHandler via the LlmPassthroughRouteHandler dispatcher, so proxy code stays provider-agnostic. Also patch the actually-called _has_post_call_guardrails_for_passthrough in the malformed-body regression test instead of the unused _has_post_call_guardrails. * fix(proxy): forward upstream headers on bedrock guardrail passthrough responses Mirror the non-guardrail passthrough path by merging the upstream response headers (via get_response_headers) into the guardrailed non-streaming and event-stream responses, so headers like x-amzn-requestid survive when a post-call guardrail rewrites the body. Drop the stray fastapi HTTPException import from the SDK-tree handler test in favor of a local sentinel exception. * fix(bedrock): scan tool definitions and additional request fields for passthrough guardrails Converse passthrough guardrails only scanned system and message content, so a key holder could route blocked or PII text through toolConfig tool names, descriptions and input schemas or through additionalModelRequestFields, all of which are still forwarded to Bedrock. Collect strings from those fields too so key/team guardrails inspect and rewrite them, matching how the chat-completions path forwards tool definitions to guardrails. * fix(bedrock): log instead of silently dropping passthrough guardrail edge cases * test(local): skip httpbin timeout probe when the service returns 5xx local_testing_part1 was failing on test_post_delay_exceeds_per_request_timeout_raises because httpbin.org/delay/10 intermittently answers 503 instead of delaying, so HTTPHandler.post raised MaskedHTTPStatusError rather than the expected Timeout. The test already means to skip when httpbin is unavailable, but its guard only probed GET /get and ignored a server error on the delay endpoint. Treat a 5xx from httpbin as 'service unavailable' and skip, which is outside this repo's control, while still asserting Timeout when httpbin genuinely delays. * fix(proxy): set content-type on buffered bedrock passthrough event-stream responses * fix(bedrock): skip passthrough output write-back when guardrail returns no texts * fix(proxy): apply response-headers hook on guardrailed bedrock passthrough responses * refactor(bedrock): import event-stream crc32 from binascii not botocore internals * fix(proxy): scope bedrock passthrough stream buffering to de-anonymizable endpoints Only buffer a passthrough event stream into a non-streaming response when the resolved provider and endpoint actually have an event-stream guardrail handler that can rewrite frames (Bedrock converse-stream). Other Bedrock event-stream endpoints such as invoke-with-response-stream keep streaming, since the Converse handler leaves their frames untouched and buffering would silently break the streaming contract for no content change. * test(ui-e2e): re-issue deep-link navigation when auth bootstrap drops the page param navigateToPage deep-links to /ui?page=<page> then proceeds once the network settles, but a fresh load can race the auth bootstrap: the app momentarily treats the session as anonymous, bounces through /ui/login, and returns to the default Virtual Keys page with the ?page= query param dropped. The helper never checked where it actually landed, so any single bounce left callers asserting against the wrong page and timing out (mcpServers, modelHub, addModel). Confirm the requested page is what rendered and re-issue the navigation when it was clobbered; auth is warm by the second load so the param sticks. Migrated path routes are left alone since they intentionally leave the legacy root. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
BerriAI#30194) * feat(proxy): enforce key/team guardrails on bedrock passthrough routes /bedrock/... passthrough routes silently skipped all guardrail hooks because CallTypes.allm_passthrough_route had no entry in guardrail_translation_mappings. Add a dispatcher (LlmPassthroughRouteHandler) registered for that call type that routes to BedrockPassthroughGuardrailHandler for Bedrock Converse endpoints; wire post_call_success_hook into both the JSON and AWS event-stream response paths in common_request_processing, including full de-anonymization for streaming responses with proportional text distribution across original event-stream delta frames. * refactor(proxy): address greptile feedback on bedrock passthrough guardrails Move botocore event-stream logic from proxy/ to BedrockPassthroughGuardrailHandler.de_anonymize_converse_stream; _handle_event_stream_allm_passthrough_route becomes a thin provider dispatcher. Pass custom_headers through _handle_non_streaming_allm_passthrough_route so early-return guardrail responses include x-litellm-call-id and related headers. * style: run black on handler and common_request_processing * refactor(proxy): dedupe non-streaming passthrough guardrail handling Replace the inline JSON/eventstream block in the streaming-request branch with a call to _handle_non_streaming_allm_passthrough_route so both paths share one implementation and cannot diverge. * fix(bedrock): preserve trailing bytes when re-encoding converse stream The event-stream re-encoder only emitted parsed frames, so any trailing bytes left after the parse loop (truncated/corrupt final frame, or fewer than 16 bytes after the last complete frame) were silently dropped from the de-anonymized output. Capture and re-append them so the transformer never truncates the stream. * fix(proxy): guard non-dict post-call hook return on bedrock passthrough JSON path * fix(proxy): guard malformed JSON body on bedrock passthrough guardrail path * fix(proxy): close guardrail bypass via tool result text and default-mode post-call guardrails on bedrock passthrough Pre-call extraction only read top-level Converse text blocks, so blocked content placed under toolResult.content[].text was forwarded to Bedrock without the key/team guardrail seeing it. Extraction now walks nested tool result text and write-back mutates the owning block in place. Post-call buffering for passthrough used _has_post_call_guardrails, which excludes event_hook=None guardrails. Those guardrails run at post_call, so their output processing was skipped and the raw upstream body was returned. Add a passthrough-specific predicate that counts them. * refactor(proxy): route bedrock event-stream de-anonymization through llm passthrough dispatcher Remove the hardcoded bedrock provider guard from common_request_processing by delegating event-stream de-anonymization to LlmPassthroughRouteHandler, which resolves the provider from the existing handler registry. Keeps proxy/ provider-agnostic and reuses the same dispatch path as the input and output guardrail handlers. Also log instead of silently dropping the result when post_call_success_hook returns a non-dict on the JSON and event-stream passthrough paths. * fix(proxy): close guardrail bypass on bedrock invoke passthrough routes Pre-call extraction and post-call output processing only handled Converse shapes, so /bedrock/model/{modelId}/invoke and invoke-with-response-stream returned unguarded. An authenticated caller could move blocked content into an InvokeModel payload and skip the key/team guardrail entirely. Non-Converse Bedrock routes now fall back to the generic passthrough handler, which scans the full request and response payloads so blocking guardrails still run, matching how other passthrough providers are guarded. * fix(proxy): keep non-bedrock passthrough streams streaming under post-call guardrails * fix(bedrock): scan non-text converse blocks for passthrough guardrails Key/team guardrails on bedrock converse passthrough only saw top-level text blocks, so a caller could hide prompt content in toolUse.input or toolResult.content[].json and have it forwarded to Bedrock without the configured guardrail inspecting it, bypassing blocking guardrails by default. Walk those arbitrary-JSON subtrees and write masked values back in place. Extend the non-streaming converse response path to the equivalent model-output fields (toolUse.input, reasoningContent text and citationsContent text) while leaving structural values such as reasoning signatures and citation sources untouched. * fix(bedrock): make passthrough guardrail string collection iterative and type-safe Rewrite _collect_strings with an explicit stack so it no longer recurses, satisfying the recursive-function CI guard, and widen the holder container type so mypy accepts indexing JSON nodes by str or int keys. * fix(proxy): scope passthrough post-call guardrail buffering to the request Buffering the Bedrock event stream into a single non-streaming response was gated on whether any post_call guardrail existed globally, so every converse-stream request lost streaming once any post_call guardrail was registered, even for keys that did not reference it. Mirror the gate used by post_call_success_hook (should_run_guardrail against the request's merged guardrails) so only requests whose key/team actually trigger a post_call guardrail are buffered. * fix(bedrock): guardrail non-text converse stream deltas on passthrough de_anonymize_event_stream only routed delta.text through the post-call guardrail, so model output streamed in reasoningContent.text, toolUse.input or citationsContent.content[].text was forwarded raw and skipped masking/blocking. Collect every user-visible text field per contentBlockDelta, concatenate per logical stream so split mask tokens still reassemble, run them through the hook, then redistribute the guardrailed text back into the matching delta fields. This brings streaming coverage in line with the non-streaming Converse output handler. * refactor(proxy): keep bedrock event-stream content-type detection in llms Move the vnd.amazon.eventstream content-type check out of the proxy passthrough path into BedrockPassthroughGuardrailHandler via the LlmPassthroughRouteHandler dispatcher, so proxy code stays provider-agnostic. Also patch the actually-called _has_post_call_guardrails_for_passthrough in the malformed-body regression test instead of the unused _has_post_call_guardrails. * fix(proxy): forward upstream headers on bedrock guardrail passthrough responses Mirror the non-guardrail passthrough path by merging the upstream response headers (via get_response_headers) into the guardrailed non-streaming and event-stream responses, so headers like x-amzn-requestid survive when a post-call guardrail rewrites the body. Drop the stray fastapi HTTPException import from the SDK-tree handler test in favor of a local sentinel exception. * fix(bedrock): scan tool definitions and additional request fields for passthrough guardrails Converse passthrough guardrails only scanned system and message content, so a key holder could route blocked or PII text through toolConfig tool names, descriptions and input schemas or through additionalModelRequestFields, all of which are still forwarded to Bedrock. Collect strings from those fields too so key/team guardrails inspect and rewrite them, matching how the chat-completions path forwards tool definitions to guardrails. * fix(bedrock): log instead of silently dropping passthrough guardrail edge cases * test(local): skip httpbin timeout probe when the service returns 5xx local_testing_part1 was failing on test_post_delay_exceeds_per_request_timeout_raises because httpbin.org/delay/10 intermittently answers 503 instead of delaying, so HTTPHandler.post raised MaskedHTTPStatusError rather than the expected Timeout. The test already means to skip when httpbin is unavailable, but its guard only probed GET /get and ignored a server error on the delay endpoint. Treat a 5xx from httpbin as 'service unavailable' and skip, which is outside this repo's control, while still asserting Timeout when httpbin genuinely delays. * fix(proxy): set content-type on buffered bedrock passthrough event-stream responses * fix(bedrock): skip passthrough output write-back when guardrail returns no texts * fix(proxy): apply response-headers hook on guardrailed bedrock passthrough responses * refactor(bedrock): import event-stream crc32 from binascii not botocore internals * fix(proxy): scope bedrock passthrough stream buffering to de-anonymizable endpoints Only buffer a passthrough event stream into a non-streaming response when the resolved provider and endpoint actually have an event-stream guardrail handler that can rewrite frames (Bedrock converse-stream). Other Bedrock event-stream endpoints such as invoke-with-response-stream keep streaming, since the Converse handler leaves their frames untouched and buffering would silently break the streaming contract for no content change. * test(ui-e2e): re-issue deep-link navigation when auth bootstrap drops the page param navigateToPage deep-links to /ui?page=<page> then proceeds once the network settles, but a fresh load can race the auth bootstrap: the app momentarily treats the session as anonymous, bounces through /ui/login, and returns to the default Virtual Keys page with the ?page= query param dropped. The helper never checked where it actually landed, so any single bounce left callers asserting against the wrong page and timing out (mcpServers, modelHub, addModel). Confirm the requested page is what rendered and re-issue the navigation when it was clobbered; auth is warm by the second load so the param sticks. Migrated path routes are left alone since they intentionally leave the legacy root. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
13 tasks
blake-hamm
added a commit
to blake-hamm/bhamm-lab
that referenced
this pull request
Jun 28, 2026
…to v1.90.0 (#232) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [https://github.com/BerriAI/litellm.git](https://github.com/BerriAI/litellm) | minor | `v1.89.4` → `v1.90.0` | --- ### Release Notes <details> <summary>BerriAI/litellm (https://github.com/BerriAI/litellm.git)</summary> ### [`v1.90.0`](https://github.com/BerriAI/litellm/releases/tag/v1.90.0) [Compare Source](BerriAI/litellm@v1.89.4...v1.90.0-rc.1) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.90.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.90.0/cosign.pub \ ghcr.io/berriai/litellm:v1.90.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - fix(responses-bridge): map system-only chat request to system input item by [@​milan-berri](https://github.com/milan-berri) in [#​29817](BerriAI/litellm#29817) - feat(bedrock): forward strict and additionalProperties to Converse toolSpec by [@​mateo-berri](https://github.com/mateo-berri) in [#​29814](BerriAI/litellm#29814) - fix(mcp): highlight MCP cards red when the logged-in user is missing per-user env vars by [@​mateo-berri](https://github.com/mateo-berri) in [#​29856](BerriAI/litellm#29856) - feat(ui): add budget duration to edit team member form by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29717](BerriAI/litellm#29717) - fix(ui): make workflow runs page fill full width by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29868](BerriAI/litellm#29868) - feat: standardize rate limit errors with category, rate\_limit\_type, model, and llm\_provider fields by [@​mateo-berri](https://github.com/mateo-berri) in [#​27687](BerriAI/litellm#27687) - fix(ui): default guardrails page to the Guardrails tab by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29872](BerriAI/litellm#29872) - docs(readme): add Deploy on AWS/GCP Terraform section and fix deploy button rendering by [@​mateo-berri](https://github.com/mateo-berri) in [#​29879](BerriAI/litellm#29879) - refactor(bedrock): build Converse toolSpec via a BedrockToolSpec dict subclass by [@​mateo-berri](https://github.com/mateo-berri) in [#​29869](BerriAI/litellm#29869) - feat(litellm): add models and repository layers by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29686](BerriAI/litellm#29686) - feat(ui): include internal routes in the dashboard's generated OpenAPI types by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29885](BerriAI/litellm#29885) - feat(proxy): publish /v2/model/info in Swagger OpenAPI spec by [@​Sameerlite](https://github.com/Sameerlite) in [#​29900](BerriAI/litellm#29900) - refactor(ui): single source of truth for migrated-page routing by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29949](BerriAI/litellm#29949) - fix(ui/model-hub): render provider icons on the public model hub by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29958](BerriAI/litellm#29958) - fix(ui): keep create guardrail modal open on outside click by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29871](BerriAI/litellm#29871) - fix(ui): label default key type as "Full Access" on key edit page by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29870](BerriAI/litellm#29870) - fix(ui): unify migrated-route URLs and migrate the API Reference page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29953](BerriAI/litellm#29953) - fix(mcp): let non-creator users OAuth into OBO-mode MCP servers from the Tools page by [@​tin-berri](https://github.com/tin-berri) in [#​29867](BerriAI/litellm#29867) - Litellm oss staging 080626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29932](BerriAI/litellm#29932) - feat(galileo): add health check support for UI callback test by [@​Sameerlite](https://github.com/Sameerlite) in [#​29908](BerriAI/litellm#29908) - fix(model-management): allow deleting a BYOK model after its team is deleted by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29875](BerriAI/litellm#29875) - feat(jwt-auth): opt-in fallback to DB team on unresolved JWT claim by [@​milan-berri](https://github.com/milan-berri) in [#​28913](BerriAI/litellm#28913) - fix(team\_endpoints): don't block /team/update on unchanged team budget by [@​milan-berri](https://github.com/milan-berri) in [#​29525](BerriAI/litellm#29525) - fix(fireworks): enable tool calling for glm-5p1 in model cost map by [@​milan-berri](https://github.com/milan-berri) in [#​29697](BerriAI/litellm#29697) - fix(vertex): propagate Vertex AI metadata in streaming success callbacks by [@​Sameerlite](https://github.com/Sameerlite) in [#​29899](BerriAI/litellm#29899) - fix(ui): show team projects to internal users on key creation by [@​milan-berri](https://github.com/milan-berri) in [#​28855](BerriAI/litellm#28855) - build(deps): bump pyjwt to 2.13.0 and ws override to 8.20.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29982](BerriAI/litellm#29982) - fix(team-management): delete a team's BYOK models when the team is deleted by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29977](BerriAI/litellm#29977) - feat(vantage): include organization metadata in FOCUS Tags export by [@​milan-berri](https://github.com/milan-berri) in [#​28184](BerriAI/litellm#28184) - fix(guardrails): read CrowdStrike AIDR identity from both metadata bags by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29991](BerriAI/litellm#29991) - fix(mcp): mirror upstream token lifetime instead of forcing a 1h OBO expiry by [@​tin-berri](https://github.com/tin-berri) in [#​29951](BerriAI/litellm#29951) - feat(azure\_ai): add MAI-Image-2.5 image generation support by [@​Sameerlite](https://github.com/Sameerlite) in [#​29688](BerriAI/litellm#29688) - fix(mcp): load MCP tool configuration tools via the OBO/passthrough-aware GET path by [@​tin-berri](https://github.com/tin-berri) in [#​29960](BerriAI/litellm#29960) - fix(team): reserve team budget raises for proxy admins on /team/update by [@​milan-berri](https://github.com/milan-berri) in [#​30030](BerriAI/litellm#30030) - test(ui): data-driven App Router migration E2E smoke (default + server-root-path) by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29974](BerriAI/litellm#29974) - fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through by [@​michelligabriele](https://github.com/michelligabriele) in [#​24232](BerriAI/litellm#24232) - chore(ui): remove dead App Router route stubs under (dashboard) by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30045](BerriAI/litellm#30045) - fix(ui/mcp): reset OAuth state on create-server modal close so a prior server's token no longer leaks into the next add-server session by [@​tin-berri](https://github.com/tin-berri) in [#​30000](BerriAI/litellm#30000) - fix(mcp): allow team access-group grants in OAuth authorize/token access check by [@​tin-berri](https://github.com/tin-berri) in [#​30041](BerriAI/litellm#30041) - docs(security): require a reproduction video for vulnerability reports by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30063](BerriAI/litellm#30063) - feat(ui): add admin flag to disable in-product UI nudges for everyone by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29796](BerriAI/litellm#29796) - chore(ui): remove dead dashboard files and unused dependencies by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30047](BerriAI/litellm#30047) - fix(proxy): authorize batch files using upload target\_model\_names (LIT-3593) by [@​Sameerlite](https://github.com/Sameerlite) in [#​30009](BerriAI/litellm#30009) - Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI by [@​mateo-berri](https://github.com/mateo-berri) in [#​30064](BerriAI/litellm#30064) - Add Claude Fable 5 cost map entries (data-only hotfix for the hosted map) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30076](BerriAI/litellm#30076) - fix(caching): restore stored prompt\_tokens on embedding cache hits instead of recomputing by [@​michelligabriele](https://github.com/michelligabriele) in [#​30046](BerriAI/litellm#30046) - Litellm oss 090626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30021](BerriAI/litellm#30021) - fix(proxy): self-heal startup/reload prisma reads on engine disconnect by [@​michelligabriele](https://github.com/michelligabriele) in [#​28803](BerriAI/litellm#28803) - chore(ui): make knip recognize .mjs scripts and openapi-typescript by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30052](BerriAI/litellm#30052) - fix(register\_model): preserve built-in cache pricing when registering custom overrides under unmapped keys by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30044](BerriAI/litellm#30044) - \[internal copy of [#​28007](BerriAI/litellm#28007)] Fix/gcp model garden streaming by [@​mateo-berri](https://github.com/mateo-berri) in [#​28363](BerriAI/litellm#28363) - feat(cli): per-agent `lite claude` / `codex` / `opencode` commands that wrap coding agents through the proxy by [@​mateo-berri](https://github.com/mateo-berri) in [#​29850](BerriAI/litellm#29850) - fix(callbacks): forward callback\_settings to callback initializers and guard consumers against non-dict values by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30161](BerriAI/litellm#30161) - fix(mcp): drop orphaned per-user credential rows when an MCP server is deleted by [@​tin-berri](https://github.com/tin-berri) in [#​30141](BerriAI/litellm#30141) - fix(proxy): recover from cached-plan errors by reconnecting the Prisma client by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29983](BerriAI/litellm#29983) - feat(proxy): add option to disable server-side prepared statements for DB lookups by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29984](BerriAI/litellm#29984) - fix(release): stop backport releases from overwriting the latest badge by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30005](BerriAI/litellm#30005) - feat: add conventional commits and coding guidelines by [@​mateo-berri](https://github.com/mateo-berri) in [#​30159](BerriAI/litellm#30159) - fix(proxy): return 5xx on DB infra errors during auth; reserve 401 for genuine auth failures by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29986](BerriAI/litellm#29986) - fix(ui): dev server 404s on migrated-page links because uiBase hardcodes /ui by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30169](BerriAI/litellm#30169) - refactor(ui): consolidate dashboard to one shell in the (dashboard) layout by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30166](BerriAI/litellm#30166) - fix(proxy): align /v1/model/info with router deployments by [@​Sameerlite](https://github.com/Sameerlite) in [#​30025](BerriAI/litellm#30025) - fix: completion\_cost AttributeError on streaming Anthropic web\_search responses ([#​26153](BerriAI/litellm#26153)) by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​27346](BerriAI/litellm#27346) - \[internal copy of [#​30137](BerriAI/litellm#30137)] perf(realtime): eliminate redundant per-frame JSON work on OpenAI realtime relay by [@​mateo-berri](https://github.com/mateo-berri) in [#​30142](BerriAI/litellm#30142) - feat(bedrock): aws\_bedrock\_project\_id for bedrock-mantle project / workspace association by [@​mateo-berri](https://github.com/mateo-berri) in [#​30163](BerriAI/litellm#30163) - chore(hooks): enforce Conventional Commits and Conventional Branches by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30174](BerriAI/litellm#30174) - feat(rate-limiter): allow opting out of v3 TPM reservation and Redis circuit breaker by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30211](BerriAI/litellm#30211) - feat(spend\_logs): opt-in native Postgres partitioning for SpendLogs retention by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29466](BerriAI/litellm#29466) - feat(ui): migrate playground to path routing and colocate its files by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30185](BerriAI/litellm#30185) - feat(ui): migrate projects and access-groups to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30226](BerriAI/litellm#30226) - fix(proxy): coalesce NULL rollup metrics in aggregated daily-activity by [@​michelligabriele](https://github.com/michelligabriele) in [#​30151](BerriAI/litellm#30151) - fix(anthropic\_passthrough): resolve costing model from message\_start chunk, litellm\_params and model\_group instead of 'unknown' by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30160](BerriAI/litellm#30160) - feat(ui): migrate budgets, workflows, and guardrails-monitor to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30236](BerriAI/litellm#30236) - feat(ui): migrate mcp-servers, search-tools, tag-management, vector-stores, and memory to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30261](BerriAI/litellm#30261) - fix(a2a): forward agent\_extra\_headers through completion bridge by [@​mateo-berri](https://github.com/mateo-berri) in [#​28277](BerriAI/litellm#28277) - fix(gemini-live): forward audio buffer commit and correct Vertex PCM rate by [@​Sameerlite](https://github.com/Sameerlite) in [#​29946](BerriAI/litellm#29946) - fix(proxy): skip double-wrapping unified batch output file ids on retrieve by [@​Sameerlite](https://github.com/Sameerlite) in [#​30011](BerriAI/litellm#30011) - feat: litellm oss 110626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30202](BerriAI/litellm#30202) - fix(docker): copy only runtime artifacts into the final image by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30243](BerriAI/litellm#30243) - feat(proxy): enforce key/team guardrails on bedrock passthrough routes by [@​Sameerlite](https://github.com/Sameerlite) in [#​30194](BerriAI/litellm#30194) - feat(gemini): forward web search tools in image generation by [@​Sameerlite](https://github.com/Sameerlite) in [#​30119](BerriAI/litellm#30119) - fix: bedrock mantle fixes by [@​Sameerlite](https://github.com/Sameerlite) in [#​30083](BerriAI/litellm#30083) - feat(proxy): add require\_managed\_files setting for file uploads by [@​Sameerlite](https://github.com/Sameerlite) in [#​30186](BerriAI/litellm#30186) - fix(mcp): honor server\_id for REST tool calls with shared upstream URLs by [@​Sameerlite](https://github.com/Sameerlite) in [#​30184](BerriAI/litellm#30184) - fix(responses): presidio PII masking for Azure WebSocket and streaming by [@​Sameerlite](https://github.com/Sameerlite) in [#​30003](BerriAI/litellm#30003) - feat(passthrough): add configurable pass-through request timeouts by [@​Sameerlite](https://github.com/Sameerlite) in [#​30266](BerriAI/litellm#30266) - fix(google\_genai): preserve complete SSE events in Vertex/Gemini image streaming by [@​Sameerlite](https://github.com/Sameerlite) in [#​30270](BerriAI/litellm#30270) - fix(proxy): populate access\_via\_team\_ids on /v1/model/info by [@​Sameerlite](https://github.com/Sameerlite) in [#​30274](BerriAI/litellm#30274) - chore(oss): litellm oss staging 120626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30292](BerriAI/litellm#30292) - feat(ui): migrate policies, guardrails, prompts, tool-policies, and skills to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30263](BerriAI/litellm#30263) - feat(ui): migrate caching, cost-tracking, transform-request, ui-theme, and logs to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30267](BerriAI/litellm#30267) - fix(ui): gate dashboard layout on ui config load so deep links work under SERVER\_ROOT\_PATH by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30312](BerriAI/litellm#30312) - feat(ui): migrate admin-panel, logging-and-alerts, model-hub-table, and usage to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30268](BerriAI/litellm#30268) - fix(otel): cap metric attribute cardinality with include/exclude lists by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30257](BerriAI/litellm#30257) - fix(proxy): grace-period key rotation 401s; return deprecated-key lookup result directly by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30327](BerriAI/litellm#30327) - chore(deps): bump vitest, brace-expansion, pypdf and tornado by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30220](BerriAI/litellm#30220) - refactor(ui): remove unreachable /chat page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30178](BerriAI/litellm#30178) - feat(ui): migrate agents and router-settings to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30323](BerriAI/litellm#30323) - feat: strengthen coding conventions in CLAUDE.md by [@​mateo-berri](https://github.com/mateo-berri) in [#​30333](BerriAI/litellm#30333) - feat(ui): cut the users page over to the /ui/users path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30334](BerriAI/litellm#30334) - feat: ruff strict-rule suppressions baseline gate by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30303](BerriAI/litellm#30303) - feat(guardrails): add Cisco AI Defense integration ([#​28249](BerriAI/litellm#28249)) by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30338](BerriAI/litellm#30338) - chore(ui): remove dead UI components unreferenced by any page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30340](BerriAI/litellm#30340) - ci: add osv-scanner lockfile scan workflow by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30222](BerriAI/litellm#30222) - fix(otel): record full error message on standard exception event in otel v2 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30380](BerriAI/litellm#30380) - test(fireworks): mock whisper transcription tests instead of live calls by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30391](BerriAI/litellm#30391) - build(ui): pin esbuild to 0.28.1 via overrides by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30390](BerriAI/litellm#30390) - feat(ui): cut the organizations page over to the /ui/organizations path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30336](BerriAI/litellm#30336) - fix(proxy): support SMTP implicit SSL (port 465) by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30395](BerriAI/litellm#30395) - fix(mcp): default Linear MCP registry entry to streamable HTTP by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30396](BerriAI/litellm#30396) - fix(ui): stop Virtual Keys page from infinite render loop by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30397](BerriAI/litellm#30397) - fix(streaming): guard raise\_on\_model\_repetition against empty choices by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30485](BerriAI/litellm#30485) - feat(otel-v2): emit the 6 gen\_ai.client.\* metrics at parity with v1 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30326](BerriAI/litellm#30326) - fix(mcp): drop phantom 401 span on delegated OAuth2 tool calls by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30494](BerriAI/litellm#30494) - feat(ui): cut the teams page over to the /ui/teams path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30343](BerriAI/litellm#30343) - fix(integrations): cap Anthropic cache\_control injection at 4 blocks by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30480](BerriAI/litellm#30480) - chore(codecov): add Batches, Videos, and Realtime components by [@​Sameerlite](https://github.com/Sameerlite) in [#​30517](BerriAI/litellm#30517) - test(batches): move orphan tests into tests/test\_litellm for CI coverage by [@​Sameerlite](https://github.com/Sameerlite) in [#​30510](BerriAI/litellm#30510) - fix(guardrails): run pre\_call hook once for model-level guardrails by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30543](BerriAI/litellm#30543) - fix(guardrails): stop re-initializing DB guardrails on every poll by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30542](BerriAI/litellm#30542) - chore(oss): litellm oss staging 150626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30463](BerriAI/litellm#30463) - ci(lint): add blanket-noqa, dataclass-default, and unused-noqa Ruff rules by [@​mateo-berri](https://github.com/mateo-berri) in [#​30516](BerriAI/litellm#30516) - ci: ratchet lint and type-check gates (ruff preview, ANN, mypy, basedpyright) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30379](BerriAI/litellm#30379) - fix(proxy): allow internal roles to access vector store CRUD routes by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30503](BerriAI/litellm#30503) - fix(otel): stamp gen\_ai.input/output.messages on v2 spans by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30548](BerriAI/litellm#30548) - fix(otel): export v2 gen\_ai client metrics to the configured meter provider by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30549](BerriAI/litellm#30549) - fix(bedrock): preserve cache\_control for ARN models in /v1/messages adapter by [@​mateo-berri](https://github.com/mateo-berri) in [#​29823](BerriAI/litellm#29823) - fix: greatly increase basedpyright slack by [@​mateo-berri](https://github.com/mateo-berri) in [#​30563](BerriAI/litellm#30563) - fix(budget): recompute budget\_reset\_at when budget\_duration changes on /budget/update by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30555](BerriAI/litellm#30555) - fix(otel): accept UPPER\_SNAKE\_CASE OTEL\_INSTRUMENTATION\_GENAI\_CAPTURE\_MESSAGE\_CONTENT in v2 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30562](BerriAI/litellm#30562) - chore(lint): remove PLR0915 too-many-statements ruff rule by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30574](BerriAI/litellm#30574) - ci(lint): ratcheted type-discipline gate (mutable collections, casts, guards, kwargs, suppressions) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30500](BerriAI/litellm#30500) - feat(proxy): add verification\_uri\_complete to CLI SSO device flow by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30571](BerriAI/litellm#30571) - chore: litellm oss staging160626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30527](BerriAI/litellm#30527) - fix(guardrails): return 400 not 500 when AIM blocks a request by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30573](BerriAI/litellm#30573) - ci(lint): grandfather any-discipline with a per-file ratchet budget (50% headroom) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30582](BerriAI/litellm#30582) - fix(audio): don't override explicit response\_format with verbose\_json by [@​mateo-berri](https://github.com/mateo-berri) in [#​30599](BerriAI/litellm#30599) - fix(anthropic): price and surface response service\_tier in cost tracking by [@​mateo-berri](https://github.com/mateo-berri) in [#​30558](BerriAI/litellm#30558) - feat: add dev and wildcard proxy configs for local testing by [@​mateo-berri](https://github.com/mateo-berri) in [#​30556](BerriAI/litellm#30556) - fix(proxy): list public team model name in /v1/models by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​30588](BerriAI/litellm#30588) - ci: drop mypy entirely, standardize type checking on basedpyright by [@​mateo-berri](https://github.com/mateo-berri) in [#​30648](BerriAI/litellm#30648) - feat(guardrails): surface OpenAI moderation violation\_categories on guardrail traces by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30659](BerriAI/litellm#30659) - fix(proxy): resolve list files credentials from team BYOK deployments by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30495](BerriAI/litellm#30495) - feat(proxy): add --max\_requests\_before\_restart\_jitter to stagger worker restarts by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30601](BerriAI/litellm#30601) - fix(health): correct bedrock embedding health checks by [@​mateo-berri](https://github.com/mateo-berri) in [#​30583](BerriAI/litellm#30583) - test: harden remaining pass-through CI flakes (image-gen spend poll, ruby assistants timeout) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30685](BerriAI/litellm#30685) - test(pass\_through): harden vertex spendlog poll against transient empty reads by [@​mateo-berri](https://github.com/mateo-berri) in [#​30683](BerriAI/litellm#30683) - fix(cost): stop non-string service\_tier from silently dropping cost tracking by [@​mateo-berri](https://github.com/mateo-berri) in [#​30690](BerriAI/litellm#30690) - feat(proxy): warn at startup when custom\_auth skips common\_checks enforcement by [@​tin-berri](https://github.com/tin-berri) in [#​30665](BerriAI/litellm#30665) - fix(pod\_lock): release cron lock by matching async\_set\_cache JSON encoding by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30600](BerriAI/litellm#30600) - ci: run a local fake OpenAI endpoint instead of the shared Railway mock by [@​mateo-berri](https://github.com/mateo-berri) in [#​30695](BerriAI/litellm#30695) - ci(windows): pin uv to Python 3.11 so it ignores the preinstalled 3.14 by [@​mateo-berri](https://github.com/mateo-berri) in [#​30704](BerriAI/litellm#30704) - feat(ui): migrate models page to App Router path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30677](BerriAI/litellm#30677) - refactor(ui): remove orphaned pass-through-settings route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30692](BerriAI/litellm#30692) - fix(cost): stop non-string response service\_tier from dropping cost tracking by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30706](BerriAI/litellm#30706) - feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle by [@​mateo-berri](https://github.com/mateo-berri) in [#​30433](BerriAI/litellm#30433) - chore: litellm oss 170626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30637](BerriAI/litellm#30637) - fix(bedrock\_mantle): add SigV4 fallback to chat completions auth by [@​mateo-berri](https://github.com/mateo-berri) in [#​30714](BerriAI/litellm#30714) - feat(search): add TinyFish as search provider by [@​simantak-dabhade](https://github.com/simantak-dabhade) in [#​30634](BerriAI/litellm#30634) - feat(ui): migrate old usage report to App Router path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30694](BerriAI/litellm#30694) - fix(proxy): enforce budgets against authoritative DB spend when the cross-pod counter is stale by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30684](BerriAI/litellm#30684) - chore(ci): remove Agent Shin pull\_request\_target workflows by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30784](BerriAI/litellm#30784) - chore: litellm oss staging by [@​Sameerlite](https://github.com/Sameerlite) in [#​30745](BerriAI/litellm#30745) - ci(zizmor): also run on litellm\_internal\_staging by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30789](BerriAI/litellm#30789) - fix(test): drop references to removed Agent Shin workflows by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30791](https://github.com/BerriAI/litellm/pull/30791) - chore: remove in-product survey and Claude Code feedback nudges by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30773](https://github.com/BerriAI/litellm/pull/30773) - feat(ui): migrate api-keys landing to App Router path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30699](https://github.com/BerriAI/litellm/pull/30699) - feat(proxy): configurable response headers and login-page hint by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​30792](https://github.com/BerriAI/litellm/pull/30792) - ci(zizmor): gate PRs on medium+ findings and clear existing ones by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30797](https://github.com/BerriAI/litellm/pull/30797) - fix(proxy): use e.request\_data for logging\_obj in ModifyResponseException streaming passthrough by [@​mateo-berri](https://github.com/mateo-berri) in [#​30800](https://github.com/BerriAI/litellm/pull/30800) - chore: make pr template linear portion clearer by [@​mateo-berri](https://github.com/mateo-berri) in [#​30766](https://github.com/BerriAI/litellm/pull/30766) - chore(typing): add boto3/botocore stubs so basedpyright resolves the AWS SDK by [@​mateo-berri](https://github.com/mateo-berri) in [#​30815](https://github.com/BerriAI/litellm/pull/30815) - fix(otel): one v2 logger owns the global provider; scope tenant OTLP creds per exporter by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​30590](https://github.com/BerriAI/litellm/pull/30590) - fix(passthrough): recover output tokens for interrupted anthropic streams by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30787](https://github.com/BerriAI/litellm/pull/30787) - fix(proxy): record partial spend on the failure row for interrupted streams by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30788](https://github.com/BerriAI/litellm/pull/30788) - fix(ui): repoint dead usage guide link to cost tracking docs by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30859](https://github.com/BerriAI/litellm/pull/30859) - fix(ui): warn that team models are deleted in the delete-team modal by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29990](https://github.com/BerriAI/litellm/pull/29990) - feat(caching): add valkey-semantic cache backend and fix semantic cache scope keys by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30675](https://github.com/BerriAI/litellm/pull/30675) - test(ui): isolate OldTeams delete-warning tests from leaked mock by [@​mateo-berri](https://github.com/mateo-berri) in [#​30871](https://github.com/BerriAI/litellm/pull/30871) - feat: add lint-gate target and truncation-proof summary to the strict ruff gate by [@​mateo-berri](https://github.com/mateo-berri) in [#​30877](https://github.com/BerriAI/litellm/pull/30877) - chore(ui): rebuild ui for release by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30894](https://github.com/BerriAI/litellm/pull/30894) - chore(ci): bump deps by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30899](https://github.com/BerriAI/litellm/pull/30899) - fix(watsonx): wrap string embedding input in array for WatsonX API by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30897](https://github.com/BerriAI/litellm/pull/30897) - test: point router/completion/triton tests at the local fake OpenAI endpoint by [@​mateo-berri](https://github.com/mateo-berri) in [#​30900](https://github.com/BerriAI/litellm/pull/30900) - feat(sandbox): e2b code execution primitive by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​30898](https://github.com/BerriAI/litellm/pull/30898) - fix(ui): source api-keys identity from useAuthorized to stop "User ID is not set" by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30903](https://github.com/BerriAI/litellm/pull/30903) - chore(ui): rebuild ui by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30906](https://github.com/BerriAI/litellm/pull/30906) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30907](https://github.com/BerriAI/litellm/pull/30907) - fix(redis): prevent forcing SSLConnection when ssl=False in connection pool by [@​Jacopos311](https://github.com/Jacopos311) in [#​30770](https://github.com/BerriAI/litellm/pull/30770) - fix(proxy): log UI setup failures instead of silently swallowing by [@​sarvesh1327](https://github.com/sarvesh1327) in [#​30819](https://github.com/BerriAI/litellm/pull/30819) #### New Contributors - [@​simantak-dabhade](https://github.com/simantak-dabhade) made their first contribution in [#​30634](BerriAI/litellm#30634) - [@​Jacopos311](https://github.com/Jacopos311) made their first contribution in [#​30770](https://github.com/BerriAI/litellm/pull/30770) - [@​sarvesh1327](https://github.com/sarvesh1327) made their first contribution in [#​30819](https://github.com/BerriAI/litellm/pull/30819) **Full Changelog**: <BerriAI/litellm@v1.89.0...v1.90.0> </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMjAuMCIsInVwZGF0ZWRJblZlciI6IjQzLjIyMC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=--> Co-authored-by: Renovate Bot <renovate@bhamm-lab.com> Reviewed-on: https://codeberg.org/blake-hamm/bhamm-lab/pulls/232
blake-hamm
added a commit
to blake-hamm/bhamm-lab
that referenced
this pull request
Jun 28, 2026
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | final | minor | `v1.85.1` → `v1.90.0` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.90.0`](https://github.com/BerriAI/litellm/releases/tag/v1.90.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.90.0...v1.90.0) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.90.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.90.0/cosign.pub \ ghcr.io/berriai/litellm:v1.90.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - fix(responses-bridge): map system-only chat request to system input item by [@​milan-berri](https://github.com/milan-berri) in [#​29817](https://github.com/BerriAI/litellm/pull/29817) - feat(bedrock): forward strict and additionalProperties to Converse toolSpec by [@​mateo-berri](https://github.com/mateo-berri) in [#​29814](https://github.com/BerriAI/litellm/pull/29814) - fix(mcp): highlight MCP cards red when the logged-in user is missing per-user env vars by [@​mateo-berri](https://github.com/mateo-berri) in [#​29856](https://github.com/BerriAI/litellm/pull/29856) - feat(ui): add budget duration to edit team member form by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29717](https://github.com/BerriAI/litellm/pull/29717) - fix(ui): make workflow runs page fill full width by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29868](https://github.com/BerriAI/litellm/pull/29868) - feat: standardize rate limit errors with category, rate\_limit\_type, model, and llm\_provider fields by [@​mateo-berri](https://github.com/mateo-berri) in [#​27687](https://github.com/BerriAI/litellm/pull/27687) - fix(ui): default guardrails page to the Guardrails tab by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29872](https://github.com/BerriAI/litellm/pull/29872) - docs(readme): add Deploy on AWS/GCP Terraform section and fix deploy button rendering by [@​mateo-berri](https://github.com/mateo-berri) in [#​29879](https://github.com/BerriAI/litellm/pull/29879) - refactor(bedrock): build Converse toolSpec via a BedrockToolSpec dict subclass by [@​mateo-berri](https://github.com/mateo-berri) in [#​29869](https://github.com/BerriAI/litellm/pull/29869) - feat(litellm): add models and repository layers by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29686](https://github.com/BerriAI/litellm/pull/29686) - feat(ui): include internal routes in the dashboard's generated OpenAPI types by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29885](https://github.com/BerriAI/litellm/pull/29885) - feat(proxy): publish /v2/model/info in Swagger OpenAPI spec by [@​Sameerlite](https://github.com/Sameerlite) in [#​29900](https://github.com/BerriAI/litellm/pull/29900) - refactor(ui): single source of truth for migrated-page routing by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29949](https://github.com/BerriAI/litellm/pull/29949) - fix(ui/model-hub): render provider icons on the public model hub by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29958](https://github.com/BerriAI/litellm/pull/29958) - fix(ui): keep create guardrail modal open on outside click by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29871](https://github.com/BerriAI/litellm/pull/29871) - fix(ui): label default key type as "Full Access" on key edit page by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29870](https://github.com/BerriAI/litellm/pull/29870) - fix(ui): unify migrated-route URLs and migrate the API Reference page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29953](https://github.com/BerriAI/litellm/pull/29953) - fix(mcp): let non-creator users OAuth into OBO-mode MCP servers from the Tools page by [@​tin-berri](https://github.com/tin-berri) in [#​29867](https://github.com/BerriAI/litellm/pull/29867) - Litellm oss staging 080626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29932](https://github.com/BerriAI/litellm/pull/29932) - feat(galileo): add health check support for UI callback test by [@​Sameerlite](https://github.com/Sameerlite) in [#​29908](https://github.com/BerriAI/litellm/pull/29908) - fix(model-management): allow deleting a BYOK model after its team is deleted by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29875](https://github.com/BerriAI/litellm/pull/29875) - feat(jwt-auth): opt-in fallback to DB team on unresolved JWT claim by [@​milan-berri](https://github.com/milan-berri) in [#​28913](https://github.com/BerriAI/litellm/pull/28913) - fix(team\_endpoints): don't block /team/update on unchanged team budget by [@​milan-berri](https://github.com/milan-berri) in [#​29525](https://github.com/BerriAI/litellm/pull/29525) - fix(fireworks): enable tool calling for glm-5p1 in model cost map by [@​milan-berri](https://github.com/milan-berri) in [#​29697](https://github.com/BerriAI/litellm/pull/29697) - fix(vertex): propagate Vertex AI metadata in streaming success callbacks by [@​Sameerlite](https://github.com/Sameerlite) in [#​29899](https://github.com/BerriAI/litellm/pull/29899) - fix(ui): show team projects to internal users on key creation by [@​milan-berri](https://github.com/milan-berri) in [#​28855](https://github.com/BerriAI/litellm/pull/28855) - build(deps): bump pyjwt to 2.13.0 and ws override to 8.20.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29982](https://github.com/BerriAI/litellm/pull/29982) - fix(team-management): delete a team's BYOK models when the team is deleted by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29977](https://github.com/BerriAI/litellm/pull/29977) - feat(vantage): include organization metadata in FOCUS Tags export by [@​milan-berri](https://github.com/milan-berri) in [#​28184](https://github.com/BerriAI/litellm/pull/28184) - fix(guardrails): read CrowdStrike AIDR identity from both metadata bags by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29991](https://github.com/BerriAI/litellm/pull/29991) - fix(mcp): mirror upstream token lifetime instead of forcing a 1h OBO expiry by [@​tin-berri](https://github.com/tin-berri) in [#​29951](https://github.com/BerriAI/litellm/pull/29951) - feat(azure\_ai): add MAI-Image-2.5 image generation support by [@​Sameerlite](https://github.com/Sameerlite) in [#​29688](https://github.com/BerriAI/litellm/pull/29688) - fix(mcp): load MCP tool configuration tools via the OBO/passthrough-aware GET path by [@​tin-berri](https://github.com/tin-berri) in [#​29960](https://github.com/BerriAI/litellm/pull/29960) - fix(team): reserve team budget raises for proxy admins on /team/update by [@​milan-berri](https://github.com/milan-berri) in [#​30030](https://github.com/BerriAI/litellm/pull/30030) - test(ui): data-driven App Router migration E2E smoke (default + server-root-path) by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29974](https://github.com/BerriAI/litellm/pull/29974) - fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through by [@​michelligabriele](https://github.com/michelligabriele) in [#​24232](https://github.com/BerriAI/litellm/pull/24232) - chore(ui): remove dead App Router route stubs under (dashboard) by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30045](https://github.com/BerriAI/litellm/pull/30045) - fix(ui/mcp): reset OAuth state on create-server modal close so a prior server's token no longer leaks into the next add-server session by [@​tin-berri](https://github.com/tin-berri) in [#​30000](https://github.com/BerriAI/litellm/pull/30000) - fix(mcp): allow team access-group grants in OAuth authorize/token access check by [@​tin-berri](https://github.com/tin-berri) in [#​30041](https://github.com/BerriAI/litellm/pull/30041) - docs(security): require a reproduction video for vulnerability reports by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30063](https://github.com/BerriAI/litellm/pull/30063) - feat(ui): add admin flag to disable in-product UI nudges for everyone by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29796](https://github.com/BerriAI/litellm/pull/29796) - chore(ui): remove dead dashboard files and unused dependencies by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30047](https://github.com/BerriAI/litellm/pull/30047) - fix(proxy): authorize batch files using upload target\_model\_names (LIT-3593) by [@​Sameerlite](https://github.com/Sameerlite) in [#​30009](https://github.com/BerriAI/litellm/pull/30009) - Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI by [@​mateo-berri](https://github.com/mateo-berri) in [#​30064](https://github.com/BerriAI/litellm/pull/30064) - Add Claude Fable 5 cost map entries (data-only hotfix for the hosted map) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30076](https://github.com/BerriAI/litellm/pull/30076) - fix(caching): restore stored prompt\_tokens on embedding cache hits instead of recomputing by [@​michelligabriele](https://github.com/michelligabriele) in [#​30046](https://github.com/BerriAI/litellm/pull/30046) - Litellm oss 090626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30021](https://github.com/BerriAI/litellm/pull/30021) - fix(proxy): self-heal startup/reload prisma reads on engine disconnect by [@​michelligabriele](https://github.com/michelligabriele) in [#​28803](https://github.com/BerriAI/litellm/pull/28803) - chore(ui): make knip recognize .mjs scripts and openapi-typescript by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30052](https://github.com/BerriAI/litellm/pull/30052) - fix(register\_model): preserve built-in cache pricing when registering custom overrides under unmapped keys by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30044](https://github.com/BerriAI/litellm/pull/30044) - \[internal copy of [#​28007](https://github.com/BerriAI/litellm/issues/28007)] Fix/gcp model garden streaming by [@​mateo-berri](https://github.com/mateo-berri) in [#​28363](https://github.com/BerriAI/litellm/pull/28363) - feat(cli): per-agent `lite claude` / `codex` / `opencode` commands that wrap coding agents through the proxy by [@​mateo-berri](https://github.com/mateo-berri) in [#​29850](https://github.com/BerriAI/litellm/pull/29850) - fix(callbacks): forward callback\_settings to callback initializers and guard consumers against non-dict values by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30161](https://github.com/BerriAI/litellm/pull/30161) - fix(mcp): drop orphaned per-user credential rows when an MCP server is deleted by [@​tin-berri](https://github.com/tin-berri) in [#​30141](https://github.com/BerriAI/litellm/pull/30141) - fix(proxy): recover from cached-plan errors by reconnecting the Prisma client by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29983](https://github.com/BerriAI/litellm/pull/29983) - feat(proxy): add option to disable server-side prepared statements for DB lookups by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29984](https://github.com/BerriAI/litellm/pull/29984) - fix(release): stop backport releases from overwriting the latest badge by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30005](https://github.com/BerriAI/litellm/pull/30005) - feat: add conventional commits and coding guidelines by [@​mateo-berri](https://github.com/mateo-berri) in [#​30159](https://github.com/BerriAI/litellm/pull/30159) - fix(proxy): return 5xx on DB infra errors during auth; reserve 401 for genuine auth failures by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29986](https://github.com/BerriAI/litellm/pull/29986) - fix(ui): dev server 404s on migrated-page links because uiBase hardcodes /ui by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30169](https://github.com/BerriAI/litellm/pull/30169) - refactor(ui): consolidate dashboard to one shell in the (dashboard) layout by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30166](https://github.com/BerriAI/litellm/pull/30166) - fix(proxy): align /v1/model/info with router deployments by [@​Sameerlite](https://github.com/Sameerlite) in [#​30025](https://github.com/BerriAI/litellm/pull/30025) - fix: completion\_cost AttributeError on streaming Anthropic web\_search responses ([#​26153](https://github.com/BerriAI/litellm/issues/26153)) by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​27346](https://github.com/BerriAI/litellm/pull/27346) - \[internal copy of [#​30137](https://github.com/BerriAI/litellm/issues/30137)] perf(realtime): eliminate redundant per-frame JSON work on OpenAI realtime relay by [@​mateo-berri](https://github.com/mateo-berri) in [#​30142](https://github.com/BerriAI/litellm/pull/30142) - feat(bedrock): aws\_bedrock\_project\_id for bedrock-mantle project / workspace association by [@​mateo-berri](https://github.com/mateo-berri) in [#​30163](https://github.com/BerriAI/litellm/pull/30163) - chore(hooks): enforce Conventional Commits and Conventional Branches by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30174](https://github.com/BerriAI/litellm/pull/30174) - feat(rate-limiter): allow opting out of v3 TPM reservation and Redis circuit breaker by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30211](https://github.com/BerriAI/litellm/pull/30211) - feat(spend\_logs): opt-in native Postgres partitioning for SpendLogs retention by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29466](https://github.com/BerriAI/litellm/pull/29466) - feat(ui): migrate playground to path routing and colocate its files by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30185](https://github.com/BerriAI/litellm/pull/30185) - feat(ui): migrate projects and access-groups to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30226](https://github.com/BerriAI/litellm/pull/30226) - fix(proxy): coalesce NULL rollup metrics in aggregated daily-activity by [@​michelligabriele](https://github.com/michelligabriele) in [#​30151](https://github.com/BerriAI/litellm/pull/30151) - fix(anthropic\_passthrough): resolve costing model from message\_start chunk, litellm\_params and model\_group instead of 'unknown' by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30160](https://github.com/BerriAI/litellm/pull/30160) - feat(ui): migrate budgets, workflows, and guardrails-monitor to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30236](https://github.com/BerriAI/litellm/pull/30236) - feat(ui): migrate mcp-servers, search-tools, tag-management, vector-stores, and memory to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30261](https://github.com/BerriAI/litellm/pull/30261) - fix(a2a): forward agent\_extra\_headers through completion bridge by [@​mateo-berri](https://github.com/mateo-berri) in [#​28277](https://github.com/BerriAI/litellm/pull/28277) - fix(gemini-live): forward audio buffer commit and correct Vertex PCM rate by [@​Sameerlite](https://github.com/Sameerlite) in [#​29946](https://github.com/BerriAI/litellm/pull/29946) - fix(proxy): skip double-wrapping unified batch output file ids on retrieve by [@​Sameerlite](https://github.com/Sameerlite) in [#​30011](https://github.com/BerriAI/litellm/pull/30011) - feat: litellm oss 110626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30202](https://github.com/BerriAI/litellm/pull/30202) - fix(docker): copy only runtime artifacts into the final image by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30243](https://github.com/BerriAI/litellm/pull/30243) - feat(proxy): enforce key/team guardrails on bedrock passthrough routes by [@​Sameerlite](https://github.com/Sameerlite) in [#​30194](https://github.com/BerriAI/litellm/pull/30194) - feat(gemini): forward web search tools in image generation by [@​Sameerlite](https://github.com/Sameerlite) in [#​30119](https://github.com/BerriAI/litellm/pull/30119) - fix: bedrock mantle fixes by [@​Sameerlite](https://github.com/Sameerlite) in [#​30083](https://github.com/BerriAI/litellm/pull/30083) - feat(proxy): add require\_managed\_files setting for file uploads by [@​Sameerlite](https://github.com/Sameerlite) in [#​30186](https://github.com/BerriAI/litellm/pull/30186) - fix(mcp): honor server\_id for REST tool calls with shared upstream URLs by [@​Sameerlite](https://github.com/Sameerlite) in [#​30184](https://github.com/BerriAI/litellm/pull/30184) - fix(responses): presidio PII masking for Azure WebSocket and streaming by [@​Sameerlite](https://github.com/Sameerlite) in [#​30003](https://github.com/BerriAI/litellm/pull/30003) - feat(passthrough): add configurable pass-through request timeouts by [@​Sameerlite](https://github.com/Sameerlite) in [#​30266](https://github.com/BerriAI/litellm/pull/30266) - fix(google\_genai): preserve complete SSE events in Vertex/Gemini image streaming by [@​Sameerlite](https://github.com/Sameerlite) in [#​30270](https://github.com/BerriAI/litellm/pull/30270) - fix(proxy): populate access\_via\_team\_ids on /v1/model/info by [@​Sameerlite](https://github.com/Sameerlite) in [#​30274](https://github.com/BerriAI/litellm/pull/30274) - chore(oss): litellm oss staging 120626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30292](https://github.com/BerriAI/litellm/pull/30292) - feat(ui): migrate policies, guardrails, prompts, tool-policies, and skills to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30263](https://github.com/BerriAI/litellm/pull/30263) - feat(ui): migrate caching, cost-tracking, transform-request, ui-theme, and logs to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30267](https://github.com/BerriAI/litellm/pull/30267) - fix(ui): gate dashboard layout on ui config load so deep links work under SERVER\_ROOT\_PATH by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30312](https://github.com/BerriAI/litellm/pull/30312) - feat(ui): migrate admin-panel, logging-and-alerts, model-hub-table, and usage to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30268](https://github.com/BerriAI/litellm/pull/30268) - fix(otel): cap metric attribute cardinality with include/exclude lists by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30257](https://github.com/BerriAI/litellm/pull/30257) - fix(proxy): grace-period key rotation 401s; return deprecated-key lookup result directly by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30327](https://github.com/BerriAI/litellm/pull/30327) - chore(deps): bump vitest, brace-expansion, pypdf and tornado by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30220](https://github.com/BerriAI/litellm/pull/30220) - refactor(ui): remove unreachable /chat page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30178](https://github.com/BerriAI/litellm/pull/30178) - feat(ui): migrate agents and router-settings to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30323](https://github.com/BerriAI/litellm/pull/30323) - feat: strengthen coding conventions in CLAUDE.md by [@​mateo-berri](https://github.com/mateo-berri) in [#​30333](https://github.com/BerriAI/litellm/pull/30333) - feat(ui): cut the users page over to the /ui/users path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30334](https://github.com/BerriAI/litellm/pull/30334) - feat: ruff strict-rule suppressions baseline gate by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30303](https://github.com/BerriAI/litellm/pull/30303) - feat(guardrails): add Cisco AI Defense integration ([#​28249](https://github.com/BerriAI/litellm/issues/28249)) by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30338](https://github.com/BerriAI/litellm/pull/30338) - chore(ui): remove dead UI components unreferenced by any page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30340](https://github.com/BerriAI/litellm/pull/30340) - ci: add osv-scanner lockfile scan workflow by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30222](https://github.com/BerriAI/litellm/pull/30222) - fix(otel): record full error message on standard exception event in otel v2 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30380](https://github.com/BerriAI/litellm/pull/30380) - test(fireworks): mock whisper transcription tests instead of live calls by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30391](https://github.com/BerriAI/litellm/pull/30391) - build(ui): pin esbuild to 0.28.1 via overrides by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30390](https://github.com/BerriAI/litellm/pull/30390) - feat(ui): cut the organizations page over to the /ui/organizations path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30336](https://github.com/BerriAI/litellm/pull/30336) - fix(proxy): support SMTP implicit SSL (port 465) by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30395](https://github.com/BerriAI/litellm/pull/30395) - fix(mcp): default Linear MCP registry entry to streamable HTTP by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30396](https://github.com/BerriAI/litellm/pull/30396) - fix(ui): stop Virtual Keys page from infinite render loop by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30397](https://github.com/BerriAI/litellm/pull/30397) - fix(streaming): guard raise\_on\_model\_repetition against empty choices by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30485](https://github.com/BerriAI/litellm/pull/30485) - feat(otel-v2): emit the 6 gen\_ai.client.\* metrics at parity with v1 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30326](https://github.com/BerriAI/litellm/pull/30326) - fix(mcp): drop phantom 401 span on delegated OAuth2 tool calls by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30494](https://github.com/BerriAI/litellm/pull/30494) - feat(ui): cut the teams page over to the /ui/teams path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30343](https://github.com/BerriAI/litellm/pull/30343) - fix(integrations): cap Anthropic cache\_control injection at 4 blocks by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30480](https://github.com/BerriAI/litellm/pull/30480) - chore(codecov): add Batches, Videos, and Realtime components by [@​Sameerlite](https://github.com/Sameerlite) in [#​30517](https://github.com/BerriAI/litellm/pull/30517) - test(batches): move orphan tests into tests/test\_litellm for CI coverage by [@​Sameerlite](https://github.com/Sameerlite) in [#​30510](https://github.com/BerriAI/litellm/pull/30510) - fix(guardrails): run pre\_call hook once for model-level guardrails by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30543](https://github.com/BerriAI/litellm/pull/30543) - fix(guardrails): stop re-initializing DB guardrails on every poll by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30542](https://github.com/BerriAI/litellm/pull/30542) - chore(oss): litellm oss staging 150626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30463](https://github.com/BerriAI/litellm/pull/30463) - ci(lint): add blanket-noqa, dataclass-default, and unused-noqa Ruff rules by [@​mateo-berri](https://github.com/mateo-berri) in [#​30516](https://github.com/BerriAI/litellm/pull/30516) - ci: ratchet lint and type-check gates (ruff preview, ANN, mypy, basedpyright) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30379](https://github.com/BerriAI/litellm/pull/30379) - fix(proxy): allow internal roles to access vector store CRUD routes by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30503](https://github.com/BerriAI/litellm/pull/30503) - fix(otel): stamp gen\_ai.input/output.messages on v2 spans by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30548](https://github.com/BerriAI/litellm/pull/30548) - fix(otel): export v2 gen\_ai client metrics to the configured meter provider by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30549](https://github.com/BerriAI/litellm/pull/30549) - fix(bedrock): preserve cache\_control for ARN models in /v1/messages adapter by [@​mateo-berri](https://github.com/mateo-berri) in [#​29823](https://github.com/BerriAI/litellm/pull/29823) - fix: greatly increase basedpyright slack by [@​mateo-berri](https://github.com/mateo-berri) in [#​30563](https://github.com/BerriAI/litellm/pull/30563) - fix(budget): recompute budget\_reset\_at when budget\_duration changes on /budget/update by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30555](https://github.com/BerriAI/litellm/pull/30555) - fix(otel): accept UPPER\_SNAKE\_CASE OTEL\_INSTRUMENTATION\_GENAI\_CAPTURE\_MESSAGE\_CONTENT in v2 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30562](https://github.com/BerriAI/litellm/pull/30562) - chore(lint): remove PLR0915 too-many-statements ruff rule by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30574](https://github.com/BerriAI/litellm/pull/30574) - ci(lint): ratcheted type-discipline gate (mutable collections, casts, guards, kwargs, suppressions) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30500](https://github.com/BerriAI/litellm/pull/30500) - feat(proxy): add verification\_uri\_complete to CLI SSO device flow by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30571](https://github.com/BerriAI/litellm/pull/30571) - chore: litellm oss staging160626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30527](https://github.com/BerriAI/litellm/pull/30527) - fix(guardrails): return 400 not 500 when AIM blocks a request by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30573](https://github.com/BerriAI/litellm/pull/30573) - ci(lint): grandfather any-discipline with a per-file ratchet budget (50% headroom) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30582](https://github.com/BerriAI/litellm/pull/30582) - fix(audio): don't override explicit response\_format with verbose\_json by [@​mateo-berri](https://github.com/mateo-berri) in [#​30599](https://github.com/BerriAI/litellm/pull/30599) - fix(anthropic): price and surface response service\_tier in cost tracking by [@​mateo-berri](https://github.com/mateo-berri) in [#​30558](https://github.com/BerriAI/litellm/pull/30558) - feat: add dev and wildcard proxy configs for local testing by [@​mateo-berri](https://github.com/mateo-berri) in [#​30556](https://github.com/BerriAI/litellm/pull/30556) - fix(proxy): list public team model name in /v1/models by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​30588](https://github.com/BerriAI/litellm/pull/30588) - ci: drop mypy entirely, standardize type checking on basedpyright by [@​mateo-berri](https://github.com/mateo-berri) in [#​30648](https://github.com/BerriAI/litellm/pull/30648) - feat(guardrails): surface OpenAI moderation violation\_categories on guardrail traces by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30659](https://github.com/BerriAI/litellm/pull/30659) - fix(proxy): resolve list files credentials from team BYOK deployments by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30495](https://github.com/BerriAI/litellm/pull/30495) - feat(proxy): add --max\_requests\_before\_restart\_jitter to stagger worker restarts by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30601](https://github.com/BerriAI/litellm/pull/30601) - fix(health): correct bedrock embedding health checks by [@​mateo-berri](https://github.com/mateo-berri) in [#​30583](https://github.com/BerriAI/litellm/pull/30583) - test: harden remaining pass-through CI flakes (image-gen spend poll, ruby assistants timeout) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30685](https://github.com/BerriAI/litellm/pull/30685) - test(pass\_through): harden vertex spendlog poll against transient empty reads by [@​mateo-berri](https://github.com/mateo-berri) in [#​30683](https://github.com/BerriAI/litellm/pull/30683) - fix(cost): stop non-string service\_tier from silently dropping cost tracking by [@​mateo-berri](https://github.com/mateo-berri) in [#​30690](https://github.com/BerriAI/litellm/pull/30690) - feat(proxy): warn at startup when custom\_auth skips common\_checks enforcement by [@​tin-berri](https://github.com/tin-berri) in [#​30665](https://github.com/BerriAI/litellm/pull/30665) - fix(pod\_lock): release cron lock by matching async\_set\_cache JSON encoding by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30600](https://github.com/BerriAI/litellm/pull/30600) - ci: run a local fake OpenAI endpoint instead of the shared Railway mock by [@​mateo-berri](https://github.com/mateo-berri) in [#​30695](https://github.com/BerriAI/litellm/pull/30695) - ci(windows): pin uv to Python 3.11 so it ignores the preinstalled 3.14 by [@​mateo-berri](https://github.com/mateo-berri) in [#​30704](https://github.com/BerriAI/litellm/pull/30704) - feat(ui): migrate models page to App Router path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30677](https://github.com/BerriAI/litellm/pull/30677) - refactor(ui): remove orphaned pass-through-settings route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30692](https://github.com/BerriAI/litellm/pull/30692) - fix(cost): stop non-string response service\_tier from dropping cost tracking by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30706](https://github.com/BerriAI/litellm/pull/30706) - feat(agent-shin): automated PR/issue triage, low-quality auto-close, and review-gate label lifecycle by [@​mateo-berri](https://github.com/mateo-berri) in [#​30433](https://github.com/BerriAI/litellm/pull/30433) - chore: litellm oss 170626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30637](https://github.com/BerriAI/litellm/pull/30637) - fix(bedrock\_mantle): add SigV4 fallback to chat completions auth by [@​mateo-berri](https://github.com/mateo-berri) in [#​30714](https://github.com/BerriAI/litellm/pull/30714) - feat(search): add TinyFish as search provider by [@​simantak-dabhade](https://github.com/simantak-dabhade) in [#​30634](https://github.com/BerriAI/litellm/pull/30634) - feat(ui): migrate old usage report to App Router path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30694](https://github.com/BerriAI/litellm/pull/30694) - fix(proxy): enforce budgets against authoritative DB spend when the cross-pod counter is stale by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30684](https://github.com/BerriAI/litellm/pull/30684) - chore(ci): remove Agent Shin pull\_request\_target workflows by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30784](https://github.com/BerriAI/litellm/pull/30784) - chore: litellm oss staging by [@​Sameerlite](https://github.com/Sameerlite) in [#​30745](https://github.com/BerriAI/litellm/pull/30745) - ci(zizmor): also run on litellm\_internal\_staging by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30789](https://github.com/BerriAI/litellm/pull/30789) - fix(test): drop references to removed Agent Shin workflows by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30791](https://github.com/BerriAI/litellm/pull/30791) - chore: remove in-product survey and Claude Code feedback nudges by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30773](https://github.com/BerriAI/litellm/pull/30773) - feat(ui): migrate api-keys landing to App Router path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30699](https://github.com/BerriAI/litellm/pull/30699) - feat(proxy): configurable response headers and login-page hint by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​30792](https://github.com/BerriAI/litellm/pull/30792) - ci(zizmor): gate PRs on medium+ findings and clear existing ones by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30797](https://github.com/BerriAI/litellm/pull/30797) - fix(proxy): use e.request\_data for logging\_obj in ModifyResponseException streaming passthrough by [@​mateo-berri](https://github.com/mateo-berri) in [#​30800](https://github.com/BerriAI/litellm/pull/30800) - chore: make pr template linear portion clearer by [@​mateo-berri](https://github.com/mateo-berri) in [#​30766](https://github.com/BerriAI/litellm/pull/30766) - chore(typing): add boto3/botocore stubs so basedpyright resolves the AWS SDK by [@​mateo-berri](https://github.com/mateo-berri) in [#​30815](https://github.com/BerriAI/litellm/pull/30815) - fix(otel): one v2 logger owns the global provider; scope tenant OTLP creds per exporter by [@​yucheng-berri](https://github.com/yucheng-berri) in [#​30590](https://github.com/BerriAI/litellm/pull/30590) - fix(passthrough): recover output tokens for interrupted anthropic streams by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30787](https://github.com/BerriAI/litellm/pull/30787) - fix(proxy): record partial spend on the failure row for interrupted streams by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30788](https://github.com/BerriAI/litellm/pull/30788) - fix(ui): repoint dead usage guide link to cost tracking docs by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30859](https://github.com/BerriAI/litellm/pull/30859) - fix(ui): warn that team models are deleted in the delete-team modal by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29990](https://github.com/BerriAI/litellm/pull/29990) - feat(caching): add valkey-semantic cache backend and fix semantic cache scope keys by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30675](https://github.com/BerriAI/litellm/pull/30675) - test(ui): isolate OldTeams delete-warning tests from leaked mock by [@​mateo-berri](https://github.com/mateo-berri) in [#​30871](https://github.com/BerriAI/litellm/pull/30871) - feat: add lint-gate target and truncation-proof summary to the strict ruff gate by [@​mateo-berri](https://github.com/mateo-berri) in [#​30877](https://github.com/BerriAI/litellm/pull/30877) - chore(ui): rebuild ui for release by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30894](https://github.com/BerriAI/litellm/pull/30894) - chore(ci): bump deps by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30899](https://github.com/BerriAI/litellm/pull/30899) - fix(watsonx): wrap string embedding input in array for WatsonX API by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30897](https://github.com/BerriAI/litellm/pull/30897) - test: point router/completion/triton tests at the local fake OpenAI endpoint by [@​mateo-berri](https://github.com/mateo-berri) in [#​30900](https://github.com/BerriAI/litellm/pull/30900) - feat(sandbox): e2b code execution primitive by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​30898](https://github.com/BerriAI/litellm/pull/30898) - fix(ui): source api-keys identity from useAuthorized to stop "User ID is not set" by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30903](https://github.com/BerriAI/litellm/pull/30903) - chore(ui): rebuild ui by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30906](https://github.com/BerriAI/litellm/pull/30906) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30907](https://github.com/BerriAI/litellm/pull/30907) - fix(redis): prevent forcing SSLConnection when ssl=False in connection pool by [@​Jacopos311](https://github.com/Jacopos311) in [#​30770](https://github.com/BerriAI/litellm/pull/30770) - fix(proxy): log UI setup failures instead of silently swallowing by [@​sarvesh1327](https://github.com/sarvesh1327) in [#​30819](https://github.com/BerriAI/litellm/pull/30819) ##### New Contributors - [@​simantak-dabhade](https://github.com/simantak-dabhade) made their first contribution in [#​30634](https://github.com/BerriAI/litellm/pull/30634) - [@​Jacopos311](https://github.com/Jacopos311) made their first contribution in [#​30770](https://github.com/BerriAI/litellm/pull/30770) - [@​sarvesh1327](https://github.com/sarvesh1327) made their first contribution in [#​30819](https://github.com/BerriAI/litellm/pull/30819) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.89.0...v1.90.0> ### [`v1.90.0`](https://github.com/BerriAI/litellm/releases/tag/v1.90.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.89.4...v1.90.0) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.90.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.90.0/cosign.pub \ ghcr.io/berriai/litellm:v1.90.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - fix(responses-bridge): map system-only chat request to system input item by [@​milan-berri](https://github.com/milan-berri) in [#​29817](https://github.com/BerriAI/litellm/pull/29817) - feat(bedrock): forward strict and additionalProperties to Converse toolSpec by [@​mateo-berri](https://github.com/mateo-berri) in [#​29814](https://github.com/BerriAI/litellm/pull/29814) - fix(mcp): highlight MCP cards red when the logged-in user is missing per-user env vars by [@​mateo-berri](https://github.com/mateo-berri) in [#​29856](https://github.com/BerriAI/litellm/pull/29856) - feat(ui): add budget duration to edit team member form by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29717](https://github.com/BerriAI/litellm/pull/29717) - fix(ui): make workflow runs page fill full width by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29868](https://github.com/BerriAI/litellm/pull/29868) - feat: standardize rate limit errors with category, rate\_limit\_type, model, and llm\_provider fields by [@​mateo-berri](https://github.com/mateo-berri) in [#​27687](https://github.com/BerriAI/litellm/pull/27687) - fix(ui): default guardrails page to the Guardrails tab by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29872](https://github.com/BerriAI/litellm/pull/29872) - docs(readme): add Deploy on AWS/GCP Terraform section and fix deploy button rendering by [@​mateo-berri](https://github.com/mateo-berri) in [#​29879](https://github.com/BerriAI/litellm/pull/29879) - refactor(bedrock): build Converse toolSpec via a BedrockToolSpec dict subclass by [@​mateo-berri](https://github.com/mateo-berri) in [#​29869](https://github.com/BerriAI/litellm/pull/29869) - feat(litellm): add models and repository layers by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29686](https://github.com/BerriAI/litellm/pull/29686) - feat(ui): include internal routes in the dashboard's generated OpenAPI types by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29885](https://github.com/BerriAI/litellm/pull/29885) - feat(proxy): publish /v2/model/info in Swagger OpenAPI spec by [@​Sameerlite](https://github.com/Sameerlite) in [#​29900](https://github.com/BerriAI/litellm/pull/29900) - refactor(ui): single source of truth for migrated-page routing by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29949](https://github.com/BerriAI/litellm/pull/29949) - fix(ui/model-hub): render provider icons on the public model hub by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29958](https://github.com/BerriAI/litellm/pull/29958) - fix(ui): keep create guardrail modal open on outside click by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29871](https://github.com/BerriAI/litellm/pull/29871) - fix(ui): label default key type as "Full Access" on key edit page by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29870](https://github.com/BerriAI/litellm/pull/29870) - fix(ui): unify migrated-route URLs and migrate the API Reference page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29953](https://github.com/BerriAI/litellm/pull/29953) - fix(mcp): let non-creator users OAuth into OBO-mode MCP servers from the Tools page by [@​tin-berri](https://github.com/tin-berri) in [#​29867](https://github.com/BerriAI/litellm/pull/29867) - Litellm oss staging 080626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29932](https://github.com/BerriAI/litellm/pull/29932) - feat(galileo): add health check support for UI callback test by [@​Sameerlite](https://github.com/Sameerlite) in [#​29908](https://github.com/BerriAI/litellm/pull/29908) - fix(model-management): allow deleting a BYOK model after its team is deleted by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29875](https://github.com/BerriAI/litellm/pull/29875) - feat(jwt-auth): opt-in fallback to DB team on unresolved JWT claim by [@​milan-berri](https://github.com/milan-berri) in [#​28913](https://github.com/BerriAI/litellm/pull/28913) - fix(team\_endpoints): don't block /team/update on unchanged team budget by [@​milan-berri](https://github.com/milan-berri) in [#​29525](https://github.com/BerriAI/litellm/pull/29525) - fix(fireworks): enable tool calling for glm-5p1 in model cost map by [@​milan-berri](https://github.com/milan-berri) in [#​29697](https://github.com/BerriAI/litellm/pull/29697) - fix(vertex): propagate Vertex AI metadata in streaming success callbacks by [@​Sameerlite](https://github.com/Sameerlite) in [#​29899](https://github.com/BerriAI/litellm/pull/29899) - fix(ui): show team projects to internal users on key creation by [@​milan-berri](https://github.com/milan-berri) in [#​28855](https://github.com/BerriAI/litellm/pull/28855) - build(deps): bump pyjwt to 2.13.0 and ws override to 8.20.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29982](https://github.com/BerriAI/litellm/pull/29982) - fix(team-management): delete a team's BYOK models when the team is deleted by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29977](https://github.com/BerriAI/litellm/pull/29977) - feat(vantage): include organization metadata in FOCUS Tags export by [@​milan-berri](https://github.com/milan-berri) in [#​28184](https://github.com/BerriAI/litellm/pull/28184) - fix(guardrails): read CrowdStrike AIDR identity from both metadata bags by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29991](https://github.com/BerriAI/litellm/pull/29991) - fix(mcp): mirror upstream token lifetime instead of forcing a 1h OBO expiry by [@​tin-berri](https://github.com/tin-berri) in [#​29951](https://github.com/BerriAI/litellm/pull/29951) - feat(azure\_ai): add MAI-Image-2.5 image generation support by [@​Sameerlite](https://github.com/Sameerlite) in [#​29688](https://github.com/BerriAI/litellm/pull/29688) - fix(mcp): load MCP tool configuration tools via the OBO/passthrough-aware GET path by [@​tin-berri](https://github.com/tin-berri) in [#​29960](https://github.com/BerriAI/litellm/pull/29960) - fix(team): reserve team budget raises for proxy admins on /team/update by [@​milan-berri](https://github.com/milan-berri) in [#​30030](https://github.com/BerriAI/litellm/pull/30030) - test(ui): data-driven App Router migration E2E smoke (default + server-root-path) by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29974](https://github.com/BerriAI/litellm/pull/29974) - fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through by [@​michelligabriele](https://github.com/michelligabriele) in [#​24232](https://github.com/BerriAI/litellm/pull/24232) - chore(ui): remove dead App Router route stubs under (dashboard) by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30045](https://github.com/BerriAI/litellm/pull/30045) - fix(ui/mcp): reset OAuth state on create-server modal close so a prior server's token no longer leaks into the next add-server session by [@​tin-berri](https://github.com/tin-berri) in [#​30000](https://github.com/BerriAI/litellm/pull/30000) - fix(mcp): allow team access-group grants in OAuth authorize/token access check by [@​tin-berri](https://github.com/tin-berri) in [#​30041](https://github.com/BerriAI/litellm/pull/30041) - docs(security): require a reproduction video for vulnerability reports by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30063](https://github.com/BerriAI/litellm/pull/30063) - feat(ui): add admin flag to disable in-product UI nudges for everyone by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29796](https://github.com/BerriAI/litellm/pull/29796) - chore(ui): remove dead dashboard files and unused dependencies by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30047](https://github.com/BerriAI/litellm/pull/30047) - fix(proxy): authorize batch files using upload target\_model\_names (LIT-3593) by [@​Sameerlite](https://github.com/Sameerlite) in [#​30009](https://github.com/BerriAI/litellm/pull/30009) - Add Claude Fable 5 across Anthropic, Bedrock, Vertex AI, and Azure AI by [@​mateo-berri](https://github.com/mateo-berri) in [#​30064](https://github.com/BerriAI/litellm/pull/30064) - Add Claude Fable 5 cost map entries (data-only hotfix for the hosted map) by [@​mateo-berri](https://github.com/mateo-berri) in [#​30076](https://github.com/BerriAI/litellm/pull/30076) - fix(caching): restore stored prompt\_tokens on embedding cache hits instead of recomputing by [@​michelligabriele](https://github.com/michelligabriele) in [#​30046](https://github.com/BerriAI/litellm/pull/30046) - Litellm oss 090626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30021](https://github.com/BerriAI/litellm/pull/30021) - fix(proxy): self-heal startup/reload prisma reads on engine disconnect by [@​michelligabriele](https://github.com/michelligabriele) in [#​28803](https://github.com/BerriAI/litellm/pull/28803) - chore(ui): make knip recognize .mjs scripts and openapi-typescript by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30052](https://github.com/BerriAI/litellm/pull/30052) - fix(register\_model): preserve built-in cache pricing when registering custom overrides under unmapped keys by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30044](https://github.com/BerriAI/litellm/pull/30044) - \[internal copy of [#​28007](https://github.com/BerriAI/litellm/issues/28007)] Fix/gcp model garden streaming by [@​mateo-berri](https://github.com/mateo-berri) in [#​28363](https://github.com/BerriAI/litellm/pull/28363) - feat(cli): per-agent `lite claude` / `codex` / `opencode` commands that wrap coding agents through the proxy by [@​mateo-berri](https://github.com/mateo-berri) in [#​29850](https://github.com/BerriAI/litellm/pull/29850) - fix(callbacks): forward callback\_settings to callback initializers and guard consumers against non-dict values by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30161](https://github.com/BerriAI/litellm/pull/30161) - fix(mcp): drop orphaned per-user credential rows when an MCP server is deleted by [@​tin-berri](https://github.com/tin-berri) in [#​30141](https://github.com/BerriAI/litellm/pull/30141) - fix(proxy): recover from cached-plan errors by reconnecting the Prisma client by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29983](https://github.com/BerriAI/litellm/pull/29983) - feat(proxy): add option to disable server-side prepared statements for DB lookups by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29984](https://github.com/BerriAI/litellm/pull/29984) - fix(release): stop backport releases from overwriting the latest badge by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30005](https://github.com/BerriAI/litellm/pull/30005) - feat: add conventional commits and coding guidelines by [@​mateo-berri](https://github.com/mateo-berri) in [#​30159](https://github.com/BerriAI/litellm/pull/30159) - fix(proxy): return 5xx on DB infra errors during auth; reserve 401 for genuine auth failures by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29986](https://github.com/BerriAI/litellm/pull/29986) - fix(ui): dev server 404s on migrated-page links because uiBase hardcodes /ui by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30169](https://github.com/BerriAI/litellm/pull/30169) - refactor(ui): consolidate dashboard to one shell in the (dashboard) layout by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30166](https://github.com/BerriAI/litellm/pull/30166) - fix(proxy): align /v1/model/info with router deployments by [@​Sameerlite](https://github.com/Sameerlite) in [#​30025](https://github.com/BerriAI/litellm/pull/30025) - fix: completion\_cost AttributeError on streaming Anthropic web\_search responses ([#​26153](https://github.com/BerriAI/litellm/issues/26153)) by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​27346](https://github.com/BerriAI/litellm/pull/27346) - \[internal copy of [#​30137](https://github.com/BerriAI/litellm/issues/30137)] perf(realtime): eliminate redundant per-frame JSON work on OpenAI realtime relay by [@​mateo-berri](https://github.com/mateo-berri) in [#​30142](https://github.com/BerriAI/litellm/pull/30142) - feat(bedrock): aws\_bedrock\_project\_id for bedrock-mantle project / workspace association by [@​mateo-berri](https://github.com/mateo-berri) in [#​30163](https://github.com/BerriAI/litellm/pull/30163) - chore(hooks): enforce Conventional Commits and Conventional Branches by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30174](https://github.com/BerriAI/litellm/pull/30174) - feat(rate-limiter): allow opting out of v3 TPM reservation and Redis circuit breaker by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30211](https://github.com/BerriAI/litellm/pull/30211) - feat(spend\_logs): opt-in native Postgres partitioning for SpendLogs retention by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29466](https://github.com/BerriAI/litellm/pull/29466) - feat(ui): migrate playground to path routing and colocate its files by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30185](https://github.com/BerriAI/litellm/pull/30185) - feat(ui): migrate projects and access-groups to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30226](https://github.com/BerriAI/litellm/pull/30226) - fix(proxy): coalesce NULL rollup metrics in aggregated daily-activity by [@​michelligabriele](https://github.com/michelligabriele) in [#​30151](https://github.com/BerriAI/litellm/pull/30151) - fix(anthropic\_passthrough): resolve costing model from message\_start chunk, litellm\_params and model\_group instead of 'unknown' by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30160](https://github.com/BerriAI/litellm/pull/30160) - feat(ui): migrate budgets, workflows, and guardrails-monitor to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30236](https://github.com/BerriAI/litellm/pull/30236) - feat(ui): migrate mcp-servers, search-tools, tag-management, vector-stores, and memory to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30261](https://github.com/BerriAI/litellm/pull/30261) - fix(a2a): forward agent\_extra\_headers through completion bridge by [@​mateo-berri](https://github.com/mateo-berri) in [#​28277](https://github.com/BerriAI/litellm/pull/28277) - fix(gemini-live): forward audio buffer commit and correct Vertex PCM rate by [@​Sameerlite](https://github.com/Sameerlite) in [#​29946](https://github.com/BerriAI/litellm/pull/29946) - fix(proxy): skip double-wrapping unified batch output file ids on retrieve by [@​Sameerlite](https://github.com/Sameerlite) in [#​30011](https://github.com/BerriAI/litellm/pull/30011) - feat: litellm oss 110626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30202](https://github.com/BerriAI/litellm/pull/30202) - fix(docker): copy only runtime artifacts into the final image by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30243](https://github.com/BerriAI/litellm/pull/30243) - feat(proxy): enforce key/team guardrails on bedrock passthrough routes by [@​Sameerlite](https://github.com/Sameerlite) in [#​30194](https://github.com/BerriAI/litellm/pull/30194) - feat(gemini): forward web search tools in image generation by [@​Sameerlite](https://github.com/Sameerlite) in [#​30119](https://github.com/BerriAI/litellm/pull/30119) - fix: bedrock mantle fixes by [@​Sameerlite](https://github.com/Sameerlite) in [#​30083](https://github.com/BerriAI/litellm/pull/30083) - feat(proxy): add require\_managed\_files setting for file uploads by [@​Sameerlite](https://github.com/Sameerlite) in [#​30186](https://github.com/BerriAI/litellm/pull/30186) - fix(mcp): honor server\_id for REST tool calls with shared upstream URLs by [@​Sameerlite](https://github.com/Sameerlite) in [#​30184](https://github.com/BerriAI/litellm/pull/30184) - fix(responses): presidio PII masking for Azure WebSocket and streaming by [@​Sameerlite](https://github.com/Sameerlite) in [#​30003](https://github.com/BerriAI/litellm/pull/30003) - feat(passthrough): add configurable pass-through request timeouts by [@​Sameerlite](https://github.com/Sameerlite) in [#​30266](https://github.com/BerriAI/litellm/pull/30266) - fix(google\_genai): preserve complete SSE events in Vertex/Gemini image streaming by [@​Sameerlite](https://github.com/Sameerlite) in [#​30270](https://github.com/BerriAI/litellm/pull/30270) - fix(proxy): populate access\_via\_team\_ids on /v1/model/info by [@​Sameerlite](https://github.com/Sameerlite) in [#​30274](https://github.com/BerriAI/litellm/pull/30274) - chore(oss): litellm oss staging 120626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​30292](https://github.com/BerriAI/litellm/pull/30292) - feat(ui): migrate policies, guardrails, prompts, tool-policies, and skills to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30263](https://github.com/BerriAI/litellm/pull/30263) - feat(ui): migrate caching, cost-tracking, transform-request, ui-theme, and logs to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30267](https://github.com/BerriAI/litellm/pull/30267) - fix(ui): gate dashboard layout on ui config load so deep links work under SERVER\_ROOT\_PATH by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30312](https://github.com/BerriAI/litellm/pull/30312) - feat(ui): migrate admin-panel, logging-and-alerts, model-hub-table, and usage to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30268](https://github.com/BerriAI/litellm/pull/30268) - fix(otel): cap metric attribute cardinality with include/exclude lists by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30257](https://github.com/BerriAI/litellm/pull/30257) - fix(proxy): grace-period key rotation 401s; return deprecated-key lookup result directly by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30327](https://github.com/BerriAI/litellm/pull/30327) - chore(deps): bump vitest, brace-expansion, pypdf and tornado by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30220](https://github.com/BerriAI/litellm/pull/30220) - refactor(ui): remove unreachable /chat page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30178](https://github.com/BerriAI/litellm/pull/30178) - feat(ui): migrate agents and router-settings to path routes by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30323](https://github.com/BerriAI/litellm/pull/30323) - feat: strengthen coding conventions in CLAUDE.md by [@​mateo-berri](https://github.com/mateo-berri) in [#​30333](https://github.com/BerriAI/litellm/pull/30333) - feat(ui): cut the users page over to the /ui/users path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30334](https://github.com/BerriAI/litellm/pull/30334) - feat: ruff strict-rule suppressions baseline gate by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30303](https://github.com/BerriAI/litellm/pull/30303) - feat(guardrails): add Cisco AI Defense integration ([#​28249](https://github.com/BerriAI/litellm/issues/28249)) by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30338](https://github.com/BerriAI/litellm/pull/30338) - chore(ui): remove dead UI components unreferenced by any page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30340](https://github.com/BerriAI/litellm/pull/30340) - ci: add osv-scanner lockfile scan workflow by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30222](https://github.com/BerriAI/litellm/pull/30222) - fix(otel): record full error message on standard exception event in otel v2 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30380](https://github.com/BerriAI/litellm/pull/30380) - test(fireworks): mock whisper transcription tests instead of live calls by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30391](https://github.com/BerriAI/litellm/pull/30391) - build(ui): pin esbuild to 0.28.1 via overrides by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30390](https://github.com/BerriAI/litellm/pull/30390) - feat(ui): cut the organizations page over to the /ui/organizations path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30336](https://github.com/BerriAI/litellm/pull/30336) - fix(proxy): support SMTP implicit SSL (port 465) by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30395](https://github.com/BerriAI/litellm/pull/30395) - fix(mcp): default Linear MCP registry entry to streamable HTTP by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30396](https://github.com/BerriAI/litellm/pull/30396) - fix(ui): stop Virtual Keys page from infinite render loop by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30397](https://github.com/BerriAI/litellm/pull/30397) - fix(streaming): guard raise\_on\_model\_repetition against empty choices by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30485](https://github.com/BerriAI/litellm/pull/30485) - feat(otel-v2): emit the 6 gen\_ai.client.\* metrics at parity with v1 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30326](https://github.com/BerriAI/litellm/pull/30326) - fix(mcp): drop phantom 401 span on delegated OAuth2 tool calls by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30494](https://github.com/BerriAI/litellm/pull/30494) - feat(ui): cut the teams page over to the /ui/teams path route by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​30343](https://github.com/BerriAI/litellm/pull/30343) - fix(integrations): cap Anthropic cache\_control injection at 4 blocks by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​30480](https://github.com/BerriAI/litellm/pull/30480) - chore(codecov): add Batches, Videos, and Realtime components by [@​Sameerlite](https://github.com/Sameerlite) in [#​30517](https://github.com/BerriAI/litellm/pull/30517) - test(batches): move orphan tests into tests/test\_litellm for CI coverage by [@​Sameerlite](https://github.com/Sameerlite) in [#​30510](https://github.com/BerriAI/litellm/pull/30510) - fix(guardrails): run pre\_call hook once for model-level guardrails by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​30543](http…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Fixes LIT-3385
Fixes LIT-3544
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
make test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewType
New Feature
Changes
Key/team guardrails were silently never executed for
/bedrock/...passthrough routes. The root cause wasCallTypes.allm_passthrough_routehaving no entry inguardrail_translation_mappings, soUnifiedLLMGuardrailsreturned data untouched. Post-call hooks also never fired for this route.This PR fixes both problems:
New: Bedrock passthrough guardrail handler (
litellm/llms/bedrock/passthrough/guardrail_translation/handler.py).BedrockPassthroughGuardrailHandlerextracts text from Bedrock Converse request/response shapes (system blocks, message content blocks), callsapply_guardrail, and writes guardrailed text back in place. Non-converse endpoints and missing message bodies are skipped.New: Route dispatcher registered for
allm_passthrough_route(litellm/llms/pass_through/guardrail_translation/handler.py).LlmPassthroughRouteHandlerresolvescustom_llm_providerfrom the request data and delegates to the provider-specific handler. Unknown providers log and skip. Registered inguardrail_translation_mappingsalongside the existingpass_throughentry.Post-call wiring in
common_request_processing.py. Theallm_passthrough_routeresponse path now callspost_call_success_hookfor both JSON (application/json) and AWS event-stream (vnd.amazon.eventstream) response content types when post-call guardrails are active. For event-stream responses (Bedrockconverse-stream), a new_handle_event_stream_allm_passthrough_routemethod parses binary AWS event-stream frames usingbotocore.eventstream, assembles a synthetic Converse response dict fromcontentBlockDeltaframes, runspost_call_success_hook(which triggers Presidio de-anonymization), then re-encodes the modified frames with corrected CRC checksums. De-anonymized text is distributed proportionally across the original delta frames to preserve streaming behavior.Screenshots / Proof of Fix
Start the proxy:
Generate a key with the presidio guardrail attached:
Non-streaming (converse) - PII masked in request to Bedrock, de-anonymized in response to caller:
Streaming (converse-stream) - same masking/de-anonymization over event-stream frames:
Blocking guardrail (e.g. Bedrock guardrail) attached to a key - request rejected before reaching Bedrock: