merge main - #22186
Merged
Merged
Conversation
The backend validator and frontend form both enforced auth_value as required when auth_type is api_key, bearer_token, or basic. Users who want to provide auth dynamically (via per-request headers or OAuth2 flows) could not skip the field. - Remove required validation from auth_value in create_mcp_server.tsx (keep whitespace-only rejection, matching the edit flow) - Remove validate_credentials_requirements in NewMCPServerRequest (all downstream code already treats auth_value as optional) - Add tests for the create MCP server component Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…n sortable Use the backend-provided request_duration_ms field instead of computing duration client-side from startTime/endTime. Add sort support for the Duration column, which sends sortBy=request_duration_ms to the API. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
[Feature] UI - Logs: Use backend request_duration_ms and make Duration sortable
[Fix] UI - MCP Servers: Make auth value optional for create flow
* Add post-call hook for Lakera guardrail and mask PII in responses * Add post-call hook for Lakera and mask PII in responses * Fix post-call hook: pass event_type to call_v2_guard * Address Greptile review: return ModelResponse, fix mutation, add header, test location, mask order - PII masking path: return ModelResponse instead of dict so deployment hook accepts it - Avoid mutating request data: deep copy original_messages and messages in _mask_pii_in_messages - Add guardrail header in PII-only return path - Add test in tests/test_litellm/ (test_lakera_ai_v2.py) per PR checklist - Sort PII payload spans by (start,end) descending so multiple spans in one message mask correctly Co-authored-by: Cursor <cursoragent@cursor.com> * Updated ponteital for index mismatch when choices have null content and inconsistent on_flagged access pattern * Update litellm/proxy/guardrails/guardrail_hooks/lakera_ai_v2.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update to explicitly state supported endpoints - chat completions * Fix minor lint error on masked_entity_count --------- Co-authored-by: Steve <steve.giguere@lakera.ai> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add claims agent guardrails with 243-case eval dataset 5 new category guardrails for healthcare claims agent chatbots: - claims_fraud_coaching: fraud coaching, exaggeration, document forgery - claims_phi_disclosure: unauthorized PHI access, bulk data extraction - claims_prior_auth_gaming: code manipulation, medical necessity misrepresentation - claims_system_override: system injection, rule bypass, role impersonation - claims_medical_advice: medical advice (claims-context-aware) Plus claims_agent_safety.yaml policy template combining all 5. All 5 eval suites pass at 100% precision/recall/F1 (243 test cases). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add claims agent chatbot safety policy template Combines the 5 claims guardrails into a single deployable policy template: fraud coaching, PHI disclosure, prior-auth gaming, system override, and medical advice. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add guardrail benchmark results and UI compliance prompts Adds benchmark results for claims, discrimination, and content filter guardrails. Updates UI compliance prompt data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove duplicate "file an appeal" exception in claims_prior_auth_gaming.yaml Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove unused claims_agent_safety.yaml policy template The claims-agent-safety template in policy_templates.json references individual category files in categories/, not this combined file. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
New Azure OpenAI Models 2026-02-25
[Chore] Update aiml model pricing
#22152) * feat(realtime): add guardrails query param to /v1/realtime WebSocket endpoint - Add 'guardrails' query param (comma-separated) to realtime_websocket_endpoint - Import websockets and websockets.exceptions at module level (fixes NameError in except clause) - Split try/except into Phase 1 (pre-call) and Phase 2 (routing) so guardrail errors send back a typed error event before closing, while upstream errors close silently with 1011 * feat(ui): pass selectedGuardrails from sidebar to RealtimePlayground WebSocket URL * docs(realtime): add guardrails section with dynamic passing examples
…uting code (#22154) Adds a new block_code_execution guardrail that detects markdown fenced code blocks in request/response content and blocks or masks them by language. Includes full UI integration, type definitions, compliance test dataset, and 26 unit tests. Key guardrail capabilities: - Regex-based fenced code block detection with configurable blocked languages - Confidence scoring with tunable threshold - Execution-intent heuristics (request-side only) with conflict resolution - Block or mask actions for detected code - Support for pre_call, post_call, and during_call event hooks Security hardening: - Response-side blocking skips intent heuristics (LLM output doesn't contain user intent phrases, so checking would silently disable post_call blocking) - No-execution short-circuit includes conflict resolution: if both no-execution and execution phrases match, execution intent wins - Tightened overly broad phrases to prevent trivial bypass - _normalize_escaped_newlines only applies to pure-escaped payloads to avoid corrupting content that discusses escape sequences Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…#22153) * feat(vertex_ai): add Vertex AI Gemini Live support via unified /realtime endpoint Adds VertexAIRealtimeConfig which translates the OpenAI Realtime WebSocket protocol to Vertex AI BidiGenerateContent. Supports voice in/voice out (16 kHz mic → 24 kHz speaker) and text in/text out through the proxy's /realtime endpoint. Key changes: - New litellm/llms/vertex_ai/realtime/transformation.py with VertexAIRealtimeConfig - Builds correct wss:// URL (regional + global) - OAuth2 Bearer token auth (not API key) - Full model path (projects/.../publishers/google/models/...) - Ignores session.update (Vertex AI only accepts one setup message) - realtime_api/main.py: vertex_ai branch resolves OAuth token + constructs config - llm_http_handler.py: auto-sends session setup before bidirectional_forward - gemini/realtime/transformation.py: fix crashes on empty turnComplete events - realtime_streaming.py: try/except guard so bad messages don't kill the loop - proxy_server.py: add missing websockets.exceptions import * docs: add vertex_realtime to sidebars * fix: drop unknown event types in Gemini transform; add vertex_ai health check * fix: propagate UUID fallback IDs from transform_content_done_event to return_additional_content_done_events * fix: route guardrail backend sends through provider transform; fix str.strip misuse for model prefix * fix: handle Vertex AI full resource path in session.created; route guardrail block sends through _send_to_backend * fix: remove unused VertexBase in transformation.py; apply UUID fallback in return_additional_content_done_events
[Infra] Fixing UI Build
…management_endpoints.py
* feat: add pretty view for realtime API logs in dashboard - Create RealtimePrettyView component that renders structured session config, conversation turns with transcripts, and token breakdowns - Update PrettyMessagesView to detect realtime responses (via isRealtimeResponse helper) and delegate to the new component - Session card shows model, voice, modalities, temperature, instructions in a collapsible panel - Conversation turns show status, per-turn token usage, and audio/text transcripts with appropriate icons - Add 24 tests for RealtimePrettyView and 3 tests for PrettyMessagesView - All 75 LogDetailsDrawer tests pass Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * chore: remove dev_config.yaml from tracked files Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * feat: show turn count in realtime pretty view session header and output header - Add purple 'N turns' tag to Session card header for at-a-glance turn count - Add 'Turns: N' to the Output section header next to tokens/cost - Extend SectionHeader to accept optional turnCount prop - Add 3 new tests for turn count display (singular, plural, output header) Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: address Greptile review feedback - Remove response.audio.done and conversation.item.created from isRealtimeResponse() detection since the view doesn't render them; prevents misleading fallback for responses with only those events - Remove dead code: index >= 0 is always true in .map() callback Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
…altime WebSocket (#22161) * fix(realtime): guardrails with pre_call/post_call mode now work on realtime WebSocket; return error directly to consumer * fix(realtime guardrails): address code review feedback - Restore session.update injection for audio/VAD path, but only when realtime_input_transcription guardrails are configured (not pre_call). Forward session.created to the client first so no error arrives before the client sees the session. - Change _swallow_next_response_create bool to int counter so consecutive blocked items are handled correctly. - Extract _build_litellm_metadata() helper to eliminate duplicated metadata-building logic across OpenAI/Azure/XAI provider branches. - Plumb litellm_metadata and user_api_key_dict to Azure and XAI handlers so guardrails work for those providers too. - Add tests for session.update injection, no-inject for pre_call-only, and consecutive-block counter. * simplify: remove response.create swallowing after guardrail block When an item is blocked, the error event is already sent to the client. The subsequent response.create from the client is fine to forward through — the LLM may respond to previous context which is acceptable behavior. Removing the swallow counter eliminates unnecessary state tracking.
…ngs wizard step (#22165) * feat(realtime guardrails): end_session_after_n_fails + Endpoint Settings wizard step Adds per-session violation thresholds and an optional endpoint-settings step to the guardrail wizard for /v1/realtime. Backend: - Add end_session_after_n_fails, on_violation, realtime_violation_message fields to BaseLitellmParams (no DB migration — stored in existing JSON column) - Store same fields on CustomGuardrail instance attrs - Pass through in litellm_content_filter initializer - Track _violation_count per RealTimeStreaming session; close backend_ws when on_violation=end_session OR violation count >= end_session_after_n_fails - Use realtime_violation_message as the spoken text (falls back to guardrail error string if not configured) UI (add_guardrail_form.tsx): - Rename "Default Categories" step to "Topics" - Add step 5 "Endpoint Settings (Optional)" for content filter guardrails - Call type dropdown shows /v1/realtime - Settings are in a collapsed accordion (closed by default) - "End session after X violations" + on_violation radio + spoken message field Tests: 2 new tests in test_realtime_streaming.py - test_end_session_after_n_fails_closes_connection - test_on_violation_end_session_closes_on_first_fail * fix(test): move inline imports to module level in realtime streaming tests * Update ui/litellm-dashboard/src/components/guardrails/add_guardrail_form.tsx Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
[Fix] CICD 26/02/26
* Adjust input and output cost per token for mistral-small-2503 Cost per million for mistral-small-2503 is not correct. In Azure Documentation: Pay-as-you-go (per 1,000 tokens) $0.0001 Model input $0.0003 Model output * Update input and output cost per token for model
- ArizePhoenixLogger now creates spans on its own dedicated TracerProvider instead of trying to reuse parent spans from the global otel TracerProvider (which were invisible in Phoenix since they go to a different exporter) - Auto-initialize ArizePhoenixLogger when otel callback is configured and Phoenix env vars (PHOENIX_API_KEY, PHOENIX_COLLECTOR_*) are detected - Use exact type check in get_custom_logger_compatible_class to prevent ArizePhoenixLogger (subclass) from being returned when looking up otel - Fix tool_permission guardrail to check non-function tools like code_interpreter (previously skipped with `type != "function"`) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…ed-traces fix: arize phoenix nested traces
…itecture The old test assumed ArizePhoenixLogger reused the global TracerProvider. With the nested traces fix, Phoenix now creates its own dedicated provider and produces litellm_proxy_request + litellm_request + raw_gen_ai_request spans independently. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…ed-traces-test-update fix(test): update Phoenix OTEL test
Bump litellm version to 1.81.16
Contributor
|
Too many files changed for review. ( |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Sameerlite
merged commit Feb 26, 2026
87b4fed
into
litellm_oss_staging_02_26_2026
11 of 58 checks passed
Sameerlite
temporarily deployed
to
github-pages
February 26, 2026 13:00 — with 
GitHub Pages
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes