chore(ci): promote internal staging to main - #32663
Merged
Merged
Conversation
Introduce two first-class MCP server auth_type values that make LiteLLM's role in upstream authentication explicit, added alongside the existing delegate_auth_to_upstream / oauth_passthrough flags without changing their behavior. true_passthrough is a transparent proxy: LiteLLM performs no admission auth, requires no x-litellm-api-key, mints/stores/refreshes nothing, and forwards the client's Authorization to the upstream exactly as received. oauth_delegate keeps normal LiteLLM admission (x-litellm-api-key / SSO / JWT) and then forwards the client's separate upstream Authorization unchanged; the admission credential is never forwarded upstream. Both modes forward the caller's token via the existing extra_headers path and defer egress credential resolution to v1 (the v2 to_server_spec returns None for them). Upstream 401/403 responses are surfaced rather than swallowed so upstream OAuth challenges are preserved. Servers in either mode require per-user auth, so userless health checks are skipped.
…OAuth discovery Both modes advertised LiteLLM as the authorization server and answered initialize locally, so a client with no token connected empty and was never driven into the upstream OAuth flow. The protected-resource discovery now proxies the upstream metadata for both modes (verbatim for true_passthrough, resource rewritten to the gateway for oauth_delegate), and the preemptive 401 emits the matching challenge: oauth_delegate uses the gateway-proxied resource_metadata once admission passes, true_passthrough probes the upstream anonymously and surfaces its WWW-Authenticate verbatim so the client authorizes directly against the upstream
oauth_delegate forwards the caller's token to the upstream, which validates its audience, so the protected-resource metadata must keep resource pointing at the upstream (returned verbatim, like true_passthrough) rather than rewriting it to the gateway. Rewriting to the gateway asks the client to mint a token bound to the gateway audience, which a strict IdP (Entra) refuses to issue for an unregistered resource and a spec-compliant upstream rejects on receipt. The legacy is_oauth_passthrough opt-in keeps the gateway rewrite unchanged.
Root cause: PR #30867 removed request-time os.environ/ expansion in BaseAWSLLM.get_credentials. That is only safe if config-load pre-resolves os.environ/ refs so the value reaching get_credentials is already the real secret. The YAML config path has always done this. The DB-load path (ProxyConfig._resolve_db_litellm_param) only re-expanded keys in a hardcoded whitelist (_DB_LITELLM_PARAM_ENV_REF_KEYS) plus short-circuited env-ref resolution entirely for team-scoped rows. PR #32256 extended that whitelist to 18 keys to unblock a customer whose Bedrock model with aws_role_name: os.environ/BEDROCK_ASSUME_ROLE_ARN broke on v1.90+, but the whitelist is structurally fragile: every future auth field breaks the same way until someone remembers to add it Fix: remove the whitelist and the team-scope short-circuit. The DB-load resolver now expands os.environ/ on every string field, matching the YAML path. Trust boundary stays on the write side: only PROXY_ADMIN can create team_id=None rows, only team admins of a team can create rows scoped to that team, and the request-body vector is still blocked by _BANNED_REQUEST_BODY_PARAMS. Team-scoped rows now resolve env refs — this is a deliberate LIT-3831 threat-model expansion trusting team admins for env-var reads Regression tests in tests/test_litellm/proxy/proxy_server/test_proxy_config.py: - test_ProxyConfig__add_deployment_resolves_env_refs_after_db_decrypt pins admin-scoped rows resolve every field (previously api_base stayed literal) - test_ProxyConfig__add_deployment_resolves_team_env_refs pins team rows resolve env refs (previously stayed literal) - test_ProxyConfig__add_deployment_resolves_env_refs_on_arbitrary_field pins the no-whitelist invariant against a made-up field name - test_ProxyConfig__add_deployment_resolves_env_refs_for_aws_bedrock_auth_params (from #32256) still passes - Path B counterparts (decrypt_model_list_from_db) mirror the above Left as followups (not fixed here): - /model/info and /v2/model/info still echo resolved values for fields not in the current pop-list (aws_role_name, aws_sts_endpoint, api_base, etc.). Fix is to extend remove_sensitive_info_from_deployment; separate PR - Master-key rotation reads DB rows via decrypt_model_list_from_db which now resolves universally, so rotation collapses env-refs into hardcoded values. Pre-existing bug for the 6 previously-whitelisted fields; wider surface after this PR. Separate PR
* fix(ui): scope key models dropdown options to the key's team A teamless key no longer offers the all-team-models option in the create and edit forms; the backend expands that sentinel to the full proxy model list when no team is attached, which is rarely what the user intended. A team key no longer surfaces the all-proxy-models sentinel that leaks in verbatim when the team's own model list carries it; the dropdown keeps All Team Models plus the team's individual models. Adds browser coverage to the management e2e suite: playwright (an optional dependency behind importorskip) drives the proxy-served dashboard at /ui, asserts the dropdown options a real user sees for teamless and team keys on both create and edit, and walks the create modal end to end, reading the persisted key back through /key/info. * fix(ui): offer all-proxy-models on teamless keys in the models dropdown A teamless key has no team allowlist to inherit, so the dropdown now offers All Proxy Models in place of All Team Models on both the create and edit forms, with the same exclusive-selection handling. Component and browser e2e tests updated to pin the swapped option pair; the teamless create case now also walks the modal end to end and reads the persisted key back through /key/info. * test(ui): update no-team key creation spec to pick All Proxy Models The create modal no longer offers All Team Models without a team; the teamless path now offers All Proxy Models, which is what this spec exercises * fix(ui): gate All Team Models on the team object being loaded When a key has a team_id but the teams prop does not yet include the matching team, availableModels stays empty and the models dropdown rendered All Team Models on its own with nothing to compare against. Gate the option on the team object being present so it only appears once team models are known, and add a regression test for the loading state Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): filter all-proxy-models from teamless model fetch in key edit form The teamless fetch path stored modelAvailableCall results without excludeProxyWideSentinel, so an all-proxy-models entry in the response rendered a second option colliding with the hardcoded All Proxy Models sentinel. Apply the same filter used on the team path and add a regression test asserting the sentinel option is not duplicated Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Mubashir Osmani <mubashir@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ng uplift (#32387) * fix(model_prices): add gpt-realtime-2.1 models with regional processing uplift * fix(model_prices): add cache_read_input_audio_token_cost to gpt-realtime-2.1
…uery params (#32404) * fix(passthrough): stop request params from clobbering merged target query params * fix(passthrough): rewrite managed ids in query params before folding them into the URL
* Split LLM e2e coverage modules * Add e2e coverage dashboard metrics * Remove dashboard brief from e2e coverage PR
In a listing fan-out over a scope containing more than one server that
consumes the caller's Authorization (true_passthrough, oauth_delegate,
or the legacy delegate/passthrough shapes), the request-wide bearer is
now withheld from the new modes instead of being replayed against every
upstream (RFC 9700 cross-resource replay). Explicitly-addressed
operations (tool call, get_prompt, read_resource, single-server routes)
keep forwarding it.
Multi-server aggregates use the per-server x-mcp-{alias}-authorization
header instead: its value now feeds the passthrough resolver arm as the
inbound token and wins over the request-wide header, binding one token
to one server.
…ateway double-encoding previous_response_id (#32034)
…ricing in get_model_info (LIT-4056) (#32389) * fix(utils): resolve bedrock regional inference profiles to regional pricing in get_model_info (LIT-4056) * test(register_model): use a triple provider prefix as the unresolvable-key fixture get_model_info now resolves bedrock/bedrock/... like a routing prefix, so the double-prefix fixture stopped exercising the register_model fallback path. Lock the new double-prefix resolution in as a model-info regression test
…fering in memory (#32386) * fix(passthrough): stream non-sse passthrough responses instead of buffering in memory Non-SSE passthrough responses were fully read into proxy memory (content = await response.aread()) before the first byte reached the client. For large non-JSON bodies such as Anthropic batch results jsonl files this ballooned proxy RSS to a multiple of the file size and produced near-total TTFB dead air, letting intermediaries kill the silent connection and truncate the download. The upstream request is now sent with httpx stream semantics and the buffering decision is made from the response headers: application/json (and +json) bodies plus upstream errors keep the buffered behavior since spend logging, guardrails and managed-id rewriting inspect them, while every other 2xx body is relayed as a StreamingResponse that iterates upstream bytes without accumulating them, preserving status code and headers (including x-litellm-*) and firing the success-handler logging with response_body=None once the stream completes. * fix(passthrough): log client disconnects mid-stream and derive test client cache key from production code * test(passthrough): intercept AsyncClient.send in legacy passthrough tests and assert final wire params * test(passthrough): fail with a clear assert when the passthrough client cache scan misses
…ates fix(proxy): resolve os.environ/ refs universally in DB-sourced models
…imeout-only failures to stop 15m no-output kills (#32420)
…provider timeout (#32424) * ci(responses): bound azure shell tool e2e call and enforce per-test timeout The azure variant of test_responses_api_shell_tool always makes a live Azure call (its skip outcome means no VCR cassette is ever persisted). When Azure held the connection instead of answering, the call sat on litellm's 6000s responses deadline until CircleCI killed the whole job via no_output_timeout after 15m of silence (job 2013288). Bound the e2e call at 90s and skip on litellm.Timeout, matching the existing InternalServerError and BadRequestError skips, and give the llm_responses_api_testing job the same pytest-timeout guard the llm_translation_testing job already uses so no single hung test can consume the 15m no-output window again. * test(responses): drop job-level pytest timeout, keep shell tool 90s bound
…32422) * ci: skip unit test workflows when only docs or ui files change Mirror the CircleCI backend path filter (.circleci/scripts/classify_changes.sh) in the GitHub Actions unit test workflows by adding paths-ignore for ui/**, docs/**, *.md and *.mdx to every test-unit-*.yml pull_request trigger * ci: drop docs/** from unit test paths-ignore since the folder no longer exists
…t forms (#32397) * feat(ui): expose MCP max_concurrent_requests in server create and edit forms The proxy has enforced a per-server outbound tool-call concurrency cap (max_concurrent_requests) across every MCP egress path since #31641, and the management API has accepted the field on create and update all along, but the dashboard offered no way to set it. Add an optional Max Concurrent Requests input to the MCP server create and edit forms; it applies to every auth type and transport, so it renders unconditionally rather than gated on auth mode. Clearing the field on edit sends null so the stored limit is unset. Also rebuild the per-server semaphore when the configured limit changes. Previously the semaphore was created once per server_id and never resized, so an edited limit only took effect after a proxy restart even though the new value was persisted and reloaded into the registry. * feat(ui): mark MCP max concurrent requests field label as optional * test(ui): stop OBO create-form tests from timing out on CI The token-exchange payload test and the Entra scope-required test filled five text fields with user.type, which dispatches a full keystroke sequence per character; every input event runs the antd form onValuesChange handler and re-renders the whole CreateMCPServer tree, roughly 120 renders per test. As the form grew the two tests reached 8s and 18s locally, which crosses the 30s vitest timeout on slower CI containers; ui_unit_tests failed twice this way. Switch the plain text fields to fireEvent.change (one input event per field), matching the existing stdio test pattern. Both tests assert form output, not keystroke behavior, and now run in about 3s each.
Add a tag_rpm_limit field to virtual keys so each request tag gets its own independent RPM counter on the v3 rate limiter. A key configured with per-tag limits tracks each tag/group separately, and requests whose tag has no configured limit fall back to the key-level limit. Includes the dashboard UI to manage per-tag limits on key create and edit. Resolves LIT-3147
…w changes (#32302) * fix(mcp): drop the cached per-user OAuth token when the credential row changes The v2 authorization_code chain Cached(Refreshing(V2PerUserTokenStore)) caches a positive token until its expires_at (or 300s without one), and CachedOAuthTokenStore.invalidate had no callers, so a re-authorization or revocation wrote the DB while egress kept serving the replaced token from the in-process cache until its TTL. LazyPerUserOAuthTokenStore now exposes invalidate, MCPServerManager threads it to the write side, and the three credential write sites (the OAuth callback, the Tools-tab persist endpoint, and the revoke endpoint) drop the cache entry after the row changes. The v2 refresher's own persist stays untouched; RefreshingTokenStore already feeds the rotated token back into the cache in the same fetch * test(mcp): pin cache invalidation on the revoke already-gone branch Greptile's review flagged that only the happy-path delete asserted the invalidate; a refactor moving the call inside the try block would silently skip the cache drop when the row was already deleted by a concurrent request while the cache still held the revoked token. The new test fails on exactly that mutation * test(mcp): cover invalidate on the redis-backed lazy store path Codecov flagged the redis fast path of LazyPerUserOAuthTokenStore.invalidate as unexercised; the existing invalidate tests only ran the no-redis chain. The new test builds the redis chain via a fetch and asserts a subsequent invalidate reaches the same store instance without a rebuild
…h to gemini-3.5-flash (#32439)
…ettes (#32390) * test(realtime): record and replay websocket traffic in redis vcr cassettes * style(realtime): ruff-format ws-vcr harness * fix(realtime): warn instead of silently disabling ws-vcr when the redis client cannot be built
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
…est content (#32533) * fix(rerank): log optional_rerank_params at debug not info to avoid leaking request content * test(rerank): exercise sync rerank path so coverage counts the log line --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…te (#30857) * refactor(ui): point invitation links at the dedicated /onboarding route Invitation and reset-password links were built as /ui?invitation_id=..., which lands on the dashboard index and renders the onboarding form inline. They now point at the standalone /ui/onboarding route, so the index no longer has to special-case invitations. Old links keep working unchanged; the index still renders onboarding inline for ?invitation_id until the migration closeout removes that branch. Updates the three generators (the enterprise email builder, bulk user create, and the invitation/reset-password modal) and extracts the modal's URL building into a pure, unit-tested buildOnboardingUrl Refs LIT-3687 * refactor(ui): guard buildOnboardingUrl against a missing invitation id Return "" instead of emitting an invitation_id=undefined link when the id is not yet available, matching the existing empty-baseUrl guard. Placed after the SSO branch so the SSO link, which does not use the id, is unaffected Refs LIT-3687
…cks (#32551) Bedrock Converse supports cachePoint ttl (1h GA for Claude 4.5+), and _get_cache_point_block maps cache_control.ttl -> cachePoint.ttl, but the model parameter its allow-list gate requires was only threaded through the system-message path. Every message-level path either called _get_cache_point_block without model= (8 call sites in _bedrock_converse_messages_pt / _pt_async) or hardcoded CachePointBlock(type="default") (tool-result blocks and _convert_to_bedrock_tool_call_invoke), so a requested 1h ttl silently degraded to the 5-minute default - exactly on the conversation-tail breakpoint that long-running agents need to survive tool calls longer than 5 minutes. - pass model= at the 8 _get_cache_point_block call sites - tool-result blocks: capture the cache_control dict (was a boolean) and route through _get_cache_point_block so ttl survives - _convert_to_bedrock_tool_call_invoke: accept optional model and route per-tool-call cache_control through _get_cache_point_block Completes the ttl support added for system messages (#19848, #20326): message-level cache_control now behaves identically. Note: message-level cache_control on a content-less assistant message emits no cachePoint at all today; that pre-existing gap is orthogonal to ttl and left out of scope (per-tool-call placement covers it). Co-authored-by: Arash <arashne@glia-ai.com>
…elpers (#32542) * fix(guardrails): walk Responses-API text taxonomy in shared content helpers Every guardrail sharing litellm/proxy/guardrails/_content_utils.py silently drops all text on the /v1/responses path. AIM turns it into a loud 422 ( {"error":"No messages in the request"}); every other guardrail (Lakera v2, Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret detection) scans an empty payload and lets the request through unscanned. Three defects, all in _content_utils.py: 1. _iter_text_parts_in_content recognised only part.type == "text", but the Responses API uses input_text (request) and output_text (assistant). 2. _coerce_input_to_messages gated on "every item has a role key"; any Responses input list containing a function_call or function_call_output item failed the check and was wrapped as one opaque blob. 3. build_inspection_messages forwarded any role through, including a bare tool role missing tool_call_id, which validators like AIM's /fw/v1/analyze reject with a schema error. Fix walks the actual Responses item taxonomy (message, function_call, function_call_output, bare content parts and strings), recognises {text, input_text, output_text} everywhere, and coerces any role outside {system, user, assistant} to user in the outbound inspection payload. * style: ruff-format changed guardrail files * test(guardrails): cover function_call_output string form; drop em-dash in new docstring * fix(guardrails): map function_call_output straight to user role Avoids ever materialising a schema-invalid bare tool message. The downstream role-safety coercion in build_inspection_messages still guards genuinely caller-supplied non-standard roles (developer, function, custom values); add a regression test covering that path so the coercion has real coverage after this simplification. * test(guardrails): pin chat-completions tool-role coercion in build_inspection_messages * docs(test): soften AIM-specific claims in LIT-4294 test docstrings Ryan's review flagged that several test docstrings assert AIM's /fw/v1/analyze validates + rejects specific schema violations. That behavior is customer-reported in the LIT-4294 writeup, not directly verified by us. Rephrase to attribute the AIM 422 to the customer's writeup and describe the underlying constraint as the OpenAI chat schema; any downstream API that validates against that schema rejects the same shape. * refactor(guardrails): move unsupported-role coercion into AIM only The generic coercion in build_inspection_messages collapsed any role outside {system, user, assistant} to user for every caller of the helper. Combined with the pre-existing apply_redacted_messages_back write-back behavior in Lakera/AIM/Cato, that turned a loud OpenAI 400 on chat-completions tool-message masking into a silent semantic corruption of the outbound request (role tool with tool_call_id got rewritten to bare role user, dropping the assistant + tool_calls sibling). AIM specifically requires the coercion because its /fw/v1/analyze validates the payload against the OpenAI chat schema; other guardrails either do not validate roles or do their own reconstruction. Move the coercion to AimGuardrail._build_aim_inspection_messages so the shared helper keeps caller roles intact and no new cross-guardrail role corruption is introduced. The pre-existing apply_redacted_messages_back structural flatten remains as separate follow-up work. function_call_output items still synthesise role user in the shared helper because they have no natural role field, which is a different concern from coercing a caller-supplied role. * refactor(guardrails): preserve role fidelity in shared _content_utils Shared inspection helpers should extract text and preserve semantic role signals; role coercion for third-party schema safety stays inside the guardrail that needs it (AIM). Three shared-helper changes: - Bare content-part dicts (input_text/output_text) with an explicit role keep it; only role-less parts default to user. - Responses message items already had their role preserved; the behavior is now covered by an explicit test. - function_call_output items default to role tool (semantic equivalent of the chat-completions tool message shape) instead of role user, so Responses and chat completions produce symmetric inspection payloads. A caller-supplied role on the item is still preserved. AIM's schema-safe coercion in _build_aim_inspection_messages already handles the resulting role tool: it collapses to user before the POST to /fw/v1/analyze so AIM's OpenAI-schema validator does not reject the bare tool message (no tool_call_id can survive the flatten). Added a regression test in test_aim.py covering that path.
…mespace (#32591) The v2 OTel integration stamped litellm-specific error details as error.code, error.stack_trace, and error.llm_provider, squatting on the semconv-owned error.* namespace. They now live at litellm.provider.error.code, litellm.provider.error.stack_trace, and litellm.provider.error.llm_provider alongside the other vendor-extension keys. error.type and error.message stay on the semconv keys.
When an IdP denies SSO access it redirects back to /sso/callback with error and error_description query params and no code param. The callback previously fell through to the provider token exchange, which failed with a generic "'code' parameter was not found in callback request" 400 that hides the real denial reason. Raise a 401 that surfaces the IdP's error and description instead. Ported from #26640 with conflicts resolved against current staging
… measurements deterministic (#32435)
fix(deps): constrain soupsieve>=2.8.4 to patch two high-severity CVEs
…d_filter feat(ui): add session id filter to request logs
…itellm_/release-version-bump-nightly-c31ca1 # Conflicts: # uv.lock
…nightly-c31ca1 chore: bump litellm-enterprise 0.1.48 -> 0.1.49
#32485) The expanded reasoning block did not constrain its width or break long unbreakable tokens, so its inline-block bubble grew past its max width and pushed the whole page wider (#32481). Mirror the message body handling by capping the container width and breaking long words/code. Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: T K Chandra Hasan <t.k.chandra.hasan@ibm.com>
* feat(models): add GPT-5.6 (sol/terra/luna) pricing and metadata Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: allow gpt-5.6 service-tier cache-write keys in model prices schema Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: floating point entry errors --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Contributor
|
Too many files changed for review. ( Bypass the limit by tagging |
|
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| export function buildOnboardingUrl({ | ||
| baseUrl, | ||
| invitationId, |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
4 tasks
Contributor
Merging this PR will degrade performance by 17.6%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | test_completion_multi_turn |
3.1 ms | 3.8 ms | -17.6% |
| 🆕 | test_logging_executor_runs_inline |
N/A | 125.7 µs | N/A |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing litellm_internal_staging (a874de6) with main (9996378)1
Footnotes
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
| and decrypted_value.startswith("os.environ/") | ||
| ): | ||
| if isinstance(decrypted_value, str) and decrypted_value.startswith("os.environ/"): | ||
| return get_secret(decrypted_value) |
Contributor
There was a problem hiding this comment.
High: Environment secret exfiltration via team models
Resolving every os.environ/ value here also resolves values stored on team-scoped DB models. A team admin can create a team model with litellm_params.api_key: "os.environ/OPENAI_API_KEY" and an attacker-controlled api_base; when the router reloads, this line replaces the reference with the process secret and subsequent requests forward that secret to the attacker-controlled endpoint. Keep environment-variable resolution limited to proxy-admin-owned model rows, or reject os.environ/ references on team-scoped rows before constructing LiteLLM_Params.
Contributor
PR overviewThis pull request promotes internal staging changes into the main branch and includes updates around proxy server model configuration and router reload handling. There is one open security issue. Team-scoped model configuration can currently resolve environment-variable references, which could let a team admin cause process secrets to be forwarded to an attacker-controlled model endpoint. No issues have been addressed yet, so the PR still carries a clear secret-exfiltration risk until environment-variable resolution is restricted to trusted proxy-admin-owned configuration or rejected for team-scoped rows. Open issues (1)
Fixed/addressed: 0 · PR risk: 8/10 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes