fix(router): mid-stream fallback 400s on models without assistant prefill (Claude Sonnet 4.6+) by mateo-berri · Pull Request #30743 · BerriAI/litellm

mateo-berri · 2026-06-18T13:17:10Z

Relevant issues

Internal CI mirror of #30242 by @cwang-otto so we can run CircleCI against the BerriAI/litellm repo. All credit for the fix goes to the original OSS contributor; the commits here preserve their authorship.

Mid-stream fallbacks fail deterministically on Claude Sonnet 4.6 / Opus 4.6+: the resume mechanism from #13149 appends the partial response as a prefixed assistant message, but Anthropic removed assistant prefill starting with these models, so every fallback hop returns This model does not support assistant message prefill. The conversation must end with a user message. A recoverable stream timeout then becomes a hard failure for the entire fallback chain.

Changes

Registry: supports_assistant_prefill: false for all *sonnet-4-6* entries in both cost maps (the opus-4-6/4-7/4-8 entries were already false; sonnet-4-6 was missed). The capability pin in test_claude_sonnet_4_6_config.py is updated accordingly.

router_utils/fallback_event_handlers.py: new build_mid_stream_continuation_messages; when the registry explicitly marks the primary model or any configured fallback target for the group as not supporting prefill, the partial response rides a trailing user message (the continuation pattern Anthropic's migration guide documents). All other models (capability true, absent, or unknown) keep the existing prefill-resume behavior byte-identical.

router.py: both injection sites (sync + async) now share the helper and pass the active fallback config.

Beyond the original PR, this branch also closes the two gaps Greptile flagged on the helper. Custom router model-group names are often aliases absent from the registry, so the router now injects a resolver that maps each candidate group to its deployment's registry model before the capability lookup; registry-keyed names keep working since the raw name is still checked. Candidate collection also gathers bare-string and dict-format fallback entries uniformly, so a prefill-rejecting entry in a mixed-format list is no longer skipped.

Pre-Submission checklist

I have added meaningful tests (tests/test_litellm/router_utils/test_fallback_event_handlers.py)
My PR's scope is as isolated as possible; it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

See above; this is a CI mirror of #30242 with the Greptile follow-ups folded in

Note

Medium Risk
Changes how continuation messages are built for all mid-stream fallbacks when any hop targets prefill-rejecting models; behavior is gated on explicit registry flags and is heavily tested, but incorrect classification could still alter resume semantics for mixed fallback chains.

Overview
Fixes mid-stream router fallbacks that always 400 on Claude Sonnet 4.6 / Opus 4.6+ because those models no longer accept assistant prefill. The old path appended a prefixed assistant message with partial output; that now fails with Anthropic’s prefill error and broke the whole fallback chain after a recoverable stream interrupt.

build_mid_stream_continuation_messages centralizes resume logic: when the model cost map explicitly sets supports_assistant_prefill: false for the primary model group, any bare-string fallback, or any dict-resolved fallback target, partial text is sent in a trailing user message (Anthropic’s documented continuation pattern). Everything else keeps the legacy system + prefixed assistant resume unchanged.

The sync and async router mid-stream fallback sites call this helper instead of inlined messages. _underlying_model_for_group maps custom router aliases to deployment registry names so capability checks work when the group name isn’t in the cost map. Dict fallback target resolution for the capability scan mirrors runtime get_fallback_model_group semantics (exact first, stripped/wildcard last-wins).

Registry updates flip supports_assistant_prefill to false on Sonnet 4.6–family entries across providers (both cost JSON files), add missing flags on a few routes, and align test_claude_sonnet_4_6_config. New test_fallback_event_handlers coverage exercises fallbacks, aliases, mixed list formats, and non-mutation of fallback lists.

^{Reviewed by Cursor Bugbot for commit 2068377. Bugbot is set up for automated code reviews on this repo. Configure here.}

…fill support Anthropic removed assistant message prefill starting with Claude Sonnet 4.6 / Opus 4.6 (per the official migration guide it returns a 400: 'This model does not support assistant message prefill'). The mid-stream fallback resume (#13149) appends the partial response as a prefixed assistant message, so every mid-stream fallback for these models fails deterministically across the whole fallback chain - converting recoverable stream timeouts into hard failures. - registry: supports_assistant_prefill=false for all *sonnet-4-6* entries in both cost maps (opus-4-6/4-7/4-8/fable entries were already false) - router_utils: build_mid_stream_continuation_messages - when the registry explicitly marks prefill unsupported, the partial response rides a trailing USER message (the continuation pattern Anthropic's migration guide documents); all other models keep the existing prefill-resume behavior unchanged - router: both (sync + async) injection sites now share the helper - utils: public supports_assistant_prefill() accessor (the registry field existed with 240 entries but had no supports_* accessor)

…ic accessor, update capability pin test - Greptile: the same continuation messages go to every fallback target, so the user-message continuation now engages when the primary OR any configured fallback target for the group is explicitly marked prefill-unsupported (via get_fallback_model_group) - Greptile: drop the supports_assistant_prefill() public accessor - _supports_factory's absent-key=False default contradicts this feature's absent-key=legacy-prefill routing; the routing reads get_model_info directly - CI: test_claude_sonnet_4_6_config pinned the stale capability value; prefill was removed in Sonnet 4.6 so the pin is now False

…ps string-format entries)

…_ai_gateway) The sweep matched the dash form (sonnet-4-6) and missed openrouter/anthropic/claude-sonnet-4.6, openrouter/anthropic/claude-opus-4.6, vercel_ai_gateway/anthropic/claude-opus-4.6 (dot form). Test param added for the dot variant.

… github_copilot opus-4.6-fast lack supports_assistant_prefill:false

…ages Swap List[...] -> list[...] in the new helper so the change adds 0 UP006 violations; the litellm_internal_staging strict-rule budget is tighter than litellm_oss_branch and flagged the 4 added annotations.

…ry flag Greptile findings: 1. build_mid_stream_continuation_messages now scans every entry of a flat string-format fallback list directly. get_fallback_model_group pops a single string mid-iteration, so a prefill-rejecting model at a non-first position was never capability-checked; since the continuation is built once and reused across all fallback hops, the chain still 400'd when it reached that model. Dict/standard format keeps using get_fallback_model_group (already returns the full group). 2. snowflake/claude-sonnet-4-6 gets supports_assistant_prefill:false in both the root map and the bundled backup (it existed only in root before; the backup is the offline/test source), completing the 4-6 sweep. Tests: +snowflake registry param, +flat-string non-first-position -> user continuation, +flat-string all-supporting -> legacy prefill.

Rebased onto current litellm_internal_staging, whose strict-rule budget now tracks UP045 (non-pep604 annotations). Convert the 4 Optional[...] annotations the helper added to X | None so the change stays within budget.

mateo-berri · 2026-06-18T13:17:30Z

@greptileai

Generated by Claude Code

greptile-apps · 2026-06-18T13:24:21Z

Greptile Summary

This PR fixes mid-stream router fallbacks that 400 deterministically on Claude Sonnet 4.6 / Opus 4.6+ because those models no longer accept assistant prefill. It is a CI mirror of OSS PR #30242 with two follow-up fixes for gaps flagged in the previous review round.

Registry: supports_assistant_prefill is flipped to false for all *sonnet-4-6* provider entries across both cost maps, and the previously-missing Snowflake/GitHub Copilot/Perplexity entries gain explicit false flags.
Router: Both sync and async mid-stream fallback paths now call build_mid_stream_continuation_messages, which checks whether the primary model group, any configured fallback target, or their resolved deployment aliases explicitly rejects prefill; if any does, the partial text rides a trailing user message instead of the old prefilled-assistant message.
Tests: A comprehensive mock-only test suite covers all fallback-list shapes (flat, dict, mixed), alias resolution, resolver error handling, mutation safety, and key-priority mirroring; the Bedrock Sonnet 4.6 capability assertion is corrected from true to false.

Confidence Score: 5/5

Safe to merge; the fix is well-scoped and degrades gracefully to legacy prefill when capability is unknown.

The registry and router changes together close a deterministic 400 on Sonnet 4.6+ mid-stream fallbacks without touching the legacy prefill path for any model that does not explicitly opt out. The only remaining edge case is _underlying_model_for_group checking only the first deployment in a heterogeneous group, which is a very narrow configuration.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/router_utils/fallback_event_handlers.py	Adds build_mid_stream_continuation_messages with full candidate-collection logic; mirrors get_fallback_model_group priority faithfully, no mutation of caller's list, resolver exceptions are swallowed safely.
litellm/router.py	Both sync and async mid-stream fallback paths now call build_mid_stream_continuation_messages; adds _underlying_model_for_group which returns only the first deployment for a group (acceptable for homogeneous groups).
tests/test_litellm/router_utils/test_fallback_event_handlers.py	New mock-only test file covering all edge cases: prefill-rejecting models, legacy models, alias resolution, mixed fallback formats, mutation safety, stripped/wildcard key priority, and resolver error handling.
tests/test_litellm/test_claude_sonnet_4_6_config.py	Correctly updates Bedrock Sonnet 4.6 assertion from supports_assistant_prefill=true to false, matching the registry change; this is a test correction, not a weakening.
tests/test_litellm/test_router.py	Adds test_underlying_model_for_group_resolves_alias verifying alias->registry-model resolution; no real network calls, uses sk-fake key.
model_prices_and_context_window.json	Flips supports_assistant_prefill from true to false on all claude-sonnet-4-6 provider entries; adds the flag to Snowflake/GitHub Copilot/Perplexity Sonnet 4.6 entries.
litellm/model_prices_and_context_window_backup.json	Mirrors the primary JSON registry changes for supports_assistant_prefill on all Sonnet 4.6 entries.

_{Reviews (5): Last reviewed commit: "fix(router): mirror last-wins for stripp..." | Re-trigger Greptile}

…-stream prefill check Greptile flagged two gaps in build_mid_stream_continuation_messages. Custom router model-group names (e.g. an alias that resolves to claude-sonnet-4-6) are absent from the registry, so the capability lookup silently fell back to the legacy prefill path and the 400 persisted; the router now injects a resolver that maps each candidate group to its deployment's registry model. Mixed dict+string fallback lists also let a prefill-rejecting bare-string entry slip through, because get_fallback_model_group only surfaces the dict target for the group; candidate collection now gathers bare strings and dict targets uniformly across every list format.

mateo-berri · 2026-06-18T13:35:55Z

@greptileai

Generated by Claude Code

The router code-coverage gate requires every router.py function to be exercised by a test file whose name contains "router". Add a focused test that builds a Router with an aliased deployment and asserts the group name resolves to the deployment's registry model, with None for unknown groups.

codecov · 2026-06-18T13:38:20Z

Codecov Report

❌ Patch coverage is 98.00000% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/router.py	85.71%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

…branches The FallbackEntry alias used typing.Dict/List, tripping the UP006 strict-rule budget; switch to builtin generics. Drop the now-unreachable None guard in _prefill_explicitly_unsupported (callers filter None before the lookup) and add tests for the malformed-fallback and resolver-raises paths so the defensive branches are exercised.

mateo-berri · 2026-06-18T13:45:53Z

@greptileai

Generated by Claude Code

…_group's pop quirk get_fallback_model_group pops bare-string entries while iterating the list, so a string entry preceding a dict entry skips that dict entry and could drop its prefill-rejecting target from the candidate set. Resolve dict-format fallback targets with direct iteration that mirrors the exact > stripped > wildcard priority, and add regression tests for the string-before-dict ordering, exact vs wildcard priority, and stripped-group matching.

mateo-berri · 2026-06-18T13:55:40Z

@greptileai

Generated by Claude Code

mateo-berri · 2026-06-18T14:02:20Z

bugbot run

Generated by Claude Code

cursor

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

✅ Fixed: Stripped fallback target mismatch
- Updated _resolve_dict_fallback_targets to pick the last stripped/wildcard match (mirroring get_fallback_model_group's last-wins overwrite within those tiers) and added regression tests covering duplicate stripped and wildcard keys.

_{You can send follow-ups to the cloud agent here.}

…olution _resolve_dict_fallback_targets returned the first stripped or wildcard match, but get_fallback_model_group keeps overwriting and uses the last match in each of those tiers. With duplicate stripped keys (or wildcards), the runtime fallback hop calls the last target while the capability scan only saw the first, so a Sonnet 4.6+ deployment hidden behind a prefill-supporting first hit was never detected and the mid-stream resume still sent assistant prefill and got a 400. Mirror the helper's per-tier semantics: exact still first-wins-and-breaks; stripped and wildcard now pick the last match in their tier.

CLAassistant · 2026-06-18T14:11:42Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ mateo-berri
✅ cwang-otto
❌ cursoragent
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

mateo-berri · 2026-06-18T14:14:34Z

@greptileai

Generated by Claude Code

mateo-berri · 2026-06-18T14:20:07Z

bugbot run

Generated by Claude Code

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 2068377. Configure here.}

Removing the continuation-prompt fallback (retrying with the partial response as a prefixed assistant message) so a stream failing after partial content always re-raises instead was a scope decision beyond what this PR's title/issue (BerriAI#31874) describe, and it directly conflicts with BerriAI#30242/BerriAI#30743, which are already fixing the same code path for Anthropic's removal of assistant-message prefill on Sonnet 4.6+/Opus 4.6+. Landing this PR's version first would delete the branch those PRs are patching; landing theirs first would have this PR undo their fix on rebase. Restores the original prefill-based continuation-resume behavior (including the is_pre_first_chunk guard already in litellm_internal_staging) in both _acompletion_streaming_iterator and _completion_streaming_iterator, and removes _stream_chunks_have_generated_content along with the tests that only existed to cover the guard. This PR now only touches the deferred-stream eager-fetch fix and the header-stripping fixes; the non-text-content re-raise idea becomes a follow-up PR built on top of whichever of BerriAI#30242/BerriAI#30743 lands.

…errors in _acompletion fallback path (#34627) * fix(router): eagerly fetch deferred stream to surface HTTP errors in fallback path Providers like Vertex AI and Bedrock defer their HTTP call until the first __anext__ on the returned CustomStreamWrapper (completion_stream=None, make_call set). Errors raised inside __anext__ (e.g. 429, 503) escape the _acompletion try/except block, so fail_calls is never incremented, deployment cooldown does not fire, and the standard fallback chain is bypassed. Call fetch_stream() on the wrapper before delegating to _acompletion_streaming_iterator when completion_stream is None and make_call is set. Any HTTP error now propagates through _acompletion's except block, increments fail_calls, and enters the normal retry/fallback chain. Strip Content-Length, Transfer-Encoding, Content-Encoding, and Content-Type from exception headers at the same point to prevent HTTP framing mismatches when LiteLLM builds its own error response body. Add a re-raise guard in _acompletion_streaming_iterator (async and sync paths) so MidStreamFallbackError with already-generated content re-raises to the caller instead of silently injecting a continuation prompt into a fresh request to a fallback model. Apply logging cleanup in async_function_with_fallbacks_common_utils: use %s-style formatting and exc_info=True instead of f-strings with traceback.format_exc(). * fix(router): undo success_calls on deferred-stream fetch failure; broaden header strip * fix(router): extract header-strip helper to keep _acompletion under strict C901 threshold * test(router): add unit tests for _strip_http_framing_headers to satisfy router coverage gate * test(router): add sync _completion_streaming_iterator re-raise test for mid-chunk MidStreamFallbackError * fix(router): restore Fallbacks context in no-fallback log; document update_team mcp_rpm_limit The log and debug message when no fallback model group is found was missing the Fallbacks list, making it hard to understand why routing failed. Also adds the missing mcp_rpm_limit documentation to update_team to fix the documentation_test_api_docs CI check. * fix(router): preserve original traceback in deferred stream fetch error re-raise Using bare `raise` instead of `raise fetch_err` keeps the full inner traceback from fetch_stream() intact so the error origin is visible in logs and debuggers without being anchored to this line. * style(test): restore black-style formatting in test_router.py An earlier commit on this branch collapsed the file's pre-existing multi-line formatting into single lines while adding the deferred-stream tests, producing a diff full of unrelated reformatting noise. Restores the untouched code to its original formatting; the actual new/changed test content is unaffected (verified via AST comparison). * fix(router): re-raise mid-stream fallback on any generated content, not just text The re-raise guard added for MidStreamFallbackError only checked generated_content, which tracks text deltas alone. A stream that emitted a tool-call or reasoning-only chunk before failing had generated_content="" despite already streaming to the client, so the router silently retried and the client saw duplicated/inconsistent output. The guard now also inspects the wrapper's raw chunks for tool_calls/reasoning_content. Also moves the deferred-stream HTTP-framing-header stripping out of Router._acompletion into the proxy's _handle_llm_api_exception: Router is used directly as an SDK as well as by the proxy, and stripping headers there dropped legitimate provider metadata (content-type, proxy-authenticate) for direct SDK callers who never see the proxy's own response construction. schema.d.ts regenerated via make pre-commit; unrelated to this change. * test(router): add direct coverage for _stream_chunks_have_generated_content CI's router_code_coverage check flags any router.py function never referenced by name in a test file; the new helper was only exercised indirectly through the mid-stream re-raise guard tests. * revert(ui): drop incidental schema.d.ts regeneration Committing router.py/common_request_processing.py touched pre_commit_lint.sh's litellm/proxy trigger for the API-type-sync check, which force-regenerated schema.d.ts even though neither file changes any route or model. The regenerated ordering of two unrelated Union/enum fields (stream_timeout, user_role) isn't stable across process invocations even against completely unmodified backend code (confirmed by regenerating twice against the pre-existing committed code and getting the same diff both times), so this reverts to the original committed file rather than chase non-deterministic output. * fix(proxy): strip framing headers on the pre-existing ProxyException branch too _handle_llm_api_exception filtered framing headers into a local `headers` dict, but for an exception that's already a ProxyException, it merged {**e.headers, **headers}: the original e.headers came first, so a framing header present there but absent from the filtered `headers` (because it was just stripped) was never overwritten and survived into the response unfiltered. Filters the merged result instead of relying on the merge order to do it implicitly. * chore: retrigger CI (no GitHub Actions check-suite was created for the previous two pushes) * fix(router): detect thinking_blocks as generated content in mid-stream guard Greptile flagged that a thinking-only delta (Anthropic extended thinking, Delta.thinking_blocks) wasn't recognized as already-streamed content, so a stream that emitted only thinking blocks before failing could still restart via fallback and append an unrelated response after content the client already received. * fix(proxy): strip browser-facing security headers from provider exceptions too veria-ai flagged that the framing-header denylist still let a malicious or misconfigured provider set browser-facing headers (Access-Control-Allow-Origin, Content-Security-Policy, Clear-Site-Data, etc.) on the proxy's own error response. Adds a dedicated _BROWSER_SECURITY_HEADERS set alongside the existing framing one and strips both wherever provider exception headers reach the client response. * refactor(router): address maintainer review mechanicals - List[ModelResponseStream] -> list[ModelResponseStream] in _stream_chunks_have_generated_content (ruff UP006 strict-budget gate) - drop _strip_http_framing_headers and its 3 tests: the proxy inlines the filter directly now, so the helper has had no production caller since the header-stripping was moved out of Router - move HTTP_FRAMING_HEADERS/BROWSER_SECURITY_HEADERS/ UNSAFE_PROXY_RESPONSE_HEADERS from router.py into litellm/constants.py, removing the router.py <-> proxy import path the two CodeQL cyclic-import alerts were pointing at - move the eager fetch_stream() call before success_calls/logging/ _track_deployment_metrics instead of incrementing then compensating with a manual decrement on failure - fix a dead assert message: `mock_fallback.assert_not_called(), "..."` built a tuple, not an assert-with-message; assert_not_called() already raises on its own so this just drops the inert string * revert(router): pull mid-stream continuation-removal out of this PR Removing the continuation-prompt fallback (retrying with the partial response as a prefixed assistant message) so a stream failing after partial content always re-raises instead was a scope decision beyond what this PR's title/issue (#31874) describe, and it directly conflicts with #30242/#30743, which are already fixing the same code path for Anthropic's removal of assistant-message prefill on Sonnet 4.6+/Opus 4.6+. Landing this PR's version first would delete the branch those PRs are patching; landing theirs first would have this PR undo their fix on rebase. Restores the original prefill-based continuation-resume behavior (including the is_pre_first_chunk guard already in litellm_internal_staging) in both _acompletion_streaming_iterator and _completion_streaming_iterator, and removes _stream_chunks_have_generated_content along with the tests that only existed to cover the guard. This PR now only touches the deferred-stream eager-fetch fix and the header-stripping fixes; the non-text-content re-raise idea becomes a follow-up PR built on top of whichever of #30242/#30743 lands. * fix(proxy): re-filter unsafe headers after the response-headers hook merge _handle_llm_api_exception filtered provider/framing headers once, then merged in post_call_response_headers_hook's return value afterward without re-filtering. The ProxyException branch happened to re-filter after its own header merge, but the HTTPException/httpx.HTTPStatusError/ generic-exception branches passed the post-hook headers straight through unfiltered, so a callback hook (any custom guardrail/logging plugin) returning an unsafe header would bypass the strip entirely for those paths. Filters once, right after the hook merge, so every branch gets the same guarantee. * Revert "revert(router): pull mid-stream continuation-removal out of this PR" This reverts commit c5ca101. * fix(router): detect reasoning_items as generated content in mid-stream guard Greptile flagged that a structured reasoning-only delta (Delta.reasoning_items, the OpenAI Responses-API-style reasoning item) wasn't recognized as already-streamed content by _stream_chunks_have_generated_content, alongside the existing thinking_blocks/tool_calls checks, so a stream that emitted only reasoning_items before failing could still restart via fallback. * fix(router): annotate _stream_chunks_have_generated_content with Sequence, not list The type_discipline_gate LIT001 check flags mutable-collection parameter annotations. chunks is only iterated, never mutated, so Sequence is the correct read-only annotation and clears the ratcheted budget ceiling. * fix(router): surface original provider exception, not the internal wrapper, when mid-stream fallback gives up When content has already streamed and MidStreamFallbackError carries original_exception (e.g. RateLimitError), both the async and sync streaming iterators bare-re-raised the wrapper itself, so the client lost the specific error type/code/provider_specific_fields instead of seeing the real provider error. The fallback-failure path a few lines below already unwraps to original_exception for the same reason; apply the same pattern here. Also extend _stream_chunks_have_generated_content to recognize audio, images, and annotations deltas as generated content, matching is_chunk_non_empty's existing annotations check and Delta's treatment of audio/images as first-class content fields — a stream carrying only one of these before failing was not recognized as already-streamed, so the router could still restart it via fallback after the client had received real content. * chore: retrigger CI (frontend-lint cancelled, schema.d.ts flake) frontend-lint's check-run shows conclusion=cancelled on 70e47f4 with no superseding run, and this PR touches no UI files. Verify schema.d.ts matches the proxy OpenAPI spec is on the previously diagnosed stream_timeout/user_role Union-ordering nondeterminism (e9fc5e5). Empty commit to force a fresh CI run for both rather than a manual rerun, which requires repo admin rights this fork PR doesn't have. --------- Co-authored-by: Deepanshu <deepanshu.lulla@alpha-sense.com>

devin-ai-integration · 2026-08-17T16:34:31Z

🚅 Hi, thanks for the PR! I'm Agent Shin, the automated triage bot for this repository. What's this and why am I getting it?

I read the description against our contribution rubric. Here's how it lined up:

What you got right:

✅ Linked a related GitHub issue
✅ Clear problem description
✅ Expected vs. actual behavior

What's still missing:

end-to-end QA proof (video, screenshot, or real command output)

The PR has strong problem context and clearly describes expected vs. actual behavior, including a linked issue. However, it contains no end-to-end QA evidence in the body—only test additions and claims—so it does not meet the contribution standard.

If the description isn't updated in the next 24 hours, I'll auto-close this PR. That's not us saying we don't care about the change; we want the open-PR list to mirror what a maintainer can act on right now, so contributors don't get lost in a backlog. A closed PR is a soft "park this for later," not a rejection. Take your time; everything below still works after the close.

During the grace period: just update the PR description with the missing pieces. No need to ping me; I'll re-check on the next sweep and skip the auto-close if it now passes. See what counts as QA proof for the full rubric (a linked issue alone isn't enough; it covers context, not proof).

If the PR does get auto-closed in 24 hours, you still have easy recovery paths:

Comment @agent-shin reconsider after updating the description. I'll re-evaluate and reopen the PR if it now passes.
Comment @greptileai to request a fresh Greptile review; that still works even after the PR is closed, and a stronger score is one of the signals that lifts the PR back into the queue. So a low Greptile score isn't a blocker either.

Internal BerriAI contributors: this rubric doesn't apply to you; ping a maintainer.

(I'm an LLM, so I'm not infallible. If you think I got this wrong, ping a maintainer; they'll override me.)

cwang-otto added 8 commits June 17, 2026 21:17

address review: copy fallbacks before get_fallback_model_group (it po…

f4f704a

…ps string-format entries)

address review: registry sweep stragglers — perplexity opus-4-6/4-7 +…

cb2b69d

… github_copilot opus-4.6-fast lack supports_assistant_prefill:false

fix(lint): use builtin generics in build_mid_stream_continuation_mess…

fa070c2

…ages Swap List[...] -> list[...] in the new helper so the change adds 0 UP006 violations; the litellm_internal_staging strict-rule budget is tighter than litellm_oss_branch and flagged the 4 added annotations.

fix(lint): use X | None over Optional in mid-stream helper (UP045)

fd2fce1

Rebased onto current litellm_internal_staging, whose strict-rule budget now tracks UP045 (non-pep604 annotations). Convert the 4 Optional[...] annotations the helper added to X | None so the change stays within budget.

greptile-apps Bot reviewed Jun 18, 2026

View reviewed changes

Comment thread litellm/router_utils/fallback_event_handlers.py Outdated

Comment thread litellm/router_utils/fallback_event_handlers.py Outdated

cursor Bot reviewed Jun 18, 2026

View reviewed changes

Comment thread litellm/router_utils/fallback_event_handlers.py Outdated

cursor Bot reviewed Jun 18, 2026

View reviewed changes

yassin-berriai mentioned this pull request Jul 28, 2026

fix(router): eagerly fetch Vertex AI deferred stream to surface HTTP errors in _acompletion fallback path #34627

Merged

5 tasks

Uh oh!

Uh oh!

Conversation

mateo-berri commented Jun 18, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Relevant issues

Changes

Pre-Submission checklist

Type

Changes

Uh oh!

mateo-berri commented Jun 18, 2026

Uh oh!

greptile-apps Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

mateo-berri commented Jun 18, 2026

Uh oh!

codecov Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mateo-berri commented Jun 18, 2026

Uh oh!

mateo-berri commented Jun 18, 2026

Uh oh!

mateo-berri commented Jun 18, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

CLAassistant commented Jun 18, 2026

Uh oh!

mateo-berri commented Jun 18, 2026

Uh oh!

mateo-berri commented Jun 18, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot commented Aug 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mateo-berri commented Jun 18, 2026 •

edited by cursor Bot

Loading

greptile-apps Bot commented Jun 18, 2026 •

edited

Loading

codecov Bot commented Jun 18, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading