Repository navigation
fix(sse): make Claude effort/no-think catalog variants dispatchable on every provider - #9006
Merged
diegosouzapw merged 19 commits intoAug 11, 2026
Conversation
wgordon17
marked this pull request as ready for review
July 30, 2026 22:16
wgordon17
force-pushed
the
fix/vertex-claude-catalog-dispatch
branch
from
August 4, 2026 13:40
19b58a9 to
6fd3f61
Compare
wgordon17
added a commit
to wgordon17/OmniRoute
that referenced
this pull request
Aug 4, 2026
Same two categories of inherited base-branch breakage as cheaper-inference#9006 (this PR shares the identical chatHelpers.ts diff), confirmed unrelated to this PR's own diff: - check:file-size: base.ts and chat.ts drifted further past their frozen caps via already-merged commits (7163081 and others) that didn't rebaseline after growing them. - chat-helpers.test.ts: two gpt-5.5 routing assertions predate cheaper-inference#9275 (fix(routing): bare model ids route to codex first), which deliberately made gpt-5.5 route to codex unconditionally. Updated both assertions and their names to match the new, intentional default.
diegosouzapw
force-pushed
the
fix/vertex-claude-catalog-dispatch
branch
from
August 11, 2026 00:27
d4848b9 to
25bb8d2
Compare
…h real streaming Claude models on Vertex AI were being sent through the generic OpenAI- compatible partner endpoint, which 404s/errors for Claude on at least some projects. Route them through Vertex's native Anthropic Messages API (publishers/anthropic/.../rawPredict) instead, stripping the body-level model field rawPredict rejects and injecting the required anthropic_version field. rawPredict only ever returns a complete JSON body, never real SSE framing, so streaming requests now get a genuine Anthropic-format SSE stream synthesized from that JSON (message_start/content_block_*/ message_delta/message_stop), which the existing claude-to-openai response translator already knows how to parse. Also fixes two response-format resolution bugs that silently dropped a custom model's DB-stored targetFormat override whenever the model id also existed in the static provider registry (as claude-sonnet-4-6 and claude-opus-4-7 do under vertex): resolveModelOrError had its own ad-hoc resolution that never consulted the override, and even once fixed, executeChatWithBreaker discarded the correctly-resolved format before handleChatCore's own resolution ran a second time.
Two categories of inherited base-branch breakage surfaced when rebasing onto release/v3.8.50's latest tip, both confirmed unrelated to this PR's own diff: - check:file-size: base.ts and chat.ts drifted further past their frozen caps via already-merged commits (7163081 and others) that didn't rebaseline after growing them. Documented and bumped in file-size-baseline.json. - chat-helpers.test.ts: two gpt-5.5 routing assertions predate cheaper-inference#9275 (fix(routing): bare model ids route to codex first), which deliberately made gpt-5.5 route to codex unconditionally, regardless of which other providers are active. Confirmed via cheaper-inference#9275's own commit message and code comments this is intentional, not a regression; verified reproducible on the raw base tip alone, with no changes from this PR involved. Updated both assertions and their names to match the new, intentional default.
The ALL_ACCOUNTS_INACTIVE->ALL_TARGETS_SKIPPED fix (a32aed738) added explanatory comments (+7 lines), pushing the file past its frozen 3457 cap. CI's PR-mode check:file-size caught it; local check-file-size.mjs was not re-run after that specific commit.
diegosouzapw
force-pushed
the
fix/vertex-claude-catalog-dispatch
branch
from
August 11, 2026 13:02
25bb8d2 to
2e92d4c
Compare
diegosouzapw
merged commit Aug 11, 2026
4795825
into
cheaper-inference:release/v3.8.50
2 of 5 checks passed
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…n every provider (cheaper-inference#9006) * fix(executors): route Claude-via-Vertex through native rawPredict with real streaming Claude models on Vertex AI were being sent through the generic OpenAI- compatible partner endpoint, which 404s/errors for Claude on at least some projects. Route them through Vertex's native Anthropic Messages API (publishers/anthropic/.../rawPredict) instead, stripping the body-level model field rawPredict rejects and injecting the required anthropic_version field. rawPredict only ever returns a complete JSON body, never real SSE framing, so streaming requests now get a genuine Anthropic-format SSE stream synthesized from that JSON (message_start/content_block_*/ message_delta/message_stop), which the existing claude-to-openai response translator already knows how to parse. Also fixes two response-format resolution bugs that silently dropped a custom model's DB-stored targetFormat override whenever the model id also existed in the static provider registry (as claude-sonnet-4-6 and claude-opus-4-7 do under vertex): resolveModelOrError had its own ad-hoc resolution that never consulted the override, and even once fixed, executeChatWithBreaker discarded the correctly-resolved format before handleChatCore's own resolution ran a second time. * docs: add changelog fragment for cheaper-inference#8909 * refactor(sse): extract shared Claude effort-model predicate * fix(sse): strip Claude effort-suffix ids for any provider serving a real Claude model * fix(sse): keep no-think and CC-discovery catalog variant roots unprefixed * fix(dashboard): re-qualify no-think playground model ids correctly * fix(sse): scope Vertex 404s to a per-model lockout via passthroughModels * docs: add changelog fragment for the Claude catalog/dispatch fix * fix(sse): align regex naming and changelog formatting * fix(sse): clarify effort-variant strip comment and add cross-module drift guard * fix(sse): disambiguate Vertex connection-wide vs per-model 403s * docs: document Vertex 403 disambiguation in changelog fragment * fix(sse): correlate reason and resource within the same ErrorInfo detail * fix(sse): extract Vertex error classifier and rebaseline frozen file sizes * test: register vertex-passthrough-model-lockout in stryker tap.testFiles * fix(sse): reconciles rebase-onto-tip drift for 9006 Two categories of inherited base-branch breakage surfaced when rebasing onto release/v3.8.50's latest tip, both confirmed unrelated to this PR's own diff: - check:file-size: base.ts and chat.ts drifted further past their frozen caps via already-merged commits (174e08b and others) that didn't rebaseline after growing them. Documented and bumped in file-size-baseline.json. - chat-helpers.test.ts: two gpt-5.5 routing assertions predate cheaper-inference#9275 (fix(routing): bare model ids route to codex first), which deliberately made gpt-5.5 route to codex unconditionally, regardless of which other providers are active. Confirmed via cheaper-inference#9275's own commit message and code comments this is intentional, not a regression; verified reproducible on the raw base tip alone, with no changes from this PR involved. Updated both assertions and their names to match the new, intentional default. * ci: re-trigger checks after GitHub Actions incident (2026-08-07, resolved) * ci: re-trigger checks (previous push event was dropped) * fix(quality): rebaseline combo-routing-engine.test.ts own-comment growth The ALL_ACCOUNTS_INACTIVE->ALL_TARGETS_SKIPPED fix (a32aed738) added explanatory comments (+7 lines), pushing the file past its frozen 3457 cap. CI's PR-mode check:file-size caught it; local check-file-size.mjs was not re-run after that specific commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
-high/-low/-medium/-xhigh) and no-thinking (no-think/) catalog model ids now strip/resolve correctly on any provider serving a real Claude model, not just the direct Anthropic provider