feat: add vertex-anthropic provider - #2178
Merged
steebchen merged 27 commits intoMay 17, 2026
Merged
Conversation
Add Claude models via Google Cloud Vertex AI as a new provider using the Anthropic Messages API format with Vertex's rawPredict endpoint. Models: Sonnet 4.5/4.6, Opus 4.5/4.6/4.7, Haiku 4.5 Closes theopenco#2146 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughMaps Vertex-hosted Claude models and a new ChangesVertex Anthropic (Claude on Vertex) Integration
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/gateway/src/chat/chat.ts (1)
575-577:⚠️ Potential issue | 🟠 Major | ⚡ Quick winDecouple Google auth-token routing from Google-compatible payload logic
Line 576 currently makes
isGoogleCompatibleProvider()inheritusesGoogleQueryToken(). After addingvertex-anthropicat Line 571, it now gets routed through Google-specific paths (e.g., thought-signature enrichment, Google usage adjustments, image-token accounting), which can mutate Anthropic flows and skew billing/usage behavior.Suggested fix
function usesGoogleQueryToken(provider: string): boolean { return ( provider === "google-ai-studio" || provider === "glacier" || provider === "google-vertex" || provider === "quartz" || provider === "vertex-anthropic" ); } function isGoogleCompatibleProvider(provider: string): boolean { - return usesGoogleQueryToken(provider); + return ( + provider === "google-ai-studio" || + provider === "glacier" || + provider === "google-vertex" || + provider === "quartz" + ); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/chat/chat.ts` around lines 575 - 577, The helper isGoogleCompatibleProvider incorrectly proxies to usesGoogleQueryToken which causes non-Google providers like "vertex-anthropic" to be treated as Google for downstream routing (thought-signature enrichment, Google usage adjustments, image-token accounting); change isGoogleCompatibleProvider to implement explicit compatibility checks (e.g., whitelist Google-specific providers only) rather than calling usesGoogleQueryToken, so that usesGoogleQueryToken remains the auth-token routing predicate and isGoogleCompatibleProvider only returns true for providers that should receive Google-specific payload handling (update the isGoogleCompatibleProvider implementation and any callers to ensure vertex-anthropic is excluded).
🧹 Nitpick comments (1)
packages/actions/src/prepare-request-body.ts (1)
931-933: 💤 Low valueVertex-Anthropic request body preparation looks correct.
vertex-anthropicis correctly added toproviderHandlesCacheControl(the Anthropic Messages APIcache_controlformat is used for Vertex AI too).- Sharing the
anthropiccase branch is the right approach.anthropic_version: "vertex-2023-10-16"anddelete requestBody.modelalign with Vertex AI'srawPredict/streamRawPredictcontract.Minor note: lines 1621–1623 (
if (stream) { requestBody.stream = true; }) are a no-op —requestBody.streamis already initialised tostreamat line 994. The block can be removed without any behavioral change.🧹 Remove redundant stream assignment
if (usedProvider === "vertex-anthropic") { requestBody.anthropic_version = "vertex-2023-10-16"; delete requestBody.model; - if (stream) { - requestBody.stream = true; - } }Also applies to: 1327-1328, 1618-1624
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/actions/src/prepare-request-body.ts` around lines 931 - 933, The if (stream) { requestBody.stream = true; } blocks are redundant because requestBody.stream is already initialized to stream (see the requestBody initialization around line 994); remove these redundant assignments (occurrences around the blocks that reference requestBody and stream at the locations you noted, e.g., the blocks near lines 1327–1328 and 1618–1624) so the code relies on the single initial assignment and avoid duplicate/no-op reassignments in prepare-request-body.ts (look for the requestBody and stream variables in the prepareRequestBody logic).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/actions/src/get-provider-endpoint.ts`:
- Around line 359-389: The region for Vertex Anthropic is being computed twice
(vaRegion inside the "vertex-anthropic" case and vertexAnthropicRegion in the
URL switch), causing mismatches when an explicit baseUrl is provided; refactor
to compute the region once (e.g., hoist a single vaRegion or
vertexAnthropicRegion via getProviderEnvValue/providerKeyOptions before the URL
switch) and then reuse that same variable when building both the baseUrl (or
validating/extracting region from an explicit baseUrl) and the path segment used
in vaBaseEndpoint, ensuring the region in the hostname and the
/locations/${vaRegion}/ path always agree.
In `@packages/models/src/models/anthropic.ts`:
- Around line 962-980: The anthropic provider entries for claude-opus-4-6 and
claude-opus-4-7 include stale pricingTiers that overbill long-context requests;
remove the pricingTiers property from the anthropic entries for
"claude-opus-4-6" and "claude-opus-4-7" (the objects where providerId ===
"anthropic" and modelName matches those strings) so their billing matches the
new vertex-anthropic flat-rate behavior—ensure you delete the entire
pricingTiers key (and any related tier-specific fields) from those model objects
and run tests/lint to confirm no references remain.
---
Outside diff comments:
In `@apps/gateway/src/chat/chat.ts`:
- Around line 575-577: The helper isGoogleCompatibleProvider incorrectly proxies
to usesGoogleQueryToken which causes non-Google providers like
"vertex-anthropic" to be treated as Google for downstream routing
(thought-signature enrichment, Google usage adjustments, image-token
accounting); change isGoogleCompatibleProvider to implement explicit
compatibility checks (e.g., whitelist Google-specific providers only) rather
than calling usesGoogleQueryToken, so that usesGoogleQueryToken remains the
auth-token routing predicate and isGoogleCompatibleProvider only returns true
for providers that should receive Google-specific payload handling (update the
isGoogleCompatibleProvider implementation and any callers to ensure
vertex-anthropic is excluded).
---
Nitpick comments:
In `@packages/actions/src/prepare-request-body.ts`:
- Around line 931-933: The if (stream) { requestBody.stream = true; } blocks are
redundant because requestBody.stream is already initialized to stream (see the
requestBody initialization around line 994); remove these redundant assignments
(occurrences around the blocks that reference requestBody and stream at the
locations you noted, e.g., the blocks near lines 1327–1328 and 1618–1624) so the
code relies on the single initial assignment and avoid duplicate/no-op
reassignments in prepare-request-body.ts (look for the requestBody and stream
variables in the prepareRequestBody logic).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: b71a32f2-8810-41ad-b916-bd9c7476ca53
📒 Files selected for processing (13)
apps/gateway/src/chat/chat.tsapps/gateway/src/chat/tools/extract-content.tsapps/gateway/src/chat/tools/extract-reasoning.tsapps/gateway/src/chat/tools/extract-tool-calls.tsapps/gateway/src/chat/tools/parse-provider-response.tsapps/gateway/src/chat/tools/resolve-provider-context.tsapps/gateway/src/chat/tools/transform-streaming-to-openai.tspackages/actions/src/get-provider-endpoint.tspackages/actions/src/get-provider-headers.tspackages/actions/src/prepare-request-body.tspackages/db/src/schema.tspackages/models/src/models/anthropic.tspackages/models/src/providers.ts
analogpvt
force-pushed
the
feat/vertex-anthropic-provider
branch
from
May 7, 2026 08:36
2cf4f75 to
c765eb1
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
apps/gateway/src/chat/chat.ts (1)
583-588: ⚡ Quick winConsider centralizing Anthropic-compatible provider detection.
You now repeat
usedProvider === "anthropic" || isVertexClaudeModel(...)across many branches. A single helper (e.g.isAnthropicCompatibleProvider) would reduce drift risk and keep future provider extensions safer.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/chat/chat.ts` around lines 583 - 588, You repeat the same condition in many places; add a single helper isAnthropicCompatibleProvider(provider: string, modelName?: string) that returns true when provider === "anthropic" OR when isVertexClaudeModel(provider, modelName) is true, replace all occurrences of (usedProvider === "anthropic" || isVertexClaudeModel(...)) with calls to this new helper, and update any call sites that only pass provider to pass modelName where needed so the helper can evaluate both inputs (keep existing isVertexClaudeModel intact and call it from the new helper).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/gateway/src/chat/tools/extract-reasoning.ts`:
- Line 9: Replace the loosened parameter type "data: any" with a discriminated
union covering the provider-specific response shapes used in this file (e.g.,
Anthropic responses with delta.thinking, Google responses with
candidates[].content.parts, and OpenAI responses with
choices[].delta.reasoning); update the function signature in
extract-reasoning.ts (the parameter named data) to use that union and add narrow
type guards (by checking provider tags or presence of unique fields) before
accessing provider-specific properties so all accesses are type-safe and the
compiler can infer correct types for Anthropic/Google/OpenAI branches.
In `@packages/actions/src/get-provider-endpoint.ts`:
- Around line 348-352: The code uses modelName ?? "claude-sonnet-4-6@20250514"
inside the if (modelName?.startsWith("claude-")) branch which makes the fallback
unreachable and also embeds the wrong model ID; change claudeModel to use the
narrowed modelName directly (const claudeModel = modelName) and remove the
nullish fallback, or replace the fallback with the correct Vertex model ID
constant if you intend a default outside this branch; update references to
claudeModel/claudeBaseEndpoint accordingly.
In `@packages/models/src/models/anthropic.ts`:
- Around line 394-395: The model entry that sets providerId: "google-vertex"
currently uses an incorrect modelName "claude-sonnet-4-6@20250514" which will
produce 404s; update that modelName to the correct Vertex AI identifier
"claude-sonnet-4-6" in the same object (the Claude Sonnet 4.6 model definition)
so it matches the releasedAt metadata and official Vertex AI docs.
---
Nitpick comments:
In `@apps/gateway/src/chat/chat.ts`:
- Around line 583-588: You repeat the same condition in many places; add a
single helper isAnthropicCompatibleProvider(provider: string, modelName?:
string) that returns true when provider === "anthropic" OR when
isVertexClaudeModel(provider, modelName) is true, replace all occurrences of
(usedProvider === "anthropic" || isVertexClaudeModel(...)) with calls to this
new helper, and update any call sites that only pass provider to pass modelName
where needed so the helper can evaluate both inputs (keep existing
isVertexClaudeModel intact and call it from the new helper).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: cafd6e67-7157-4f0c-a3b3-044103d44656
📒 Files selected for processing (10)
apps/gateway/src/chat/chat.tsapps/gateway/src/chat/tools/extract-content.tsapps/gateway/src/chat/tools/extract-reasoning.tsapps/gateway/src/chat/tools/extract-tool-calls.tsapps/gateway/src/chat/tools/parse-provider-response.tsapps/gateway/src/chat/tools/resolve-provider-context.tsapps/gateway/src/chat/tools/transform-streaming-to-openai.tspackages/actions/src/get-provider-endpoint.tspackages/actions/src/prepare-request-body.tspackages/models/src/models/anthropic.ts
🚧 Files skipped from review as they are similar to previous changes (5)
- apps/gateway/src/chat/tools/parse-provider-response.ts
- apps/gateway/src/chat/tools/resolve-provider-context.ts
- apps/gateway/src/chat/tools/transform-streaming-to-openai.ts
- apps/gateway/src/chat/tools/extract-tool-calls.ts
- packages/actions/src/prepare-request-body.ts
- Decouple isGoogleCompatibleProvider from usesGoogleQueryToken (vertex-anthropic uses Bearer auth, not Google query token) - Remove API key query param from vertex-anthropic endpoint (rawPredict requires OAuth2 Bearer token via header) - Fix model IDs: remove date suffixes per Vertex AI docs - Type extractReasoning parameter (replace `any` with union) - Remove redundant stream assignment in prepare-request-body Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
apps/gateway/src/chat/tools/extract-reasoning.ts (1)
38-39: ⚡ Quick winPrefer type guards over
asassertions here.
StreamingChunkis a good step, but these casts sidestep compiler narrowing and can mask shape regressions. Narrow by provider/field presence instead of asserting.Suggested refactor
+function isAnthropicStreamChunk(data: StreamingChunk): data is AnthropicStreamChunk { + return "type" in data || "delta" in data; +} + +function isGoogleStreamChunk(data: StreamingChunk): data is GoogleStreamChunk { + return "candidates" in data; +} + +function isOpenAIStreamChunk(data: StreamingChunk): data is OpenAIStreamChunk { + return "choices" in data; +} + export function extractReasoning( data: StreamingChunk, provider: Provider, ): string { switch (provider) { case "anthropic": case "vertex-anthropic": { - const chunk = data as AnthropicStreamChunk; + if (!isAnthropicStreamChunk(data)) return ""; + const chunk = data; if ( chunk.type === "content_block_delta" && chunk.delta?.type === "thinking_delta" && chunk.delta?.thinking ) { return chunk.delta.thinking; } return ""; } @@ case "quartz": { - const chunk = data as GoogleStreamChunk; + if (!isGoogleStreamChunk(data)) return ""; + const chunk = data; const parts = chunk.candidates?.[0]?.content?.parts ?? []; const reasoningParts = parts.filter((part) => part.thought); return reasoningParts.map((part) => part.text).join("") ?? ""; } default: { - const chunk = data as OpenAIStreamChunk; + if (!isOpenAIStreamChunk(data)) return ""; + const chunk = data; return ( chunk.choices?.[0]?.delta?.reasoning ??Also applies to: 52-53, 58-60
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/chat/tools/extract-reasoning.ts` around lines 38 - 39, Replace the unsafe "as" assertions (e.g., "const chunk = data as AnthropicStreamChunk") with proper type guards that narrow the union by checking provider-specific discriminators/required fields instead of casting; add functions like isAnthropicStreamChunk(obj): obj is AnthropicStreamChunk and isStreamingChunk(obj): obj is StreamingChunk that test for the presence and shape of the provider/field(s) you rely on (e.g., provider/type/delta fields) and use those guards in the code paths where you currently use the casts (the "chunk" assignments and the other casts around the StreamingChunk checks) so the compiler can safely narrow types and catch shape regressions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@apps/gateway/src/chat/tools/extract-reasoning.ts`:
- Around line 38-39: Replace the unsafe "as" assertions (e.g., "const chunk =
data as AnthropicStreamChunk") with proper type guards that narrow the union by
checking provider-specific discriminators/required fields instead of casting;
add functions like isAnthropicStreamChunk(obj): obj is AnthropicStreamChunk and
isStreamingChunk(obj): obj is StreamingChunk that test for the presence and
shape of the provider/field(s) you rely on (e.g., provider/type/delta fields)
and use those guards in the code paths where you currently use the casts (the
"chunk" assignments and the other casts around the StreamingChunk checks) so the
compiler can safely narrow types and catch shape regressions.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 551c5ad2-0c83-4437-bbdd-b08231c4a5a6
📒 Files selected for processing (5)
apps/gateway/src/chat/chat.tsapps/gateway/src/chat/tools/extract-reasoning.tspackages/actions/src/get-provider-endpoint.tspackages/actions/src/prepare-request-body.tspackages/models/src/models/anthropic.ts
🚧 Files skipped from review as they are similar to previous changes (3)
- packages/actions/src/prepare-request-body.ts
- packages/actions/src/get-provider-endpoint.ts
- apps/gateway/src/chat/chat.ts
- Add vertex-anthropic to non-streaming response transform - Add vertex-anthropic to finish reason mapping - Migrate output_format to output_config.format (deprecated) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
apps/gateway/src/chat/tools/transform-response-to-openai.ts (1)
277-381: ⚖️ Poor tradeoffConsider extracting a shared
buildChatCompletionResponsehelper to reduce duplication.The
google-ai-studio/google-vertexblock (lines 281–329) and the newanthropic/vertex-anthropicblock (lines 333–380) are structurally identical. The only difference is that the Google block spreadsimagesinto the message. Extracting a small helper would eliminate the duplication and make future provider additions one-liners.♻️ Sketch of the extraction
+function buildChatCompletionResponse(params: { + usedProvider: Provider; + baseModelName: string; + content: string | null; + reasoningContent: string | null; + toolResults: any; + images?: ImageObject[]; + annotations: Annotation[] | null; + finishReason: string | null; + promptTokens: number | null; + completionTokens: number | null; + totalTokens: number | null; + reasoningTokens: number | null; + cachedTokens: number | null; + costs: CostData | null; + showUpgradeMessage: boolean; + cacheCreationTokens: number | null; + imageInputTokens: number | null; + imageOutputTokens: number | null; + requestedModel: string; + requestedProvider: string | null; + usedModel: string; + requestId: string; + routing: RoutingAttempt[] | null; + usedRegion?: string; +}) { ... } case "google-ai-studio": case "glacier": case "google-vertex": -case "quartz": { ... } +case "quartz": + transformedResponse = buildChatCompletionResponse({ ...params, images }); + break; case "anthropic": -case "vertex-anthropic": { ... } +case "vertex-anthropic": + transformedResponse = buildChatCompletionResponse({ ...params }); + break;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/chat/tools/transform-response-to-openai.ts` around lines 277 - 381, The two case blocks in transform-response-to-openai.ts create identical chat completion objects (same id/object/created/model/choices/usage/metadata) except the google block may include images, so extract a shared helper (e.g., buildChatCompletionResponse) that takes parameters like content, reasoningContent, toolResults, annotations, images, finishReason, usedProvider, baseModelName, usedModel, requestId, routing, usedRegion and token/cost params passed to buildUsageObject; use mapFinishReasonToOpenai, buildUsageObject and buildMetadata inside the helper and add an includeImages flag (or pass images array) so both the google-ai-studio/glacier/google-vertex/quartz and anthropic/vertex-anthropic cases call this helper to set transformedResponse, removing the duplicated object construction.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@apps/gateway/src/chat/tools/transform-response-to-openai.ts`:
- Around line 277-381: The two case blocks in transform-response-to-openai.ts
create identical chat completion objects (same
id/object/created/model/choices/usage/metadata) except the google block may
include images, so extract a shared helper (e.g., buildChatCompletionResponse)
that takes parameters like content, reasoningContent, toolResults, annotations,
images, finishReason, usedProvider, baseModelName, usedModel, requestId,
routing, usedRegion and token/cost params passed to buildUsageObject; use
mapFinishReasonToOpenai, buildUsageObject and buildMetadata inside the helper
and add an includeImages flag (or pass images array) so both the
google-ai-studio/glacier/google-vertex/quartz and anthropic/vertex-anthropic
cases call this helper to set transformedResponse, removing the duplicated
object construction.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: ebc0e94a-5f6e-4135-95a8-77747f85d065
📒 Files selected for processing (3)
apps/gateway/src/chat/tools/map-finish-reason-to-openai.tsapps/gateway/src/chat/tools/transform-response-to-openai.tspackages/actions/src/prepare-request-body.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/actions/src/prepare-request-body.ts
Lite models (gemini-2.0-flash-lite, gemini-2.5-flash-lite) use key= query param on /v1/publishers/ path. All project-based models (non-lite Gemini, Claude) use OAuth2 Bearer auth header. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/actions/src/get-provider-headers.ts`:
- Around line 43-52: The validation flow calls getProviderHeaders(provider,
token) without passing modelName, causing lite models to incorrectly get an
Authorization header; update the call site in validate-provider-key.ts to pass
the resolved validationModel via the options parameter (e.g.,
getProviderHeaders(provider, token, { modelName: validationModel })) so
getProviderHeaders can detect gemini-*-flash-lite/quartz lite models and omit
the Bearer header accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 6ae6c724-ecfc-4420-9dc7-02fc282e4af7
📒 Files selected for processing (3)
apps/gateway/src/chat/chat.tspackages/actions/src/get-provider-endpoint.tspackages/actions/src/get-provider-headers.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/actions/src/get-provider-endpoint.ts
- apps/gateway/src/chat/chat.ts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generates OAuth2 Bearer tokens from a GCP service account JSON file (GCP_SERVICE_ACCOUNT_KEY_FILE env var). Tokens are cached in Redis with a 50-minute TTL (GCP tokens expire after 60 min) and in-memory as a fast-path fallback. Eliminates manual token management for Vertex AI providers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Step-by-step guide for running Claude models on Vertex AI with automatic OAuth2 token refresh via service account. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace GCP_SERVICE_ACCOUNT_KEY_FILE + LLM_VERTEX_ANTHROPIC_PROJECT with LLM_VERTEX_ANTHROPIC_SERVICE_ACCOUNT_JSON. Project ID auto- extracted from JSON. GCP token refresh scoped to vertex-anthropic. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add step showing how to convert SA JSON to single-line env var. Fix BYOK callout to reference inline JSON instead of file path. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
steebchen
enabled auto-merge
May 12, 2026 13:37
steebchen
disabled auto-merge
May 12, 2026 13:37
analogpvt
force-pushed
the
feat/vertex-anthropic-provider
branch
from
May 14, 2026 17:45
9aed57c to
0f15778
Compare
analogpvt
force-pushed
the
feat/vertex-anthropic-provider
branch
2 times, most recently
from
May 14, 2026 17:54
426c835 to
4ac56ec
Compare
analogpvt
force-pushed
the
feat/vertex-anthropic-provider
branch
from
May 14, 2026 18:08
0467093 to
7121ddd
Compare
1 task
Move the vertex-anthropic doc from guides/ to integrations/ and adapt it to the provider integration format (Dashboard Provider Keys flow with self-host env vars as a secondary section). Wire LLM_VERTEX_ANTHROPIC_SERVICE_ACCOUNT_JSON and LLM_VERTEX_ANTHROPIC_REGION into the e2e workflow so the new provider is exercised in CI. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Anthropic recommends the global endpoint: dynamic routing, no pricing premium. Regional/multi-region endpoints add a 10% premium for Sonnet 4.5+. Users who need data residency or provisioned throughput can still set LLM_VERTEX_ANTHROPIC_REGION (or the Provider Key region field) to override. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Google's global endpoint is at aiplatform.googleapis.com (no region prefix), not global-aiplatform.googleapis.com. The latter returns 404 for the rawPredict path. Confirmed by hitting Vertex AI directly in e2e tests: requests to https://global-aiplatform.googleapis.com/v1/... returned the generic Google 404 HTML, while requests to https://aiplatform.googleapis.com/v1/projects/.../locations/global/... succeed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced May 17, 2026
This was referenced May 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
vertex-anthropicprovider to route Claude models through Google Cloud Vertex AIrawPredict/streamRawPredictendpointLLM_VERTEX_ANTHROPIC_SERVICE_ACCOUNT_JSON+LLM_VERTEX_ANTHROPIC_REGIONneededImplementation
New provider:
vertex-anthropichttps://{region}-aiplatform.googleapis.com/v1/projects/{project}/locations/{region}/publishers/anthropic/models/{model}:rawPredict(orstreamRawPredict)Automatic token refresh (
apps/gateway/src/lib/gcp-token.ts)LLM_VERTEX_ANTHROPIC_SERVICE_ACCOUNT_JSON(inline JSON string, no file path needed)LLM_VERTEX_ANTHROPIC_PROJECTneededConfig (minimal)
LLM_VERTEX_ANTHROPIC_SERVICE_ACCOUNT_JSON={"type":"service_account","project_id":"...","private_key":"...","client_email":"...","token_uri":"..."} LLM_VERTEX_ANTHROPIC_REGION=us-east5Docs
apps/docs/content/guides/vertex-anthropicTest plan
vertex-anthropic/claude-sonnet-4-6TEST_MODELS="vertex-anthropic/claude-sonnet-4-6"google-vertex(Gemini) — GCP token refresh scoped tovertex-anthropiconly🤖 Generated with Claude Code