fix(provider): enable 1M context for Vertex AI Anthropic models#14055
fix(provider): enable 1M context for Vertex AI Anthropic models#14055lanej wants to merge 3 commits intoanomalyco:devfrom
Conversation
1bfa676 to
7026729
Compare
There was a problem hiding this comment.
Pull request overview
Updates the google-vertex-anthropic provider integration so Anthropic beta flags can be forwarded in a way Vertex AI honors (as anthropic_beta in the JSON request body rather than via HTTP headers).
Changes:
- Adds a
fetchwrapper inProvider.getSDK()to injectanthropic_betainto POST request bodies forgoogle-vertex-anthropic. - Adds provider config tests for
options.headers["anthropic-beta"]andoptions.betas. - Adds a test intended to validate beta deduplication behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| packages/opencode/src/provider/provider.ts | Adds a fetch interceptor to inject anthropic_beta into Vertex Anthropic POST request bodies. |
| packages/opencode/test/provider/provider.test.ts | Adds tests around config parsing for Vertex Anthropic headers/betas and a deduplication-focused test. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
39e5bca to
c8e03fb
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Follow-up to my reply on the |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
57ae22a to
50a8f10
Compare
50a8f10 to
df96183
Compare
…ontext
Registers claude-sonnet-4-6-1m@default and claude-sonnet-4-5-1m@20250929
as explicit model variants in the google-vertex-anthropic provider with
headers: { "anthropic-beta": "context-1m-2025-08-07" } and a 1M context
limit.
Vertex AI honors the anthropic-beta HTTP header and forwards it to the
Anthropic backend, enabling the 1M context window. The body param
anthropic_beta is silently ignored by Vertex. An earlier implementation
stripped the header (which the SDK correctly sets via model.headers) and
replaced it with a body param, breaking the feature entirely. The fix is
to remove the interceptor and let the header pass through unmodified.
Also removes the options.betas extraction and transform.ts betas injection,
both of which were based on the same incorrect assumption.
b1c6ddd to
5fb733c
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if (init.headers && typeof init.headers === "object" && !Array.isArray(init.headers)) { | ||
| Object.assign(headers, init.headers) |
There was a problem hiding this comment.
The header capture logic only handles plain object headers and will miss headers if the SDK passes a Headers instance or an array of tuples. The condition typeof init.headers === "object" && !Array.isArray(init.headers) excludes arrays but will still try to Object.assign a Headers instance, which may not copy all headers correctly. Consider using Headers.prototype.forEach or converting headers to a plain object more robustly to ensure the test captures the anthropic-beta header regardless of the format used by the SDK.
| if (init.headers && typeof init.headers === "object" && !Array.isArray(init.headers)) { | |
| Object.assign(headers, init.headers) | |
| const rawHeaders = init?.headers | |
| if (rawHeaders instanceof Headers) { | |
| rawHeaders.forEach((value, key) => { | |
| headers[key] = value | |
| }) | |
| } else if (Array.isArray(rawHeaders)) { | |
| for (const [key, value] of rawHeaders as Array<[string, string]>) { | |
| headers[key] = value | |
| } | |
| } else if (rawHeaders && typeof rawHeaders === "object") { | |
| for (const [key, value] of Object.entries(rawHeaders as Record<string, string>)) { | |
| headers[key] = String(value) | |
| } |
| test("google-vertex-anthropic accepts betas array from config", async () => { | ||
| await using tmp = await tmpdir({ | ||
| init: async (dir) => { | ||
| await Bun.write( | ||
| path.join(dir, "opencode.json"), | ||
| JSON.stringify({ | ||
| $schema: "https://opencode.ai/config.json", | ||
| provider: { | ||
| "google-vertex-anthropic": { | ||
| options: { | ||
| betas: ["context-1m-2025-08-07"], | ||
| }, | ||
| }, | ||
| }, | ||
| }), | ||
| ) | ||
| }, | ||
| }) | ||
| await Instance.provide({ | ||
| directory: tmp.path, | ||
| init: async () => { | ||
| Env.set("GOOGLE_CLOUD_PROJECT", "test-project") | ||
| }, | ||
| fn: async () => { | ||
| const providers = await Provider.list() | ||
| expect(providers["google-vertex-anthropic"]).toBeDefined() | ||
| expect(providers["google-vertex-anthropic"].options.betas).toEqual(["context-1m-2025-08-07"]) | ||
| }, | ||
| }) | ||
| }) |
There was a problem hiding this comment.
This test verifies that the betas array config option is accepted and stored, but there is no code in provider.ts that reads or uses options["betas"]. Based on the PR description, Vertex AI requires the anthropic-beta header (not a betas array), and the 1M variants are implemented using model.headers. This test creates a misleading impression that the betas config option is functional when it appears to be unused. Consider removing this test or documenting that the betas option is not currently implemented for google-vertex-anthropic.
| test("google-vertex-anthropic accepts betas array from config", async () => { | |
| await using tmp = await tmpdir({ | |
| init: async (dir) => { | |
| await Bun.write( | |
| path.join(dir, "opencode.json"), | |
| JSON.stringify({ | |
| $schema: "https://opencode.ai/config.json", | |
| provider: { | |
| "google-vertex-anthropic": { | |
| options: { | |
| betas: ["context-1m-2025-08-07"], | |
| }, | |
| }, | |
| }, | |
| }), | |
| ) | |
| }, | |
| }) | |
| await Instance.provide({ | |
| directory: tmp.path, | |
| init: async () => { | |
| Env.set("GOOGLE_CLOUD_PROJECT", "test-project") | |
| }, | |
| fn: async () => { | |
| const providers = await Provider.list() | |
| expect(providers["google-vertex-anthropic"]).toBeDefined() | |
| expect(providers["google-vertex-anthropic"].options.betas).toEqual(["context-1m-2025-08-07"]) | |
| }, | |
| }) | |
| }) |
| // Vertex AI 1M context window support via the context-1m-2025-08-07 beta flag. | ||
| // Vertex reads and forwards the anthropic-beta HTTP header to the Anthropic backend; | ||
| // the header is set in model.headers and flows through the SDK automatically. | ||
| { | ||
| const VERTEX_1M_BETA = "context-1m-2025-08-07" | ||
| const VERTEX_1M_MODELS: Array<{ sourceID: string; variantID: string; name: string }> = [ | ||
| { | ||
| sourceID: "claude-sonnet-4-6@default", | ||
| variantID: "claude-sonnet-4-6-1m@default", | ||
| name: "Claude Sonnet 4.6 (1M)", | ||
| }, | ||
| { | ||
| sourceID: "claude-sonnet-4-5@20250929", | ||
| variantID: "claude-sonnet-4-5-1m@20250929", | ||
| name: "Claude Sonnet 4.5 (1M)", | ||
| }, | ||
| ] | ||
| const vertexProvider = database["google-vertex-anthropic"] | ||
| if (vertexProvider) { | ||
| for (const { sourceID, variantID, name } of VERTEX_1M_MODELS) { | ||
| const source = vertexProvider.models[sourceID] | ||
| if (source) { | ||
| vertexProvider.models[variantID] = { | ||
| ...source, | ||
| id: variantID, | ||
| name, | ||
| api: { ...source.api, id: sourceID }, | ||
| limit: { ...source.limit, context: 1000000, input: 0 }, | ||
| headers: { ...source.headers, "anthropic-beta": VERTEX_1M_BETA }, | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
The PR description states "Remove result["betas"] providerOptions injection (was a no-op; anthropicProviderOptions schema has no betas field)" under the "Changes" section for transform.ts, but there are no changes to transform.ts in this diff, and searching the file shows no such code exists. Either the code was already removed in a previous commit (and shouldn't be mentioned in this PR's changes), or the PR description is inaccurate. Please clarify or update the PR description to accurately reflect the changes in this specific PR.
Summary
Adds 1M context window support for Vertex AI Anthropic models (
claude-sonnet-4-6-1m@default,claude-sonnet-4-5-1m@20250929, andclaude-opus-4-6@default).Key Finding
Vertex AI honors the
anthropic-betaHTTP header and forwards it to the Anthropic backend. An earlier implementation stripped the header and substituted ananthropic_betabody param — which Vertex silently ignores, causing 1M models to fail at 200k tokens.Verified by direct API calls:
The correct implementation: set
anthropic-beta: context-1m-2025-08-07inmodel.headers. The SDK picks it up viacreateVertexAnthropicconfig headers automatically. No fetch interception needed.Changes
provider.ts:claude-sonnet-4-6-1m@defaultandclaude-sonnet-4-5-1m@20250929as shadow model variants withlimit.context: 1_000_000andanthropic-betainmodel.headersanthropic-betadirectly intoclaude-opus-4-6@default, which already has 1M context in models.dev but still requires the beta header on Vertex — no separate variant neededprovider.test.ts:Provider.list()with correctlimit.context,api.id,name, andanthropic-betaheaderglobalThis.fetchto assertanthropic-betaheader reaches Vertex unstripped and noanthropic_betabody param is injectedKnown Limitation
Plugin-supplied betas via
chat.paramshook (which flow throughproviderOptionsat call-time) are not covered — they don't reach the provider config headers the SDK uses at construction time. This is a separate gap; the 1M shadow model approach bakes the beta intomodel.headersat model registration, which does flow through correctly.Test Plan
bun test test/provider/provider.test.ts --test-name-pattern "Vertex|vertex|1M|1m"— 5 tests passProvider.list()with correct metadataanthropic-betaheader passes through unstripped; noanthropic_betabody injection