Skip to content

fix(provider): enable 1M context for Vertex AI Anthropic models#14055

Open
lanej wants to merge 3 commits intoanomalyco:devfrom
lanej:fix/vertex-ai-1m-context
Open

fix(provider): enable 1M context for Vertex AI Anthropic models#14055
lanej wants to merge 3 commits intoanomalyco:devfrom
lanej:fix/vertex-ai-1m-context

Conversation

@lanej
Copy link

@lanej lanej commented Feb 18, 2026

Summary

Adds 1M context window support for Vertex AI Anthropic models (claude-sonnet-4-6-1m@default, claude-sonnet-4-5-1m@20250929, and claude-opus-4-6@default).

Key Finding

Vertex AI honors the anthropic-beta HTTP header and forwards it to the Anthropic backend. An earlier implementation stripped the header and substituted an anthropic_beta body param — which Vertex silently ignores, causing 1M models to fail at 200k tokens.

Verified by direct API calls:

# Header approach (works):
curl -H "anthropic-beta: context-1m-2025-08-07" ... → 230k tokens accepted

# Body param approach (fails):
curl -d '{"anthropic_beta": "context-1m-2025-08-07", ...}' ... → 400 "prompt too long: 230k > 200k"

The correct implementation: set anthropic-beta: context-1m-2025-08-07 in model.headers. The SDK picks it up via createVertexAnthropic config headers automatically. No fetch interception needed.

Changes

provider.ts:

  • Registers claude-sonnet-4-6-1m@default and claude-sonnet-4-5-1m@20250929 as shadow model variants with limit.context: 1_000_000 and anthropic-beta in model.headers
  • Injects anthropic-beta directly into claude-opus-4-6@default, which already has 1M context in models.dev but still requires the beta header on Vertex — no separate variant needed
  • No fetch interceptor; the header flows through the SDK naturally

provider.test.ts:

  • Integration test: asserts 1M variants are registered in Provider.list() with correct limit.context, api.id, name, and anthropic-beta header
  • Integration test: asserts Opus 4.6's beta header is injected without creating a separate variant
  • Regression test: mocks globalThis.fetch to assert anthropic-beta header reaches Vertex unstripped and no anthropic_beta body param is injected

Known Limitation

Plugin-supplied betas via chat.params hook (which flow through providerOptions at call-time) are not covered — they don't reach the provider config headers the SDK uses at construction time. This is a separate gap; the 1M shadow model approach bakes the beta into model.headers at model registration, which does flow through correctly.

Test Plan

  • bun test test/provider/provider.test.ts --test-name-pattern "Vertex|vertex|1M|1m" — 5 tests pass
  • 1M variants present in Provider.list() with correct metadata
  • anthropic-beta header passes through unstripped; no anthropic_beta body injection

@lanej lanej marked this pull request as draft February 18, 2026 01:08
@lanej lanej force-pushed the fix/vertex-ai-1m-context branch 3 times, most recently from 1bfa676 to 7026729 Compare February 18, 2026 16:54
@lanej lanej marked this pull request as ready for review February 19, 2026 01:01
@lanej lanej changed the title fix(provider): add 1M context beta header for google-vertex-anthropic fix(provider): inject anthropic_beta as request body param for Vertex AI Feb 19, 2026
Copilot AI review requested due to automatic review settings February 19, 2026 02:59
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the google-vertex-anthropic provider integration so Anthropic beta flags can be forwarded in a way Vertex AI honors (as anthropic_beta in the JSON request body rather than via HTTP headers).

Changes:

  • Adds a fetch wrapper in Provider.getSDK() to inject anthropic_beta into POST request bodies for google-vertex-anthropic.
  • Adds provider config tests for options.headers["anthropic-beta"] and options.betas.
  • Adds a test intended to validate beta deduplication behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
packages/opencode/src/provider/provider.ts Adds a fetch interceptor to inject anthropic_beta into Vertex Anthropic POST request bodies.
packages/opencode/test/provider/provider.test.ts Adds tests around config parsing for Vertex Anthropic headers/betas and a deduplication-focused test.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@lanej
Copy link
Author

lanej commented Feb 19, 2026

Follow-up to my reply on the chat.params betas gap: the fix would be to also read betas from opts.headers in the fetch interceptor (normalized across Headers object, plain object, and array-of-tuples forms), in addition to options.betas and options.headers. That would cover plugin-supplied betas without requiring shadow model entries. Happy to open a follow-up PR for that if useful.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@lanej lanej force-pushed the fix/vertex-ai-1m-context branch from 57ae22a to 50a8f10 Compare February 20, 2026 23:08
@lanej lanej requested a review from adamdotdevin as a code owner February 20, 2026 23:08
@lanej lanej force-pushed the fix/vertex-ai-1m-context branch from 50a8f10 to df96183 Compare February 20, 2026 23:20
…ontext

Registers claude-sonnet-4-6-1m@default and claude-sonnet-4-5-1m@20250929
as explicit model variants in the google-vertex-anthropic provider with
headers: { "anthropic-beta": "context-1m-2025-08-07" } and a 1M context
limit.

Vertex AI honors the anthropic-beta HTTP header and forwards it to the
Anthropic backend, enabling the 1M context window. The body param
anthropic_beta is silently ignored by Vertex. An earlier implementation
stripped the header (which the SDK correctly sets via model.headers) and
replaced it with a body param, breaking the feature entirely. The fix is
to remove the interceptor and let the header pass through unmodified.

Also removes the options.betas extraction and transform.ts betas injection,
both of which were based on the same incorrect assumption.
@lanej lanej force-pushed the fix/vertex-ai-1m-context branch from b1c6ddd to 5fb733c Compare February 26, 2026 18:12
@lanej lanej changed the title fix(provider): inject anthropic_beta as request body param for Vertex AI fix(provider): enable 1M context for Vertex AI Anthropic models Feb 26, 2026
@lanej lanej requested a review from Copilot February 26, 2026 19:38
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +2353 to +2354
if (init.headers && typeof init.headers === "object" && !Array.isArray(init.headers)) {
Object.assign(headers, init.headers)
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The header capture logic only handles plain object headers and will miss headers if the SDK passes a Headers instance or an array of tuples. The condition typeof init.headers === "object" && !Array.isArray(init.headers) excludes arrays but will still try to Object.assign a Headers instance, which may not copy all headers correctly. Consider using Headers.prototype.forEach or converting headers to a plain object more robustly to ensure the test captures the anthropic-beta header regardless of the format used by the SDK.

Suggested change
if (init.headers && typeof init.headers === "object" && !Array.isArray(init.headers)) {
Object.assign(headers, init.headers)
const rawHeaders = init?.headers
if (rawHeaders instanceof Headers) {
rawHeaders.forEach((value, key) => {
headers[key] = value
})
} else if (Array.isArray(rawHeaders)) {
for (const [key, value] of rawHeaders as Array<[string, string]>) {
headers[key] = value
}
} else if (rawHeaders && typeof rawHeaders === "object") {
for (const [key, value] of Object.entries(rawHeaders as Record<string, string>)) {
headers[key] = String(value)
}

Copilot uses AI. Check for mistakes.
Comment on lines +2304 to +2333
test("google-vertex-anthropic accepts betas array from config", async () => {
await using tmp = await tmpdir({
init: async (dir) => {
await Bun.write(
path.join(dir, "opencode.json"),
JSON.stringify({
$schema: "https://opencode.ai/config.json",
provider: {
"google-vertex-anthropic": {
options: {
betas: ["context-1m-2025-08-07"],
},
},
},
}),
)
},
})
await Instance.provide({
directory: tmp.path,
init: async () => {
Env.set("GOOGLE_CLOUD_PROJECT", "test-project")
},
fn: async () => {
const providers = await Provider.list()
expect(providers["google-vertex-anthropic"]).toBeDefined()
expect(providers["google-vertex-anthropic"].options.betas).toEqual(["context-1m-2025-08-07"])
},
})
})
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test verifies that the betas array config option is accepted and stored, but there is no code in provider.ts that reads or uses options["betas"]. Based on the PR description, Vertex AI requires the anthropic-beta header (not a betas array), and the 1M variants are implemented using model.headers. This test creates a misleading impression that the betas config option is functional when it appears to be unused. Consider removing this test or documenting that the betas option is not currently implemented for google-vertex-anthropic.

Suggested change
test("google-vertex-anthropic accepts betas array from config", async () => {
await using tmp = await tmpdir({
init: async (dir) => {
await Bun.write(
path.join(dir, "opencode.json"),
JSON.stringify({
$schema: "https://opencode.ai/config.json",
provider: {
"google-vertex-anthropic": {
options: {
betas: ["context-1m-2025-08-07"],
},
},
},
}),
)
},
})
await Instance.provide({
directory: tmp.path,
init: async () => {
Env.set("GOOGLE_CLOUD_PROJECT", "test-project")
},
fn: async () => {
const providers = await Provider.list()
expect(providers["google-vertex-anthropic"]).toBeDefined()
expect(providers["google-vertex-anthropic"].options.betas).toEqual(["context-1m-2025-08-07"])
},
})
})

Copilot uses AI. Check for mistakes.
Comment on lines +762 to +795
// Vertex AI 1M context window support via the context-1m-2025-08-07 beta flag.
// Vertex reads and forwards the anthropic-beta HTTP header to the Anthropic backend;
// the header is set in model.headers and flows through the SDK automatically.
{
const VERTEX_1M_BETA = "context-1m-2025-08-07"
const VERTEX_1M_MODELS: Array<{ sourceID: string; variantID: string; name: string }> = [
{
sourceID: "claude-sonnet-4-6@default",
variantID: "claude-sonnet-4-6-1m@default",
name: "Claude Sonnet 4.6 (1M)",
},
{
sourceID: "claude-sonnet-4-5@20250929",
variantID: "claude-sonnet-4-5-1m@20250929",
name: "Claude Sonnet 4.5 (1M)",
},
]
const vertexProvider = database["google-vertex-anthropic"]
if (vertexProvider) {
for (const { sourceID, variantID, name } of VERTEX_1M_MODELS) {
const source = vertexProvider.models[sourceID]
if (source) {
vertexProvider.models[variantID] = {
...source,
id: variantID,
name,
api: { ...source.api, id: sourceID },
limit: { ...source.limit, context: 1000000, input: 0 },
headers: { ...source.headers, "anthropic-beta": VERTEX_1M_BETA },
}
}
}
}
}
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description states "Remove result["betas"] providerOptions injection (was a no-op; anthropicProviderOptions schema has no betas field)" under the "Changes" section for transform.ts, but there are no changes to transform.ts in this diff, and searching the file shows no such code exists. Either the code was already removed in a previous commit (and shouldn't be mentioned in this PR's changes), or the PR description is inaccurate. Please clarify or update the PR description to accurately reflect the changes in this specific PR.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants