Skip to content

bedrock + anthropic patches for adaptive thinking - #5821

Merged
akshaydeo merged 1 commit into
mainfrom
08-03-bedrock_anthropic_patches_for_adaptive_thinking
Aug 4, 2026
Merged

bedrock + anthropic patches for adaptive thinking#5821
akshaydeo merged 1 commit into
mainfrom
08-03-bedrock_anthropic_patches_for_adaptive_thinking

Conversation

@akshaydeo

@akshaydeo akshaydeo commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR fixes several correctness bugs across the Anthropic and Bedrock providers: reasoning/thinking max_tokens validation errors now surface as HTTP 400 instead of 500; tool.defer_loading is re-gated on its own tool-search-tool-2025-10-19 beta header rather than the advanced-tool-use bundle; Bedrock's classic Converse path no longer claims to support tool_search_tool_* (which AWS restricts to InvokeModel only); Nova2 web search and code execution tools are now correctly handled on the chat completions path (not just the Responses path); reasoning signatures are preserved through the /invoke egress path; and message_start streaming events no longer fabricate all-zero usage when the upstream hasn't reported it yet.

Closes #5638

Changes

  • Reasoning max_tokens → 400 not 500: Introduced ErrReasoningMaxTokensTooLow sentinel error, wrapped at every reasoning-budget validation site in chat.go and responses.go. requestbuilder.go now detects it via errors.Is and calls NewBifrostBadRequestError (new helper, sets StatusCode=400) instead of the generic operation error that defaulted to 500.
  • defer_loading beta header split: tool.defer_loading now requires tool-search-tool-2025-10-19 (AnthropicToolSearchBetaHeader), not the advanced-tool-use-2025-11-20 bundle. ProviderFeatureSupport.ToolSearch gates it; AdvancedToolUse is narrowed to allowed_callers only. stripUnsupportedAnthropicFields, StripUnsupportedFieldsFromRawBody, AddMissingBetaHeadersToContext, and betaHeaderPrefixToFeature are all updated consistently.
  • Bedrock drops ToolSearch: ProviderFeatures[schemas.Bedrock].ToolSearch is set to false because AWS restricts tool-search-tool-2025-10-19 to InvokeModel/InvokeModelWithResponseStream; Bifrost's Bedrock provider always dispatches tool-bearing requests via Converse. convertAnthropicTools in invoke.go now explicitly skips tool_search_tool_* entries rather than building a broken schema-less function tool.
  • Nova2 web search/code execution on chat completions path: ValidateChatToolsForProvider's prefix map now ORs in WebSearchNova/CodeExecNova for web_search_/code_execution_ prefixes, matching the Responses-path validator. convertToolConfigFromFiltered gains a new branch that converts these to nova_grounding/nova_code_interpreter system tools for Nova2 models and records them as model-dropped for non-Nova2 models. A shared convertToNovaSystemTool helper is extracted so both the Chat and Responses builders use one rule.
  • Dropped tools surfaced on response: BifrostContextKeyDroppedUnsupportedTools context key and BifrostResponseExtraFields.DroppedUnsupportedTools field are added. Both ChatCompletion and Responses in bedrock.go call a new applyDroppedUnsupportedTools helper that reads the context value and logs a warning.
  • Reasoning signature through /invoke: toBedrockInvokeAnthropicResponse now reads item.Content.ContentBlocks first (where Bedrock-originated reasoning lives) and carries the Signature field through to BedrockInvokeMessagesContentBlock (new field). The streaming path emits a signature_delta event type instead of nesting the signature inside thinking_delta, matching Anthropic's documented streaming contract.
  • message_start usage omission: ToAnthropicResponsesStreamResponse no longer fabricates an all-zero AnthropicUsage when bifrostResp.Response.Usage is nil. The field is omitted entirely, which Anthropic's streaming spec permits and which avoids misrepresenting cost telemetry to clients (e.g. Claude Code) that read input_tokens from message_start.
  • collectBedrockServerToolsFromFiltered: Skips web_search_/code_execution_ prefixes to avoid sending Bedrock two conflicting representations of the same Nova system tool.

Type of change

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Chore/CI

Affected areas

  • Core (Go)
  • Transports (HTTP)
  • Providers/Integrations
  • Plugins
  • UI (React)
  • Docs

How to test

go test ./core/providers/anthropic/... ./core/providers/bedrock/... ./core/providers/utils/...

Key scenarios to validate:

  • Send a request with reasoning.effort=high and max_tokens=500 to an Anthropic model — expect a 400 response, not 500.
  • Send a request with tool.defer_loading=true to Vertex — expect tool-search-tool-2025-10-19 in the beta headers, not advanced-tool-use-2025-11-20.
  • Send a tool_search_tool_* tool to classic Bedrock — expect it to be silently dropped, not forwarded as a broken function tool.
  • Send web_search/code_execution tools to a Nova2 Bedrock model via chat completions — expect nova_grounding/nova_code_interpreter system tools in the Converse request.
  • Verify ExtraFields.DroppedUnsupportedTools is populated when tools are dropped on Bedrock.
  • Stream a reasoning response through /invoke — expect a signature_delta event carrying the signature, and the thinking text present in the non-streaming response.

Breaking changes

  • No

DroppedUnsupportedTools is a new additive field. The defer_loading beta header change is a correctness fix; clients that were relying on advanced-tool-use to gate defer_loading on Bedrock were already broken (Bedrock doesn't support it). The message_start usage omission is spec-compliant and only removes fabricated zeros.

Security considerations

None. No auth, secrets, PII, or sandboxing changes.

Checklist

  • I read docs/contributing/README.md and followed the guidelines
  • I added/updated tests where appropriate
  • I updated documentation where needed
  • I verified builds succeed (Go and UI)
  • I verified the CI pipeline passes locally if applicable

@akshaydeo
akshaydeo marked this pull request as ready for review August 4, 2026 03:04
@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 414e3d5a-ec19-49f6-8450-57708901b55b

📥 Commits

Reviewing files that changed from the base of the PR and between c97b720 and e58ed94.

📒 Files selected for processing (18)
  • core/providers/anthropic/chat.go
  • core/providers/anthropic/messagestartusage_test.go
  • core/providers/anthropic/requestbuilder.go
  • core/providers/anthropic/requestbuilder_test.go
  • core/providers/anthropic/responses.go
  • core/providers/anthropic/types.go
  • core/providers/anthropic/utils.go
  • core/providers/anthropic/utils_test.go
  • core/providers/anthropic/validatechattools_test.go
  • core/providers/bedrock/bedrock.go
  • core/providers/bedrock/convert_tool_config_test.go
  • core/providers/bedrock/invoke.go
  • core/providers/bedrock/invoke_test.go
  • core/providers/bedrock/responses.go
  • core/providers/bedrock/types.go
  • core/providers/bedrock/utils.go
  • core/providers/utils/utils.go
  • core/schemas/bifrost.go

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added Nova web search and code execution support through Bedrock.
    • Improved Anthropic tool-search compatibility across supported providers.
    • Responses now identify tools omitted when unsupported by the provider or model.
    • Preserved reasoning signatures in Bedrock streaming and non-streaming responses.
  • Bug Fixes

    • Invalid reasoning token budgets now return clear HTTP 400 errors.
    • Anthropic streaming usage is omitted when unavailable instead of reporting zero values.
    • Improved conversion of supported search tools for Nova models.

Walkthrough

This PR updates Anthropic reasoning validation to wrap errors with a sentinel for HTTP 400 classification. It separates tool-search from advanced tool-use support with separate beta headers and gating. It adds Bedrock Nova tool conversion, tracks dropped tools in request context, and exposes them in responses. It also preserves Bedrock reasoning content-block signatures and emits streaming signatures as separate events.

Changes

Anthropic Reasoning Error Handling and Streaming Fixes

Layer / File(s) Summary
Reasoning validation and bad-request classification
core/providers/anthropic/chat.go, core/providers/anthropic/responses.go, core/providers/anthropic/requestbuilder.go, core/providers/utils/utils.go
Reasoning budget errors wrap ErrReasoningMaxTokensTooLow sentinel. Request builders detect this sentinel and return enriched HTTP 400 errors instead of generic conversion failures.
message_start usage handling
core/providers/anthropic/responses.go, core/providers/anthropic/messagestartusage_test.go
Unknown response usage is omitted instead of using zero values. Known input and output token counts are preserved in generated message_start events.

Anthropic Tool-Search Beta Header Split

Layer / File(s) Summary
Tool-search feature contract
core/providers/anthropic/types.go
Tool-search and advanced-tool-use headers are separated. Advanced-tool-use covers only allowed_callers. Tool-search covers deferred loading and tool-search server tools. Classic Bedrock no longer supports tool-search because it uses Converse.
Tool-search gating and validation
core/providers/anthropic/utils.go, core/providers/anthropic/utils_test.go
Request sanitization gates defer_loading with ToolSearch instead of AdvancedToolUse. Beta-header injection uses the tool-search header for deferred loading. Server-tool validation accepts Nova web search and code execution alongside existing capabilities.
Tool-search conversion filtering
core/providers/anthropic/validatechattools_test.go, core/providers/bedrock/invoke.go, core/providers/bedrock/invoke_test.go
Provider filtering tests confirm tool-search gating behavior. Bedrock conversion skips Anthropic tool_search_tool_* server tools instead of converting them to schema-less function tools.

Bedrock Nova Tool Support and Dropped-Tools Tracking

Layer / File(s) Summary
Dropped-tools contract and Nova conversion
core/schemas/bifrost.go, core/providers/bedrock/utils.go, core/providers/bedrock/responses.go
Bifrost context and response schema define dropped-tool tracking. Bedrock tool conversion uses canonical model names, maps supported Nova tools (web_search, code_execution) to system tools, and records provider- and model-dropped tool types in request context.
Response population and regression coverage
core/providers/bedrock/bedrock.go, core/providers/bedrock/convert_tool_config_test.go
Chat and Responses API results copy dropped tool names from context into response metadata. Regression tests cover Nova system-tool mappings, alias resolution, unsupported-tool recording, and Responses conversion of web_search_preview to Nova grounding.

Bedrock Reasoning Signature Handling

Layer / File(s) Summary
Signature contract and conversion
core/providers/bedrock/types.go, core/providers/bedrock/invoke.go
Bedrock invoke content blocks add an optional Signature response field. Non-streaming conversion preserves signatures from Bedrock reasoning blocks. Streaming conversion emits signatures as separate signature_delta events instead of embedding them in thinking_delta.
Regression coverage
core/providers/bedrock/invoke_test.go
Tests verify signature preservation in non-streaming responses, fallback to reasoning summaries when content blocks are unusable, retention of empty-text reasoning blocks when they contain signatures, and emission of streaming signatures as standalone events.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant AnthropicRequestBuilder
  participant AnthropicConverter
  participant BifrostResponse
  Client->>AnthropicRequestBuilder: Submit request
  AnthropicRequestBuilder->>AnthropicConverter: Convert request
  AnthropicConverter-->>AnthropicRequestBuilder: Return wrapped reasoning budget error
  AnthropicRequestBuilder->>BifrostResponse: Create HTTP 400 error
  BifrostResponse-->>Client: Return validation error
Loading
sequenceDiagram
  participant Client
  participant BedrockProvider
  participant ToolConverter
  participant BifrostContext
  participant BifrostResponse
  Client->>BedrockProvider: Submit request with tools
  BedrockProvider->>ToolConverter: Convert and filter tools
  ToolConverter->>BifrostContext: Store dropped tool names
  BedrockProvider->>BifrostContext: Read dropped tool names
  BedrockProvider->>BifrostResponse: Add dropped tools to response
  BifrostResponse-->>Client: Return response metadata
Loading

Possibly related issues

  • maximhq/bifrost#5274: Addresses Anthropic thinking-signature replay paths and reasoning-detail conversion that relate to signature handling in this PR.

Possibly related PRs

  • maximhq/bifrost#3227: Both PRs modify Anthropic request-conversion logic, including reasoning-budget handling and provider capability filtering in shared files.
  • maximhq/bifrost#5286: Both PRs modify Anthropic reasoning response conversion and signature handling in responses.go.

Suggested reviewers: pratham-mishra04, roroghost17, tejasghatte, sammaji

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 90.70% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title identifies Anthropic and Bedrock adaptive-thinking patches, but it does not cover the broader tool, signature, and streaming fixes.
Description check ✅ Passed The description covers the required sections, explains the changes, links the issue, and provides targeted validation steps.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 08-03-bedrock_anthropic_patches_for_adaptive_thinking

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/providers/bedrock/invoke.go`:
- Around line 1230-1255: Update the reasoning conversion around the
Content.ContentBlocks loop to track whether a non-empty reasoning block was
emitted. If no thinking block was appended, fall back to iterating
item.ResponsesReasoning.Summary and preserve its non-empty entries; retain the
existing block output when usable reasoning content exists. Add a regression
case covering non-reasoning or empty content blocks with a populated summary.

In `@core/providers/bedrock/responses.go`:
- Around line 2512-2526: Update the tool-type condition in the Nova conversion
loop around convertToNovaSystemTool to handle both ResponsesToolTypeWebSearch
and ResponsesToolTypeWebSearchPreview, mapping either to
BedrockSystemToolNovaGrounding. Preserve the existing code-interpreter mapping
and modelDroppedTools behavior, and add Nova2 and non-Nova2 coverage for
web_search_preview.

In `@core/providers/bedrock/utils.go`:
- Line 264: Use the resolved canonical model variable capModel, rather than
bifrostReq.Model, when calling convertToolConfigFromFiltered in the Chat path so
Nova2 aliases trigger the correct capability handling; update the related
IsNova2Model check context as needed and add a regression test covering an alias
that resolves to Nova2 and preserves the required web_search or code_execution
system tool.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b636d8b9-d59b-4ce0-a56a-5da785fb55a6

📥 Commits

Reviewing files that changed from the base of the PR and between 55d2790 and efd2b41.

📒 Files selected for processing (18)
  • core/providers/anthropic/chat.go
  • core/providers/anthropic/messagestartusage_test.go
  • core/providers/anthropic/requestbuilder.go
  • core/providers/anthropic/requestbuilder_test.go
  • core/providers/anthropic/responses.go
  • core/providers/anthropic/types.go
  • core/providers/anthropic/utils.go
  • core/providers/anthropic/utils_test.go
  • core/providers/anthropic/validatechattools_test.go
  • core/providers/bedrock/bedrock.go
  • core/providers/bedrock/convert_tool_config_test.go
  • core/providers/bedrock/invoke.go
  • core/providers/bedrock/invoke_test.go
  • core/providers/bedrock/responses.go
  • core/providers/bedrock/types.go
  • core/providers/bedrock/utils.go
  • core/providers/utils/utils.go
  • core/schemas/bifrost.go

Comment thread core/providers/bedrock/invoke.go Outdated
Comment thread core/providers/bedrock/responses.go
Comment thread core/providers/bedrock/utils.go Outdated
@akshaydeo
akshaydeo force-pushed the 08-03-bedrock_anthropic_patches_for_adaptive_thinking branch from efd2b41 to bce1722 Compare August 4, 2026 03:35
@coderabbitai
coderabbitai Bot requested review from roroghost17 and sammaji August 4, 2026 03:37
coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 4, 2026
@akshaydeo
akshaydeo force-pushed the 08-03-bedrock_anthropic_patches_for_adaptive_thinking branch from bce1722 to d9e928e Compare August 4, 2026 04:12
@akshaydeo
akshaydeo force-pushed the 08-03-fixes_5186_-_encrypted_reasoning branch from 55d2790 to d506a89 Compare August 4, 2026 04:12
coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 4, 2026
@akshaydeo
akshaydeo force-pushed the 08-03-bedrock_anthropic_patches_for_adaptive_thinking branch from d9e928e to e2a7d8b Compare August 4, 2026 04:15
@akshaydeo
akshaydeo force-pushed the 08-03-fixes_5186_-_encrypted_reasoning branch from d506a89 to 2bfd9ce Compare August 4, 2026 04:15
coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 4, 2026
@akshaydeo akshaydeo mentioned this pull request Aug 4, 2026
9 tasks
@akshaydeo
akshaydeo force-pushed the 08-03-fixes_5186_-_encrypted_reasoning branch from 2bfd9ce to 709c216 Compare August 4, 2026 07:47
@akshaydeo
akshaydeo force-pushed the 08-03-bedrock_anthropic_patches_for_adaptive_thinking branch from e2a7d8b to c97b720 Compare August 4, 2026 07:47
coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 4, 2026

akshaydeo commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Merge activity

@akshaydeo
akshaydeo changed the base branch from 08-03-fixes_5186_-_encrypted_reasoning to graphite-base/5821 August 4, 2026 09:17
@akshaydeo
akshaydeo changed the base branch from graphite-base/5821 to main August 4, 2026 09:20
@akshaydeo
akshaydeo dismissed coderabbitai[bot]’s stale review August 4, 2026 09:20

The base branch was changed.

@akshaydeo
akshaydeo force-pushed the 08-03-bedrock_anthropic_patches_for_adaptive_thinking branch from c97b720 to e58ed94 Compare August 4, 2026 09:20
@akshaydeo
akshaydeo merged commit cbe2100 into main Aug 4, 2026
13 of 14 checks passed
@akshaydeo
akshaydeo deleted the 08-03-bedrock_anthropic_patches_for_adaptive_thinking branch August 4, 2026 09:22
akshaydeo added a commit that referenced this pull request Aug 6, 2026
## Summary

Fixes #5885. `@ai-sdk/anthropic` (versions 4.0.6–4.0.32) marks `message.usage` and `message.usage.input_tokens` as required fields in its `message_start` schema, while `id`, `model`, and `role` are nullable. When Bifrost omitted the `usage` key entirely on `message_start` frames — which happened for every non-Anthropic provider, including Bedrock Converse, because usage figures aren't available until the terminal event — the SDK aborted the stream before the first token reached the client.

The fix emits an all-zero `usage` object on `message_start` when no real figures are available. Zeros are chosen over Anthropic's own `output_tokens: 1` placeholder because Bedrock's `message_delta` carries absolute totals rather than cumulative deltas; a client summing both frames would over-count by one. Zero is neutral under both interpretations, and Bifrost's passthrough accumulator uses max-merge, so a zero can never displace a real figure later in the stream.

## Changes

- `responses.go`: The `else` branch that previously left `messageUsage` nil now populates an all-zero `AnthropicUsage` struct, ensuring the `usage` key is always present on the wire regardless of `omitempty`.
- `invoke.go`: `message_start` frames built for the Bedrock `/invoke-with-response-stream` path now include a `usage` map with zero placeholders by default. When `resp.Response.Usage` is non-nil, real figures replace the zeros and cache token fields are conditionally included.
- Tests updated to assert that `usage` is present and zero when unknown, and that real figures are preferred when available. A second test marshals the struct to raw JSON and validates the wire representation directly, since the original `omitempty` bug was invisible to struct-level nil checks.

## Type of change

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/...
```

Validate that streaming through a Bedrock-backed model with an `@ai-sdk/anthropic` client no longer aborts on the first frame. The `message_start` event should contain `message.usage.input_tokens = 0` and `message.usage.output_tokens = 0`, with authoritative totals arriving on `message_delta`.

## Breaking changes

- [x] No

## Related issues

Closes #5885
See also #5821 (the prior fix that introduced the omission)

## Security considerations

None.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
akshaydeo added a commit that referenced this pull request Aug 7, 2026
## Summary

Fixes #5885. `@ai-sdk/anthropic` (versions 4.0.6–4.0.32) marks `message.usage` and `message.usage.input_tokens` as required fields in its `message_start` schema, while `id`, `model`, and `role` are nullable. When Bifrost omitted the `usage` key entirely on `message_start` frames — which happened for every non-Anthropic provider, including Bedrock Converse, because usage figures aren't available until the terminal event — the SDK aborted the stream before the first token reached the client.

The fix emits an all-zero `usage` object on `message_start` when no real figures are available. Zeros are chosen over Anthropic's own `output_tokens: 1` placeholder because Bedrock's `message_delta` carries absolute totals rather than cumulative deltas; a client summing both frames would over-count by one. Zero is neutral under both interpretations, and Bifrost's passthrough accumulator uses max-merge, so a zero can never displace a real figure later in the stream.

## Changes

- `responses.go`: The `else` branch that previously left `messageUsage` nil now populates an all-zero `AnthropicUsage` struct, ensuring the `usage` key is always present on the wire regardless of `omitempty`.
- `invoke.go`: `message_start` frames built for the Bedrock `/invoke-with-response-stream` path now include a `usage` map with zero placeholders by default. When `resp.Response.Usage` is non-nil, real figures replace the zeros and cache token fields are conditionally included.
- Tests updated to assert that `usage` is present and zero when unknown, and that real figures are preferred when available. A second test marshals the struct to raw JSON and validates the wire representation directly, since the original `omitempty` bug was invisible to struct-level nil checks.

## Type of change

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/...
```

Validate that streaming through a Bedrock-backed model with an `@ai-sdk/anthropic` client no longer aborts on the first frame. The `message_start` event should contain `message.usage.input_tokens = 0` and `message.usage.output_tokens = 0`, with authoritative totals arriving on `message_delta`.

## Breaking changes

- [x] No

## Related issues

Closes #5885
See also #5821 (the prior fix that introduced the omission)

## Security considerations

None.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
atharvamhaske pushed a commit to atharvamhaske/bifrost that referenced this pull request Aug 13, 2026
## Summary

This PR fixes several correctness bugs across the Anthropic and Bedrock providers: reasoning/thinking `max_tokens` validation errors now surface as HTTP 400 instead of 500; `tool.defer_loading` is re-gated on its own `tool-search-tool-2025-10-19` beta header rather than the `advanced-tool-use` bundle; Bedrock's classic Converse path no longer claims to support `tool_search_tool_*` (which AWS restricts to InvokeModel only); Nova2 web search and code execution tools are now correctly handled on the chat completions path (not just the Responses path); reasoning signatures are preserved through the `/invoke` egress path; and `message_start` streaming events no longer fabricate all-zero usage when the upstream hasn't reported it yet.

Closes maximhq#5638

## Changes

- **Reasoning** **`max_tokens`** **→ 400 not 500**: Introduced `ErrReasoningMaxTokensTooLow` sentinel error, wrapped at every reasoning-budget validation site in `chat.go` and `responses.go`. `requestbuilder.go` now detects it via `errors.Is` and calls `NewBifrostBadRequestError` (new helper, sets `StatusCode=400`) instead of the generic operation error that defaulted to 500.
- **`defer_loading`** **beta header split**: `tool.defer_loading` now requires `tool-search-tool-2025-10-19` (`AnthropicToolSearchBetaHeader`), not the `advanced-tool-use-2025-11-20` bundle. `ProviderFeatureSupport.ToolSearch` gates it; `AdvancedToolUse` is narrowed to `allowed_callers` only. `stripUnsupportedAnthropicFields`, `StripUnsupportedFieldsFromRawBody`, `AddMissingBetaHeadersToContext`, and `betaHeaderPrefixToFeature` are all updated consistently.
- **Bedrock drops** **`ToolSearch`**: `ProviderFeatures[schemas.Bedrock].ToolSearch` is set to `false` because AWS restricts `tool-search-tool-2025-10-19` to InvokeModel/InvokeModelWithResponseStream; Bifrost's Bedrock provider always dispatches tool-bearing requests via Converse. `convertAnthropicTools` in `invoke.go` now explicitly skips `tool_search_tool_*` entries rather than building a broken schema-less function tool.
- **Nova2 web search/code execution on chat completions path**: `ValidateChatToolsForProvider`'s prefix map now ORs in `WebSearchNova`/`CodeExecNova` for `web_search_`/`code_execution_` prefixes, matching the Responses-path validator. `convertToolConfigFromFiltered` gains a new branch that converts these to `nova_grounding`/`nova_code_interpreter` system tools for Nova2 models and records them as model-dropped for non-Nova2 models. A shared `convertToNovaSystemTool` helper is extracted so both the Chat and Responses builders use one rule.
- **Dropped tools surfaced on response**: `BifrostContextKeyDroppedUnsupportedTools` context key and `BifrostResponseExtraFields.DroppedUnsupportedTools` field are added. Both `ChatCompletion` and `Responses` in `bedrock.go` call a new `applyDroppedUnsupportedTools` helper that reads the context value and logs a warning.
- **Reasoning signature through** **`/invoke`**: `toBedrockInvokeAnthropicResponse` now reads `item.Content.ContentBlocks` first (where Bedrock-originated reasoning lives) and carries the `Signature` field through to `BedrockInvokeMessagesContentBlock` (new field). The streaming path emits a `signature_delta` event type instead of nesting the signature inside `thinking_delta`, matching Anthropic's documented streaming contract.
- **`message_start`** **usage omission**: `ToAnthropicResponsesStreamResponse` no longer fabricates an all-zero `AnthropicUsage` when `bifrostResp.Response.Usage` is nil. The field is omitted entirely, which Anthropic's streaming spec permits and which avoids misrepresenting cost telemetry to clients (e.g. Claude Code) that read `input_tokens` from `message_start`.
- **`collectBedrockServerToolsFromFiltered`**: Skips `web_search_`/`code_execution_` prefixes to avoid sending Bedrock two conflicting representations of the same Nova system tool.

## Type of change

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/... ./core/providers/utils/...
```

Key scenarios to validate:

- Send a request with `reasoning.effort=high` and `max_tokens=500` to an Anthropic model — expect a `400` response, not `500`.
- Send a request with `tool.defer_loading=true` to Vertex — expect `tool-search-tool-2025-10-19` in the beta headers, not `advanced-tool-use-2025-11-20`.
- Send a `tool_search_tool_*` tool to classic Bedrock — expect it to be silently dropped, not forwarded as a broken function tool.
- Send `web_search`/`code_execution` tools to a Nova2 Bedrock model via chat completions — expect `nova_grounding`/`nova_code_interpreter` system tools in the Converse request.
- Verify `ExtraFields.DroppedUnsupportedTools` is populated when tools are dropped on Bedrock.
- Stream a reasoning response through `/invoke` — expect a `signature_delta` event carrying the signature, and the thinking text present in the non-streaming response.

## Breaking changes

- [x] No

`DroppedUnsupportedTools` is a new additive field. The `defer_loading` beta header change is a correctness fix; clients that were relying on `advanced-tool-use` to gate `defer_loading` on Bedrock were already broken (Bedrock doesn't support it). The `message_start` usage omission is spec-compliant and only removes fabricated zeros.

## Security considerations

None. No auth, secrets, PII, or sandboxing changes.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
atharvamhaske pushed a commit to atharvamhaske/bifrost that referenced this pull request Aug 13, 2026
…mhq#5886)

## Summary

Fixes maximhq#5885. `@ai-sdk/anthropic` (versions 4.0.6–4.0.32) marks `message.usage` and `message.usage.input_tokens` as required fields in its `message_start` schema, while `id`, `model`, and `role` are nullable. When Bifrost omitted the `usage` key entirely on `message_start` frames — which happened for every non-Anthropic provider, including Bedrock Converse, because usage figures aren't available until the terminal event — the SDK aborted the stream before the first token reached the client.

The fix emits an all-zero `usage` object on `message_start` when no real figures are available. Zeros are chosen over Anthropic's own `output_tokens: 1` placeholder because Bedrock's `message_delta` carries absolute totals rather than cumulative deltas; a client summing both frames would over-count by one. Zero is neutral under both interpretations, and Bifrost's passthrough accumulator uses max-merge, so a zero can never displace a real figure later in the stream.

## Changes

- `responses.go`: The `else` branch that previously left `messageUsage` nil now populates an all-zero `AnthropicUsage` struct, ensuring the `usage` key is always present on the wire regardless of `omitempty`.
- `invoke.go`: `message_start` frames built for the Bedrock `/invoke-with-response-stream` path now include a `usage` map with zero placeholders by default. When `resp.Response.Usage` is non-nil, real figures replace the zeros and cache token fields are conditionally included.
- Tests updated to assert that `usage` is present and zero when unknown, and that real figures are preferred when available. A second test marshals the struct to raw JSON and validates the wire representation directly, since the original `omitempty` bug was invisible to struct-level nil checks.

## Type of change

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/...
```

Validate that streaming through a Bedrock-backed model with an `@ai-sdk/anthropic` client no longer aborts on the first frame. The `message_start` event should contain `message.usage.input_tokens = 0` and `message.usage.output_tokens = 0`, with authoritative totals arriving on `message_delta`.

## Breaking changes

- [x] No

## Related issues

Closes maximhq#5885
See also maximhq#5821 (the prior fix that introduced the omission)

## Security considerations

None.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
atharvamhaske pushed a commit to atharvamhaske/bifrost that referenced this pull request Aug 13, 2026
This PR fixes several correctness bugs across the Anthropic and Bedrock providers: reasoning/thinking `max_tokens` validation errors now surface as HTTP 400 instead of 500; `tool.defer_loading` is re-gated on its own `tool-search-tool-2025-10-19` beta header rather than the `advanced-tool-use` bundle; Bedrock's classic Converse path no longer claims to support `tool_search_tool_*` (which AWS restricts to InvokeModel only); Nova2 web search and code execution tools are now correctly handled on the chat completions path (not just the Responses path); reasoning signatures are preserved through the `/invoke` egress path; and `message_start` streaming events no longer fabricate all-zero usage when the upstream hasn't reported it yet.

Closes maximhq#5638

- **Reasoning** **`max_tokens`** **→ 400 not 500**: Introduced `ErrReasoningMaxTokensTooLow` sentinel error, wrapped at every reasoning-budget validation site in `chat.go` and `responses.go`. `requestbuilder.go` now detects it via `errors.Is` and calls `NewBifrostBadRequestError` (new helper, sets `StatusCode=400`) instead of the generic operation error that defaulted to 500.
- **`defer_loading`** **beta header split**: `tool.defer_loading` now requires `tool-search-tool-2025-10-19` (`AnthropicToolSearchBetaHeader`), not the `advanced-tool-use-2025-11-20` bundle. `ProviderFeatureSupport.ToolSearch` gates it; `AdvancedToolUse` is narrowed to `allowed_callers` only. `stripUnsupportedAnthropicFields`, `StripUnsupportedFieldsFromRawBody`, `AddMissingBetaHeadersToContext`, and `betaHeaderPrefixToFeature` are all updated consistently.
- **Bedrock drops** **`ToolSearch`**: `ProviderFeatures[schemas.Bedrock].ToolSearch` is set to `false` because AWS restricts `tool-search-tool-2025-10-19` to InvokeModel/InvokeModelWithResponseStream; Bifrost's Bedrock provider always dispatches tool-bearing requests via Converse. `convertAnthropicTools` in `invoke.go` now explicitly skips `tool_search_tool_*` entries rather than building a broken schema-less function tool.
- **Nova2 web search/code execution on chat completions path**: `ValidateChatToolsForProvider`'s prefix map now ORs in `WebSearchNova`/`CodeExecNova` for `web_search_`/`code_execution_` prefixes, matching the Responses-path validator. `convertToolConfigFromFiltered` gains a new branch that converts these to `nova_grounding`/`nova_code_interpreter` system tools for Nova2 models and records them as model-dropped for non-Nova2 models. A shared `convertToNovaSystemTool` helper is extracted so both the Chat and Responses builders use one rule.
- **Dropped tools surfaced on response**: `BifrostContextKeyDroppedUnsupportedTools` context key and `BifrostResponseExtraFields.DroppedUnsupportedTools` field are added. Both `ChatCompletion` and `Responses` in `bedrock.go` call a new `applyDroppedUnsupportedTools` helper that reads the context value and logs a warning.
- **Reasoning signature through** **`/invoke`**: `toBedrockInvokeAnthropicResponse` now reads `item.Content.ContentBlocks` first (where Bedrock-originated reasoning lives) and carries the `Signature` field through to `BedrockInvokeMessagesContentBlock` (new field). The streaming path emits a `signature_delta` event type instead of nesting the signature inside `thinking_delta`, matching Anthropic's documented streaming contract.
- **`message_start`** **usage omission**: `ToAnthropicResponsesStreamResponse` no longer fabricates an all-zero `AnthropicUsage` when `bifrostResp.Response.Usage` is nil. The field is omitted entirely, which Anthropic's streaming spec permits and which avoids misrepresenting cost telemetry to clients (e.g. Claude Code) that read `input_tokens` from `message_start`.
- **`collectBedrockServerToolsFromFiltered`**: Skips `web_search_`/`code_execution_` prefixes to avoid sending Bedrock two conflicting representations of the same Nova system tool.

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/... ./core/providers/utils/...
```

Key scenarios to validate:

- Send a request with `reasoning.effort=high` and `max_tokens=500` to an Anthropic model — expect a `400` response, not `500`.
- Send a request with `tool.defer_loading=true` to Vertex — expect `tool-search-tool-2025-10-19` in the beta headers, not `advanced-tool-use-2025-11-20`.
- Send a `tool_search_tool_*` tool to classic Bedrock — expect it to be silently dropped, not forwarded as a broken function tool.
- Send `web_search`/`code_execution` tools to a Nova2 Bedrock model via chat completions — expect `nova_grounding`/`nova_code_interpreter` system tools in the Converse request.
- Verify `ExtraFields.DroppedUnsupportedTools` is populated when tools are dropped on Bedrock.
- Stream a reasoning response through `/invoke` — expect a `signature_delta` event carrying the signature, and the thinking text present in the non-streaming response.

- [x] No

`DroppedUnsupportedTools` is a new additive field. The `defer_loading` beta header change is a correctness fix; clients that were relying on `advanced-tool-use` to gate `defer_loading` on Bedrock were already broken (Bedrock doesn't support it). The `message_start` usage omission is spec-compliant and only removes fabricated zeros.

None. No auth, secrets, PII, or sandboxing changes.

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
atharvamhaske pushed a commit to atharvamhaske/bifrost that referenced this pull request Aug 13, 2026
This PR fixes several correctness bugs across the Anthropic and Bedrock providers: reasoning/thinking `max_tokens` validation errors now surface as HTTP 400 instead of 500; `tool.defer_loading` is re-gated on its own `tool-search-tool-2025-10-19` beta header rather than the `advanced-tool-use` bundle; Bedrock's classic Converse path no longer claims to support `tool_search_tool_*` (which AWS restricts to InvokeModel only); Nova2 web search and code execution tools are now correctly handled on the chat completions path (not just the Responses path); reasoning signatures are preserved through the `/invoke` egress path; and `message_start` streaming events no longer fabricate all-zero usage when the upstream hasn't reported it yet.

Closes maximhq#5638

- **Reasoning** **`max_tokens`** **→ 400 not 500**: Introduced `ErrReasoningMaxTokensTooLow` sentinel error, wrapped at every reasoning-budget validation site in `chat.go` and `responses.go`. `requestbuilder.go` now detects it via `errors.Is` and calls `NewBifrostBadRequestError` (new helper, sets `StatusCode=400`) instead of the generic operation error that defaulted to 500.
- **`defer_loading`** **beta header split**: `tool.defer_loading` now requires `tool-search-tool-2025-10-19` (`AnthropicToolSearchBetaHeader`), not the `advanced-tool-use-2025-11-20` bundle. `ProviderFeatureSupport.ToolSearch` gates it; `AdvancedToolUse` is narrowed to `allowed_callers` only. `stripUnsupportedAnthropicFields`, `StripUnsupportedFieldsFromRawBody`, `AddMissingBetaHeadersToContext`, and `betaHeaderPrefixToFeature` are all updated consistently.
- **Bedrock drops** **`ToolSearch`**: `ProviderFeatures[schemas.Bedrock].ToolSearch` is set to `false` because AWS restricts `tool-search-tool-2025-10-19` to InvokeModel/InvokeModelWithResponseStream; Bifrost's Bedrock provider always dispatches tool-bearing requests via Converse. `convertAnthropicTools` in `invoke.go` now explicitly skips `tool_search_tool_*` entries rather than building a broken schema-less function tool.
- **Nova2 web search/code execution on chat completions path**: `ValidateChatToolsForProvider`'s prefix map now ORs in `WebSearchNova`/`CodeExecNova` for `web_search_`/`code_execution_` prefixes, matching the Responses-path validator. `convertToolConfigFromFiltered` gains a new branch that converts these to `nova_grounding`/`nova_code_interpreter` system tools for Nova2 models and records them as model-dropped for non-Nova2 models. A shared `convertToNovaSystemTool` helper is extracted so both the Chat and Responses builders use one rule.
- **Dropped tools surfaced on response**: `BifrostContextKeyDroppedUnsupportedTools` context key and `BifrostResponseExtraFields.DroppedUnsupportedTools` field are added. Both `ChatCompletion` and `Responses` in `bedrock.go` call a new `applyDroppedUnsupportedTools` helper that reads the context value and logs a warning.
- **Reasoning signature through** **`/invoke`**: `toBedrockInvokeAnthropicResponse` now reads `item.Content.ContentBlocks` first (where Bedrock-originated reasoning lives) and carries the `Signature` field through to `BedrockInvokeMessagesContentBlock` (new field). The streaming path emits a `signature_delta` event type instead of nesting the signature inside `thinking_delta`, matching Anthropic's documented streaming contract.
- **`message_start`** **usage omission**: `ToAnthropicResponsesStreamResponse` no longer fabricates an all-zero `AnthropicUsage` when `bifrostResp.Response.Usage` is nil. The field is omitted entirely, which Anthropic's streaming spec permits and which avoids misrepresenting cost telemetry to clients (e.g. Claude Code) that read `input_tokens` from `message_start`.
- **`collectBedrockServerToolsFromFiltered`**: Skips `web_search_`/`code_execution_` prefixes to avoid sending Bedrock two conflicting representations of the same Nova system tool.

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/... ./core/providers/utils/...
```

Key scenarios to validate:

- Send a request with `reasoning.effort=high` and `max_tokens=500` to an Anthropic model — expect a `400` response, not `500`.
- Send a request with `tool.defer_loading=true` to Vertex — expect `tool-search-tool-2025-10-19` in the beta headers, not `advanced-tool-use-2025-11-20`.
- Send a `tool_search_tool_*` tool to classic Bedrock — expect it to be silently dropped, not forwarded as a broken function tool.
- Send `web_search`/`code_execution` tools to a Nova2 Bedrock model via chat completions — expect `nova_grounding`/`nova_code_interpreter` system tools in the Converse request.
- Verify `ExtraFields.DroppedUnsupportedTools` is populated when tools are dropped on Bedrock.
- Stream a reasoning response through `/invoke` — expect a `signature_delta` event carrying the signature, and the thinking text present in the non-streaming response.

- [x] No

`DroppedUnsupportedTools` is a new additive field. The `defer_loading` beta header change is a correctness fix; clients that were relying on `advanced-tool-use` to gate `defer_loading` on Bedrock were already broken (Bedrock doesn't support it). The `message_start` usage omission is spec-compliant and only removes fabricated zeros.

None. No auth, secrets, PII, or sandboxing changes.

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
akhsaul pushed a commit to akhsaul/bifrost that referenced this pull request Aug 27, 2026
## Summary

This PR fixes several correctness bugs across the Anthropic and Bedrock providers: reasoning/thinking `max_tokens` validation errors now surface as HTTP 400 instead of 500; `tool.defer_loading` is re-gated on its own `tool-search-tool-2025-10-19` beta header rather than the `advanced-tool-use` bundle; Bedrock's classic Converse path no longer claims to support `tool_search_tool_*` (which AWS restricts to InvokeModel only); Nova2 web search and code execution tools are now correctly handled on the chat completions path (not just the Responses path); reasoning signatures are preserved through the `/invoke` egress path; and `message_start` streaming events no longer fabricate all-zero usage when the upstream hasn't reported it yet.

Closes maximhq#5638

## Changes

- **Reasoning** **`max_tokens`** **→ 400 not 500**: Introduced `ErrReasoningMaxTokensTooLow` sentinel error, wrapped at every reasoning-budget validation site in `chat.go` and `responses.go`. `requestbuilder.go` now detects it via `errors.Is` and calls `NewBifrostBadRequestError` (new helper, sets `StatusCode=400`) instead of the generic operation error that defaulted to 500.
- **`defer_loading`** **beta header split**: `tool.defer_loading` now requires `tool-search-tool-2025-10-19` (`AnthropicToolSearchBetaHeader`), not the `advanced-tool-use-2025-11-20` bundle. `ProviderFeatureSupport.ToolSearch` gates it; `AdvancedToolUse` is narrowed to `allowed_callers` only. `stripUnsupportedAnthropicFields`, `StripUnsupportedFieldsFromRawBody`, `AddMissingBetaHeadersToContext`, and `betaHeaderPrefixToFeature` are all updated consistently.
- **Bedrock drops** **`ToolSearch`**: `ProviderFeatures[schemas.Bedrock].ToolSearch` is set to `false` because AWS restricts `tool-search-tool-2025-10-19` to InvokeModel/InvokeModelWithResponseStream; Bifrost's Bedrock provider always dispatches tool-bearing requests via Converse. `convertAnthropicTools` in `invoke.go` now explicitly skips `tool_search_tool_*` entries rather than building a broken schema-less function tool.
- **Nova2 web search/code execution on chat completions path**: `ValidateChatToolsForProvider`'s prefix map now ORs in `WebSearchNova`/`CodeExecNova` for `web_search_`/`code_execution_` prefixes, matching the Responses-path validator. `convertToolConfigFromFiltered` gains a new branch that converts these to `nova_grounding`/`nova_code_interpreter` system tools for Nova2 models and records them as model-dropped for non-Nova2 models. A shared `convertToNovaSystemTool` helper is extracted so both the Chat and Responses builders use one rule.
- **Dropped tools surfaced on response**: `BifrostContextKeyDroppedUnsupportedTools` context key and `BifrostResponseExtraFields.DroppedUnsupportedTools` field are added. Both `ChatCompletion` and `Responses` in `bedrock.go` call a new `applyDroppedUnsupportedTools` helper that reads the context value and logs a warning.
- **Reasoning signature through** **`/invoke`**: `toBedrockInvokeAnthropicResponse` now reads `item.Content.ContentBlocks` first (where Bedrock-originated reasoning lives) and carries the `Signature` field through to `BedrockInvokeMessagesContentBlock` (new field). The streaming path emits a `signature_delta` event type instead of nesting the signature inside `thinking_delta`, matching Anthropic's documented streaming contract.
- **`message_start`** **usage omission**: `ToAnthropicResponsesStreamResponse` no longer fabricates an all-zero `AnthropicUsage` when `bifrostResp.Response.Usage` is nil. The field is omitted entirely, which Anthropic's streaming spec permits and which avoids misrepresenting cost telemetry to clients (e.g. Claude Code) that read `input_tokens` from `message_start`.
- **`collectBedrockServerToolsFromFiltered`**: Skips `web_search_`/`code_execution_` prefixes to avoid sending Bedrock two conflicting representations of the same Nova system tool.

## Type of change

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/... ./core/providers/utils/...
```

Key scenarios to validate:

- Send a request with `reasoning.effort=high` and `max_tokens=500` to an Anthropic model — expect a `400` response, not `500`.
- Send a request with `tool.defer_loading=true` to Vertex — expect `tool-search-tool-2025-10-19` in the beta headers, not `advanced-tool-use-2025-11-20`.
- Send a `tool_search_tool_*` tool to classic Bedrock — expect it to be silently dropped, not forwarded as a broken function tool.
- Send `web_search`/`code_execution` tools to a Nova2 Bedrock model via chat completions — expect `nova_grounding`/`nova_code_interpreter` system tools in the Converse request.
- Verify `ExtraFields.DroppedUnsupportedTools` is populated when tools are dropped on Bedrock.
- Stream a reasoning response through `/invoke` — expect a `signature_delta` event carrying the signature, and the thinking text present in the non-streaming response.

## Breaking changes

- [x] No

`DroppedUnsupportedTools` is a new additive field. The `defer_loading` beta header change is a correctness fix; clients that were relying on `advanced-tool-use` to gate `defer_loading` on Bedrock were already broken (Bedrock doesn't support it). The `message_start` usage omission is spec-compliant and only removes fabricated zeros.

## Security considerations

None. No auth, secrets, PII, or sandboxing changes.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
akhsaul pushed a commit to akhsaul/bifrost that referenced this pull request Aug 27, 2026
…mhq#5886)

## Summary

Fixes maximhq#5885. `@ai-sdk/anthropic` (versions 4.0.6–4.0.32) marks `message.usage` and `message.usage.input_tokens` as required fields in its `message_start` schema, while `id`, `model`, and `role` are nullable. When Bifrost omitted the `usage` key entirely on `message_start` frames — which happened for every non-Anthropic provider, including Bedrock Converse, because usage figures aren't available until the terminal event — the SDK aborted the stream before the first token reached the client.

The fix emits an all-zero `usage` object on `message_start` when no real figures are available. Zeros are chosen over Anthropic's own `output_tokens: 1` placeholder because Bedrock's `message_delta` carries absolute totals rather than cumulative deltas; a client summing both frames would over-count by one. Zero is neutral under both interpretations, and Bifrost's passthrough accumulator uses max-merge, so a zero can never displace a real figure later in the stream.

## Changes

- `responses.go`: The `else` branch that previously left `messageUsage` nil now populates an all-zero `AnthropicUsage` struct, ensuring the `usage` key is always present on the wire regardless of `omitempty`.
- `invoke.go`: `message_start` frames built for the Bedrock `/invoke-with-response-stream` path now include a `usage` map with zero placeholders by default. When `resp.Response.Usage` is non-nil, real figures replace the zeros and cache token fields are conditionally included.
- Tests updated to assert that `usage` is present and zero when unknown, and that real figures are preferred when available. A second test marshals the struct to raw JSON and validates the wire representation directly, since the original `omitempty` bug was invisible to struct-level nil checks.

## Type of change

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/...
```

Validate that streaming through a Bedrock-backed model with an `@ai-sdk/anthropic` client no longer aborts on the first frame. The `message_start` event should contain `message.usage.input_tokens = 0` and `message.usage.output_tokens = 0`, with authoritative totals arriving on `message_delta`.

## Breaking changes

- [x] No

## Related issues

Closes maximhq#5885
See also maximhq#5821 (the prior fix that introduced the omission)

## Security considerations

None.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
occcat pushed a commit to occcat/bifrost that referenced this pull request Sep 2, 2026
## Summary

This PR fixes several correctness bugs across the Anthropic and Bedrock providers: reasoning/thinking `max_tokens` validation errors now surface as HTTP 400 instead of 500; `tool.defer_loading` is re-gated on its own `tool-search-tool-2025-10-19` beta header rather than the `advanced-tool-use` bundle; Bedrock's classic Converse path no longer claims to support `tool_search_tool_*` (which AWS restricts to InvokeModel only); Nova2 web search and code execution tools are now correctly handled on the chat completions path (not just the Responses path); reasoning signatures are preserved through the `/invoke` egress path; and `message_start` streaming events no longer fabricate all-zero usage when the upstream hasn't reported it yet.

Closes maximhq#5638

## Changes

- **Reasoning** **`max_tokens`** **→ 400 not 500**: Introduced `ErrReasoningMaxTokensTooLow` sentinel error, wrapped at every reasoning-budget validation site in `chat.go` and `responses.go`. `requestbuilder.go` now detects it via `errors.Is` and calls `NewBifrostBadRequestError` (new helper, sets `StatusCode=400`) instead of the generic operation error that defaulted to 500.
- **`defer_loading`** **beta header split**: `tool.defer_loading` now requires `tool-search-tool-2025-10-19` (`AnthropicToolSearchBetaHeader`), not the `advanced-tool-use-2025-11-20` bundle. `ProviderFeatureSupport.ToolSearch` gates it; `AdvancedToolUse` is narrowed to `allowed_callers` only. `stripUnsupportedAnthropicFields`, `StripUnsupportedFieldsFromRawBody`, `AddMissingBetaHeadersToContext`, and `betaHeaderPrefixToFeature` are all updated consistently.
- **Bedrock drops** **`ToolSearch`**: `ProviderFeatures[schemas.Bedrock].ToolSearch` is set to `false` because AWS restricts `tool-search-tool-2025-10-19` to InvokeModel/InvokeModelWithResponseStream; Bifrost's Bedrock provider always dispatches tool-bearing requests via Converse. `convertAnthropicTools` in `invoke.go` now explicitly skips `tool_search_tool_*` entries rather than building a broken schema-less function tool.
- **Nova2 web search/code execution on chat completions path**: `ValidateChatToolsForProvider`'s prefix map now ORs in `WebSearchNova`/`CodeExecNova` for `web_search_`/`code_execution_` prefixes, matching the Responses-path validator. `convertToolConfigFromFiltered` gains a new branch that converts these to `nova_grounding`/`nova_code_interpreter` system tools for Nova2 models and records them as model-dropped for non-Nova2 models. A shared `convertToNovaSystemTool` helper is extracted so both the Chat and Responses builders use one rule.
- **Dropped tools surfaced on response**: `BifrostContextKeyDroppedUnsupportedTools` context key and `BifrostResponseExtraFields.DroppedUnsupportedTools` field are added. Both `ChatCompletion` and `Responses` in `bedrock.go` call a new `applyDroppedUnsupportedTools` helper that reads the context value and logs a warning.
- **Reasoning signature through** **`/invoke`**: `toBedrockInvokeAnthropicResponse` now reads `item.Content.ContentBlocks` first (where Bedrock-originated reasoning lives) and carries the `Signature` field through to `BedrockInvokeMessagesContentBlock` (new field). The streaming path emits a `signature_delta` event type instead of nesting the signature inside `thinking_delta`, matching Anthropic's documented streaming contract.
- **`message_start`** **usage omission**: `ToAnthropicResponsesStreamResponse` no longer fabricates an all-zero `AnthropicUsage` when `bifrostResp.Response.Usage` is nil. The field is omitted entirely, which Anthropic's streaming spec permits and which avoids misrepresenting cost telemetry to clients (e.g. Claude Code) that read `input_tokens` from `message_start`.
- **`collectBedrockServerToolsFromFiltered`**: Skips `web_search_`/`code_execution_` prefixes to avoid sending Bedrock two conflicting representations of the same Nova system tool.

## Type of change

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/... ./core/providers/utils/...
```

Key scenarios to validate:

- Send a request with `reasoning.effort=high` and `max_tokens=500` to an Anthropic model — expect a `400` response, not `500`.
- Send a request with `tool.defer_loading=true` to Vertex — expect `tool-search-tool-2025-10-19` in the beta headers, not `advanced-tool-use-2025-11-20`.
- Send a `tool_search_tool_*` tool to classic Bedrock — expect it to be silently dropped, not forwarded as a broken function tool.
- Send `web_search`/`code_execution` tools to a Nova2 Bedrock model via chat completions — expect `nova_grounding`/`nova_code_interpreter` system tools in the Converse request.
- Verify `ExtraFields.DroppedUnsupportedTools` is populated when tools are dropped on Bedrock.
- Stream a reasoning response through `/invoke` — expect a `signature_delta` event carrying the signature, and the thinking text present in the non-streaming response.

## Breaking changes

- [x] No

`DroppedUnsupportedTools` is a new additive field. The `defer_loading` beta header change is a correctness fix; clients that were relying on `advanced-tool-use` to gate `defer_loading` on Bedrock were already broken (Bedrock doesn't support it). The `message_start` usage omission is spec-compliant and only removes fabricated zeros.

## Security considerations

None. No auth, secrets, PII, or sandboxing changes.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
occcat pushed a commit to occcat/bifrost that referenced this pull request Sep 2, 2026
…mhq#5886)

## Summary

Fixes maximhq#5885. `@ai-sdk/anthropic` (versions 4.0.6–4.0.32) marks `message.usage` and `message.usage.input_tokens` as required fields in its `message_start` schema, while `id`, `model`, and `role` are nullable. When Bifrost omitted the `usage` key entirely on `message_start` frames — which happened for every non-Anthropic provider, including Bedrock Converse, because usage figures aren't available until the terminal event — the SDK aborted the stream before the first token reached the client.

The fix emits an all-zero `usage` object on `message_start` when no real figures are available. Zeros are chosen over Anthropic's own `output_tokens: 1` placeholder because Bedrock's `message_delta` carries absolute totals rather than cumulative deltas; a client summing both frames would over-count by one. Zero is neutral under both interpretations, and Bifrost's passthrough accumulator uses max-merge, so a zero can never displace a real figure later in the stream.

## Changes

- `responses.go`: The `else` branch that previously left `messageUsage` nil now populates an all-zero `AnthropicUsage` struct, ensuring the `usage` key is always present on the wire regardless of `omitempty`.
- `invoke.go`: `message_start` frames built for the Bedrock `/invoke-with-response-stream` path now include a `usage` map with zero placeholders by default. When `resp.Response.Usage` is non-nil, real figures replace the zeros and cache token fields are conditionally included.
- Tests updated to assert that `usage` is present and zero when unknown, and that real figures are preferred when available. A second test marshals the struct to raw JSON and validates the wire representation directly, since the original `omitempty` bug was invisible to struct-level nil checks.

## Type of change

- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

```sh
go test ./core/providers/anthropic/... ./core/providers/bedrock/...
```

Validate that streaming through a Bedrock-backed model with an `@ai-sdk/anthropic` client no longer aborts on the first frame. The `message_start` event should contain `message.usage.input_tokens = 0` and `message.usage.output_tokens = 0`, with authoritative totals arriving on `message_delta`.

## Breaking changes

- [x] No

## Related issues

Closes maximhq#5885
See also maximhq#5821 (the prior fix that introduced the omission)

## Security considerations

None.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant