Skip to content

fix: signal Bedrock max_output_tokens truncation on Responses API - #4680

Merged
akshaydeo merged 1 commit into
maximhq:devfrom
jeremym-tanium:fix/bedrock-responses-incomplete-details
Jul 1, 2026
Merged

fix: signal Bedrock max_output_tokens truncation on Responses API#4680
akshaydeo merged 1 commit into
maximhq:devfrom
jeremym-tanium:fix/bedrock-responses-incomplete-details

Conversation

@jeremym-tanium

Copy link
Copy Markdown
Contributor

Summary

When a /v1/responses request to a Bedrock model is truncated by max_output_tokens, Bifrost drops the OpenAI-canonical truncation signals on the wire response:

  • response.status is left unset (OpenAI sets "incomplete")
  • response.incomplete_details is left unset (OpenAI sets {"reason": "max_output_tokens"})
  • On the streaming path, the terminal SSE event is response.completed (OpenAI emits response.incomplete)

Consumers on the OpenAI Responses-API contract therefore cannot detect Bedrock-side truncation. This is especially damaging for streaming tool calls, where the truncated bytes parse as valid-but-incomplete JSON (required fields silently missing or {}), so a downstream parser cannot fall back on a parse error.

Changes

Two narrow edits in core/providers/bedrock/responses.go:

  • Non-streaming path (BedrockConverseResponse.ToBifrostResponsesResponse): after the existing bifrostResp.StopReason = &stopReason, set Status/IncompleteDetails from the mapped stop reason ("length"incomplete + max_output_tokens; clean reasons → completed). Mirrors core/schemas/mux.go::responsesStatusFromChatFinishReason.
  • Streaming path (FinalizeBedrockStream): the terminal chunk's hard-coded ResponsesStreamResponseTypeCompleted becomes conditional — when the mapped stop reason is "length", emit ResponsesStreamResponseTypeIncomplete and set the same fields on the embedded Response.

Value flow: Bedrock stopReason "max_tokens" → Bifrost finish reason "length" → OpenAI status "incomplete" + incomplete_details.reason "max_output_tokens". No new code paths or API surface; status/incomplete_details are existing optional fields.

Changelog entry added to core/changelog.md.

Type of change

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Chore/CI

Affected areas

  • Core (Go)
  • Transports (HTTP)
  • Providers/Integrations
  • Plugins
  • UI (React)
  • Docs

How to test

cd core
go test ./providers/bedrock/ \
  -run 'TestBedrockStopReasonMappingResponsesPath|TestFinalizeBedrockStream' -count=1 -v

Covered by TestBedrockStopReasonMappingResponsesPath (extended with Status/IncompleteDetails per stop reason), TestFinalizeBedrockStream_MaxTokensTruncation, and TestFinalizeBedrockStream_CleanCompletionUnaffected. AWS reports stopReason: "max_tokens" for the same input (aws bedrock-runtime converse-stream); this brings the Bifrost translation in line with that signal.

Breaking changes

  • Yes
  • No

status and incomplete_details are existing optional fields. The streaming terminal event type changes from response.completed to response.incomplete only when Bedrock reports max_tokens — which is the bug fix (consumers now see the OpenAI-canonical type).

Related issues

Closes #4679

Security considerations

None — no auth, secret, or PII surface changed.

Checklist

  • I read the contributing guidelines and followed them
  • I added/updated tests where appropriate
  • I updated documentation where needed (no behavior outside the existing OpenAI contract)
  • I verified Go builds and tests succeed (UI not touched)
  • I verified the full CI pipeline locally (provider suite needs live credentials; ran gofmt, go vet, go build, and the bedrock tests above)

@CLAassistant

CLAassistant commented Jun 24, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a1a085dc-7c56-46db-b902-d6dd4360567d

📥 Commits

Reviewing files that changed from the base of the PR and between 23b134c and eb48fd6.

📒 Files selected for processing (4)
  • core/changelog.md
  • core/providers/bedrock/bedrock_test.go
  • core/providers/bedrock/responses.go
  • core/schemas/responses.go
✅ Files skipped from review due to trivial changes (1)
  • core/changelog.md
🚧 Files skipped from review as they are similar to previous changes (3)
  • core/schemas/responses.go
  • core/providers/bedrock/responses.go
  • core/providers/bedrock/bedrock_test.go

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes
    • Correctly reflect Bedrock Responses API truncation as incomplete with IncompleteDetails.Reason = "max_output_tokens" for both streaming terminal events and non-streaming conversions.
    • Ensure clean/completion-like stop reasons map to completed, while unmapped reasons leave status/incomplete details unset.
  • Tests
    • Added/expanded coverage for stop-reason → terminal SSE event behavior and status normalization, including truncation and unmapped scenarios.
  • Documentation
    • Updated the changelog with the truncation fix note.

Walkthrough

Bedrock Responses API now marks length stops as incomplete in non-streaming responses and streamed terminal events. A changelog note and schema constants were added for the fix.

Changes

Bedrock truncation signaling

Layer / File(s) Summary
Responses status constants
core/schemas/responses.go
ResponsesResponse status and incomplete-reason constants are added.
Responses status mapping
core/providers/bedrock/responses.go, core/providers/bedrock/bedrock_test.go, core/changelog.md
ToBifrostResponsesResponse now sets status and incomplete_details from Bedrock stop reasons, and the tests cover completed, incomplete, and unset outcomes.
Stream terminal event
core/providers/bedrock/responses.go, core/providers/bedrock/bedrock_test.go
FinalizeBedrockStream now emits response.incomplete for length and response.completed for clean endings or unmapped reasons, with tests covering incomplete, completed, and unset response fields.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • maximhq/bifrost#3506: Also changes Bedrock stop-reason normalization in core/providers/bedrock/responses.go and related tests around Responses-path behavior.
  • maximhq/bifrost#3838: Also touches FinalizeBedrockStream in core/providers/bedrock/responses.go, but for stream text assembly rather than stop-reason signaling.

Suggested reviewers

  • akshaydeo
  • danpiths

Poem

🐰 I hopped through Bedrock, soft and spry,
When tokens run short, we say so outright.
response.incomplete twinkles bright,
With max_output_tokens tucked in sight.
Thump-thump — the stream feels just right!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the main fix: Bedrock Responses API now signals max_output_tokens truncation.
Description check ✅ Passed The PR description follows the template well and fills the required sections with substantive details.
Linked Issues check ✅ Passed The changes implement the linked bug fix by setting incomplete status/details and emitting response.incomplete on truncation.
Out of Scope Changes check ✅ Passed The extra changes are tests, docs, and supporting constants directly tied to the truncation fix.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Confidence Score: 4/5

The core fix is correct and well-tested, but the reverse-conversion function ToBedrockConverseStreamResponse (used on the Nova-model Bedrock SDK compatibility path) still has no case for the new ResponsesStreamResponseTypeIncomplete event type, meaning a Nova model that hits max_tokens will silently drop its terminal messageStop on that path.

Both the non-streaming and streaming Bifrost-Responses paths now correctly surface max_output_tokens truncation, the new switch mirrors mux.go's responsesStatusFromChatFinishReason, and the test coverage is solid. The gap in ToBedrockConverseStreamResponse (no handling for ResponsesStreamResponseTypeIncomplete, which silently drops the messageStop for Nova models on truncation) was flagged in a previous review cycle and remains unaddressed in this revision.

core/providers/bedrock/responses.go — specifically ToBedrockConverseStreamResponse, which needs a case for ResponsesStreamResponseTypeIncomplete to handle the new terminal event type for Nova models.

Important Files Changed

Filename Overview
core/providers/bedrock/responses.go Core logic for the fix: adds Status/IncompleteDetails/terminal-event-type switch to both FinalizeBedrockStream (streaming) and ToBifrostResponsesResponse (non-streaming). The reverse-conversion function ToBedrockConverseStreamResponse still has no case for ResponsesStreamResponseTypeIncomplete (previously flagged), so Nova-model streams hitting max_tokens will silently drop the messageStop event on the Bedrock SDK compatibility path.
core/providers/bedrock/bedrock_test.go Extends the non-streaming stop-reason table test with Status/IncompleteDetails checks and adds three targeted streaming tests (truncation, clean completion, unmapped reason). Coverage is thorough for the changed paths.
core/schemas/responses.go Adds ResponsesResponseStatus* and ResponsesResponseIncompleteReason* constant blocks for the OpenAI Responses API status values. ResponsesResponseIncompleteReasonContentFilter is defined here but not yet used anywhere in the codebase (safe forward-declaration). Minor alignment fix for ResponsesMessageTypeAdvisorCall.
core/changelog.md Adds a changelog entry for the fix.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Bedrock
    participant FinalizeBedrockStream
    participant BifrostResponsesStreamResponse
    participant Consumer

    Bedrock->>FinalizeBedrockStream: "stopReason = "max_tokens""
    Note over FinalizeBedrockStream: map "max_tokens" → "length"
    FinalizeBedrockStream->>FinalizeBedrockStream: "switch "length":<br/>terminalEventType = Incomplete<br/>Status = "incomplete"<br/>IncompleteDetails.Reason = "max_output_tokens""
    FinalizeBedrockStream->>BifrostResponsesStreamResponse: "Type=response.incomplete<br/>Response.Status="incomplete"<br/>Response.IncompleteDetails.Reason="max_output_tokens""
    BifrostResponsesStreamResponse->>Consumer: SSE: response.incomplete ✅

    Note over Bedrock,Consumer: Non-streaming path
    Bedrock->>FinalizeBedrockStream: "stopReason = "max_tokens" (non-streaming)"
    Note over FinalizeBedrockStream: ToBifrostResponsesResponse
    FinalizeBedrockStream->>Consumer: "Status="incomplete"<br/>IncompleteDetails.Reason="max_output_tokens" ✅"
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Bedrock
    participant FinalizeBedrockStream
    participant BifrostResponsesStreamResponse
    participant Consumer

    Bedrock->>FinalizeBedrockStream: "stopReason = "max_tokens""
    Note over FinalizeBedrockStream: map "max_tokens" → "length"
    FinalizeBedrockStream->>FinalizeBedrockStream: "switch "length":<br/>terminalEventType = Incomplete<br/>Status = "incomplete"<br/>IncompleteDetails.Reason = "max_output_tokens""
    FinalizeBedrockStream->>BifrostResponsesStreamResponse: "Type=response.incomplete<br/>Response.Status="incomplete"<br/>Response.IncompleteDetails.Reason="max_output_tokens""
    BifrostResponsesStreamResponse->>Consumer: SSE: response.incomplete ✅

    Note over Bedrock,Consumer: Non-streaming path
    Bedrock->>FinalizeBedrockStream: "stopReason = "max_tokens" (non-streaming)"
    Note over FinalizeBedrockStream: ToBifrostResponsesResponse
    FinalizeBedrockStream->>Consumer: "Status="incomplete"<br/>IncompleteDetails.Reason="max_output_tokens" ✅"
Loading

Reviews (7): Last reviewed commit: "fix: signal Bedrock max_output_tokens tr..." | Re-trigger Greptile

Comment thread core/providers/bedrock/responses.go Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/providers/bedrock/responses.go`:
- Around line 1513-1524: The streaming path in responses.go is incorrectly
defaulting every non-length stop reason to response.Status="completed", which
causes unmapped Bedrock reasons like content_filter and guardrail_intervened to
diverge from ToBifrostResponsesResponse. Update the terminal handling around the
Responses stream response mapping so only the explicit length truncation case
sets status to "incomplete", and the completed status is assigned only for known
completed cases; leave Status unset for unmapped stop reasons so streaming and
non-streaming behavior stay aligned.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 1b0bf952-5100-4874-875e-af6e0d7f6529

📥 Commits

Reviewing files that changed from the base of the PR and between 7131032 and 0fb8d82.

📒 Files selected for processing (3)
  • core/changelog.md
  • core/providers/bedrock/bedrock_test.go
  • core/providers/bedrock/responses.go

Comment thread core/providers/bedrock/responses.go Outdated
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jun 24, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jun 25, 2026
Comment thread core/providers/bedrock/responses.go Outdated
switch *response.StopReason {
case string(schemas.BifrostFinishReasonLength):
terminalEventType = schemas.ResponsesStreamResponseTypeIncomplete
response.Status = schemas.Ptr("incomplete")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move them to constants for better reusability?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 8918d56 — moved them into schemas constants (ResponsesResponseStatusCompleted / ResponsesResponseStatusIncomplete and ResponsesResponseIncompleteReasonMaxOutputTokens, plus the sibling status/reason values for reuse) and used them on both the streaming and non-streaming Bedrock Responses paths.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jun 25, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jun 29, 2026
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jun 29, 2026
@akshaydeo
akshaydeo dismissed coderabbitai[bot]’s stale review June 30, 2026 03:04

The merge-base changed after approval.

@akshaydeo
akshaydeo requested a review from a team as a code owner June 30, 2026 03:04
@jeremym-tanium

Copy link
Copy Markdown
Contributor Author

Hi @akshaydeo it looks like the most recent merges have caused some conflicts. Ok if I rebase?

When a /v1/responses request to a Bedrock model is truncated by
max_output_tokens, the OpenAI-canonical truncation signals were dropped:
response.status and response.incomplete_details were left unset, and the
terminal streaming event was response.completed. Consumers on the OpenAI
Responses-API contract could not detect Bedrock-side truncation.

Set Status="incomplete" + IncompleteDetails.Reason="max_output_tokens" on
the non-streaming path, and emit response.incomplete on the streaming
terminal event, when the mapped stop reason is "length". Unmapped reasons
(content_filter, guardrail_intervened, ...) leave Status unset on both
paths.

Affected packages:
- core/providers/bedrock/ - Status/IncompleteDetails + terminal event type
- core/schemas/ - Responses status + incomplete-reason constants

Closes maximhq#4679
@jeremym-tanium
jeremym-tanium force-pushed the fix/bedrock-responses-incomplete-details branch from 23b134c to eb48fd6 Compare July 1, 2026 00:58
@akshaydeo
akshaydeo merged commit af0b891 into maximhq:dev Jul 1, 2026
6 checks passed
nghodkicisco pushed a commit to nghodkicisco/bifrost-06162026 that referenced this pull request Jul 1, 2026
…ximhq#4680)

When a /v1/responses request to a Bedrock model is truncated by
max_output_tokens, the OpenAI-canonical truncation signals were dropped:
response.status and response.incomplete_details were left unset, and the
terminal streaming event was response.completed. Consumers on the OpenAI
Responses-API contract could not detect Bedrock-side truncation.

Set Status="incomplete" + IncompleteDetails.Reason="max_output_tokens" on
the non-streaming path, and emit response.incomplete on the streaming
terminal event, when the mapped stop reason is "length". Unmapped reasons
(content_filter, guardrail_intervened, ...) leave Status unset on both
paths.

Affected packages:
- core/providers/bedrock/ - Status/IncompleteDetails + terminal event type
- core/schemas/ - Responses status + incomplete-reason constants

Closes maximhq#4679
akshaydeo pushed a commit that referenced this pull request Jul 1, 2026
)

When a /v1/responses request to a Bedrock model is truncated by
max_output_tokens, the OpenAI-canonical truncation signals were dropped:
response.status and response.incomplete_details were left unset, and the
terminal streaming event was response.completed. Consumers on the OpenAI
Responses-API contract could not detect Bedrock-side truncation.

Set Status="incomplete" + IncompleteDetails.Reason="max_output_tokens" on
the non-streaming path, and emit response.incomplete on the streaming
terminal event, when the mapped stop reason is "length". Unmapped reasons
(content_filter, guardrail_intervened, ...) leave Status unset on both
paths.

Affected packages:
- core/providers/bedrock/ - Status/IncompleteDetails + terminal event type
- core/schemas/ - Responses status + incomplete-reason constants

Closes #4679
@jeremym-tanium
jeremym-tanium deleted the fix/bedrock-responses-incomplete-details branch July 2, 2026 19:08
@TejasGhatte TejasGhatte mentioned this pull request Jul 6, 2026
17 tasks
Pratham-Mishra04 pushed a commit that referenced this pull request Jul 6, 2026
## Summary

Adds end-to-end test coverage for a set of provider egress streaming and truncation correctness bugs. The new tests assert that Bedrock `converse-stream` properly closes content blocks before terminating, that Anthropic normalized and Claude Code passthrough streams emit contiguous `content_block_start` indices starting from 0, and that Bedrock Responses API truncated responses correctly signal `status=incomplete` with `reason=max_output_tokens` in both streaming and non-streaming modes.

## Changes

- Added a `contentBlockStop`-before-`messageStop` assertion to the existing Bedrock `converse-stream` basic test to catch #4923.
- Added a `content_block_start` index contiguity check to the existing Anthropic normalized streaming test to catch gaps introduced by server-tool rewrites (#4890 / #4932).
- Added a new **section 17 – Provider Egress Streaming/Truncation Guards** with four requests:
  - Bedrock forced-tool `converse-stream` verifies `toolUse`, `contentBlockStop`, and `messageStop` ordering (#4923).
  - Anthropic normalized `web_fetch` streaming verifies contiguous `content_block_start` indices (#4932).
  - Bedrock Responses non-streaming truncation verifies `status=incomplete` and `incomplete_details.reason=max_output_tokens` (#4680).
  - Bedrock Responses streaming truncation verifies `response.incomplete` is emitted and `response.completed` is absent (#4680).
- Added a new **section 18 – Claude Code Passthrough server-tool streaming index contiguity** with three requests covering `web_search` (normal results), `web_search` (zero results), and `web_fetch` via the `claude-cli` User-Agent passthrough path (#4890).

## Type of change

- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [x] Chore/CI

## Affected areas

- [ ] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

Run the Postman/Newman collection against a live Bifrost instance:

```sh
newman run tests/e2e/api/collections/provider-harness.json \
  --env-var baseUrl=<BIFROST_URL> \
  --env-var bedrockModel=<BEDROCK_MODEL_ID> \
  --env-var anthropicKey=<ANTHROPIC_API_KEY>
```

All tests in sections 17 and 18 should pass. Specifically:
- Bedrock `converse-stream` responses must contain `contentBlockStop` before `messageStop`.
- All Anthropic streaming responses must have `content_block_start` indices `[0, 1, 2, …]` with no gaps.
- Bedrock Responses truncated (non-streaming) must return `status=incomplete` with `incomplete_details.reason=max_output_tokens`.
- Bedrock Responses truncated (streaming) must emit `response.incomplete` and must **not** emit `response.completed`.

## Breaking changes

- [x] No

## Related issues

Closes #4923, #4932, #4890, #4680

## Security considerations

None. These are read-only test assertions against existing API endpoints; no new credentials or secrets are introduced beyond those already required by the collection.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Bedrock Responses API does not signal max_output_tokens truncation (status/incomplete_details unset; stream emits response.completed)

5 participants