Skip to content

fix: signal Bedrock max_output_tokens truncation on Responses API - #1

Closed
jeremym-tanium wants to merge 1 commit into
devfrom
fix/bedrock-responses-incomplete-details
Closed

jeremym-tanium wants to merge 1 commit into
devfrom
fix/bedrock-responses-incomplete-details

Conversation

@jeremym-tanium

@jeremym-tanium jeremym-tanium commented Jun 24, 2026 •

Copy link
Copy Markdown
Owner

Summary

When a /v1/responses request to a Bedrock model is truncated by max_output_tokens, Bifrost drops the OpenAI-canonical truncation signals on the wire response:

  • response.status is left unset (OpenAI sets "incomplete")
  • response.incomplete_details is left unset (OpenAI sets {"reason": "max_output_tokens"})
  • The streaming terminal SSE event is response.completed (OpenAI emits response.incomplete)

Consumers using the OpenAI Responses-API contract therefore cannot detect Bedrock-side truncation. This is especially damaging for streaming tool calls, where the truncated bytes parse as valid-but-incomplete JSON (required fields silently missing or {}), so a downstream parser cannot fall back on a parse error.

Changes

Two narrow edits in core/providers/bedrock/responses.go:

  • Non-streaming path (BedrockConverseResponse.ToBifrostResponsesResponse): after the existing bifrostResp.StopReason = &stopReason, set Status/IncompleteDetails from the mapped stop reason ("length" → incomplete + max_output_tokens; clean reasons → completed). Mirrors core/schemas/mux.go::responsesStatusFromChatFinishReason.
  • Streaming path (FinalizeBedrockStream): the terminal chunk's hard-coded ResponsesStreamResponseTypeCompleted becomes conditional — when the mapped stop reason is "length", emit ResponsesStreamResponseTypeIncomplete and set the same fields on the embedded Response.

Value flow: Bedrock stopReason "max_tokens" → Bifrost finish reason "length" → OpenAI status "incomplete" + incomplete_details.reason "max_output_tokens". No new code paths or API surface; status/incomplete_details are existing optional fields, so consumers that ignore them are unaffected.

Changelog entry added to core/changelog.md.

Type of change

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Chore/CI

Affected areas

  • Core (Go)
  • Transports (HTTP)
  • Providers/Integrations
  • Plugins
  • UI (React)
  • Docs

How to test

cd core
go test ./providers/bedrock/ \
  -run 'TestBedrockStopReasonMappingResponsesPath|TestFinalizeBedrockStream' -count=1 -v

Covered by:

  • TestBedrockStopReasonMappingResponsesPath — extended with Status/IncompleteDetails expectations per stop reason (truncation → incomplete, clean → completed, unmapped → unset)
  • TestFinalizeBedrockStream_MaxTokensTruncation — terminal event is response.incomplete with the fields set
  • TestFinalizeBedrockStream_CleanCompletionUnaffected — non-truncation stop reasons still produce response.completed

Breaking changes

  • Yes
  • No

status and incomplete_details are existing optional fields. The streaming terminal event type changes from response.completed to response.incomplete only when Bedrock reports max_tokens — which is the bug fix (consumers now see the OpenAI-canonical type).

Related issues

None currently tracks this. Happy to open one first if maintainers prefer.

Security considerations

None — no auth, secret, or PII surface changed.

Checklist

  • I read the contributing guidelines and followed them
  • I added/updated tests where appropriate
  • I updated documentation where needed (no behavior outside the existing OpenAI contract)
  • I verified Go builds and tests succeed (UI not touched)
  • I verified the full CI pipeline locally (the provider suite needs live credentials; ran gofmt -s, goimports, go vet, go build ./..., and the bedrock tests above)

@jeremym-tanium jeremym-tanium changed the title fix(bedrock): signal max_output_tokens truncation on Responses-API fix: signal Bedrock max_output_tokens truncation on Responses API Jun 24, 2026
@jeremym-tanium
jeremym-tanium force-pushed the fix/bedrock-responses-incomplete-details branch from 30b1d81 to d691c34 Compare June 24, 2026 18:11
@jeremym-tanium jeremym-tanium changed the title fix: signal Bedrock max_output_tokens truncation on Responses API fix(bedrock): signal max_output_tokens truncation on Responses API Jun 24, 2026
@jeremym-tanium
jeremym-tanium force-pushed the fix/bedrock-responses-incomplete-details branch from d691c34 to 669a552 Compare June 24, 2026 18:16
When a /v1/responses request to a Bedrock model is truncated by
max_output_tokens, the OpenAI-canonical truncation signals were dropped:
response.status and response.incomplete_details were left unset, and the
terminal streaming event was response.completed. Consumers on the OpenAI
Responses-API contract could not detect Bedrock-side truncation.

Set Status="incomplete" + IncompleteDetails.Reason="max_output_tokens" on
the non-streaming path, and emit response.incomplete on the streaming
terminal event, when the mapped stop reason is "length".

Affected packages:
- core/providers/bedrock/ - Status/IncompleteDetails + terminal event type
@jeremym-tanium
jeremym-tanium force-pushed the fix/bedrock-responses-incomplete-details branch from 669a552 to 0fb8d82 Compare June 24, 2026 18:18
@jeremym-tanium jeremym-tanium changed the title fix(bedrock): signal max_output_tokens truncation on Responses API fix: signal Bedrock max_output_tokens truncation on Responses API Jun 24, 2026
@jeremym-tanium
jeremym-tanium marked this pull request as draft June 24, 2026 18:25
@jeremym-tanium

Copy link
Copy Markdown
Owner Author

Superseded by the upstream PR against the main repo: maximhq#4680. Closing this fork-internal review PR; the same branch backs the upstream one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant