fix: signal Bedrock max_output_tokens truncation on Responses API - #1
Closed
jeremym-tanium wants to merge 1 commit into
Closed
jeremym-tanium wants to merge 1 commit into
jeremym-tanium wants to merge 1 commit into
Conversation
jeremym-tanium
force-pushed
the
fix/bedrock-responses-incomplete-details
branch
from
June 24, 2026 18:11
30b1d81 to
d691c34
Compare
jeremym-tanium
force-pushed
the
fix/bedrock-responses-incomplete-details
branch
from
June 24, 2026 18:16
d691c34 to
669a552
Compare
When a /v1/responses request to a Bedrock model is truncated by max_output_tokens, the OpenAI-canonical truncation signals were dropped: response.status and response.incomplete_details were left unset, and the terminal streaming event was response.completed. Consumers on the OpenAI Responses-API contract could not detect Bedrock-side truncation. Set Status="incomplete" + IncompleteDetails.Reason="max_output_tokens" on the non-streaming path, and emit response.incomplete on the streaming terminal event, when the mapped stop reason is "length". Affected packages: - core/providers/bedrock/ - Status/IncompleteDetails + terminal event type
jeremym-tanium
force-pushed
the
fix/bedrock-responses-incomplete-details
branch
from
June 24, 2026 18:18
669a552 to
0fb8d82
Compare
jeremym-tanium
marked this pull request as draft
June 24, 2026 18:25
Owner
Author
|
Superseded by the upstream PR against the main repo: maximhq#4680. Closing this fork-internal review PR; the same branch backs the upstream one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a
/v1/responsesrequest to a Bedrock model is truncated bymax_output_tokens, Bifrost drops the OpenAI-canonical truncation signals on the wire response:response.statusis left unset (OpenAI sets"incomplete")response.incomplete_detailsis left unset (OpenAI sets{"reason": "max_output_tokens"})response.completed(OpenAI emitsresponse.incomplete)Consumers using the OpenAI Responses-API contract therefore cannot detect Bedrock-side truncation. This is especially damaging for streaming tool calls, where the truncated bytes parse as valid-but-incomplete JSON (required fields silently missing or
{}), so a downstream parser cannot fall back on a parse error.Changes
Two narrow edits in
core/providers/bedrock/responses.go:BedrockConverseResponse.ToBifrostResponsesResponse): after the existingbifrostResp.StopReason = &stopReason, setStatus/IncompleteDetailsfrom the mapped stop reason ("length"→incomplete+max_output_tokens; clean reasons →completed). Mirrorscore/schemas/mux.go::responsesStatusFromChatFinishReason.FinalizeBedrockStream): the terminal chunk's hard-codedResponsesStreamResponseTypeCompletedbecomes conditional — when the mapped stop reason is"length", emitResponsesStreamResponseTypeIncompleteand set the same fields on the embeddedResponse.Value flow: Bedrock
stopReason "max_tokens"→ Bifrost finish reason"length"→ OpenAIstatus "incomplete"+incomplete_details.reason "max_output_tokens". No new code paths or API surface;status/incomplete_detailsare existing optional fields, so consumers that ignore them are unaffected.Changelog entry added to
core/changelog.md.Type of change
Affected areas
How to test
Covered by:
TestBedrockStopReasonMappingResponsesPath— extended withStatus/IncompleteDetailsexpectations per stop reason (truncation →incomplete, clean →completed, unmapped → unset)TestFinalizeBedrockStream_MaxTokensTruncation— terminal event isresponse.incompletewith the fields setTestFinalizeBedrockStream_CleanCompletionUnaffected— non-truncation stop reasons still produceresponse.completedBreaking changes
statusandincomplete_detailsare existing optional fields. The streaming terminal event type changes fromresponse.completedtoresponse.incompleteonly when Bedrock reportsmax_tokens— which is the bug fix (consumers now see the OpenAI-canonical type).Related issues
None currently tracks this. Happy to open one first if maintainers prefer.
Security considerations
None — no auth, secret, or PII surface changed.
Checklist
gofmt -s,goimports,go vet,go build ./..., and the bedrock tests above)