fixes buffering flow for bedrock

[akshaydeo]

Summary

Fixes a streaming buffering issue (Closes #4542) where Bedrock streaming responses arrived in a single burst at the end of generation rather than incrementally. Go's net/http transport automatically negotiates gzip encoding, which causes the eventstream to be buffered until the stream completes, collapsing time-to-first-byte (TTFB) to the total generation time.

Changes

• Added Accept-Encoding: identity header to all Bedrock streaming requests, preventing Go's net/http transport from auto-negotiating gzip compression on the eventstream connection
• Removed a duplicate Accept header set that was only applied to the IAM auth path
• Added TestChatCompletionStream_StreamsIncrementally_NotBuffered: a deterministic integration test using a channel-gated fake Bedrock server that verifies the first chunk arrives while the upstream is still held open (pre-fix this times out; post-fix it succeeds)
• Added TestMakeStreamingRequest_SendsIdentityAcceptEncoding: a guard test that asserts the outbound request carries Accept-Encoding: identity rather than Go's default gzip

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

go test ./core/providers/bedrock/... -v -run TestChatCompletionStream_StreamsIncrementally_NotBuffered
go test ./core/providers/bedrock/... -v -run TestMakeStreamingRequest_SendsIdentityAcceptEncoding
go test ./...

The buffering test is deterministic and channel-gated — no wall-clock sleeps. Pre-fix, TestChatCompletionStream_StreamsIncrementally_NotBuffered fails with a 2-second timeout; post-fix it passes immediately upon receiving the first flushed chunk.

Screenshots/Recordings

N/A

Breaking changes

• No

Related issues

Closes #4542

Security considerations

No security implications. The change only affects HTTP content-encoding negotiation for the Bedrock streaming endpoint.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[coderabbitai]

Warning

Review limit reached

@akshaydeo, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 16 minutes and 16 seconds. Learn how PR review limits work.

To continue reviewing without waiting, enable usage-based billing in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 61a4de89-5e61-4f7d-bf7e-4b900c243d3e

📥 Commits

Reviewing files that changed from the base of the PR and between 2ab70fa and f03a04a.

📒 Files selected for processing (3)

• core/providers/bedrock/bedrock.go
• core/providers/bedrock/streambuffering_test.go
• tests/e2e/api/collections/provider-harness.json

📝 Walkthrough

Walkthrough

makeStreamingRequest in the Bedrock provider now sets Accept-Encoding: identity on streaming requests and removes the previous Accept: application/vnd.amazon.eventstream header. Two regression tests are added to verify that event-stream chunks are delivered incrementally and that the outgoing header value is exactly identity.

Changes

Bedrock streaming gzip fix

Layer / File(s)	Summary
Force identity encoding in makeStreamingRequest core/providers/bedrock/bedrock.go	Sets Accept-Encoding: identity on the outgoing streaming request header, replacing the prior Accept: application/vnd.amazon.eventstream header.
Regression tests for incremental streaming and identity header core/providers/bedrock/streambuffering_test.go	Adds writeEventStreamEvent helper and two tests: one asserts chunks arrive before stream completion via a channel-gated httptest server; the other asserts the outgoing Accept-Encoding header is exactly identity.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

• danpiths

Poem

🐇 A rabbit hops through event streams so fine,
No gzip to buffer, the chunks arrive in line!
Identity encoding, the header now set,
The frames flow like carrots — no lag, no regret.
Each test a bright tunnel, each chunk arrives first,
The stream never staggers, no buffered-up burst! 🌟

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	Linked issue #123 is for Files API Support, which is unrelated to Bedrock streaming buffering fixes; the PR implements streaming fixes for issue #4542 (mentioned in description) rather than issue #123.	Verify the correct issue link; #4542 (the actual issue being fixed) should be linked instead of #123 (Files API Support).

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'fixes buffering flow for bedrock' is directly related to the main change: fixing Bedrock streaming response buffering by setting Accept-Encoding headers.
Description check	✅ Passed	The description is comprehensive and complete, covering summary, changes, type, affected areas, testing, related issues, and security considerations with appropriate detail.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to fixing Bedrock streaming buffering: headers for gzip prevention and tests validating incremental streaming without buffering.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 06-21-fixes_buffering_flow_for_bedrock

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[akshaydeo]

• fixes buffering flow for bedrock #4578 👈 (View in Graphite)
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[greptile-apps]

Confidence Score: 5/5

Safe to merge — the change is a single header write on the streaming path that corrects a well-understood Go transport behavior, and the fix is covered by both a channel-gated unit test and an e2e collection entry.

The production change is minimal: one req.Header.Set call added unconditionally before the auth branch, and one redundant Accept header removed from the IAM-only branch. The header placement before signAWSRequest is correct — Accept-Encoding is set on the request that gets signed, so the signature and the wire header agree. The new test file reproduces the exact failure mode with a fake Bedrock server and verifies recovery without any wall-clock coupling in the success path. No concurrency, pooling, or provider-interface concerns are touched.

No files require special attention.

Important Files Changed

Filename	Overview
core/providers/bedrock/bedrock.go	Adds Accept-Encoding: identity before auth branching and removes a duplicate Accept header from the IAM path — targeted, correct one-liner fix with no regressions visible in surrounding logic.
core/providers/bedrock/streambuffering_test.go	New test file with two well-scoped unit tests: a channel-gated fake-server integration test that reproduces and verifies the fix for the gzip buffering issue, and a guard test that asserts the header on the wire. Tests cover the bearer-token auth path only; IAM path has no coverage gap since the header is set unconditionally before auth branching.
tests/e2e/api/collections/provider-harness.json	Adds two Postman collection entries for the native Bedrock EventStream and OpenAI-compatible SSE streaming paths. Comments correctly acknowledge that Newman buffers the full response so these tests validate frame structure/framing correctness rather than real-time TTFB.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Client
    participant BedrockProvider
    participant GoHTTPTransport
    participant BedrockRuntime

    Note over GoHTTPTransport: Pre-fix: transport auto-adds Accept-Encoding: gzip
    Note over GoHTTPTransport: Post-fix: request carries Accept-Encoding: identity

    Client->>BedrockProvider: ChatCompletionStream(...)
    BedrockProvider->>BedrockProvider: makeStreamingRequest()
    BedrockProvider->>BedrockProvider: Set Accept: application/vnd.amazon.eventstream
    BedrockProvider->>BedrockProvider: Set Accept-Encoding: identity (NEW)
    BedrockProvider->>BedrockProvider: Set auth header (bearer or IAM sign)
    BedrockProvider->>GoHTTPTransport: streamingClient.Do(req)
    GoHTTPTransport->>BedrockRuntime: "POST /model/{id}/converse-stream Accept-Encoding: identity"
    BedrockRuntime-->>GoHTTPTransport: 200 OK (raw EventStream, no gzip)
    loop Per EventStream frame (incremental)
        GoHTTPTransport-->>BedrockProvider: chunk bytes (flushed immediately)
        BedrockProvider-->>Client: BifrostStreamChunk
    end
    BedrockRuntime-->>GoHTTPTransport: messageStop frame
    GoHTTPTransport-->>BedrockProvider: EOF
    BedrockProvider-->>Client: close(streamChan)

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Client
    participant BedrockProvider
    participant GoHTTPTransport
    participant BedrockRuntime

    Note over GoHTTPTransport: Pre-fix: transport auto-adds Accept-Encoding: gzip
    Note over GoHTTPTransport: Post-fix: request carries Accept-Encoding: identity

    Client->>BedrockProvider: ChatCompletionStream(...)
    BedrockProvider->>BedrockProvider: makeStreamingRequest()
    BedrockProvider->>BedrockProvider: Set Accept: application/vnd.amazon.eventstream
    BedrockProvider->>BedrockProvider: Set Accept-Encoding: identity (NEW)
    BedrockProvider->>BedrockProvider: Set auth header (bearer or IAM sign)
    BedrockProvider->>GoHTTPTransport: streamingClient.Do(req)
    GoHTTPTransport->>BedrockRuntime: "POST /model/{id}/converse-stream Accept-Encoding: identity"
    BedrockRuntime-->>GoHTTPTransport: 200 OK (raw EventStream, no gzip)
    loop Per EventStream frame (incremental)
        GoHTTPTransport-->>BedrockProvider: chunk bytes (flushed immediately)
        BedrockProvider-->>Client: BifrostStreamChunk
    end
    BedrockRuntime-->>GoHTTPTransport: messageStop frame
    GoHTTPTransport-->>BedrockProvider: EOF
    BedrockProvider-->>Client: close(streamChan)

Loading

_{Reviews (2): Last reviewed commit: "fixes buffering flow for bedrock" | Re-trigger Greptile}

[akshaydeo]

Merge activity

• Jun 21, 5:39 AM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jun 21, 5:40 AM UTC: @akshaydeo merged this pull request with Graphite.