Skip to content

fix: stop attaching accumulated reasoning_details to reasoning-delta events (#413) - #477

Merged
robert-j-y merged 1 commit into
mainfrom
devin/1776216594-fix-413-reasoning-metadata-bloat
Apr 15, 2026
Merged

fix: stop attaching accumulated reasoning_details to reasoning-delta events (#413)#477
robert-j-y merged 1 commit into
mainfrom
devin/1776216594-fix-413-reasoning-metadata-bloat

Conversation

@robert-j-y

Copy link
Copy Markdown
Contributor

Summary

Fixes #413 — reasoning metadata O(N²) payload bloat.

Problem

Each reasoning-delta SSE event carried a full snapshot of all accumulatedReasoningDetails in its providerMetadata. For N reasoning chunks, total serialized payload grew O(N²) instead of O(N).

For example, with 20 reasoning chunks the total payload was 10,537 bytes — well above the 6,000-byte linear threshold (300 bytes × 20 chunks).

Fix

reasoning-start and reasoning-delta events no longer carry providerMetadata. The full accumulated reasoning_details are still available on reasoning-end, tool-call, and finish events (unchanged).

Before / After

Before: Every reasoning-delta included a growing snapshot:

reasoning-delta { delta: "chunk 1", providerMetadata: { reasoning_details: [detail1] } }
reasoning-delta { delta: "chunk 2", providerMetadata: { reasoning_details: [detail1, detail2] } }
reasoning-delta { delta: "chunk 3", providerMetadata: { reasoning_details: [detail1, detail2, detail3] } }
// ... O(N²) total payload

After: Deltas carry only the text:

reasoning-delta { delta: "chunk 1" }
reasoning-delta { delta: "chunk 2" }
reasoning-delta { delta: "chunk 3" }
// reasoning_details still available on reasoning-end and finish events

Changes

  • src/chat/index.ts — Removed providerMetadata parameter from emitReasoningChunk, removed snapshot creation
  • src/chat/index.test.ts — Updated 3 test expectations to verify providerMetadata is undefined on reasoning-delta/start events
  • e2e/issues/issue-413-reasoning-metadata-bloat.test.ts — 7 new regression tests
  • .changeset/fix-413-reasoning-metadata-bloat.md — Patch changeset

Test Results

  • Unit tests: 400 passed
  • Regression tests (issue-413): 7/7 passed
  • pnpm stylecheck: passed
  • pnpm typecheck: passed

Human Review Checklist

  • Verify that consumers relying on providerMetadata from reasoning-delta events are not broken (this was undocumented behavior)
  • Confirm reasoning-end and finish events still carry the full reasoning_details
  • Check that multi-turn reasoning with tool calls still works correctly

Generated by Devin

…events (#413)

Previously, each reasoning-delta chunk carried a full snapshot of all
accumulatedReasoningDetails in its providerMetadata. For N reasoning
chunks, this caused O(N²) total payload size.

Changes:
- Remove providerMetadata parameter from emitReasoningChunk()
- reasoning-delta and reasoning-start events no longer carry metadata
- reasoning-end, tool-call, and finish events still carry the full
  accumulated reasoning_details (unchanged)
- Remove unused SharedV3ProviderMetadata import

Add regression test with 7 cases including edge cases for single chunk,
mixed detail types, and payload size verification.

Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
@robert-j-y
robert-j-y force-pushed the devin/1776216594-fix-413-reasoning-metadata-bloat branch from 24318a9 to 2d90052 Compare April 15, 2026 21:17
@robert-j-y
robert-j-y merged commit 918d564 into main Apr 15, 2026
2 checks passed
@robert-j-y
robert-j-y deleted the devin/1776216594-fix-413-reasoning-metadata-bloat branch April 15, 2026 21:44
@github-actions github-actions Bot mentioned this pull request Apr 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool call events are buffered until stream ends (flush) causing perceived streaming delay

1 participant