Skip to content

fix(responses-api): prevent output_index collision and data truncation - #9183

Closed
hartmark wants to merge 3 commits into
cheaper-inference:release/v3.8.50from
hartmark:fix/responses-api-index-collision
Closed

hartmark wants to merge 3 commits into
cheaper-inference:release/v3.8.50from
hartmark:fix/responses-api-index-collision

Conversation

@hartmark

@hartmark hartmark commented Aug 2, 2026 •

Copy link
Copy Markdown
Contributor

This PR fixes several critical bugs in the Responses API implementation that caused data loss, truncated responses, and incorrect turn grouping in multi-turn conversations.

Key Fixes:

  1. Index Collisions: Fixed a bug where reasoning, message, and tool calls were assigned colliding 'output_index' values, causing clients to ignore tool calls.
  2. Data Truncation: Fixed a race condition in chunk processing where tool calls and finish signals were lost if they arrived in the same chunk as reasoning or text content.
  3. Multi-Choice Support: Enabled full processing of multi-choice chunks, which were previously truncated to a single choice.
  4. Reasoning Replay: Added support for capturing and replaying 'reasoning_content' for DeepSeek-based models (like 'big-pickle'). This prevents history corruption with generic placeholders that cause models to lose context and stop prematurely.
  5. Reasoning Extraction: Added 'big-pickle' to models recognized for native textual reasoning tags, ensuring blocks are correctly converted to reasoning items.
  6. Turn Grouping: Improved the Responses-to-Chat translation to group reasoning, content, and tool calls into a single assistant turn, as required by strict OpenAI-compatible upstreams.
  7. Encrypted Reasoning: Added a descriptive placeholder for encrypted Responses API reasoning blocks to maintain context in downgraded Chat API requests.

Verification:

  • Added 'tests/integration/live-responses-api.test.ts' to verify tool calls and multi-turn reasoning in a live environment.
  • Added 'tests/unit/responses-api-truncation.test.ts' for truncation and multi-choice scenarios.
  • Added 'tests/unit/responses-request-translation.test.ts' for reasoning conversion and turn grouping.
  • Verified reasoning replay logic for 'big-pickle' via system-level logic tests.
  • All Responses API unit and integration tests pass.

@hartmark
hartmark requested a review from diegosouzapw as a code owner August 2, 2026 02:19
@hartmark hartmark changed the title fix(responses-api): prevent output_index collision between reasoning, message, and tool calls fix(responses-api): prevent output_index collision and data truncation Aug 2, 2026
@hartmark
hartmark marked this pull request as draft August 3, 2026 12:38
@hartmark

hartmark commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Still a work in progress — more testing in progress, not ready for review yet.

@hartmark
hartmark force-pushed the fix/responses-api-index-collision branch from 74fc86a to b491d13 Compare August 3, 2026 12:59
@hartmark
hartmark marked this pull request as ready for review August 3, 2026 19:19
…and empty tool_calls in reasoning replay

Squashes the working history for the big-pickle multi-turn tool-calling
fixes into a single commit:

- Prevent output_index collision between reasoning, message, and tool
  calls in the Responses API stream.
- Prevent data truncation in multi-choice and reasoning chunks.
- Preserve reasoning history and improve turn grouping across turns.
- Enable reasoning capture/extraction and replay for big-pickle.
- Fix multi-turn reasoning replay and turn index hardcoding.
- Strip empty `tool_calls: []` from assistant turns in the
  Responses -> Chat Completions downgrade — upstreams like
  opencode-zen/big-pickle reject the empty array with a 400.

Still WIP: further live agentic-workflow testing against the "default"
combo in progress.
…ead side

The turn-index-hardcoding fix updated the reasoning-cache read side
(translator/index.ts) to key lookups by the assistant message's real
position in the messages array, but chatCore.ts's write side (both the
streaming and non-streaming completion paths) still cached every
response under the old hardcoded `messageIndex: 0`. Past the first
assistant turn the two conventions no longer matched, so DeepSeek/
Xiaomi-mimo plain-turn reasoning replay silently missed the cache and
fell back to the placeholder in ordinary multi-turn conversations.

Compute the write-side index from the incoming request's message count
instead, matching the position the response will occupy once the client
appends it to history for the next turn. Found via coverage analysis
surfacing two now-fixed pre-existing tests (reasoning-cache.test.ts,
translator-helper-branches.test.ts) whose fixture keys assumed the old
hardcoded convention; added an end-to-end write/read regression test.
@hartmark
hartmark changed the base branch from release/v3.8.49 to release/v3.8.50 August 3, 2026 20:05
@hartmark
hartmark force-pushed the fix/responses-api-index-collision branch from e1cb77e to f681467 Compare August 3, 2026 20:06
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@hartmark

hartmark commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

Folding this into #9439 at the operator's request — the responses-api index-collision/truncation/reasoning-replay fixes are now applied there (same diff, verified with the full 82-test suite passing) alongside the agentic conversation tracking work. Closing in favor of #9439.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant