fix(responses): preserve replay data across reasoning phases by jinwukong · Pull Request #40673 · BerriAI/litellm

jinwukong · 2026-09-11T03:30:29Z

TLDR

Problem this solves:

Reasoning done events omit signed thinking needed for replay
Resumed reasoning leaves later blocks out of saved items

How it solves it:

Emit a reasoning item for each asynchronous thinking phase
Preserve signed and redacted blocks with the shared encoder
Keep streamed item indexes aligned with completed output

Intentional product change: Resumed thinking and intervening text now produce separate output items, preserving their chronological order. Clients should retain every completed item by its output_index. Text continues streaming immediately; message completion events wait for final citations

User Flow

Before: a client saving reasoning done items cannot retain all signed thinking from a response that resumes reasoning after a tool

POST http://localhost:44100/v1/responses with streaming, thinking, web search, and reasoning.encrypted_content included
Save each response.output_item.done item by output_index as the server executes its search
Compare the saved reasoning with response.completed: later signed blocks are absent from the saved items

After: the same client retains the replay data for every reasoning phase

POST http://localhost:44101/v1/responses with the same request
Save each response.output_item.done item by output_index, including reasoning that resumes after the search
Compare the saved reasoning with response.completed: replay blocks and item indexes agree

Relevant issues

Builds on #44127 and the shared replay encoder from #44132. The provider-side duplicate-text fix is retained

Community review before this rebase: PrimeFoldTools approved 4292140a, reporting 409 passing tests and 120 synthetic reasoning/tool/text combinations. Their recheck covers item identity, output indexes, completion order, message text, exact reasoning-block ownership, JSON persistence, SDK parsing, and replay. They did not repeat live-provider calls; the real API evidence below is from my runs

Rebased onto cb138ba to retain the upstream public helper names, hidden-parameter accessors, incomplete terminal events and request-parameter echo. The replay logic is unchanged, including pending thinking across empty, usage-only and metadata-only chunks. Annotation conversion uses the shared public helper. The approval above refers to the pre-rebase commit

The earlier #43010 duplication report was closed after #44127. #43196 was closed by its author in favor of this PR before the scope changed

Pre-Submission checklist

I have added meaningful tests
The test files covering my change pass locally
My PR passes all required CI/CD checks
My PR's scope is isolated to replay data in reasoning events
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Local validation: 446 related tests and full make lint passed. All 94 iterator cases pass on this branch. The merge base has 50 failing cases and 44 passing controls. Eighteen targeted mutations were detected, with all iterator controls passing before mutation and after restoration

The regression cases cover resumed visible, signature-only and redacted thinking, ordinary tools, successful and failed web search, tool-first responses, combined result/reasoning chunks, upstream exhaustion, empty-message placeholders, unsigned-only blocks, empty/usage/metadata chunks before a signature, message chronology, mixed chunks, late and positionless citations, and converted code-tool indexes. Saved items are JSON-round-tripped, reasoning items are parsed with the OpenAI SDK, and decoded replay data is checked

Screenshots / Proof of Fix

Fresh requests against real Anthropic-backed proxies, using claude-opus-5-5. Each column below is a separate model response; the comparison checks preservation within that response

Before cb138ba, after f429906

Case	Before: saved / completed blocks	After: saved / completed blocks	After indexes match
client-tool, visible	0 / 1	1 / 1	True
client-tool, omitted	0 / 1	1 / 1	True
server-tool, visible	0 / 2	2 / 2	True
server-tool, omitted	0 / 2	2 / 2	True

The four client-tool follow-up requests used the saved done items and returned HTTP 200 with 7418. Server-tool cases verify same-response resumption and payload/index consistency; they are not separate tool-result round trips

Reproduce the real requests and inspect their captured output

Run the base checkout on port 44100 and this branch on 44101 with the same locked dependencies. Set ANTHROPIC_API_KEY in the environment and use this proxy.yaml

model_list:
  - model_name: thinking-model
    litellm_params:
      model: anthropic/claude-opus-5-5
      api_key: os.environ/ANTHROPIC_API_KEY
      thinking:
        type: adaptive
      max_tokens: 4096
litellm_settings:
  num_retries: 0
  request_timeout: 120
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Generate one temporary proxy key and export it to both proxy processes and the curl shell

export LITELLM_RUST=0 LITELLM_LOCAL_MODEL_COST_MAP=True
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
python -c 'from litellm.proxy.proxy_cli import run_server; run_server()' \
  --config proxy.yaml --host 127.0.0.1 --port 44100 --telemetry False
# Use --port 44101 in the patched checkout

Save as client-tool-omitted.json

{
  "model": "thinking-model",
  "input": [
    {
      "role": "user",
      "content": "Find the smallest positive integer n such that n mod 17 = 5, n mod 19 = 7, and n mod 23 = 11. Work out n before calling the add tool with a=n and b=1. Wait for the tool result, then reply with only the total."
    }
  ],
  "tools": [
    {
      "type": "function",
      "name": "add",
      "description": "Return the sum of two integers",
      "parameters": {
        "type": "object",
        "properties": {
          "a": {
            "type": "integer"
          },
          "b": {
            "type": "integer"
          }
        },
        "required": [
          "a",
          "b"
        ],
        "additionalProperties": false
      }
    }
  ],
  "reasoning": {
    "effort": "max"
  },
  "include": [
    "reasoning.encrypted_content"
  ],
  "store": false,
  "stream": true,
  "thinking": {
    "type": "adaptive",
    "display": "omitted"
  }
}

Create client-tool-visible.json

jq '.thinking.display = "summarized"' client-tool-omitted.json > client-tool-visible.json

Save as server-tool-omitted.json

{
  "model": "thinking-model",
  "input": [
    {
      "role": "user",
      "content": "Use web search once to find the latest stable CPython release on python.org. First decide which official source to search, then inspect the returned evidence and explain whether that release is newer than Python 3.12. Keep your final answer under 80 words."
    }
  ],
  "tools": [
    {
      "type": "web_search",
      "search_context_size": "low"
    }
  ],
  "reasoning": {
    "effort": "high"
  },
  "include": [
    "reasoning.encrypted_content"
  ],
  "store": false,
  "stream": true,
  "thinking": {
    "type": "adaptive",
    "display": "omitted"
  }
}

Create server-tool-visible.json

jq '.thinking.display = "summarized"' server-tool-omitted.json > server-tool-visible.json

Save as inspect-stream.jq. It prints counts and comparisons without exposing signatures

def replay_blocks($items):
  [$items[] | select(.type == "reasoning") | .encrypted_content // empty | fromjson[]];
def chronology($events):
  reduce range(0; $events | length) as $i
    ({reasoning: -1, messages: {}, ok: true};
      $events[$i] as $event |
      if $event.type == "response.reasoning_summary_text.delta" or
         ($event.type == "response.output_item.added" and $event.item.type == "reasoning")
      then .reasoning = $i
      elif $event.type == "response.output_item.added" and $event.item.type == "message"
      then .messages[$event.item.id] = $i
      elif $event.type == "response.output_text.delta"
      then .ok = (.ok and ((.messages[$event.item_id] // -1) > .reasoning))
      else . end) | .ok;
[split("\n")[] | select(startswith("data: ")) | ltrimstr("data: ") |
 select(. != "[DONE]") | fromjson] as $events |
([$events[] | select(.type == "response.completed") | .response][0]) as $completed |
([$events[] | select(.type == "response.output_item.done")] | sort_by(.output_index)) as $done |
replay_blocks([$done[].item]) as $saved |
replay_blocks($completed.output) as $final |
{
  reasoning_done_items: ([$done[] | select(.item.type == "reasoning")] | length),
  completed_reasoning_items: ([$completed.output[] | select(.type == "reasoning")] | length),
  done_replay_blocks: ($saved | length),
  completed_replay_blocks: ($final | length),
  done_payload_matches_completed: (($saved | length) > 0 and $saved == $final),
  done_item_indexes_match_completed: all($done[]; .item.id == $completed.output[.output_index].id),
  message_chronology_preserved: chronology($events),
  output_item_types: [$completed.output[].type],
  server_tool_statuses: [$completed.output[] | select(.type == "web_search_call") | .status]
}

For client-tool cases, define this helper to collect done items in their output order

make_replay() {
  jq -Rsc '[split("\n")[] | select(startswith("data: ")) |
    ltrimstr("data: ") | select(. != "[DONE]") | fromjson |
    select(.type == "response.output_item.done")] |
    sort_by(.output_index) | map(.item)' "$1.sse" > "$1-output.json"
  jq --slurpfile output "$1-output.json"     '. + {stream: false, input: (.input + $output[0] +
      [$output[0][] | select(.type == "function_call") |
       {type: "function_call_output", call_id: .call_id,
        output: (.arguments | fromjson | .a + .b | tostring)}])}'     "$2" > "$1-replay.json"
}

Before (`cb138ba`)

Client-tool, visible thinking

Stream the request

curl -sS -N http://127.0.0.1:44100/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @client-tool-visible.json \
  -o before-client-tool-visible.sse -w '%{http_code}\n'

Inspect the saved replay payload and item positions

jq -Rsc -f inspect-stream.jq before-client-tool-visible.sse

{
  "reasoning_done_items": 1,
  "completed_reasoning_items": 1,
  "done_replay_blocks": 0,
  "completed_replay_blocks": 1,
  "done_payload_matches_completed": false,
  "done_item_indexes_match_completed": false,
  "message_chronology_preserved": true,
  "output_item_types": [
    "reasoning",
    "message",
    "function_call"
  ],
  "server_tool_statuses": []
}

Replay the saved items with the tool result

make_replay before-client-tool-visible client-tool-visible.json
curl -sS http://127.0.0.1:44100/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @before-client-tool-visible-replay.json \
  -o before-client-tool-visible-replay-response.json -w '%{http_code}\n'
jq -r '[.output[] | select(.type == "message") | .content[] |
  select(.type == "output_text") | .text] | join("")' before-client-tool-visible-replay-response.json

200
7418

Client-tool, omitted thinking

Stream the request

curl -sS -N http://127.0.0.1:44100/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @client-tool-omitted.json \
  -o before-client-tool-omitted.sse -w '%{http_code}\n'

Inspect the saved replay payload and item positions

jq -Rsc -f inspect-stream.jq before-client-tool-omitted.sse

{
  "reasoning_done_items": 1,
  "completed_reasoning_items": 1,
  "done_replay_blocks": 0,
  "completed_replay_blocks": 1,
  "done_payload_matches_completed": false,
  "done_item_indexes_match_completed": false,
  "message_chronology_preserved": true,
  "output_item_types": [
    "reasoning",
    "message",
    "function_call"
  ],
  "server_tool_statuses": []
}

Replay the saved items with the tool result

make_replay before-client-tool-omitted client-tool-omitted.json
curl -sS http://127.0.0.1:44100/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @before-client-tool-omitted-replay.json \
  -o before-client-tool-omitted-replay-response.json -w '%{http_code}\n'
jq -r '[.output[] | select(.type == "message") | .content[] |
  select(.type == "output_text") | .text] | join("")' before-client-tool-omitted-replay-response.json

200
7418

Server-tool, visible thinking

Stream the request

curl -sS -N http://127.0.0.1:44100/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @server-tool-visible.json \
  -o before-server-tool-visible.sse -w '%{http_code}\n'

Inspect the saved replay payload and item positions

jq -Rsc -f inspect-stream.jq before-server-tool-visible.sse

{
  "reasoning_done_items": 1,
  "completed_reasoning_items": 1,
  "done_replay_blocks": 0,
  "completed_replay_blocks": 2,
  "done_payload_matches_completed": false,
  "done_item_indexes_match_completed": true,
  "message_chronology_preserved": false,
  "output_item_types": [
    "reasoning",
    "message",
    "web_search_call"
  ],
  "server_tool_statuses": [
    "completed"
  ]
}

Server-tool, omitted thinking

Stream the request

curl -sS -N http://127.0.0.1:44100/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @server-tool-omitted.json \
  -o before-server-tool-omitted.sse -w '%{http_code}\n'

Inspect the saved replay payload and item positions

jq -Rsc -f inspect-stream.jq before-server-tool-omitted.sse

{
  "reasoning_done_items": 1,
  "completed_reasoning_items": 1,
  "done_replay_blocks": 0,
  "completed_replay_blocks": 2,
  "done_payload_matches_completed": false,
  "done_item_indexes_match_completed": true,
  "message_chronology_preserved": true,
  "output_item_types": [
    "reasoning",
    "message",
    "web_search_call"
  ],
  "server_tool_statuses": [
    "completed"
  ]
}

After (`f429906`)

Client-tool, visible thinking

Stream the request

curl -sS -N http://127.0.0.1:44101/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @client-tool-visible.json \
  -o after-client-tool-visible.sse -w '%{http_code}\n'

Inspect the saved replay payload and item positions

jq -Rsc -f inspect-stream.jq after-client-tool-visible.sse

{
  "reasoning_done_items": 1,
  "completed_reasoning_items": 1,
  "done_replay_blocks": 1,
  "completed_replay_blocks": 1,
  "done_payload_matches_completed": true,
  "done_item_indexes_match_completed": true,
  "message_chronology_preserved": true,
  "output_item_types": [
    "reasoning",
    "function_call"
  ],
  "server_tool_statuses": []
}

Replay the saved items with the tool result

make_replay after-client-tool-visible client-tool-visible.json
curl -sS http://127.0.0.1:44101/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @after-client-tool-visible-replay.json \
  -o after-client-tool-visible-replay-response.json -w '%{http_code}\n'
jq -r '[.output[] | select(.type == "message") | .content[] |
  select(.type == "output_text") | .text] | join("")' after-client-tool-visible-replay-response.json

200
7418

Client-tool, omitted thinking

Stream the request

curl -sS -N http://127.0.0.1:44101/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @client-tool-omitted.json \
  -o after-client-tool-omitted.sse -w '%{http_code}\n'

Inspect the saved replay payload and item positions

jq -Rsc -f inspect-stream.jq after-client-tool-omitted.sse

{
  "reasoning_done_items": 1,
  "completed_reasoning_items": 1,
  "done_replay_blocks": 1,
  "completed_replay_blocks": 1,
  "done_payload_matches_completed": true,
  "done_item_indexes_match_completed": true,
  "message_chronology_preserved": true,
  "output_item_types": [
    "reasoning",
    "function_call"
  ],
  "server_tool_statuses": []
}

Replay the saved items with the tool result

make_replay after-client-tool-omitted client-tool-omitted.json
curl -sS http://127.0.0.1:44101/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @after-client-tool-omitted-replay.json \
  -o after-client-tool-omitted-replay-response.json -w '%{http_code}\n'
jq -r '[.output[] | select(.type == "message") | .content[] |
  select(.type == "output_text") | .text] | join("")' after-client-tool-omitted-replay-response.json

200
7418

Server-tool, visible thinking

Stream the request

curl -sS -N http://127.0.0.1:44101/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @server-tool-visible.json \
  -o after-server-tool-visible.sse -w '%{http_code}\n'

Inspect the saved replay payload and item positions

jq -Rsc -f inspect-stream.jq after-server-tool-visible.sse

{
  "reasoning_done_items": 2,
  "completed_reasoning_items": 2,
  "done_replay_blocks": 2,
  "completed_replay_blocks": 2,
  "done_payload_matches_completed": true,
  "done_item_indexes_match_completed": true,
  "message_chronology_preserved": true,
  "output_item_types": [
    "reasoning",
    "message",
    "web_search_call",
    "reasoning",
    "message"
  ],
  "server_tool_statuses": [
    "completed"
  ]
}

Server-tool, omitted thinking

Stream the request

curl -sS -N http://127.0.0.1:44101/v1/responses \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H 'Content-Type: application/json' --data-binary @server-tool-omitted.json \
  -o after-server-tool-omitted.sse -w '%{http_code}\n'

Inspect the saved replay payload and item positions

jq -Rsc -f inspect-stream.jq after-server-tool-omitted.sse

{
  "reasoning_done_items": 2,
  "completed_reasoning_items": 2,
  "done_replay_blocks": 2,
  "completed_replay_blocks": 2,
  "done_payload_matches_completed": true,
  "done_item_indexes_match_completed": true,
  "message_chronology_preserved": true,
  "output_item_types": [
    "reasoning",
    "message",
    "web_search_call",
    "reasoning",
    "message"
  ],
  "server_tool_statuses": [
    "completed"
  ]
}

Type

Bug Fix

Caveats

Medium

Synchronous streams still replay reasoning from terminal snapshots

This change handles asynchronous reasoning phases. It does not add synchronous reasoning done events

As in the merge base, message done events wait until upstream exhaustion so providers can deliver final citations. Text deltas remain immediate; clients acting only on message done will see those segments finalized later

Final Attestation

The tests cover the replay regression and its described edge cases

CLAassistant · 2026-09-11T03:30:36Z

All committers have signed the CLA.

codspeed · 2026-09-11T03:33:33Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing jinwukong:bugfix/responses-thinking-replay (f429906) with main (cb138ba)}

greptile-apps · 2026-09-11T03:34:53Z

Confidence Score: 5/5

[Medium risk] Refactors streaming response handling for reasoning phases.

The PR appears safe to merge, with no outstanding previous findings or accepted new issues

Summary

This PR preserves replayable signed and redacted reasoning across asynchronous reasoning phases while keeping streamed output indexes aligned with completed output

Emits and retains a separate reasoning item for each asynchronous reasoning phase
Splits intervening message segments to preserve chronological output ordering and citation ownership
Uses the shared reasoning encoder and public response transformation helpers
Adds extensive regression coverage for reasoning, tools, annotations, indexes, terminal events, and replay persistence

_{Reviews (22) · Last reviewed commit: "fix(responses): retain pending thinking ..." · Reviewed by Greptile}

codecov · 2026-09-11T03:39:31Z

Codecov Report

❌ Patch coverage is 95.10870% with 9 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...lm_completion_transformation/streaming_iterator.py	95.10%	9 Missing ⚠️

📢 Thoughts on this report? Let us know!

jinwukong · 2026-09-11T03:59:45Z

@greptileai Added a regression for reasoning events before response assembly and refreshed the live replay evidence for the latest commit. Please review again

jinwukong · 2026-09-11T04:25:48Z

@greptileai Addressed the mutation review with an immutable grouping implementation. Full lint, 244 tests, and updated live replay evidence pass. Please review again

jinwukong · 2026-09-11T04:28:50Z

@Sameerlite Required checks pass, coverage is complete, and Greptile reports 5/5. Could you review and rerun cost-map-guard, which timed out downloading multidict?

jinwukong · 2026-09-13T14:18:18Z

@greptileai Rebased onto main with all 27 required checks passing and refreshed live replay evidence matching the current tip. Please review again

jinwukong · 2026-09-13T14:20:29Z

bugbot run Please review the rebased main diff after all required checks and live replay validation passed successfully

cursor

Stale Bugbot comment from a previous run.

jinwukong · 2026-09-13T14:25:56Z

@Sameerlite The PR is rebased onto main, mergeable, and green across required CI, coverage, Greptile, Veria, and Bugbot. Ready for review

jinwukong · 2026-09-19T14:45:39Z

@greptileai Rebased onto current main, preserved upstream lifecycle tests, and refreshed 327 regressions plus live Anthropic replay evidence. Please review again

jinwukong · 2026-09-19T14:45:41Z

bugbot run Please review the current main diff after 327 regressions, full lint, and refreshed live Anthropic replay validation passed

cursor

Stale Bugbot comment from a previous run.

jinwukong · 2026-09-20T06:44:44Z

@greptileai Rebased onto main with the upstream CI fix. Local lint, 332 tests, and refreshed live replay pass. Please review again

jinwukong · 2026-09-20T06:44:47Z

bugbot run Please review the updated main diff with the upstream CI fix, passing local checks, and refreshed live replay evidence

cursor

Stale Bugbot comment from a previous run.

jinwukong · 2026-09-20T07:02:31Z

@Sameerlite All 27 required checks pass after rebasing onto main. Patch coverage is complete, with Greptile 5/5, Veria, and Bugbot clear. Please review

jinwukong · 2026-09-22T22:48:54Z

@greptileai Resolved an upstream test import conflict. Full lint, 342 tests, and refreshed Opus 5.5 replay evidence pass. Please review the updated commit

jinwukong · 2026-09-22T22:48:56Z

bugbot run Please review the updated commit after resolving the upstream import conflict, passing 342 tests, and refreshing live replay evidence

jinwukong · 2026-10-04T03:53:44Z

@greptileai Please review 069f5a1. Converted tool indexes and late citations are fixed; 433 regressions, full lint, and refreshed provider checks pass

jinwukong · 2026-10-04T03:53:46Z

bugbot run Please review 069f5a1 for converted tool indexes and late annotations. Regression, mutation, lint, and refreshed real-provider checks pass

cursor

Stale Bugbot comment from a previous run.

jinwukong · 2026-10-04T04:14:50Z

@greptileai Please review 4292140. Positionless citations now retain segment ownership; completion timing matches the merge base. Regression, lint, and live checks pass

jinwukong · 2026-10-04T04:14:52Z

bugbot run Please review 4292140 for positionless citation preservation and replay consistency. All 434 regressions, full lint, and refreshed live checks pass

cursor

Stale Bugbot comment from a previous run.

PrimeFoldTools

Rechecked current commit 4292140a after the resumed-reasoning changes.

Independent verification: 409 relevant tests passed. Separately, 120 synthetic reasoning/tool/text combinations passed checks covering item identity, output indexes, completion order, message text, and exact reasoning-block ownership.

The original reasoning → tool → reasoning boundary now produces two distinct reasoning done items, each with its own signed block and each matching the corresponding completed output item. JSON persistence, OpenAI SDK parsing, and replay also continue to pass.

I haven’t independently repeated the live-provider calls, but the local regression coverage no longer shows a blocker in the expanded revision.

Approved from my review scope.

jinwukong · 2026-10-06T05:18:25Z

@greptileai Please review acb1e3a after rebasing onto current main. Incomplete events are retained; 441 regressions, full lint, and fresh provider checks pass

jinwukong · 2026-10-06T05:18:28Z

bugbot run Please review acb1e3a for replay preservation after the main rebase and annotation consolidation. Regression, lint, and live checks pass

cursor

Stale Bugbot comment from a previous run.

jinwukong · 2026-10-06T06:07:26Z

@greptileai Please review cc233e3. Pending thinking survives empty, usage-only, and metadata-only chunks; 444 regressions, 18 mutations, lint, and refreshed live proof pass

jinwukong · 2026-10-06T06:07:28Z

bugbot run Please review cc233e3. The reported unsigned-thinking buffer loss is fixed and covered by regressions, mutation checks, and refreshed live validation

cursor

Stale Bugbot comment from a previous run.

jinwukong · 2026-10-07T12:51:57Z

@greptileai Please review f429906 after resolving upstream helper renames. Replay behavior is unchanged; 446 regressions, lint, 18 mutations and refreshed live proof pass

jinwukong · 2026-10-07T12:52:00Z

bugbot run Please review f429906 after the upstream rebase. Public helper names are retained; regression, mutation, lint and real API checks pass

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit f429906. Configure here.}

jinwukong · 2026-10-07T13:54:27Z

Superseded by #45044, which keeps only the reasoning done-event replay fix in 119 lines. Thanks @PrimeFoldTools for the earlier review.

jinwukong requested a review from a team September 11, 2026 03:30

greptile-apps Bot reviewed Sep 11, 2026

View reviewed changes

Comment thread litellm/litellm_core_utils/streaming_chunk_builder_utils.py Outdated

yuneng-berri deleted the branch BerriAI:main September 13, 2026 04:52

yuneng-berri closed this Sep 13, 2026

yuneng-berri reopened this Sep 13, 2026

jinwukong force-pushed the bugfix/responses-thinking-replay branch from 236df8e to 1086b1c Compare September 13, 2026 14:08

jinwukong changed the base branch from litellm_internal_staging to main September 13, 2026 14:08

cursor Bot reviewed Sep 13, 2026

View reviewed changes

jinwukong force-pushed the bugfix/responses-thinking-replay branch from 1086b1c to ec81208 Compare September 19, 2026 14:45

cursor Bot reviewed Sep 19, 2026

View reviewed changes

jinwukong force-pushed the bugfix/responses-thinking-replay branch from ec81208 to 2f19d21 Compare September 20, 2026 06:44

cursor Bot reviewed Sep 20, 2026

View reviewed changes

jinwukong force-pushed the bugfix/responses-thinking-replay branch from 2f19d21 to 8b61968 Compare September 22, 2026 22:48

greptile-apps Bot reviewed Oct 4, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/streaming_iterator.py

Comment thread litellm/responses/litellm_completion_transformation/streaming_iterator.py Outdated

cursor Bot reviewed Oct 4, 2026

View reviewed changes

greptile-apps Bot reviewed Oct 4, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/streaming_iterator.py

cursor Bot reviewed Oct 4, 2026

View reviewed changes

PrimeFoldTools approved these changes Oct 4, 2026

View reviewed changes

jinwukong force-pushed the bugfix/responses-thinking-replay branch from 4292140 to acb1e3a Compare October 6, 2026 05:17

cursor Bot reviewed Oct 6, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/streaming_iterator.py

cursor Bot reviewed Oct 6, 2026

View reviewed changes

jinwukong added 7 commits October 7, 2026 20:39

fix(responses): include replay data in reasoning done events

ff82d36

fix(responses): preserve resumed reasoning phases

8e030af

fix(responses): preserve interleaved message history

39dd7a0

fix(responses): retain final citations and converted tool indexes

83b4f3e

fix(responses): preserve citations without text positions

2551344

refactor(responses): share streamed annotation conversion

dd16ab9

fix(responses): retain pending thinking across empty chunks

f429906

jinwukong force-pushed the bugfix/responses-thinking-replay branch from cc233e3 to f429906 Compare October 7, 2026 12:50

cursor Bot reviewed Oct 7, 2026

View reviewed changes

jinwukong mentioned this pull request Oct 7, 2026

fix(responses): include encrypted_content in streamed reasoning done events #45044

Open

5 tasks done

jinwukong closed this Oct 7, 2026

Conversation

jinwukong commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Pre-Submission checklist

Screenshots / Proof of Fix

Before (cb138ba)

Client-tool, visible thinking

Client-tool, omitted thinking

Server-tool, visible thinking

Server-tool, omitted thinking

After (f429906)

Client-tool, visible thinking

Client-tool, omitted thinking

Server-tool, visible thinking

Server-tool, omitted thinking

Type

Caveats

Medium

Final Attestation

Uh oh!

CLAassistant commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed Bot commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

greptile-apps Bot commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

codecov Bot commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jinwukong commented Sep 11, 2026

Uh oh!

Uh oh!

jinwukong commented Sep 11, 2026

Uh oh!

jinwukong commented Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jinwukong commented Sep 13, 2026

Uh oh!

jinwukong commented Sep 13, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jinwukong commented Sep 13, 2026

Uh oh!

jinwukong commented Sep 19, 2026

Uh oh!

jinwukong commented Sep 19, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jinwukong commented Sep 20, 2026

Uh oh!

jinwukong commented Sep 20, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jinwukong commented Sep 20, 2026

Uh oh!

jinwukong commented Sep 22, 2026

Uh oh!

jinwukong commented Sep 22, 2026

Uh oh!

jinwukong commented Oct 4, 2026

jinwukong commented Sep 11, 2026 •

edited

Loading

Before (`cb138ba`)

After (`f429906`)

CLAassistant commented Sep 11, 2026 •

edited

Loading

codspeed Bot commented Sep 11, 2026 •

edited

Loading

greptile-apps Bot commented Sep 11, 2026 •

edited

Loading

codecov Bot commented Sep 11, 2026 •

edited

Loading

jinwukong commented Sep 11, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading