Skip to content

Fix dropped Inkling reasoning at stream end - #31787

Merged
ispobock merged 3 commits into
mainfrom
fix-inkling-stream-end-reasoning-flush
Jul 20, 2026
Merged

ispobock merged 3 commits into
mainfrom
fix-inkling-stream-end-reasoning-flush

Conversation

@ispobock

@ispobock ispobock commented Jul 20, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

With stream_reasoning=False, the Inkling reasoning detector buffers the thinking block into _pending_reasoning and only flushes it when a control/end token closes the block. When generation is cut mid-block (e.g. max_tokens truncates a <|content_thinking|> block), the stream ends with no end token, so the buffered trace is dropped entirely — the whole reasoning disappears from the streamed response.

The non-streaming detect_and_parse path already flushes _pending_reasoning at the end, but InklingDetector had no finish() override, so at stream end parse_stream_end() fell through to the base finish() (which only handles _force_nonempty_content/_in_reasoning, neither used by this detector) and emitted nothing.

Modifications

  • Override InklingDetector.finish() to emit any buffered _pending_reasoning (when mid-reasoning under stream_reasoning=False) and reset the detector's block state, mirroring the non-streaming flush.
  • Add a regression test that streams a thinking block truncated before its end token and asserts finish() surfaces the trailing reasoning (fails on the pre-fix code, passes after).

Verified on a live server that BFCL live tool-calling accuracy is unchanged (tool_parse_errors=0), i.e. the added stream-end flush does not perturb the tool-call path.


CI States

Latest PR Test (Base): 🚫 Run #29753147981
Latest PR Test (Extra): ❌ Run #29753146335

@ispobock
ispobock requested a review from JustinTong0323 as a code owner July 20, 2026 14:21
@ispobock

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/unit/parser/test_reasoning_parser.py

@github-actions

github-actions Bot commented Jul 20, 2026 •

Copy link
Copy Markdown
Contributor

Results for /rerun-test test/registered/unit/parser/test_reasoning_parser.py:

🚀 ubuntu-latest (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/parser/test_reasoning_parser.py

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a finish method to the reasoning parser to flush buffered reasoning when stream_reasoning=False and the stream ends before a control token closes the block, and includes a unit test to verify this behavior. The reviewer points out that the base class BaseReasoningFormatDetector suffers from the same issue and suggests updating its finish method to ensure consistent behavior across all reasoning detectors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread python/sglang/srt/parser/reasoning_parser.py
@ispobock ispobock added the run-ci CI: run the baseline test suite on this PR label Jul 20, 2026
@ispobock
ispobock merged commit 5ab3d90 into main Jul 20, 2026
75 of 92 checks passed
@ispobock
ispobock deleted the fix-inkling-stream-end-reasoning-flush branch July 20, 2026 15:23
Zhylkaaa pushed a commit to Zhylkaaa/sglang that referenced this pull request Jul 29, 2026
Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant