Repository navigation
Conversation
AdaAibaby
requested review from
aarnphm,
bbrowning,
chaunceyjiang and
sfeng33
as code owners
September 23, 2026 12:01
6 tasks done
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
AdaAibaby
force-pushed
the
fix/internlm2-streaming-two-action-blocks
branch
from
October 26, 2026 03:57
325ea68 to
3a4a669
Compare
…nlm2 streaming
Internlm2ToolParser.extract_tool_calls_streaming splits the accumulated
delta on "<|action_start|><|plugin|>" with an unbounded str.split and
unpacks the result into exactly two names:
text, action = new_delta.split("<|action_start|><|plugin|>")
self.position is only advanced past text that precedes the first marker,
and self.current_tool_id never exceeds 0, so the `current_tool_id > 0`
early-return never fires on the tool-call path. Once the model streams a
second "<|action_start|><|plugin|>" block, current_text[last_pos:]
contains two markers, split returns three parts, and the unpack raises
`ValueError: too many values to unpack (expected 2)`. This runs before
the method's try/except (which starts later), so it propagates and aborts
the stream.
Use str.partition to split only on the first marker. The following
`action.split("<|action_end|>")[0]` already isolates the first tool call,
so single-tool behavior is unchanged.
Signed-off-by: shaolila <shaolila@buaa.edu.cn>
AdaAibaby
force-pushed
the
fix/internlm2-streaming-two-action-blocks
branch
from
October 26, 2026 04:06
3a4a669 to
b1d572a
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Internlm2ToolParser.extract_tool_calls_streamingsplits the accumulated text on the tool-call marker with an unboundedstr.splitand unpacks the result into exactly two values:self.positionis only advanced past text that precedes the first marker, andself.current_tool_idnever exceeds0, so theif self.current_tool_id > 0early-return never fires on the tool-call path. Once the model streams a second<|action_start|><|plugin|>block,current_text[last_pos:]contains two markers,splitreturns three parts, and the unpack raises:This line runs before the method's
try/except(which starts a few lines later), so the exception propagates and aborts the streaming request rather than being swallowed.Trigger
A model emitting two consecutive tool-call blocks, e.g. accumulated text:
InternLM2 is documented as not supporting parallel tool calls, but that is a limitation, not a guard — nothing constrains the decoder from emitting a second block, and a misprompted/hallucinating model realistically can.
Fix
Split only on the first marker with
str.partition:The subsequent
action.split("<|action_end|>")[0]already isolates the first tool call, so single-tool-call behavior is unchanged. Verified that for single-block input the extracted(text, action)is identical to the old code.Test
Added
test_streaming_two_action_blocks_no_unpack_errorintests/tool_parsers/test_internlm2_tool_parser.py, which streams two action blocks and asserts the extractor does not raise. It fails onmain(ValueError) and passes with this change.