fix(agent): split concatenated tool_call args in streaming assembler - #63015
fix(agent): split concatenated tool_call args in streaming assembler#63015liuhao1024 wants to merge 1 commit into
Conversation
Gemini's OpenAI-compat endpoint sends parallel tool calls with index=None,
causing the streaming assembler to merge all argument fragments into a single
entry. json.loads rejects the result with 'Extra data' and the repair pipeline
replaces it with '{}' — silently dropping every parallel call.
Add _split_concatenated_json_objects() to detect this pattern and split it back
into individual tool calls at assembly time. Falls through to the existing
repair path for other malformations.
Fixes NousResearch#62937
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tracing this to the streaming assembler. The current-main failure is real: index=None is normalized to slot 0 at agent/chat_completion_helpers.py:2389, fragments are appended at :2435-2437, and unrepairable arguments fall through to {} at :2478-2494.
Problems
- The new recovery emits every decoded object with the one accumulated
tool_name(agent/chat_completion_helpers.py:2495in this diff). The current accumulator overwrites that name whenever a non-empty name delta arrives (:2425-2435), so a mixed-function parallel batch cannot retain its argument-to-function mapping and may dispatch recovered arguments to the wrong function. - The added tests cover only the pure splitter. They do not exercise the changed streaming
mock_tool_callsassembly path or verify IDs, names, and arguments together.
Suggested changes
- Preserve per-call metadata for
index=Nonewhile accumulating, or only recover where a same-function guarantee is established. - Add an assembler-level regression with synthetic
index=Nonedeltas, including a mixed-function case.
Automated hermes-sweeper review.
| type=tc["type"], | ||
| extra_content=tc.get("extra_content") if _i == 0 else None, | ||
| function=SimpleNamespace( | ||
| name=tool_name, |
There was a problem hiding this comment.
All recovered objects use the one accumulator-level tool_name. The current assembler overwrites that field for each non-empty name delta, so a mixed-function index=None batch cannot preserve the argument-to-function mapping and may invoke the wrong tool. Preserve per-call names before splitting, or reject recovery unless a same-function guarantee is established.
|
Thanks for the thorough review! I've addressed both concerns: 1. Mixed-function batch guard This ensures we only split when all parallel calls belong to the same function, preventing incorrect argument dispatch. Mixed-function batches fall through to the repair path instead. 2. Test coverage
Also fixed a typo in The fix now handles single-function parallel batches safely while explicitly guarding against the mixed-function edge case you called out. |
What does this PR do?
When Gemini's OpenAI-compatible endpoint delivers parallel tool calls, it sends them with
index=None. The streaming assembler treats all deltas at the same raw index as one call, so the argument fragments get concatenated into a single string (e.g.{"date":"2026-07-01"}{"date":"2026-07-05"}{"date":"2026-07-09"}).json.loadsrejects this with "Extra data",_repair_tool_call_argumentscan't fix concatenated objects, and replaces the whole blob with{}— silently dropping every parallel tool call. The agent responds claiming it performed actions but nothing actually happens.This PR adds
_split_concatenated_json_objects()which detects the "Extra data" pattern, peels off each top-level JSON object usingJSONDecoder.raw_decode, and emits onemock_tool_callper object at assembly time. This recovers all N parallel calls instead of losing all of them. The function returnsNonefor single objects or non-concatenated malformations, so the existing repair path handles those as before.Related Issue
Fixes #62937
Type of Change
Changes Made
agent/message_sanitization.py: Added_split_concatenated_json_objects(raw_args)helper that splits a string of 2+ concatenated top-level JSON objects into individual compact JSON strings; returnsNonefor anything else (single object, garbage, partial concatenation).agent/chat_completion_helpers.py: Ininterruptible_streaming_api_call'smock_tool_callsassembly loop, whenjson.loadsfails on tool call arguments, try_split_concatenated_json_objectsfirst. On success, emit onemock_tool_callper object (with_splitNid suffixes for uniqueness) andcontinue. On failure, fall through to the existing_repair_tool_call_argumentspath unchanged.run_agent.py: Added_split_concatenated_json_objectsto the re-export list (mirrors_repair_tool_call_arguments).How to Test
Run the unit tests for the new helper and existing repair tests:
Observed result: 39 tests should pass (33 existing + 6 new).
Verify the split function handles the exact payload from the issue:
Observed result: 3 objects correctly split, all individually valid JSON.
Verify non-concatenated malformations still go through the existing repair path (no regression):
Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/A