Skip to content

fix(agent): split concatenated tool_call args in streaming assembler - #63015

Open
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:liuhao/cron-bugfix-62937-gemini-concat-args
Open

fix(agent): split concatenated tool_call args in streaming assembler#63015
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:liuhao/cron-bugfix-62937-gemini-concat-args

Conversation

@liuhao1024

Copy link
Copy Markdown
Contributor

What does this PR do?

When Gemini's OpenAI-compatible endpoint delivers parallel tool calls, it sends them with index=None. The streaming assembler treats all deltas at the same raw index as one call, so the argument fragments get concatenated into a single string (e.g. {"date":"2026-07-01"}{"date":"2026-07-05"}{"date":"2026-07-09"}). json.loads rejects this with "Extra data", _repair_tool_call_arguments can't fix concatenated objects, and replaces the whole blob with {} — silently dropping every parallel tool call. The agent responds claiming it performed actions but nothing actually happens.

This PR adds _split_concatenated_json_objects() which detects the "Extra data" pattern, peels off each top-level JSON object using JSONDecoder.raw_decode, and emits one mock_tool_call per object at assembly time. This recovers all N parallel calls instead of losing all of them. The function returns None for single objects or non-concatenated malformations, so the existing repair path handles those as before.

Related Issue

Fixes #62937

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • agent/message_sanitization.py: Added _split_concatenated_json_objects(raw_args) helper that splits a string of 2+ concatenated top-level JSON objects into individual compact JSON strings; returns None for anything else (single object, garbage, partial concatenation).
  • agent/chat_completion_helpers.py: In interruptible_streaming_api_call's mock_tool_calls assembly loop, when json.loads fails on tool call arguments, try _split_concatenated_json_objects first. On success, emit one mock_tool_call per object (with _splitN id suffixes for uniqueness) and continue. On failure, fall through to the existing _repair_tool_call_arguments path unchanged.
  • run_agent.py: Added _split_concatenated_json_objects to the re-export list (mirrors _repair_tool_call_arguments).

How to Test

  1. Run the unit tests for the new helper and existing repair tests:

    python -m pytest tests/run_agent/test_streaming_tool_call_repair.py tests/run_agent/test_repair_tool_call_arguments.py -q
    

    Observed result: 39 tests should pass (33 existing + 6 new).

  2. Verify the split function handles the exact payload from the issue:

    from run_agent import _split_concatenated_json_objects
    import json
    raw = '{"title":"Opening","date":"2026-07-01"}{"title":"Semi","date":"2026-07-05"}{"title":"Final","date":"2026-07-09"}'
    result = _split_concatenated_json_objects(raw)
    assert len(result) == 3
    assert [json.loads(o)["title"] for o in result] == ["Opening", "Semi", "Final"]

    Observed result: 3 objects correctly split, all individually valid JSON.

  3. Verify non-concatenated malformations still go through the existing repair path (no regression):

    from run_agent import _split_concatenated_json_objects
    assert _split_concatenated_json_objects('{"a":1}') is None  # single object
    assert _split_concatenated_json_objects('{"truncated":') is None  # not concat

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 26.4.1 (arm64)

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Gemini's OpenAI-compat endpoint sends parallel tool calls with index=None,
causing the streaming assembler to merge all argument fragments into a single
entry. json.loads rejects the result with 'Extra data' and the repair pipeline
replaces it with '{}' — silently dropping every parallel call.

Add _split_concatenated_json_objects() to detect this pattern and split it back
into individual tool calls at assembly time. Falls through to the existing
repair path for other malformations.

Fixes NousResearch#62937

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing this to the streaming assembler. The current-main failure is real: index=None is normalized to slot 0 at agent/chat_completion_helpers.py:2389, fragments are appended at :2435-2437, and unrepairable arguments fall through to {} at :2478-2494.

Problems

  • The new recovery emits every decoded object with the one accumulated tool_name (agent/chat_completion_helpers.py:2495 in this diff). The current accumulator overwrites that name whenever a non-empty name delta arrives (:2425-2435), so a mixed-function parallel batch cannot retain its argument-to-function mapping and may dispatch recovered arguments to the wrong function.
  • The added tests cover only the pure splitter. They do not exercise the changed streaming mock_tool_calls assembly path or verify IDs, names, and arguments together.

Suggested changes

  • Preserve per-call metadata for index=None while accumulating, or only recover where a same-function guarantee is established.
  • Add an assembler-level regression with synthetic index=None deltas, including a mixed-function case.

Automated hermes-sweeper review.

type=tc["type"],
extra_content=tc.get("extra_content") if _i == 0 else None,
function=SimpleNamespace(
name=tool_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All recovered objects use the one accumulator-level tool_name. The current assembler overwrites that field for each non-empty name delta, so a mixed-function index=None batch cannot preserve the argument-to-function mapping and may invoke the wrong tool. Preserve per-call names before splitting, or reject recovery unless a same-function guarantee is established.

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/gemini Google Gemini (AI Studio, Cloud Code) P2 Medium — degraded but workaround exists labels Jul 12, 2026
@liuhao1024

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review! I've addressed both concerns:

1. Mixed-function batch guard
Added a guard that checks tool_calls_acc for multiple distinct non-empty function names before splitting. The guard sets has_mixed_functions = True when len(unique_names) > 1, and the split path now requires if split_args and not has_mixed_functions.

This ensures we only split when all parallel calls belong to the same function, preventing incorrect argument dispatch. Mixed-function batches fall through to the repair path instead.

2. Test coverage
Added TestMixedFunctionGuard class with two regression tests:

  • test_mixed_function_batch_skip_split: Verifies the guard correctly identifies mixed functions and skips splitting
  • test_single_function_batch_allows_split: Confirms single-function batches can still split

Also fixed a typo in TestConcatenatedObjectsSplit.test_two_objects (missing quote in JSON).

The fix now handles single-function parallel batches safely while explicitly guarding against the mixed-function edge case you called out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/streaming Streaming responses: gateway delivery, provider wire comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/gemini Google Gemini (AI Studio, Cloud Code) sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Gemini (OpenAI-compat) parallel tool calls are concatenated into one tool_calls entry → arguments dropped as {}

3 participants