Skip to content

fix(gemini): give a new native tool call its own slot instead of colliding on part_index - #75528

Open
jmiguellucas wants to merge 2 commits into
NousResearch:mainfrom
jmiguellucas:fix/gemini-native-parallel-tool-call-slots
Open

fix(gemini): give a new native tool call its own slot instead of colliding on part_index#75528
jmiguellucas wants to merge 2 commits into
NousResearch:mainfrom
jmiguellucas:fix/gemini-native-parallel-tool-call-slots

Conversation

@jmiguellucas

@jmiguellucas jmiguellucas commented Jul 31, 2026

Copy link
Copy Markdown

What does this PR do?

Two different calls to the same tool arriving in separate stream events collide in one
accumulator slot in the native Gemini adapter. Their arguments are emitted under the same
index and concatenated downstream into unparseable JSON ({"query": "A"}{"query": "B"}),
and the call is dropped.

This is the source-side defect pointed at when #72489 was closed under the standing
model-output-repair policy:

The linked discussion on #72488 usefully identifies an upstream native-Gemini
slot-collision candidate in agent/gemini_native_adapter.py:726-765; a narrowly scoped
source-side fix for a verified deterministic adapter defect would be the appropriate
re-scope rather than reconstructing malformed arguments downstream.

That is what this PR is. No repair or reconstruction pass is added — the call boundary is
simply never discarded in the first place.

Where it manifests on current main

In translate_stream_event (agent/gemini_native_adapter.py:718), the dedup key is built
from part_index (742-749):

call_key = json.dumps(
    {
        "part_index": part_index,
        "name": name,
        "thought_signature": thought_signature,
    },
    sort_keys=True,
)

part_index comes from for part_index, part in enumerate(parts) — it restarts at 0 on
every stream event
. Two different calls to the same tool, arriving in two events, produce
an identical call_key and land in the same slot.

The accumulator (759-764) then emits both under one index:

last_arguments = str(slot.get("last_arguments") or "")
if last_arguments:
    if args_str == last_arguments:
        emitted_arguments = ""
    elif args_str.startswith(last_arguments):
        emitted_arguments = args_str[len(last_arguments):]

A genuinely different call is neither equal nor a prefix extension, so the full
{"query": "B"} is emitted under the same index as {"query": "A"}.

The tell: same-event parallel calls work fine — each part gets its own part_index,
hence a distinct key. Only calls split across events collide. If the model were the one
concatenating, both cases would fail identically.

Why this approach

When a payload arrives for an existing slot, is not an extension of what is already there,
and what is already there is a complete JSON object, it is a new call and gets its own slot.
The "already parses as complete JSON" guard is what keeps partial-argument streaming working:
a half-sent object does not parse, so it stays in its slot and keeps accumulating.

This mirrors the existing Ollama workaround in agent/chat_completion_helpers.py
(_last_id_at_idx / _active_slot_by_idx, 3244-3260), which solves the sibling problem of an
endpoint reusing index 0 for a whole parallel batch.

Scope — what this deliberately does not claim

  • The OpenAI-compat path is already covered by that Ollama workaround: a differing id on
    the same raw index already redirects to a fresh slot. This PR does not touch it. Whether
    Gemini's OpenAI-compat endpoint can still collide when it sends index=None and omits ids
    is not something I can demonstrate — I run the native provider and have no trace of that
    endpoint, so I am not asserting it.
  • This does not prove the model never concatenates. It proves there is a path inside Hermes
    that produces the symptom on its own, from well-formed input. Both causes can coexist.
  • On the native path args is always serialized from a complete dict
    (json.dumps(fc.get("args") or {}, ..., sort_keys=True)), so in practice startswith is only
    ever true for an identical resend and _prev always parses. The two guards are therefore
    defensive rather than load-bearing today — they are what keeps this safe if a partial
    argument string ever reaches this accumulator. Behaviour changes only in cases that currently
    produce invalid JSON; every case that works today still works.

Related Issue

Fixes #72488

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • agent/gemini_native_adapter.py — in translate_stream_event, when an incoming payload is
    not an extension of the arguments already accumulated in the slot and those arguments are a
    complete JSON object, allocate a fresh slot instead of reusing the colliding one (16 lines).
  • tests/agent/test_gemini_native_adapter.py — six tests covering the defect and the
    behaviours that must not regress.

How to Test

Deterministic, no network and no model — synthetic well-formed stream events, each carrying one
valid functionCall part.

  1. Check out this branch and revert only the adapter hunk, keeping the new tests:
    git checkout HEAD~1 -- agent/gemini_native_adapter.py
  2. pytest tests/agent/test_gemini_native_adapter.py -q2 failed, 12 passed.
  3. Restore the adapter (git checkout HEAD -- agent/gemini_native_adapter.py) and re-run
    14 passed (8 pre-existing + 6 new).

Test matrix (behaviour on main, without this change):

Test On main
test_same_tool_called_twice_across_events_gets_distinct_slots fails
test_three_calls_to_same_tool_across_events_each_get_a_slot fails
test_parallel_calls_in_one_event_keep_working passes (regression guard)
test_different_tools_across_events_keep_working passes (regression guard)
test_identical_resend_is_still_deduplicated_into_one_slot passes (regression guard)
test_partial_json_arguments_keep_accumulating_in_one_slot passes (regression guard)

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass — partially, see note
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)

Note on the box above: I ran scripts/run_tests.sh tests/agent/ — the directory this change
lives in — on this branch: 325 files, 3351 tests passed, 1 failed in 38.7 min. Three things
worth stating plainly, all checked against main without this change:

  • The one failure is
    test_credential_pool_routing.py::TestFailureAttribution::test_unmatched_key_does_not_retry_only_pool_entry.
    It reproduces identically on main1 failed, 14 passed on both sides, same assertion — so
    it is pre-existing and unrelated.
  • test_compression_concurrent_fork.py hit the runner's 600 s per-file timeout without
    collecting. That was contention: run on its own it is 16 passed in 87 s on this branch, while
    main gives 1 failed, 15 passed on a threading.Event.wait(timeout=5).
  • test_subagent_lifecycle.py was flagged flaky (failed once, passed on retry). Run alone on
    main it passes 3/3.

My box has 2 cores and also runs a production gateway, which is where the last two come from. I
have not run the full ~20k-test suite locally — a full pass pins the machine for hours — so I am
leaving the box unchecked rather than checking it on a partial run. Happy to run any specific
area CI flags.

  • I've tested on my platform: Ubuntu 24.04, Python 3.11, native Gemini provider via the Telegram gateway

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Reproduction against main at 126ff7071 — this branch with the adapter hunk reverted, so the
only thing running is stock translate_stream_event fed synthetic, well-formed stream events:

$ git checkout HEAD~1 -- agent/gemini_native_adapter.py
$ pytest tests/agent/test_gemini_native_adapter.py -q -k across_events

>       assert len(acc) == 2, acc
E       AssertionError: {0: '{"query": "A"}{"query": "B"}'}
E       assert 1 == 2
E        +  where 1 = len({0: '{"query": "A"}{"query": "B"}'})

>       assert len(acc) == 3, acc
E       AssertionError: {0: '{"path": "a"}{"path": "b"}{"path": "c"}'}
E       assert 1 == 3
E        +  where 1 = len({0: '{"path": "a"}{"path": "b"}{"path": "c"}'})

2 failed, 1 passed, 11 deselected in 2.24s

One slot where there should be two, and one where there should be three — with the arguments of
distinct calls concatenated into a string that no JSON parser will accept. Restoring the adapter
hunk gives 14 passed.

This fix has been running in production here since 2026-07-29 (Telegram gateway, native Gemini
provider) with no regressions observed.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating this to the native adapter rather than adding a downstream repair pass. The current remote main still constructs its stream key from the per-event part_index at agent/gemini_native_adapter.py:742-750, so the source-side collision is real.

Problems

  • agent/gemini_native_adapter.py:765 only derives a new suffix from the base key. After calls A then B create base and base#1, a resend of B starts from base again and allocates base#2; it cannot reach base#1 for the existing dedup path. Please preserve deduplication for collision-created slots as well.
  • tests/agent/test_gemini_native_adapter.py:357-368 covers [A, A], but needs [A, B, B] to exercise that secondary-slot replay.

Suggested changes

  • Remove the leftover insertion instructions at tests/agent/test_gemini_native_adapter.py:262-265.

The PR base is only one commit behind remote main and neither modified file changed, so this remains a focused, high-salvageability fix. This is an automated hermes-sweeper review.

Comment thread agent/gemini_native_adapter.py Outdated
except (json.JSONDecodeError, TypeError, ValueError):
pass
else:
call_key = f"{call_key}#{len(tool_call_indices)}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allocates base#N without checking collision-created slots. With events [A, B, B], the second B starts from base (whose arguments are A) and becomes base#2, so it is not deduplicated with base#1. Please retain/locate the secondary slot before allocating another one, and add that replay regression.

@@ -259,3 +259,137 @@ def test_stream_event_translation_emits_tool_call_delta_with_stable_index():



# Bloque de tests para anadir al final de

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove these leftover Spanish instructions for inserting the test block; they are contributor-process notes rather than test documentation.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/gemini Google Gemini (AI Studio, Cloud Code) area/streaming Streaming responses: gateway delivery, provider wire duplicate This issue or pull request already exists labels Jul 31, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Same mechanism as the earliest-open canonical fix #24676 (identical call_key/part_index slot-collision in translate_stream_event, value-based fresh-slot disambiguation with a prefix-continuation guard). Marking as duplicate of #24676; the parallel-Gemini-tool-call fix cluster (#57941, #59871, #54355) already anchors there. Maintainer picks the canonical implementation.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 31, 2026
@jmiguellucas

Copy link
Copy Markdown
Author

Both review points are addressed in f1e0f8c.

[A, B, B] — collision-created slots were unreachable. Confirmed, and it is worse than a missed de-duplication: replaying [A, B, B] emitted three tool calls for two. After A and B open base and base#1, the resent B restarts its lookup at base, whose accumulated arguments are A's, fails to match there, and allocated base#2 instead of reaching base#1.

The lookup now walks the slots already derived from the same key before allocating another one, so a continuation or a resend lands on the slot that call opened. The rule deciding it is unchanged and now lives in one helper, _tool_call_slot_accepts: a slot takes a payload that extends or repeats the arguments it already holds, or that follows arguments which are not yet complete JSON.

Replay regression added. test_resend_of_the_second_call_reuses_its_collision_created_slot replays [A, B, B] and pins the slot count (2), the emitted indices ([0, 1, 1]), the empty argument delta on the resend, and its id matching the call it repeats. It fails on the previous commit with assert 3 == 2 and passes on this one; tests/agent/test_gemini_native_adapter.py is 15 passed.

Leftover notes removed from the top of the test block. I also corrected the _accumulate docstring in the same file: it named run_agent.py as the streaming loop, but the loop that owns tool_call_indices is GeminiNativeClient._stream_completion.

On the duplicate label — happy for a maintainer to pick the canonical implementation. One data point in case it is useful: the 2026-07-13 review on #24676 found that its regression test sends the same complete {"q": ""} dictionary twice, so it exercises equality de-duplication rather than the differing-argument collision, and that PR has not been updated since. This branch now covers both as replay regressions — differing arguments across events, and the resend of a collision-created slot. Whichever implementation is chosen, the case worth keeping in the test suite is the differing-argument one.

CI has never run on this PR: every workflow is action_required, waiting on maintainer approval.

@andrexibiza

Copy link
Copy Markdown
Contributor

Verified against current main (75901a2) as part of the duplicate-cluster triage for #72488. Premise holds and the fix works:

Premise (reproduced, no network): call_key is built from part_index which restarts at 0 on every stream event (agent/gemini_native_adapter.py:742-750), and the accumulator emits the full args string under the same slot index when a payload is neither equal nor a prefix-extension (:759-764). Two synthetic same-name functionCall events in separate events yield slot 0 = {"query": "A"}{"query": "B"} (invalid JSON); three write_file events concatenate all three. Same-event parallel calls (distinct part_index) correctly produce 2 slots — the tell that the collision is adapter-side, not model output.

Fix (cherry-picked onto current main, additive, no conflicts): all three synthetic cases now yield one valid slot per call ({"query": "A"}, {"query": "B"}, …) and the same-event control is unchanged.

Tests (tests/agent/test_gemini_native_adapter.py, current main + this head): 15 passed. The three regression tests here (test_same_tool_called_twice_across_events_gets_distinct_slots, test_three_calls_to_same_tool_across_events_each_get_a_slot, test_resend_of_the_second_call_reuses_its_collision_created_slot) each FAIL when the adapter change is reverted and pass with it — they assert the absence, not just the new behavior.

Green-light from the #72488 cluster triage: this is the source-side fix the #72489 close verdict pointed to (native adapter, not downstream reconstruction).

jmiguellucas and others added 2 commits August 20, 2026 08:26
…iding on part_index

Two different calls to the same tool arriving in separate stream events
collide in one accumulator slot in the native Gemini adapter, because
`call_key` is built from `part_index`, which restarts at 0 on every
event. Their arguments are then emitted under the same index and
concatenated downstream into unparseable JSON (`{"query": "A"}{"query":
"B"}`), and the call is dropped.

Same-event parallel calls are unaffected: each part gets its own
`part_index`, hence a distinct key. Only calls split across events
collide, which is the tell that this concatenation is produced inside
Hermes rather than by the model.

When a payload arrives for an existing slot, is not an extension of the
arguments already accumulated there, and those arguments are a complete
JSON object, it is a new call and gets its own slot. The "already parses
as complete JSON" guard keeps partial-argument streaming working: a
half-sent object does not parse, so it stays in its slot and keeps
accumulating.

Refs NousResearch#72488

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review on NousResearch#75528: after calls A and B open `base` and `base#1`, a resend
of B restarts its lookup from `base`, whose accumulated arguments are
A's. It matched nothing there and allocated `base#2`, so the resent call
was emitted under a third index instead of being deduplicated into the
slot it had already opened — replaying `[A, B, B]` produced three tool
calls for two.

The slot lookup now walks the slots derived from the same key before
allocating another one, so a continuation or a resend lands on the slot
that call opened. The acceptance rule that decides it is unchanged and
now lives in one place: a slot takes a payload that extends or repeats
the arguments it already holds, or that follows arguments which are not
yet complete JSON.

Also drop the leftover contributor notes at the top of the new test
block, and correct the `_accumulate` docstring, which named `run_agent.py`
as the streaming loop instead of `_stream_completion`.

Refs NousResearch#72488

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jmiguellucas
jmiguellucas force-pushed the fix/gemini-native-parallel-tool-call-slots branch from f1e0f8c to 653f626 Compare August 20, 2026 08:42
@jmiguellucas

Copy link
Copy Markdown
Author

Rebased onto current main (27562ad5f); the branch was 4445 commits behind. The fix itself is unchanged — the two adapter hunks are byte-identical to f1e0f8c, the revision @andrexibiza verified.

What the conflict was. 4be4b9866 ("preserve Gemini 3 tool call IDs") rewrites the "id": line inside the same if slot is None: block this PR inserts in front of, so the two edits are adjacent but independent. Both textual conflicts were in tests/agent/test_gemini_native_adapter.py, where that commit's new TestGemini3ToolCallIds class landed at the end of the file alongside this PR's block — resolved by keeping both. No conflict in the adapter; it auto-merged, and upstream's provider-supplied call id stays intact.

The premise still holds on today's main: call_key is still built from part_index, which restarts at 0 on every stream event, and the last_arguments/startswith accumulator is unchanged.

Re-verified against 27562ad5f:

tests/agent/test_gemini_native_adapter.py with the fix 25 passed
same file, adapter alone reverted to main 3 failed, 22 passed
the failures exactly the three regression tests, nothing else
8 adjacent gemini/streaming/tool-call test files 44 passed

So the defect still reproduces on current main, and the three tests fail for the reason they were written rather than detecting a change.

Still purely additive: +198 / −0 across two files, no existing line modified or deleted.

CI has never run on this PR — every workflow sits at action_required, which needs a maintainer to approve a fork run.

@alt-glitch alt-glitch removed sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/streaming Streaming responses: gateway delivery, provider wire comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists provider/gemini Google Gemini (AI Studio, Cloud Code) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gemini 3.5 Flash occasionally concatenates multiple JSON objects into one tool_call instead of emitting separate tool_calls

4 participants