Skip to content

fix(agent): close the empty tool_calls 400 class at all three chokepoints (#83312, #77921) - #86654

Merged
teknium1 merged 5 commits into
mainfrom
fix-empty-toolcalls-class
Aug 15, 2026
Merged

fix(agent): close the empty tool_calls 400 class at all three chokepoints (#83312, #77921)#86654
teknium1 merged 5 commits into
mainfrom
fix-empty-toolcalls-class

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Fixes #83312. Fixes #77921. Closes #64335-class empty-tool_calls 400s on strict providers (DeepSeek v4, Moonshot/Kimi, Qwen/onerouter) at every layer that can create or forward the invalid shape.

The bug class

Strict OpenAI-compatible providers reject an assistant message carrying tool_calls: [] (or null) with a non-retryable HTTP 400 (Invalid 'messages[N].tool_calls': empty array). Because the poisoned message stays in the replayed in-memory history, every subsequent turn in the session fails the same way — the session is permanently wedged until the agent cache is cleared (#83312 observed this across 6 sessions; #77921 reproduced it three times in v0.19.1 despite the #59110 chokepoint fix).

The #58755/#59110 sanitizer strips tool_calls: [] where it sees it — but two later passes re-create the empty array after the strip has run, and some routes bypass the sanitizer entirely:

  1. sanitize_api_messages tool_call_id dedup (runs after the empty-array drop): when every call on an assistant turn is a duplicate, kept_tcs collapses to [] and was written back as tool_calls: [] — re-creating the exact payload deleted a few passes earlier. This is the mechanism behind DeepSeek 400: assistant messages with empty tool_calls:[] wedge sessions permanently #83312's "sanitizer works in isolation but the array still reaches the wire", and it correlates with the Repaired N message-alternation violations log line because repair_message_sequence's consecutive-assistant merge is what produces the all-duplicates shape.
  2. repair_message_sequence consecutive-assistant merge: the union branch preserved a stale tool_calls: [] on the surviving turn (and the repaired list is the live/persisted trajectory, so the [] was replayed every turn — Still reproducing in v0.19.1: empty tool_calls after repair_message_sequence (follow-up to #58755) #77921's finding that tool_calls = NULL rows become [] on load and survive the merge).
  3. Transport boundary: auxiliary/custom-provider routes can reach ChatCompletionsTransport.convert_messages() without passing through sanitize_api_messages at all.

The fix (three chokepoints, salvaged from the three earliest PRs for each site)

All three commits are cherry-picked with original authorship preserved.

Verification

tests/run_agent/test_message_sequence_repair.py            — 40 passed (incl. 3 new regressions)
tests/agent/transports/test_chat_completions_empty_tool_calls.py — new, passed
tests/agent/transports/test_chat_completions.py            — 46 passed (no regressions)
tests/run_agent/test_run_agent_codex_responses.py          — 47 passed (codex interim-turn exemption intact)

Duplicate/overlapping PRs covered by this class fix

#64345 (salvaged), #77944 (salvaged, was already closed into #78063), #72591 (salvaged), #83600, #83315, #86020, #77377 (agent_runtime_helpers portion), #64843, #82252 — all address subsets of the same class at one of these three sites.

Infographic

empty-toolcalls-class

liuhao1024 and others added 5 commits August 14, 2026 20:51
The dedup pass in sanitize_api_messages (introduced by #58327) can
produce an empty tool_calls array when all tool_call_ids in a message
are duplicates of earlier messages in a long conversation history.

DeepSeek v4 and newer OpenAI reject empty tool_calls with HTTP 400:
'Invalid messages[N].tool_calls: empty array'.

When kept_tcs is empty after dedup, drop the tool_calls key entirely
instead of writing tool_calls: [].

Fixes #64335
Strict OpenAI-compatible providers (onerouter / Qwen, DeepSeek v4) reject
an assistant message carrying tool_calls: [] (or null) with HTTP 400
'Empty tool_calls is not supported in message.'

The pre-API sanitizer in agent_runtime_helpers.sanitize_api_messages already
drops these on the conversation_loop path, but auxiliary / custom-provider
routes that bypass that sanitizer can still reach the wire with an invalid
empty array and abort the whole session (non-retryable 400).

Normalize at the transport layer too: detect an empty-list / null
tool_calls on assistant messages, strip the key on the per-call copy (never
mutate the stored history), and keep real tool_calls untouched. Includes
unit tests covering empty-list, null, real-call preservation, mixed batches,
user-role non-mutation, copy-on-write, and cross-provider parity.

Follow-up to #58755.
…ping, not empty array

Reviewer noted the test name suggested an empty-array case but the fixture
has one tool call; renamed to match actual behavior.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/deepseek DeepSeek API provider/kimi Kimi / Moonshot provider/qwen Qwen / Alibaba Cloud (OAuth) sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Aug 15, 2026
@webtecnica

Copy link
Copy Markdown
Contributor

Thanks @teknium1 for consolidating the empty-tool_calls class into one fix and for salvaging our #77944 (repair-merge site) with authorship preserved — nice to see the three chokepoints (dedup, repair merge, wire boundary) covered from the earliest PRs for each. This closes the class properly!

@github-actions

github-actions Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

૮ >ﻌ< ა ci review

running on 8e73134 — fix(transport): scope empty tool_calls comment to transport-


Still running 1 job: CI timing report

⚠️ Warnings

OSV vulnerability scan · View job

5 known vulnerabilities found in pinned dependencies.

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.

@teknium1
teknium1 merged commit 4e60771 into main Aug 15, 2026
50 checks passed
@teknium1
teknium1 deleted the fix-empty-toolcalls-class branch August 15, 2026 04:25
JoaoMarcos44 added a commit to JoaoMarcos44/hermes-agent that referenced this pull request Aug 16, 2026
Rebased onto current main to drop the empty-tool_calls fix (already on
main via NousResearch#86654, cherry-picked from NousResearch#77944 with @webtecnica's
authorship). This PR now carries only the two fixes unique to it:

1. A pre-existing api_content sidecar left stale on the consecutive-
   assistant merge. The sidecar takes priority over content at
   API-build time, so a merge could silently discard its own freshly
   concatenated content on the next call. Only dropped when the merge
   actually changes the resulting value (wz-heng, NousResearch#78063 review) --
   content_rewritten compares before/after value, not just whether an
   assignment branch fired, so a falsy new_content (e.g. "") that
   strips to nothing no longer trips a spurious sidecar drop.

2. sanitize_api_messages never flagged a tool result with a missing/
   empty tool_call_id -- its orphan-detection set only ever collected
   truthy ids, so an unpaired result with no id passed the final
   chokepoint untouched.

Addresses teknium1's rebase request and wz-heng's review findings on
NousResearch#78063.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/deepseek DeepSeek API provider/kimi Kimi / Moonshot provider/qwen Qwen / Alibaba Cloud (OAuth) sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

5 participants