Skip to content

fix(agent): normalize empty tool-call content for strict chat APIs - #31615

Open
Nigelkil01 wants to merge 1 commit into
NousResearch:mainfrom
Nigelkil01:fix/strict-tool-call-content
Open

Nigelkil01 wants to merge 1 commit into
NousResearch:mainfrom
Nigelkil01:fix/strict-tool-call-content

Conversation

@Nigelkil01

Copy link
Copy Markdown

Summary

  • Normalize empty/whitespace assistant content to null for pure tool-call turns sent through strict Chat Completions paths
  • Apply the same normalization in the strict tool-call sanitizer used before provider replay
  • Add regression coverage for empty, whitespace-only, missing, and substantive assistant content with tool calls

Fixes #31583.

Test Plan

  • ../hermes-upstream-full-20260513-201914/.venv/bin/pytest -o addopts='' tests/run_agent/test_strict_api_validation.py -q
  • ../hermes-upstream-full-20260513-201914/.venv/bin/pytest -o addopts='' tests/run_agent/test_provider_parity.py -q
  • ../hermes-upstream-full-20260513-201914/.venv/bin/pytest -o addopts='' tests/agent/transports/test_transport.py -q
  • ../hermes-upstream-full-20260513-201914/.venv/bin/python -m py_compile run_agent.py agent/transports/chat_completions.py tests/run_agent/test_strict_api_validation.py
  • git diff --check

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 24, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

Related: #31175 (same root cause — empty assistant content sanitization in chat_completions.py). This PR also patches run_agent.py sanitizer and adds regression tests. Likely supersedes #31175.

@Nigelkil01

Copy link
Copy Markdown
Author

Maintainer ping: can you confirm which PR you want to carry this fix — #31615 or #31582?

#31615 was intended to supersede #31175. It fixes the same empty-tool-call-content bug, but also covers the strict run_agent.py sanitizer and adds regression tests. Right now there are multiple overlapping PRs for the same bug, so I don’t want to keep rebasing the wrong one.

If #31615 is the preferred path, I’ll rebase it and clean up the conflicts. If not, I’m happy to fold the work into the PR you want to keep.

@poorel

poorel commented Jul 6, 2026

Copy link
Copy Markdown

I independently reproduced this against a Bedrock-backed OpenAI-compatible Claude/Sonnet endpoint, and this transport-layer normalization matches the observed provider behavior.

Repro shape:

{
  "role": "assistant",
  "content": "",
  "tool_calls": [
    {
      "id": "call_probe_123",
      "type": "function",
      "function": {
        "name": "probe_tool",
        "arguments": "{\"x\":1}"
      }
    }
  ]
}

When replayed through the Bedrock-backed Claude/Sonnet OpenAI-compatible gateway, the request fails with:

ValidationException: messages: text content blocks must be non-empty

Changing only the assistant tool-call turn from content: "" to content: null makes the same request succeed.

I also verified this in a real Hermes tool-call flow using a custom OpenAI-compatible endpoint routing to Claude/Sonnet on Bedrock: the model called the terminal tool, the tool result was replayed, and the follow-up assistant response succeeded after normalizing the empty tool-call content to null.

So this affects custom OpenAI-compatible endpoints that route Claude/Sonnet traffic to AWS Bedrock, not just synthetic strict-schema validators.

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for the focused strict-provider compatibility fix. The underlying bug is still present on current main: agent/chat_completion_helpers.py:1077 constructs empty assistant content, and :1202-1266 attaches the tool calls; agent/transports/chat_completions.py:196-210 currently returns that otherwise-clean history unchanged.

Problems

  • The transport has since moved to copy-on-write sanitization. The proposed direct mutation needs to be ported through needs_sanitize and mutable_msg() in agent/transports/chat_completions.py:180-255; otherwise the identity fast path at :209-210 remains active.
  • The submitted tests exercise _build_api_kwargs(), but the max-iteration summary request bypasses the transport and instead calls _sanitize_tool_calls_for_strict_api() at agent/chat_completion_helpers.py:1688-1711.

Suggested changes

  • Salvage the normalization into both current outgoing boundaries and add direct regression coverage for each boundary, including empty, whitespace-only, missing, substantive, and no-tool-call content.

Automated hermes-sweeper review.

@alt-glitch alt-glitch added provider/anthropic Anthropic native Messages API provider/bedrock AWS Bedrock (boto3, IAM) needs-decision Awaiting maintainer decision before any implementation labels Jul 13, 2026
@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jul 13, 2026
History stores a textless tool-call turn as content:"" (build_assistant_message
coerces absent text). Strict OpenAI-compatible validators reject that beside
tool_calls — Bedrock-backed Claude via a compatible gateway fails with
"messages: text content blocks must be non-empty"; Mistral/Fireworks 400 — and
null is the schema-compatible form.

Normalize at both outgoing boundaries (per the sweeper review on NousResearch#31615):
the transport's copy-on-write sanitizer (`_sanitize_message`, used by
convert_messages/build_kwargs) and `_sanitize_tool_calls_for_strict_api`, which
the max-iteration summary request uses instead of the transport. One shared
predicate, `_blank_tool_call_content`; a missing content key is already valid
and left alone; substantive text and tool-call-free turns are untouched; the
history copy is never mutated. The mapping is deterministic on every send, so
prompt-cache prefixes stay byte-stable across turns.

The send-time placeholder pad still exempts tool-call turns (the tool_calls are
the payload); its test now asserts that instead of content == "".

Regression: tests/agent/transports/test_chat_completions_empty_tool_call_content.py
covers empty, whitespace-only, missing, substantive and no-tool-call content at
both boundaries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@Nigelkil01
Nigelkil01 force-pushed the fix/strict-tool-call-content branch from 805eda0 to 268c5a2 Compare September 21, 2026 00:15

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists provider/anthropic Anthropic native Messages API provider/bedrock AWS Bedrock (boto3, IAM) sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Empty assistant content with tool_calls causes HTTP 400 from strict OpenAI-compat upstreams

4 participants