Skip to content

fix(agent): coordinate truncated tool call argument repair - #56399

Closed
konsisumer wants to merge 1 commit into
NousResearch:mainfrom
konsisumer:fix/tool-call-arg-truncation
Closed

konsisumer wants to merge 1 commit into
NousResearch:mainfrom
konsisumer:fix/tool-call-arg-truncation

Conversation

@konsisumer

Copy link
Copy Markdown
Contributor

What does this PR do?

Coordinates truncated tool-call argument handling around one repair contract: repairable malformed JSON continues normally, while unrepairable truncated JSON is reported back to the model as a tool error instead of executing with {} or aborting prematurely.

Related Issue

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Security fix
  • Documentation update
  • Tests (adding or improving test coverage)
  • Refactor (no behavior change)
  • New skill (bundled or hub)

Shared root cause

  • agent/message_sanitization.py returned only repaired argument text, so callers could not distinguish a legitimate empty {} from the last-resort {} used after failed repair.
  • agent/conversation_loop.py had a separate invalid-JSON truncation branch that rejected router-mislabeled truncated tool calls before trying the shared repair routine.
  • Streaming and pre-request transcript sanitation therefore made inconsistent decisions for the same broken tool-call arguments.

How this fixes each issue

Changes Made

  • Added repair_tool_call_arguments_with_status() so callers can tell whether repair succeeded even when the returned JSON is {}.
  • Reused that status-bearing repair in the conversation loop, streaming assembly, and pre-request tool-call argument sanitizer.
  • Converted unrepairable truncated live tool calls into assistant/tool recovery messages instead of a partial abort.
  • Added regression coverage for repair success/failure status, pre-request sanitizer behavior, and router-mislabeled truncated tool calls.

How to Test

  1. HERMES_HOME=/private/tmp/hermes-test-home /opt/homebrew/bin/timeout -k 30 480 sh -c 'pytest tests/run_agent/test_repair_tool_call_arguments.py tests/run_agent/test_tool_call_args_sanitizer.py tests/run_agent/test_run_agent.py -q -x --timeout=60'
  2. BASE=$(git merge-base origin/main HEAD); { git diff --name-only --diff-filter=d "$BASE"; git ls-files --others --exclude-standard; } | grep -E '\.pyi?$' | sort -u | xargs ruff check
  3. python scripts/check-windows-footguns.py run_agent.py agent/message_sanitization.py agent/agent_runtime_helpers.py agent/chat_completion_helpers.py agent/conversation_loop.py tests/run_agent/test_repair_tool_call_arguments.py tests/run_agent/test_tool_call_args_sanitizer.py tests/run_agent/test_run_agent.py
  4. Broad suite attempted with /opt/homebrew/bin/timeout -k 30 480 sh -c 'pytest tests/ -q -x --timeout=60 "$@"' sh; it aborted during collection because fastapi/uvicorn are not installed and this Homebrew Python is externally managed, so the lazy dependency installer cannot install them.

What platforms tested on

  • macOS on darwin-arm64 (local)

Checklist

Code

  • I've read the Contributing Guide / AGENTS instructions
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched the provided issue cluster context for duplicate/member PRs
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass; broad collection is blocked locally by missing dashboard dependencies as noted above
  • I've added tests for my changes
  • I've tested on my platform: macOS darwin-arm64

Documentation & Housekeeping

  • Documentation update N/A
  • cli-config.yaml.example update N/A
  • CONTRIBUTING.md / AGENTS.md update N/A
  • Cross-platform impact considered; changed-file Windows footgun scan passed
  • Tool descriptions/schemas update N/A

Screenshots / Logs

  • Focused tests: 446 passed
  • Ruff changed files: passed (with a pre-existing warning about run_agent.py:107 # noqa directive format)
  • Windows footgun scan: passed

Refs #35151
Refs #35574

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Jul 1, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: LGTM

Improves tool call argument repair by distinguishing successful normalization from unrepairable failures. Truncated args now return tool errors instead of empty-object execution.

Looks Good

  • ToolCallArgumentsRepair dataclass cleanly separates repair status
  • Truncated args now return model-visible errors instead of silent empty execution
  • Repairable truncations still get one attempt before fallback
  • Comprehensive tests covering repair success, failure, and truncation paths

Reviewed by Hermes Agent

swissly added a commit to swissly/hermes-agent that referenced this pull request Jul 11, 2026
Structured stats collection for the existing tool-call repair pipeline.
Records RepairEvent (pattern, tool, model, timestamp) at each repair
pass in message_sanitization.py and model_tools.py coerce_tool_args.

New module: agent/tool_repair_stats.py
- RepairPattern enum (20 known failure patterns)
- ToolRepairStats singleton: thread-safe, ring-buffer (10k events)
- record_repair() convenience function
- summary() for CLI display

Hooks added (1-2 lines each, zero-overhead when unused):
- message_sanitization.py: 6 hooks in _repair_tool_call_arguments
  (empty_args, none_literal, control_char_escape, trailing_comma,
   unrepairable)
- model_tools.py: 2 hooks in coerce_tool_args
  (bare_string_wrap, bare_object_wrap)

Design constraints:
- No new model tools (zero API cost impact)
- No prompt caching impact
- No new config keys
- Import failure → no-op (never breaks repair pipeline)
- Thread-safe with threading.Lock
- Bounded memory (ring buffer caps at 10k events)

Tests: 19 new tests (stats, thread-safety, ring-buffer, resilience)
Regression: 82 existing repair/coercion tests still pass

Complementary to existing repair PRs (NousResearch#62578, NousResearch#56399, NousResearch#61550, NousResearch#59267,
NousResearch#52747, NousResearch#55620, NousResearch#56557, NousResearch#21696) — adds observability, not repairs.
@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for the careful investigation and focused tests.

This automated hermes-sweeper review is closing the PR as not planned under the standing model-output-repair policy.

  • The PR adds new reconstruction/repair behavior for malformed and truncated model-emitted tool-call JSON, including a new trailing-comma repair and routing repairable truncations through execution.
  • Current main instead fail-closes at the execution boundary: agent/tool_executor.py:78-94 accepts only a valid JSON object and produces a model-visible Invalid tool arguments result stating that the tool was not executed.
  • Commit 5e50f18b (fix(agent): reject malformed tool call arguments (#61784)) is an ancestor of current main; tests/run_agent/test_malformed_tool_arguments.py:50-98 covers truncated arguments and verifies they are not dispatched while valid siblings still run.
  • The linked issue [RFC] Attempt JSON repair on truncated tool call args before aborting #35574 was already closed under this same policy. As its maintainer comment notes, a focused change that preserves invalid-argument rejection while improving a model-visible recovery/error path, without reconstructing malformed content, can be proposed separately.

This is a design-direction decision, not a judgment on the quality of the contribution. Thanks again for the clear root-cause analysis and regression coverage.


Automated hermes-sweeper review.


Closed as not-planned per standing maintainer policy (model-output-repair). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.

@teknium1 teknium1 closed this Jul 15, 2026
@teknium1 teknium1 added the sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) label Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants