Skip to content

feat: Add support for multiple tool calls in a single message - #147

Closed
PastaPastaPasta wants to merge 3 commits into
PrimeIntellect-ai:mainfrom
PastaPastaPasta:feat/multiple-tool-calls
Closed

feat: Add support for multiple tool calls in a single message#147
PastaPastaPasta wants to merge 3 commits into
PrimeIntellect-ai:mainfrom
PastaPastaPasta:feat/multiple-tool-calls

Conversation

@PastaPastaPasta

Copy link
Copy Markdown

Description

<!-- Provide a brief description of the changes in this PR -->

This PR adds support for executing multiple tool calls within a single message, significantly improving efficiency for tool-based environments and agent workflows. Agents can now make multiple tool calls in one turn instead of requiring separate round-trips for each tool.

Type of Change

<!-- Mark the relevant option with an "x" -->

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Test improvement

Testing

<!-- Describe the tests you ran to verify your changes -->

  • All existing tests pass
  • New tests have been added to cover the changes
  • Tests have been run locally with python -m pytest tests/

Test Coverage

<!-- If applicable, mention the test coverage for new code -->

  • Current coverage: 100% for new functionality
  • Coverage after changes: Comprehensive edge case coverage including error handling, malformed input, and performance scenarios

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings (tested with -W error flags and manual verification)
  • Any dependent changes have been merged and published

Additional Notes

<!-- Add any additional notes, screenshots, or context about the PR here -->

Key Features

  • Multiple tool execution: Parse and execute multiple <tool> tags in one message
  • Backward compatibility: Single tool calls work exactly as before (no breaking changes)
  • Error resilience: If one tool fails, others continue executing
  • Smart formatting: Multiple results labeled with tool names for clarity

Error Handling Details

When one tool fails among multiple tools:

  1. Execution continues: Remaining tools are still executed sequentially
  2. Error isolation: Failed tool returns error message, but doesn't stop processing
  3. Complete results: All results (successful and failed) are included in the response
  4. Clear identification: Each tool result is labeled with the actual tool name

Example behavior with mixed success/failure:

add_tool result:
15

invalid_tool result:
Error: Unknown tool 'invalid_tool'. Please format your tool call as...

search_tool result:
Found results for: example query

Technical Implementation

  • Added XMLParser.parse_all() method using re.findall() for multiple tag extraction
  • Enhanced ToolEnv.env_response() to handle sequential tool execution with per-tool error handling
  • Tool results labeled with actual tool names (e.g., "add_tool result:" vs "Tool 1 result:")
  • Maintains state consistency through sequential execution
  • Comprehensive error handling for mixed valid/invalid tool scenarios

Usage Example

<think>I need to use multiple tools efficiently</think>
<tool>{"name": "search_tool", "args": {"query": "example"}}</tool>
<tool>{"name": "calculate_tool", "args": {"a": 5, "b": 10}}</tool>
<tool>{"name": "format_tool", "args": {"text": "result"}}</tool>

Results in:

search_tool result:
Search results for example

calculate_tool result:
15

format_tool result:
Formatted: RESULT

Performance

Tested with 15+ concurrent tool calls with no performance degradation. Sequential execution ensures tool state consistency while providing significant efficiency gains for agent workflows.

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

PastaPastaPasta and others added 2 commits July 17, 2025 23:31
- Add XMLParser.parse_all() method to find all occurrences of XML tags
- Update ToolEnv.env_response() to execute multiple tools sequentially
- Maintain backward compatibility with single tool calls
- Add comprehensive test suite covering edge cases and error handling

Features:
- Parse and execute multiple <tool> tags in one message
- Combine results with "Tool N result:" prefix for multiple calls
- Handle mixed valid/invalid tool calls gracefully
- Support for error recovery (one tool fails, others continue)
- Performance tested with 15+ concurrent tool calls

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Replace 'Tool 1 result:', 'Tool 2 result:' with actual tool names
- Example: 'add_tool result:', 'search_tool result:'
- More semantic and self-documenting for agents
- Easier to reference specific tool outputs in complex workflows
- Update all tests to expect tool name labels
- Maintain single tool backward compatibility (no labels for single calls)
@CLAassistant

CLAassistant commented Jul 18, 2025

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
ronaldnetawat pushed a commit to ronaldnetawat/verifiers that referenced this pull request Nov 13, 2025
* add xtra logging to shardcast downloader start

* add resume with shardcast

* make it possible to repeat

* seed random correctly when we repeat step

* Update src/zeroband/inference.py

Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>

* clarification comments

---------

Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
@xeophon

xeophon commented Jul 13, 2026

Copy link
Copy Markdown
Member

This was added after the PR was opened, thus this PR can be closed

@xeophon xeophon closed this Jul 13, 2026
mikasenghaas added a commit that referenced this pull request Sep 1, 2026
## Summary

- add the same optional `CompactionConfig` to the RLM harness as the
bash harness carries
- cross ACP with a flat policy: a `compaction` toggle plus
`summarize_at_tokens`
- pin merged nano-rlm compaction commit `4ef3438`

Builds on merged
[#2454](#2454), which
adds Bash compaction and the interception `/v1/models` relay.
Companion [nano-rlm
#147](PrimeIntellect-ai/nano-rlm#147) is merged.

## Breaking

- `RLMHarnessConfig.summarize_at_tokens` moves to
`RLMHarnessConfig.compaction.summarize_at_tokens` and no longer accepts
a `(lo, hi)` range.
- Leave `compaction` unset to disable proactive and reactive compaction.

## Verification

- `uv run pytest -q tests/v1` — passed; live E2E tests skipped without
`PRIME_API_KEY`
- `uv run pytest -q tests/v1/test_configs.py` — 12 passed
- `uv run ruff check verifiers/v1/harnesses/rlm/harness.py`
- `uv run ruff format --check verifiers/v1/harnesses/rlm/harness.py`

Terminal-Bench 2 e2e: 8 tasks, local vLLM `poolside/Laguna-XS-2.1` at
32k (glm45 reasoning + glm47 tool parsers), `compaction = {}` so the
engine discovers the threshold itself (`32768 − 16384 = 16384`). Trace
analysis of the pinned engine:

- Threshold discovery and the proactive trigger work through ACP:
compaction fired on the 3 episodes whose context crossed ~17k; episodes
that stayed below (0.7k-6.7k peaks) never compacted; 20KB tool
truncation visible where tool output was large.
- The runs surfaced and the pin fixes three integration bugs, each
verified against the failing trace: `/models` discovery crashing on
Python 3.10 containers (raw `cast_to` parse; now `models.list()`), and
two interactions with semantic-edge bookkeeping. A failed checkpoint
attempt and a resampled unusable reply each left the compaction's
summary-request claim held, which killed the retry with "compaction
already has a summary request".
- Laguna answers checkpoint prompts entirely in the reasoning channel,
so under the summaries-are-content-only rule its compactions exercise
the resample-then-end-cleanly path; summary carry-over across branches
was demonstrated on content-channel models (Qwen3-0.6B,
deepseek-v4-flash).
- A final combined verification run on a content-channel model is
pending before merge.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Breaking harness configuration and ACP runtime policy shape affect
rollout behavior and compaction timing; changes are localized to the RLM
harness but alter long-running agent context management.
> 
> **Overview**
> **RLM harness compaction** is restructured to match the bash harness:
a nested `CompactionConfig` with optional `summarize_at_tokens`, exposed
over ACP as `policy.compaction` (on/off) plus the threshold when set.
> 
> **Breaking config change:** `RLMHarnessConfig.summarize_at_tokens` is
removed in favor of `compaction`; the `(lo, hi)` per-task random range
and `summarize_threshold()` are dropped. **`compaction` unset** means
compaction is off; an **empty** `compaction` object enables automatic
thresholding (e.g. context window minus 16k when advertised).
> 
> The pinned **nano-rlm** ref updates to **`4ef3438`** for the merged
compaction engine. `_runtime_metadata` no longer takes `TaskData` since
thresholds are no longer task-index–seeded.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
5f39bb2. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- Macroscope's pull request summary starts here -->
<!-- Macroscope will only edit the content between these invisible
markers, and the markers themselves will not be visible in the GitHub
rendered markdown. -->
<!-- If you delete either of the start / end markers from your PR's
description, Macroscope will append its summary at the bottom of the
description. -->
> [!NOTE]
> ### Add `CompactionConfig` to `RLMHarness` and remove per-task
threshold randomization
> - Introduces `CompactionConfig` (derived from `BaseConfig`) with an
optional `summarize_at_tokens: PositiveInt` field, replacing the flat
`summarize_at_tokens` tuple-range field on `RLMHarnessConfig`.
> - Removes `RLMHarness.summarize_threshold`, which previously computed
per-task randomized thresholds seeded by task index. `_runtime_metadata`
now reads thresholds directly from `self.config.compaction` and emits a
boolean `compaction` flag in the session policy.
> - Changes the default `version` git ref for `RLMHarnessConfig` to
`4ef3438`.
> - Behavioral Change: `summarize_at_tokens` no longer accepts `(lo,
hi)` tuple ranges; callers must provide a single `PositiveInt` inside
`CompactionConfig`. The `data` parameter was removed from
`RLMHarness._runtime_metadata`.
>
> <!-- Macroscope's review summary starts here -->
>
> <sup><a href="https://app.macroscope.com">Macroscope</a> summarized
5f39bb2.</sup>
> <!-- Macroscope's review summary ends here -->
>
<!-- Macroscope's pull request summary ends here -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants