feat: Add support for multiple tool calls in a single message - #147
Closed
PastaPastaPasta wants to merge 3 commits into
Closed
feat: Add support for multiple tool calls in a single message#147PastaPastaPasta wants to merge 3 commits into
PastaPastaPasta wants to merge 3 commits into
Conversation
- Add XMLParser.parse_all() method to find all occurrences of XML tags - Update ToolEnv.env_response() to execute multiple tools sequentially - Maintain backward compatibility with single tool calls - Add comprehensive test suite covering edge cases and error handling Features: - Parse and execute multiple <tool> tags in one message - Combine results with "Tool N result:" prefix for multiple calls - Handle mixed valid/invalid tool calls gracefully - Support for error recovery (one tool fails, others continue) - Performance tested with 15+ concurrent tool calls 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Replace 'Tool 1 result:', 'Tool 2 result:' with actual tool names - Example: 'add_tool result:', 'search_tool result:' - More semantic and self-documenting for agents - Easier to reference specific tool outputs in complex workflows - Update all tests to expect tool name labels - Maintain single tool backward compatibility (no labels for single calls)
🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
ronaldnetawat
pushed a commit
to ronaldnetawat/verifiers
that referenced
this pull request
Nov 13, 2025
* add xtra logging to shardcast downloader start * add resume with shardcast * make it possible to repeat * seed random correctly when we repeat step * Update src/zeroband/inference.py Co-authored-by: samsja <55492238+samsja@users.noreply.github.com> * clarification comments --------- Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
Member
|
This was added after the PR was opened, thus this PR can be closed |
mikasenghaas
added a commit
that referenced
this pull request
Sep 1, 2026
## Summary - add the same optional `CompactionConfig` to the RLM harness as the bash harness carries - cross ACP with a flat policy: a `compaction` toggle plus `summarize_at_tokens` - pin merged nano-rlm compaction commit `4ef3438` Builds on merged [#2454](#2454), which adds Bash compaction and the interception `/v1/models` relay. Companion [nano-rlm #147](PrimeIntellect-ai/nano-rlm#147) is merged. ## Breaking - `RLMHarnessConfig.summarize_at_tokens` moves to `RLMHarnessConfig.compaction.summarize_at_tokens` and no longer accepts a `(lo, hi)` range. - Leave `compaction` unset to disable proactive and reactive compaction. ## Verification - `uv run pytest -q tests/v1` — passed; live E2E tests skipped without `PRIME_API_KEY` - `uv run pytest -q tests/v1/test_configs.py` — 12 passed - `uv run ruff check verifiers/v1/harnesses/rlm/harness.py` - `uv run ruff format --check verifiers/v1/harnesses/rlm/harness.py` Terminal-Bench 2 e2e: 8 tasks, local vLLM `poolside/Laguna-XS-2.1` at 32k (glm45 reasoning + glm47 tool parsers), `compaction = {}` so the engine discovers the threshold itself (`32768 − 16384 = 16384`). Trace analysis of the pinned engine: - Threshold discovery and the proactive trigger work through ACP: compaction fired on the 3 episodes whose context crossed ~17k; episodes that stayed below (0.7k-6.7k peaks) never compacted; 20KB tool truncation visible where tool output was large. - The runs surfaced and the pin fixes three integration bugs, each verified against the failing trace: `/models` discovery crashing on Python 3.10 containers (raw `cast_to` parse; now `models.list()`), and two interactions with semantic-edge bookkeeping. A failed checkpoint attempt and a resampled unusable reply each left the compaction's summary-request claim held, which killed the retry with "compaction already has a summary request". - Laguna answers checkpoint prompts entirely in the reasoning channel, so under the summaries-are-content-only rule its compactions exercise the resample-then-end-cleanly path; summary carry-over across branches was demonstrated on content-channel models (Qwen3-0.6B, deepseek-v4-flash). - A final combined verification run on a content-channel model is pending before merge. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Breaking harness configuration and ACP runtime policy shape affect rollout behavior and compaction timing; changes are localized to the RLM harness but alter long-running agent context management. > > **Overview** > **RLM harness compaction** is restructured to match the bash harness: a nested `CompactionConfig` with optional `summarize_at_tokens`, exposed over ACP as `policy.compaction` (on/off) plus the threshold when set. > > **Breaking config change:** `RLMHarnessConfig.summarize_at_tokens` is removed in favor of `compaction`; the `(lo, hi)` per-task random range and `summarize_threshold()` are dropped. **`compaction` unset** means compaction is off; an **empty** `compaction` object enables automatic thresholding (e.g. context window minus 16k when advertised). > > The pinned **nano-rlm** ref updates to **`4ef3438`** for the merged compaction engine. `_runtime_metadata` no longer takes `TaskData` since thresholds are no longer task-index–seeded. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 5f39bb2. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- Macroscope's pull request summary starts here --> <!-- Macroscope will only edit the content between these invisible markers, and the markers themselves will not be visible in the GitHub rendered markdown. --> <!-- If you delete either of the start / end markers from your PR's description, Macroscope will append its summary at the bottom of the description. --> > [!NOTE] > ### Add `CompactionConfig` to `RLMHarness` and remove per-task threshold randomization > - Introduces `CompactionConfig` (derived from `BaseConfig`) with an optional `summarize_at_tokens: PositiveInt` field, replacing the flat `summarize_at_tokens` tuple-range field on `RLMHarnessConfig`. > - Removes `RLMHarness.summarize_threshold`, which previously computed per-task randomized thresholds seeded by task index. `_runtime_metadata` now reads thresholds directly from `self.config.compaction` and emits a boolean `compaction` flag in the session policy. > - Changes the default `version` git ref for `RLMHarnessConfig` to `4ef3438`. > - Behavioral Change: `summarize_at_tokens` no longer accepts `(lo, hi)` tuple ranges; callers must provide a single `PositiveInt` inside `CompactionConfig`. The `data` parameter was removed from `RLMHarness._runtime_metadata`. > > <!-- Macroscope's review summary starts here --> > > <sup><a href="https://app.macroscope.com">Macroscope</a> summarized 5f39bb2.</sup> > <!-- Macroscope's review summary ends here --> > <!-- Macroscope's pull request summary ends here -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
<!-- Provide a brief description of the changes in this PR -->
This PR adds support for executing multiple tool calls within a single message, significantly improving efficiency for tool-based environments and agent workflows. Agents can now make multiple tool calls in one turn instead of requiring separate round-trips for each tool.
Type of Change
<!-- Mark the relevant option with an "x" -->
Testing
<!-- Describe the tests you ran to verify your changes -->
python -m pytest tests/Test Coverage
<!-- If applicable, mention the test coverage for new code -->
Checklist
-W errorflags and manual verification)Additional Notes
<!-- Add any additional notes, screenshots, or context about the PR here -->
Key Features
<tool>tags in one messageError Handling Details
When one tool fails among multiple tools:
Example behavior with mixed success/failure:
Technical Implementation
XMLParser.parse_all()method usingre.findall()for multiple tag extractionToolEnv.env_response()to handle sequential tool execution with per-tool error handlingUsage Example
Results in:
Performance
Tested with 15+ concurrent tool calls with no performance degradation. Sequential execution ensures tool state consistency while providing significant efficiency gains for agent workflows.
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com