feat(compression): prune oversized tool results in place before summarization - #85479
Draft
pablo-afterlife wants to merge 1 commit into
Draft
feat(compression): prune oversized tool results in place before summarization#85479pablo-afterlife wants to merge 1 commit into
pablo-afterlife wants to merge 1 commit into
Conversation
…rization Add compression.tool_result_prune (opt-in, default off): oversized tool results (role=tool, content > threshold_chars) are pruned in their OWN message node — same position, same role, same tool_call_id — down to head_chars + marker + tail_chars, as a no-LLM pre-pass that runs BEFORE the summarization region is selected in the in-loop compression. Shrinking oversized tool bodies often makes the transcript fit the tail budget, skipping the summarizer entirely; the commit rewrites the persisted history through the same archive_and_compact mechanism as in-place compaction, so the reclaimed state is durable even when no summarization follows. Marker is a standalone line between blank lines: '[... tool result middle pruned ...]' — pinned verbatim; it also doubles as the immunity marker that keeps the deterministic demote pass from re-replacing an already-pruned result. Conservative by design: enabled defaults to false; an unsatisfiable budget (head + marker + tail > threshold) disables the feature; string bodies are sliced by Unicode code point; list bodies prune only text parts, preserving non-text parts and order; tail protection mirrors the region selection (recent tool output stays verbatim); a bound session store without archive_and_compact (or a failed commit) makes the prune a no-op so the in-memory transcript never drifts from the DB. Ports the pattern from the deepseek-harness compaction tool-result pruner. Tests: unit coverage of the pure prune function (threshold, head/tail, marker, unicode, list content, idempotence), config resolution, node-identity/persistence/rollback of the prune method, and integration of the prune running before region selection and the summarizer (with the feasibility-skip path avoiding the LLM call).
alt-glitch
added
type/feature
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prunes oversized tool results in their own message node — same position, same role, same
tool_call_id— down tohead + marker + tail, as a no-LLM pre-pass that runs BEFORE the summarization region is selected. Shrinking oversized tool bodies often makes the transcript fit the tail budget, skipping the summarizer entirely.Root cause: giant tool outputs (web extraction, terminal dumps) inflate the persisted transcript and force LLM summarization, which invalidates the cached prefix. In-place pruning is the most cache-friendly compaction form: the message count, roles, and call IDs never change — only one node's text shrinks.
Changes
agent/context_compressor.py:PRUNE_MARKER(pinned verbatim, also doubles as immunity marker for the demote pass), pureprune_tool_result_content(code-point safe; supports str and multimodal list bodies, pruning only text parts),resolve_tool_result_prune_config(unsatisfiable budget disables the feature),_tool_prune_boundaryshared by both passes, integration before region selectionagent/agent_init.py: config wiringhermes_cli/config_defaults.py:compression.tool_result_prunedefaults —enabled: false(opt-in),threshold_chars: 8192,head_chars: 4096,tail_chars: 1024tests/agent/test_tool_result_prune.py(new): unit coverage of the pure function (threshold, head/tail, marker, unicode, list content, idempotence), config resolution, node-identity/persistence/rollback, and integration before region selection with the feasibility-skip path avoiding the LLM callValidation
pytest tests/agent/test_tool_result_prune.py tests/agent/test_context_compressor.pyenabled: false) — conservative, opt-in