Conversation
|
Verification on commit
No live tool handler or external service was invoked; all side-effect assertions use mocked dispatch and synthetic |
Related to #87985: both prevent compacted historical content from reaching effectful calls, while this PR adds the model-tools/registry read-only metadata path and middleware recheck. They are complementary rather than duplicates. |
trevorgordon981
left a comment
There was a problem hiding this comment.
Strong, well-scoped fix for a genuinely nasty failure mode: after context compression, a model turn could echo a compactor's synthetic marker back into a side-effecting tool call, persisting truncated/corrupt content as if it were the real thing. Blocking that is the right call, and the design is careful.
Detection is precise, not a blunt substring match. _SYNTHETIC_TRUNCATION_MARKER_RE requires either the compressor's exact sentinel (⟪HERMES-CONTEXT-COMPRESSION: …⟫) or an ellipsis-bracket shape (...[truncated]), and _contains_synthetic_truncation_marker walks nested dicts/lists/tuples/sets with cycle protection, so a marker buried in a nested arg is caught. The false-positive tests are the right ones: test_legitimate_truncation_prose_is_not_blocked ("…the preview was truncated by the UI" — the .../[truncated] adjacency requirement is what distinguishes it, correctly) and test_read_only_tool_is_not_blocked_by_compaction_marker (a legit path containing ... [truncated]).
Fail-closed across the whole write surface. read_only metadata is added to ToolEntry (registry.py:221, read_only = read_only is True — anything other than exactly True falls through as write-capable), and it's plumbed through read_file/search_files/web_search/web_extract plus MCP readOnlyHint/resource-tool inference. So unknown/plugin tools, MCP tools without a read hint, and any future tool fail closed by default — the safe direction. The gate runs at three dispatch boundaries (_return_bridge_result:1519, and both _dispatch closures at 1618/1629) so a marker can't slip in via execution middleware either (covered by test_execution_middleware_cannot_inject_marker_into_write).
Two small notes (non-blocking):
- The check happens after some argument normalization but the block path correctly returns before any handler runs, and
_emit_post_tool_call_hookis called withstatus="blocked"+error_type="compacted_tool_arguments"— good observability. Thetest_guard_does_not_mutate_existing_role_alternationtest also pins that dropping the call doesn't corrupt the message history. _SYNTHETIC_TRUNCATION_MARKER_REis now the second encoder-aware copy of the compressor's marker format (the compressor emits it, this scans for it). If the marker shape ever changes, both must change in lockstep — worth a cross-referencing comment or a shared constant. Minor.
Tests pass at head (test_model_tools.py, test_registry.py) and the MCP-trust/registry updates are consistent. Merge-ready from my side.
|
Production hit of this exact boundary on v0.21.0, with a write target that isn't a file — posting it because the failure mode is silent rather than loud. A Timeline from Control: a fresh 803-char prompt in a single call stored intact (803, no marker). Consistent with this PR's framing that the transport is fine and reuse of compacted history is the unsafe boundary. The cron handler stores verbatim and has no length cap, so it faithfully persisted the damaged value. Why this target class is worse than the file cases in #83714: The fail-closed default for unknown tools already covers |
kshitijk4poor
left a comment
There was a problem hiding this comment.
PR Review — #96592
Verdict: Request changes / do not merge as-is. The bug is real and reproduced on main, the guard tests are meaningful (mutation-verified), but the design turns a targeted replay guard into a global content filter with a high false-positive rate and no override. A maintainer choice between this and #87985 (same bug, agent layer) is needed first.
Premise (verified on origin/main): agent/context_compressor.py:1265-1283 _truncate_tool_call_args_json still shrinks string leaves of historical tool_calls[].function.arguments to 200 chars + ...[truncated] (via _truncate_tool_call_args_at :2618-2630 for args > 500 chars), leaving valid JSON a later turn can replay into write_file/patch. No guard exists on main; tools/registry.py has no read_only field.
Conflicts: model_tools.py, tools/file_tools.py, tests/tools/test_mcp_trust_gating.py — mechanical (main moved the dispatch pre-checks into _pre_dispatch_guards); port applied for testing, 88 passed. Mutation (revert model_tools.py guard to main): 5 red — the guard tests bite. 4 of the 9 tests (byte-for-byte, read_only passthrough, prose, role-alternation) are non-discriminating by design.
Critical
model_tools.py:798-804(head) —_SYNTHETIC_TRUNCATION_MARKER_REmatched 5 of 7 realistic legit payloads in a probe: pasted pip output... [truncated] 40 more lines, a markdown note quoting`...[truncated]`, a grep hit ofcontext_compressor.pysource,assert out.endswith('...[truncated]')in a test file,(…[truncated]…). Anywrite_file/patch/terminalcarrying those is blocked with no override — it will hit developers editing this repo, log readers, and anyone quoting a tool error.- Fail-closed default: with
read_onlyunset on 91 of 95 tools, the guard is effectively global. If a dispatch guard is kept, the default must ship with a broadread_onlymarking pass (skill_view, skills_list, session_search, read_terminal, browser snapshot/console, kanban list/show, …). - The PR pre-guards a not-yet-existing unique sentinel, which concedes the right fix is at the producer.
Warnings
- Middleware re-check duplicates the dispatch-side check; one seam is enough.
- 9 tests vs the ≤2-invariant bar.
Preferred path: fix at the single producer — have _truncate_tool_call_args_json emit an unambiguous structured placeholder (that's #87985's layer, and AGENTS.md's "narrow waist" rule points there). Then, if a dispatch guard is still wanted, salvage this PR's read_only registry/MCP readOnlyHint propagation (clean, matches _annotation_read_only_hint in tools/mcp_tool_registration.py:43) with credit, rebase into _pre_dispatch_guards, narrow the regex to the exact producer form, mark the obvious read tools, drop the middleware recheck and the speculative sentinel, cut tests to 2.
|
Closing in favour of #87985 (@LevSky22, 2026-08-16, earlier submitter) as the carrier for the compacted-content-in-effectful-calls class; the verdict from my 09-07 review stands (5/7 legitimate payloads blocked by the regex; 91/95 tools fail closed with no override), and the branch has not moved since 08-27. What is worth keeping from here, as a co-authored follow-up once #87985 lands: the |
Summary
Add a fail-closed dispatch guard for synthetic context-compaction truncation markers in potentially side-effecting tool arguments.
Compaction can shorten historical
tool_calls[].function.argumentsand insert...[truncated]. That historical copy remains valid JSON, so a later model turn can mistake it for complete content and reuse it in a fresh write/send/publish call. There is no evidence that the deferred tool transport truncates a fresh complete argument; the unsafe boundary is reuse of synthetic compacted content.Security boundary
HERMES-CONTEXT-COMPRESSIONsentinel recursively in string/nested arguments.read_onlyregistry metadata; it does not alter serialized tool schemas or prompt-cache bytes.readOnlyHint=True; missing/malformed hints remain write-capable.Reproduction
On unmodified
origin/main, the regression test generated compacted args via_truncate_tool_call_args_json(), calledwrite_file, and reached the mocked write handler. The focused red run failed 2 tests because dispatch occurred.Tests
Canonical focused suites and full-suite results are included in the PR checks / author comment.
Existing work
This complements rather than duplicates:
Those do not provide a generic boundary for messaging, mail, publishing, payments, plugins, deferred MCP tools, or already-compacted history. This PR addresses the validation-hook ask in #83435 without mutating conversation history, role order, or tool schemas.
Refs #83435