fix(context-compressor): replayed tool-call args stop carrying a marker models copy (#83714, salvage #83843) - #116169
Merged
kshitijk4poor merged 5 commits intoSep 19, 2026
Conversation
… into replayed tool_calls Root cause for NousResearch#83714 (write_file/patch_tool writing literal "...[truncated]" into files, PR NousResearch#83752's guard is the safety net, not the fix): _truncate_tool_call_args_json() in the compression pass shrinks long string values inside a PAST assistant message's tool_calls[].function.arguments — the exact field that represents the model's own prior generated output, replayed back to it verbatim on every subsequent turn. The old marker, a bare "...[truncated]" suffix, is indistinguishable from something the model itself could have written (it's exactly the kind of terse ellipsis abbreviation models already produce). A model conditioned on seeing itself "get away with" that pattern in its own history imitates it in a new tool call, writing the literal marker instead of real content. This is the second bug from the same root text. The first (NousResearch#11762, MiniMax 400s from unterminated JSON) was fixed by shrinking inside the parsed structure so the JSON stays valid, but kept the same visible marker text — fixing the syntax problem while leaving the imitation problem untouched. Fix: replace the marker with one deliberately NOT shaped like prose a model would write — distinctive non-ASCII delimiters, an explicit "not part of the original tool call" disclaimer, and a per-instance char-count that won't match the next omission point even if copied verbatim. The shrunk value stays a plain string (not a nested object) so the NousResearch#11762 valid-JSON/matching-shape contract is unchanged — only the marker text changed. Checked context_compressor.py's other "...[truncated]" call sites (_serialize_for_summary, _compact_fallback_turn, the user-message-only one near _ACTIVE_TASK_MAX_CHARS) — none of them write into a value that gets replayed as the main model's own assistant/tool_calls history, so they don't share this priming risk and were left as-is. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…shrink
The anti-imitation marker is ~220 chars, longer than the 200-char head it
follows, which made the replayed-arg rewrite unsound in two ways:
- A leaf just over `head_chars` came back LONGER: a 628-char args blob
shrank to 455 chars before this change and grew to 863 after it.
- The result was not a fixed point. `_shrink` re-ran on every later
compaction, so a leaf's marker was rewritten from the true count
("2,800 of 3,000 chars omitted") to a self-referential one
("223 of 423"), destroying the per-instance count property the marker
relies on and churning those bytes on each pass.
A leaf now keeps its head+marker replacement only when that is strictly
shorter, and an already-marked leaf is left alone. The two tests pin the
behaviour contract: never grow, and `shrink(shrink(x)) == shrink(x)`.
… byte-identical Review findings on the previous commit, all reproduced: - The "already marked" guard was a substring test, so a leaf that merely contains the marker — including one a model imitated into a new call, the NousResearch#83714 failure mode itself — was exempt from shrinking forever. The marker is always written at `head_chars`, so the guard now tests that position: a 1,550-char imitated leaf shrank to 423 again. - The helper re-serialised even when nothing was replaced, so compact wire JSON came back with inserted spaces and callers read it as a change (rewriting replayed history and counting a pressure hit for zero reclaim). Nothing replaced now returns the original string: a 547-char compact blob is byte-identical.
Second review round on the previous commit, both findings reproduced: - `startswith(prefix, head_chars)` still exempted the imitation shape NousResearch#83714 is about: a replayed leaf of head + marker followed by new content was never shrunk again (a 5,278-char leaf stayed 5,278). The guard now also requires the marker to close the leaf, so that shape shrinks to head + marker with true counts. - Per-leaf savings were compared in characters, but the final re-serialise adds separator whitespace, so compact args with many keys could come back LONGER (measured 3,511 -> 4,110 chars) and be counted as reclaimed pressure. The helper now returns the caller's string unless the whole rewrite is a net reduction. Tests cover both shapes plus a many-key compact payload.
alt-glitch
added
type/bug
This was referenced Sep 19, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compaction no longer rewrites a replayed tool call's arguments with a marker the model copies.
The marker appended to a shrunk argument leaf is now
⟪HERMES-CONTEXT-COMPRESSION: N of M chars omitted here by Hermes's context compressor. This is NOT part of the original tool call and must never be reproduced in new output — always write full, untruncated content.⟫instead of
"...[truncated]". The old text sat in the model's own history as its prior output, and a model imitated it into new tool calls whosenew_stringwas then written to disk (#83714). The replacement also only happens when it is a net reduction, never twice on the same leaf, and the helper returns the caller's exact string when nothing was replaced.Validation — this branch vs
origin/main, through the realcompress()path and the helper directly:...[truncated](226 chars)f(f(x)) == f(x)(counts stable)Tests: 186 passed, 0 failed across
tests/agent/test_context_compressor.py,tests/agent/test_protected_tail_pressure.py,tests/agent/test_compression_boundary.py,tests/test_trajectory_compressor.py. With_shrinkmutated back to the earlier "always append" behaviour, 2 of the 3 new tests fail (the ones that pin this change) and the class-regression test fails on unmodified main.ruff check,scripts/check-windows-footguns.py --all,scripts/check_compat_pointers.pyandgit diff --checkare clean.Deliberate trade-offs, documented in the function docstring: the marker costs ~60 tokens per shrunk leaf against ~4 for the old one, and leaves shorter than the break-even (
head_chars+ marker, ~420 chars) are now left intact instead of being head-truncated.Based on #83843 by @djbclark — cherry-picked so authorship survives; the follow-up commits (only replace when it reclaims, never re-shrink, position-based marker guard, byte-identical no-op, tests) are ours.