fix(compression): prune stale codex_reasoning_items during compaction (#71058)

[webtecnica]

Summary

Fixes #71058. Compaction never pruned codex_reasoning_items — encrypted reasoning blobs from Codex/Responses that ride on every retained assistant message. On long-lived sessions these blobs become the largest incompressible component (~36% of the payload), causing compaction to bottom out at ~2× the configured target_ratio and re-fire every 30–60 minutes.

Changes

agent/context_compressor.py — 3 additive changes, no behavioral change to non-Codex sessions:

1. _prune_stale_reasoning_replay(messages) — New module-level function that scans the fully-assembled compacted message list right-to-left, locates the last assistant turn, and strips codex_reasoning_items (and any future keys in _STALE_REPLAY_PRUNE_KEYS) from all assistant messages before it. The final assistant message retains its items because its replay chain may still be active.

2. _STALE_REPLAY_PRUNE_KEYS — New tuple alongside _REPLAY_BUDGET_KEYS, listing fields safe to strip from stale turns. Currently: codex_reasoning_items only.

3. Wired into compress() — Called after _strip_persistence_markers(compressed), just before the return. Logs pruned count at INFO when non-zero and not in quiet mode.

Safety

• Compaction already invalidates the prompt-cache prefix — stripping blobs at this point costs nothing extra cache-wise.
• conversation_loop.py already drops these wholesale when api_mode != "codex_responses" — this is a scoped strip consistent with existing semantics.
• Conservative boundary — only items older than the last assistant turn are pruned; the most recent assistant message keeps its replay items for the current turn.
• No config knob needed — the behavior is always correct and safe during compaction. A config option would add complexity for no benefit given the cache-invalidation guarantee above.

Testing

• Unit tests of _prune_stale_reasoning_replay pass (empty list, single asst, multi-asst, no asst, three-asst cases).
• All 43 existing test_context_compressor.py tests pass (including TestCompress full integration tests).
• from agent.context_compressor import _prune_stale_reasoning_replay, _STALE_REPLAY_PRUNE_KEYS succeeds.

Closes #71058.

[PRATHAMESH75]

Nice diagnosis and a genuinely additive shape — the _STALE_REPLAY_PRUNE_KEYS tuple + single assembly-pass hook read well, and updating the _estimate_msg_budget_tokens docstring to reflect the new prune is a good touch. Two things I'd want resolved before this is safe to land, one correctness and one coverage.

1. The prune boundary is the last assistant message, but a Codex turn is a multi-message chain — this can strip reasoning mid-turn.

_prune_stale_reasoning_replay keeps items only on the last assistant message (for i in range(last_asst_idx)). But with protect_last_n=20 (default, context_compressor.py:1530), the protected tail routinely retains a full in-flight chain:

assistant(reasoning A, tool_call 1) → tool_result 1
assistant(reasoning B, tool_call 2) → tool_result 2
assistant(reasoning C, final answer)   ← only this keeps its items

This strips reasoning A and B while keeping tool_call 1/2. On the next request the codex_responses adapter replays each assistant message's codex_reasoning_items to "maintain coherent reasoning chains" (codex_responses_adapter.py:387-397); a replayed function_call whose preceding reasoning item is gone is exactly the "bridge function calls within a turn" hazard #71058 itself flagged. The blast radius is real: if the provider rejects it as invalid_encrypted_content, the recovery at conversation_loop.py:3118-3146 fires — but that path disables replay for the entire session and strips all items, which defeats the feature you're trying to preserve, and it only triggers for that one error class.

Worth stressing: the two precedents the PR cites as making this safe both differ in the load-bearing way. conversation_loop.py dropping items when api_mode != \"codex_responses\" and _disable_codex_reasoning_replay() are both wholesale strips, and the latter also turns replay off. This PR is the first place that strips items partially while replay stays enabled — so the "consistent with existing semantics" argument doesn't quite cover the new case.

Suggested boundary: prune older than the last turn / tool-call chain, not the last message — i.e. walk back from the end over the contiguous assistant/tool run to the last user message (or the start of the last tool_call chain) and only strip assistant messages before that. That matches the issue's own wording ("older than the most recent assistant turn (or older than the last tool-call chain)") and leaves the current turn's replay intact.

2. No regression test is committed. The diff touches only agent/context_compressor.py; the "unit tests of _prune_stale_reasoning_replay pass" from the description aren't in the PR. Given this mutates compaction output for a replay-sensitive provider, it needs a committed test — ideally one that builds a multi-assistant-message turn in the protected tail and asserts (a) prior-turn items are stripped, (b) the current turn's items survive on every assistant message in the chain, and (c) the codex_responses adapter still emits a valid input where each replayed function_call is preceded by its reasoning item.

Happy to be wrong on (1) if there's provider behavior I'm missing — if the Responses endpoint tolerates a function_call with a missing preceding reasoning item under store:false, a test demonstrating that would settle it and the message-level boundary is fine as-is.

[teknium1]

Thanks for targeting the replay-payload floor; current main still accounts for these fields without pruning them during compaction (agent/context_compressor.py:722-767, 4806-4854).

Problems

• The proposed for i in range(last_asst_idx) boundary removes replay state from earlier assistant messages in a retained active tool-call chain. The Responses converter replays each assistant message's encrypted reasoning before that message's tool calls (agent/codex_responses_adapter.py:410-570), and Codex interim state is explicitly required to be replayed verbatim (agent/agent_runtime_helpers.py:555-561).
• The PR changes only agent/context_compressor.py; no regression coverage accompanies it. Existing tests cover replay-field budgeting (tests/agent/test_context_compressor.py:267-346) and single-message replay (tests/run_agent/test_provider_parity.py:884-902), not this chain boundary.

Suggested changes

• Preserve all assistant messages in the latest active assistant/tool chain; prune only before that chain.
• Add helper, compression, and adapter-output tests for that boundary.

Automated hermes-sweeper review.

[teknium1]

This treats every earlier assistant message as stale, but a retained Codex tool turn can contain assistant → tool → assistant rounds. The Responses converter replays encrypted reasoning before each assistant message's tool calls (agent/codex_responses_adapter.py:410-570), so retain the whole latest active chain rather than only the final assistant message.

[GottZ]

This was generated by AI during triage.

Summary

One open PR addresses Issue #71058 by pruning stale codex_reasoning_items during compaction, directly targeting the reported incompressible replay-payload floor. Its diff removes replay fields from assistant messages before the final assistant message, but does not yet preserve a complete active multi-message Codex tool chain or add regression coverage.

Related pull requests

• fix(compression): prune stale codex_reasoning_items during compaction (#71058) #71077 best fix — (+74/-2) — partial fix: adds _prune_stale_reasoning_replay to remove codex_reasoning_items from retained assistant messages preceding the last assistant message, reducing stale replay payload during compaction. The [contributor:7624 commits] review identifies a correctness gap because this boundary can strip reasoning from earlier messages in the active assistant/tool chain, and notes missing chain-boundary and adapter-output tests; the PR remains relevant as the sole direct implementation of the requested pruning.

Suggested consolidation

keep open with a salvage path: retain #71077’s post-compaction pruning approach, but preserve the entire latest active assistant/tool chain rather than only the final assistant message, and add helper, compression, and adapter-output regression tests. This follows the [PR review:COMMENTED] keep_open verdict and the blocking [contributor:7624 commits] review; do not merge or close #71077 until that replay-boundary correctness issue is explicitly addressed.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I71058(["issue #71058 (open)"])
    P71077["PR #71077 (open)"]
    P71077 -->|best fix| I71058
    class I71058 open
    class P71077 open
    class P71077 best
    class P71077 target
    click I71058 "https://github.com/NousResearch/hermes-agent/issues/71058"
    click P71077 "https://github.com/NousResearch/hermes-agent/pull/71077"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 1 pull request and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 6 kB of PR diffs, 5 kB of issue/PR text, 5 kB of discussion (3 comments), 2 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[webtecnica]

Closing in favor of #81944 (salvage by @teknium1 with authorship preserved — fixes #71058 with the corrected prune boundary + checkpoint exemption + sibling-site fix). Thanks for completing the gaps from the original review.

[teknium1]

Landed on main via PR #81944 with your base commit cherry-picked intact — thank you @webtecnica, the diagnosis and the prune shape were exactly right.

Two corrections rode on top: (1) the prune boundary is now the last USER message rather than the last assistant message, so a multi-message Codex tool-call chain in the active turn keeps its reasoning items (the gap @PRATHAMESH75 flagged in review); (2) type="compaction" items are exempt — those are native server-side compaction checkpoints (PR #81747, merged after this PR was opened) that carry already-pruned history and must survive on every retained message. A sibling site in the continuation-dedup path got the same checkpoint-preserving treatment.

Closing this PR as superseded by the merged salvage.

[webtecnica]

Thanks @teknium1 — thrilled to see it land! The last-USER-message boundary (keeping the active turn's multi-message Codex chain intact) and the compaction-checkpoint exemption are exactly the right corrections, and the sibling continuation-dedup treatment closes the class. Appreciate the cherry-picked authorship and the kind words on the diagnosis.