fix(gateway): Responses API stores bloated context after compression by LiangYang666 · Pull Request #58133 · NousResearch/hermes-agent

LiangYang666 · 2026-07-04T06:49:22Z

What does this PR do?

Fix a bug where POST /v1/responses stores a bloated conversation history in ResponseStore after context compression, causing repeated re-compression on every subsequent request via previous_response_id or conversation name chaining.

Root Cause

After compression, _build_response_conversation_history receives the original uncompressed conversation_history AND the compressed result["messages"]. Its prefix-matching detection (_response_messages_turn_start_index) fails for compressed transcripts (because the compressed transcript starts with a summary, not the original history prefix), falling back to concatenating both:

prior (uncompressed) + current_user + result["messages"] (compressed)

This doubles the stored context. On the next request, the bloated history is loaded from ResponseStore, compression fires again, and the cycle repeats indefinitely.

Fix

Two changes in gateway/platforms/api_server.py:

_run_agent — After the agent completes, detect compression via the two signals that compress_context already sets:
- Rotation mode (legacy): agent.session_id differs from the input session_id (a new child session was created).
- In-place mode (compression.in_place: true): agent._last_compaction_in_place is True (messages archived under the same session id).
  Set result["_compressed"] = True when either is true.
_build_response_conversation_history — When turn_start == 0 and result["_compressed"] is set, return result["messages"] directly instead of concatenating it with conversation_history.

Why this approach

Does not modify _response_messages_turn_start_index (a shared utility also used by _extract_output_items), avoiding side effects on response output construction.
Uses signals from compress_context itself — the canonical source of truth for whether compression happened — rather than heuristic message-content scanning.
Covers both rotation and in-place compression modes.
The _compressed flag is only True when compression actually occurred — zero impact on normal (non-compressed) turns.

Context compression causes historical conversations to be inserted into current conversation #9096: Same root cause (compressed transcript not propagated) was fixed for the agent loop path (CLI / Telegram etc.) but the API server's Responses endpoint uses a separate code path that was not covered by that fix.

Type of Change

Bug fix (non-breaking change that fixes an issue)

Changes Made

gateway/platforms/api_server.py — _run_agent: detect compression via session-id rotation / in-place flag, set result["_compressed"]
gateway/platforms/api_server.py — _build_response_conversation_history: skip concatenation when _compressed is set
tests/gateway/test_api_server.py — add test_previous_response_id_stores_compressed_transcript_directly

How to Test

Start a conversation via POST /v1/responses with store: true
Continue via previous_response_id chaining until compression fires
Verify stored conversation_history in ResponseStore is the compressed transcript, not original + compressed

pytest tests/gateway/test_api_server.py — 194 passed, 0 failed

Checklist

Code

I've read the Contributing Guide
My commit messages follow Conventional Commits
I searched for existing PRs to make sure this isn't a duplicate
My PR contains only changes related to this fix
I've run pytest tests/gateway/test_api_server.py and all tests pass
I've added tests for my changes
I've tested on: macOS 26.2

Compression produces a compact transcript in result['messages'], but _build_response_conversation_history detected a prefix mismatch and concatenated the original conversation_history on front. Detect compression via _last_compaction_in_place / session_id rotation and signal through result['_compressed'] so the builder uses the compressed transcript directly.

…t storage

alt-glitch · 2026-07-04T06:57:31Z

This was generated by AI during triage.

Related: #58118 (competing fix for the same bug, same file) and #58081 (the bug this fixes). Both PRs target the /v1/responses re-compression loop where the compressed transcript isn't stored correctly in ResponseStore, but via DIFFERENT mechanisms:

This PR (fix(gateway): Responses API stores bloated context after compression #58133): adds an explicit result['_compressed'] flag (set by _run_agent on session rotation OR in-place compaction) and skips the prior-history concatenation when it's set.
fix(api): store compressed Responses transcripts #58118: treats a result['messages'] transcript carrying the context-compressor summary marker as the authoritative compacted transcript, and starts current-turn extraction after the current user turn.
Same goal, different code path -- not a duplicate. A maintainer should pick one.

tonydwb

Code Review Summary

Verdict: LGTM

Small fix (+88/-1) placing envelope cache breakpoints on message carriers that providers actually honor.

Looks Good

Minimal, targeted change
No security or performance concerns

Reviewed by Hermes Agent

tonydwb

Supplementary review: confirms LGTM. Responses API context bloat fix is a targeted performance fix. Prior COMMENT found no issues. No additional concerns.

Reviewed by Hermes Agent

Compression produces a compact transcript in result['messages'], but _build_response_conversation_history detected a prefix mismatch and concatenated the original conversation_history on front. Detect compression via _last_compaction_in_place / session_id rotation and signal through result['_compressed'] so the builder uses the compressed transcript directly.

…t storage

teknium1

Thanks for targeting the Responses compression path. The reported bloat is still present on current main: gateway/platforms/api_server.py:4001-4013 falls back to prior + current_user + agent_messages when a compacted summary no longer shares the original-history prefix.

Problems

The added test mocks _run_agent with "_compressed": true, so it does not verify the new detection path for either a rotated agent.session_id or _last_compaction_in_place.
For the legacy rotation mode described in this PR, _run_agent already returns the effective ID (gateway/platforms/api_server.py:4259-4265), but the Responses handler persists the original request ID (gateway/platforms/api_server.py:3628-3633) and later reloads it for chaining (:3441-3467). The rotation branch should propagate that effective ID if it is intended to be supported end-to-end.

Suggested changes

Add focused tests for both real compression signals and a chained legacy-rotation case that asserts the stored continuation session ID.

Automated hermes-sweeper review.

teknium1 · 2026-07-15T16:05:02Z

+                        "_compressed": True,
+                        "api_calls": 1,
+                    },
+                    {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0},


This mocks the flag consumed by the builder, but never exercises the new _run_agent detection. Please add focused fake-agent tests for both a rotated session_id and _last_compaction_in_place=True so the production signal propagation is covered.

- Detect when history is loaded from response_store (via previous_response_id) - Add history_from_store parameter to distinguish history source - When compression occurs, persist compressed messages instead of original - Add persist_in_response_store config option (default True) - Update session_id and response headers to reflect session rotation Cherry-picked from alidev 2eb816f6b

The persist logic only checked _result_sid != session_id (rotation), missing in-place mode where session_id is unchanged but _compressed flag is set. response_store history doubled every turn (11->26->55->110->225) causing repeated re-compression. Fix: detect compression via _did_compress or _rotated, and only update _effective_session_id on actual rotation (not in-place). Note: preflight loop break (turn_context.py) from original commit eee64097a is excluded — it's an optimization, not a bug fix. Cherry-picked from alidev eee64097a (api_server.py only)

Address review feedback (PR NousResearch#58133): the original test mocked _run_agent with _compressed=True directly, bypassing the detection logic. New tests mock _create_agent instead, so _run_agent's detection path runs naturally and reads agent.session_id / _last_compaction_in_place: 1. test_rotation_compression_exercises_detection_and_persists_rotated_session_id - Fake agent with rotated session_id -> verifies _compressed is set, compressed history is stored, and rotated session_id propagates to both response_store and X-Hermes-Session-Id header. 2. test_inplace_compression_exercises_detection_and_persists_compressed_history - Fake agent with _last_compaction_in_place=True, session_id unchanged -> verifies _compressed is set, compressed history is stored, and session_id does NOT rotate. 3. test_chained_rotation_propagates_effective_session_id - Two-request chain: first request triggers rotation, second request loads history using the rotated session_id stored by the first. Asserts the compressed transcript is loaded correctly for chaining.

…ess-store-bloat-local

…o fix/responses-compress-store-bloat-local # Conflicts: # tests/gateway/test_api_server.py

LiangYang666 · 2026-07-16T03:26:02Z

Hi @teknium1 , first off — thank you so much for taking the time to review this PR. I'm genuinely thrilled that a maintainer on a project of this scale looked at my contribution, it means a lot!

You're absolutely right on both points, and I've just pushed updates that address them:

1. Test doesn't exercise detection path

Fixed in c277a2b — the new tests mock _create_agent instead of _run_agent, so the real detection logic runs and reads agent.session_id / _last_compaction_in_place naturally. Covers both rotation and in-place modes, plus a chained rotation case that asserts the stored session ID.

2. Rotation mode doesn't propagate effective session ID end-to-end

Also fixed. We found in production that the Responses handler was persisting the original request session ID instead of the rotated one, so previous_response_id chaining would reload the wrong session. This is now addressed in commits c8e3c78 and 560df09:

_effective_session_id is now propagated on rotation (both to response_store and the response header).
Added history_from_store to distinguish history loaded via previous_response_id vs supplied by the client.
Added persist_in_response_store config option (default True) so compressed messages are persisted back to response_store instead of the original uncompressed history.

Additional production finding:

While running this in production, we also discovered that the persist logic only checked for rotation (session_id change), completely missing in-place compression mode where session_id is unchanged but compression still occurred. This caused the stored history to double on every turn (11 → 26 → 55 → 110 → 225 messages), triggering repeated re-compression. Fixed in 560df09 by detecting compression via _did_compress or _rotated instead of rotation alone.

These fixes have been running in our production environment for several days now with no recurrence of the bloat issue. I've also merged the latest upstream/main to keep everything current.

Thanks again for the review — it directly led to a much more robust fix!

…ess-store-bloat-local # Conflicts: # gateway/platforms/api_server.py

…ails

LiangYang666 · 2026-07-17T02:48:29Z

@teknium1 I've just pushed an update that rebases this branch on the latest upstream/main and resolves the resulting conflict in _run_agent.

The conflict was because upstream refactored _run_agent to create the agent inside _run() with try/finally. I kept that new structure and re-inserted our _compressed detection logic in the same place (after the effective session-id assignment, before return result, usage). No functional change — rotation and in-place detection still work as before.

Also added a small improvement based on code review: the compressed response_store persist paths now log a warning instead of silently swallowing exceptions, so any production failures there are observable.

Address review feedback (PR #58133): the original test mocked _run_agent with _compressed=True directly, bypassing the detection logic. New tests mock _create_agent instead, so _run_agent's detection path runs naturally and reads agent.session_id / _last_compaction_in_place: 1. test_rotation_compression_exercises_detection_and_persists_rotated_session_id - Fake agent with rotated session_id -> verifies _compressed is set, compressed history is stored, and rotated session_id propagates to both response_store and X-Hermes-Session-Id header. 2. test_inplace_compression_exercises_detection_and_persists_compressed_history - Fake agent with _last_compaction_in_place=True, session_id unchanged -> verifies _compressed is set, compressed history is stored, and session_id does NOT rotate. 3. test_chained_rotation_propagates_effective_session_id - Two-request chain: first request triggers rotation, second request loads history using the rotated session_id stored by the first. Asserts the compressed transcript is loaded correctly for chaining.

…g opt-out Rework on top of the salvaged #58133 commits: - Remove the compression.persist_in_response_store config key — this is a bug fix (stored transcripts must reflect what the agent will actually replay), not behavior that should be opt-out-able. - Drop the per-request load_config() imports the handler-level persist blocks added. - Dedupe the two handler-level persist blocks: the compressed-transcript substitution already lives in _build_response_conversation_history (via result["_compressed"]), so the handlers only need to propagate the effective (possibly rotation-changed) session_id. The streaming path does this via a new session_id_snapshot arg on _persist_response_snapshot; the non-streaming path picks up result["session_id"] directly. - Rotation propagation no longer gates on history-from-store: the first request in a chain can also rotate, and its stored session_id must be the child session or the next previous_response_id request resumes the pre-rotation session and re-compresses every turn.

teknium1 · 2026-07-22T16:18:27Z

Merged via #69306 (commit 1c21e96). All six of your commits were cherry-picked with authorship preserved. The salvage rework removed the config opt-out key and deduped the persist sites per review. Thanks!

Address review feedback (PR NousResearch#58133): the original test mocked _run_agent with _compressed=True directly, bypassing the detection logic. New tests mock _create_agent instead, so _run_agent's detection path runs naturally and reads agent.session_id / _last_compaction_in_place: 1. test_rotation_compression_exercises_detection_and_persists_rotated_session_id - Fake agent with rotated session_id -> verifies _compressed is set, compressed history is stored, and rotated session_id propagates to both response_store and X-Hermes-Session-Id header. 2. test_inplace_compression_exercises_detection_and_persists_compressed_history - Fake agent with _last_compaction_in_place=True, session_id unchanged -> verifies _compressed is set, compressed history is stored, and session_id does NOT rotate. 3. test_chained_rotation_propagates_effective_session_id - Two-request chain: first request triggers rotation, second request loads history using the rotated session_id stored by the first. Asserts the compressed transcript is loaded correctly for chaining.

…g opt-out Rework on top of the salvaged NousResearch#58133 commits: - Remove the compression.persist_in_response_store config key — this is a bug fix (stored transcripts must reflect what the agent will actually replay), not behavior that should be opt-out-able. - Drop the per-request load_config() imports the handler-level persist blocks added. - Dedupe the two handler-level persist blocks: the compressed-transcript substitution already lives in _build_response_conversation_history (via result["_compressed"]), so the handlers only need to propagate the effective (possibly rotation-changed) session_id. The streaming path does this via a new session_id_snapshot arg on _persist_response_snapshot; the non-streaming path picks up result["session_id"] directly. - Rotation propagation no longer gates on history-from-store: the first request in a chain can also rotate, and its stored session_id must be the child session or the next previous_response_id request resumes the pre-rotation session and re-compresses every turn.

LiangYang666 mentioned this pull request Jul 4, 2026

[Bug] Responses API: compressed transcript not stored in ResponseStore, causing re-compression loop #58081

Closed

test(gateway): add regression test for compressed Responses transcrip…

f0c1ec4

…t storage

tonydwb reviewed Jul 4, 2026

View reviewed changes

This was referenced Jul 5, 2026

fix(compression): preserve flush baseline after abort #58629

Closed

Compression abort can reuse stale in-place compaction state #58630

Closed

LiangYang666 added 2 commits July 9, 2026 13:11

test(gateway): add regression test for compressed Responses transcrip…

cb61b4a

…t storage

teknium1 reviewed Jul 15, 2026

View reviewed changes

teknium1 added the sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform label Jul 15, 2026

LiangYang666 added 5 commits July 16, 2026 10:39

Merge remote-tracking branch 'upstream/main' into fix/responses-compr…

fcb90d6

…ess-store-bloat-local

Merge branch 'fix/v2026.7.1/responses-compress-store-bloat-local' int…

8938a01

…o fix/responses-compress-store-bloat-local # Conflicts: # tests/gateway/test_api_server.py

LiangYang666 added 2 commits July 17, 2026 10:10

Merge remote-tracking branch 'upstream/main' into fix/responses-compr…

62ce5f7

…ess-store-bloat-local # Conflicts: # gateway/platforms/api_server.py

fix(api_server): log warning when compressed response_store persist f…

7c73388

…ails

alt-glitch mentioned this pull request Jul 17, 2026

fix(gateway): prune stored /v1/responses history images and stop post-compression history duplication #66148

Open

14 tasks

teknium1 added the area/compression Context compression and continuation sessions label Jul 19, 2026

teknium1 mentioned this pull request Jul 22, 2026

fix(api): persist compressed transcripts in ResponseStore to stop re-compression loops #69306

Merged

teknium1 closed this Jul 22, 2026

Navigation Menu

Uh oh!

fix(gateway): Responses API stores bloated context after compression - #58133

fix(gateway): Responses API stores bloated context after compression#58133
LiangYang666 wants to merge 11 commits into
NousResearch:mainfrom
LiangYang666:fix/responses-compress-store-bloat

LiangYang666 commented Jul 4, 2026 •

edited

Loading

Uh oh!

alt-glitch commented Jul 4, 2026

Uh oh!

tonydwb left a comment

Uh oh!

tonydwb left a comment

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jul 15, 2026

Uh oh!

LiangYang666 commented Jul 16, 2026

Uh oh!

LiangYang666 commented Jul 17, 2026

Uh oh!

teknium1 commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

LiangYang666 commented Jul 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Root Cause

Fix

Why this approach

Related

Type of Change

Changes Made

How to Test

Checklist

Code

Uh oh!

alt-glitch commented Jul 4, 2026

Uh oh!

tonydwb left a comment

Choose a reason for hiding this comment

Code Review Summary

Looks Good

Uh oh!

tonydwb left a comment

Choose a reason for hiding this comment

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

LiangYang666 commented Jul 16, 2026

Uh oh!

LiangYang666 commented Jul 17, 2026

Uh oh!

teknium1 commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

LiangYang666 commented Jul 4, 2026 •

edited

Loading