fix(api): persist compressed transcripts in ResponseStore to stop re-compression loops - #69306
Merged
Conversation
Compression produces a compact transcript in result['messages'], but _build_response_conversation_history detected a prefix mismatch and concatenated the original conversation_history on front. Detect compression via _last_compaction_in_place / session_id rotation and signal through result['_compressed'] so the builder uses the compressed transcript directly.
- Detect when history is loaded from response_store (via previous_response_id) - Add history_from_store parameter to distinguish history source - When compression occurs, persist compressed messages instead of original - Add persist_in_response_store config option (default True) - Update session_id and response headers to reflect session rotation Cherry-picked from alidev 2eb816f6b
The persist logic only checked _result_sid != session_id (rotation), missing in-place mode where session_id is unchanged but _compressed flag is set. response_store history doubled every turn (11->26->55->110->225) causing repeated re-compression. Fix: detect compression via _did_compress or _rotated, and only update _effective_session_id on actual rotation (not in-place). Note: preflight loop break (turn_context.py) from original commit eee64097a is excluded — it's an optimization, not a bug fix. Cherry-picked from alidev eee64097a (api_server.py only)
Address review feedback (PR #58133): the original test mocked _run_agent with _compressed=True directly, bypassing the detection logic. New tests mock _create_agent instead, so _run_agent's detection path runs naturally and reads agent.session_id / _last_compaction_in_place: 1. test_rotation_compression_exercises_detection_and_persists_rotated_session_id - Fake agent with rotated session_id -> verifies _compressed is set, compressed history is stored, and rotated session_id propagates to both response_store and X-Hermes-Session-Id header. 2. test_inplace_compression_exercises_detection_and_persists_compressed_history - Fake agent with _last_compaction_in_place=True, session_id unchanged -> verifies _compressed is set, compressed history is stored, and session_id does NOT rotate. 3. test_chained_rotation_propagates_effective_session_id - Two-request chain: first request triggers rotation, second request loads history using the rotated session_id stored by the first. Asserts the compressed transcript is loaded correctly for chaining.
…g opt-out Rework on top of the salvaged #58133 commits: - Remove the compression.persist_in_response_store config key — this is a bug fix (stored transcripts must reflect what the agent will actually replay), not behavior that should be opt-out-able. - Drop the per-request load_config() imports the handler-level persist blocks added. - Dedupe the two handler-level persist blocks: the compressed-transcript substitution already lives in _build_response_conversation_history (via result["_compressed"]), so the handlers only need to propagate the effective (possibly rotation-changed) session_id. The streaming path does this via a new session_id_snapshot arg on _persist_response_snapshot; the non-streaming path picks up result["session_id"] directly. - Rotation propagation no longer gates on history-from-store: the first request in a chain can also rotate, and its stored session_id must be the child session or the next previous_response_id request resumes the pre-rotation session and re-compresses every turn.
Contributor
૮ >ﻌ< ა ci reviewrunning on 2c313a8 looks good to me! |
alt-glitch
added
type/bug
This was referenced Jul 22, 2026
Closed
11 tasks
1 task
14 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/v1/responsesnow persists the compressed transcript (and the compression-rotated session id) in the ResponseStore, so chainedprevious_response_idrequests stop re-compressing the same bloated history every turn. Root cause:_response_messages_turn_start_indexdoes exact prefix equality, so a compressed transcript never matches the stored history and the fallback concatenatedprior + current + agent messages— bloating the snapshot and re-triggering compression on every subsequent request, while the store also kept the pre-rotationsession_id.Changes
gateway/platforms/api_server.py:_run_agentdetects compression via the two authoritative signals — session rotation (agent.session_idchange) and in-place compaction (agent._last_compaction_in_place) — and setsresult["_compressed"]._build_response_conversation_historyuses_compressedto store the compressed transcript directly instead of concatenating the uncompressed prior history on front (single persist site for both streaming and non-streaming paths).X-Hermes-Session-Id, so chaining resumes the child session instead of the pre-rotation parent.tests/gateway/test_api_server.py: regression tests for compressed-transcript storage, rotation and in-place fake-agent detection paths, and two-request chained rotation propagation.Rework applied during salvage: removed the
compression.persist_in_response_storeconfig opt-out (bug fixes don't get opt-outs), removed the per-requestload_config()imports, and deduped the two handler-level persist blocks into the existing_build_response_conversation_history_compressedcheck.Validation
X-Hermes-Session-Id)Targeted tests:
tests/gateway/test_api_server*.py— 394 passed, 0 failed (includes the 4 new regression tests);tests/hermes_cli/test_config.py,test_config_validation.py,tests/run_agent/test_in_place_compaction.py— 218 passed.Credit
Salvaged from #58133 by @LiangYang666. The same root cause was first identified and fixed by @xiaolongzhao2015 in #41700 (Jun 8) and independently by @izumi0uu in #56933 — first-submitter credit to both. Also fixes the bug reported in #56996 by @ZzzSimon and #58118 by @tianma-if. Fixes #56895.
Infographic