Release M — v0.51.37 — Compression / lineage backend (6 PRs) - #2020
Conversation
When context compression fires, the agent rotates to a new session_id.
The compression migration block correctly migrates the session lock,
SESSION_AGENT_CACHE, SESSIONS dict, and the session file rename, but
does not ensure s.profile is set on the continuation session.
On the next request, _run_agent_streaming resolves the profile via:
get_hermes_home_for_profile(getattr(s, 'profile', None))
With s.profile == None this falls back to the default profile's
HERMES_HOME. Memory tool calls then read and write the wrong profile's
MEMORY.md — confirmed by investigation: session 0dfefb (continuation
after compression from a troubleshooting profile session) read memory
at 16% / 1,184 chars with 4 entries, while the troubleshooting profile's
actual state was 72-77% / 5,000+ chars. That reading could only come
from the default profile's bank. Subsequent replace operations failed
because the target entries existed only in the troubleshooting profile.
There are two failure paths:
1. In-memory: if s.profile was None from the start (legacy session or
one created before this fix), the continuation session object carries
null through the current request.
2. Persistence: s.save() persists "profile": null to the continuation
session's JSON file (profile is in METADATA_FIELDS, models.py ~408).
On the next request, Session.load(new_sid) reads it back as null and
get_hermes_home_for_profile(None) falls back to the default profile.
Fix: capture _resolved_profile_name at request entry (~line 2019),
immediately after profile home resolution. This is the only point where
profile context is reliable: s.profile if already set, otherwise
get_active_profile_name() — which at that point reads thread-local
storage (_tls.profile) correctly set by the HTTP handler thread via
set_request_profile(). Calling get_active_profile_name() at compression
time instead would be unsafe: the streaming thread is a separate
threading.Thread, does not inherit TLS, and the call would fall back to
the process-global _active_profile which may belong to a different
concurrent tab.
Stamp s.profile in the compression migration block immediately after
s.session_id = new_sid. Guarded by `if not s.profile` so sessions that
already have a profile set are unaffected. A logger.info line records
when the stamp fires, making future investigation straightforward.
Fixes: memory writes bleeding into default profile after compression
Reproduces: reliably on any long non-default profile session that hits
the compression threshold (default: 0.80 context fill)
…summary for reload UI by @franksong2702
…of compression lineage by @ai-ag2026
Opus advisor verdict — SHIP-WITH-CAVEATSFull review run on stage diff with brief at Summary of findings#2004 (anchor metadata persistence) — Field added to
#2006 (profile stamping) —
#2012 (lineage report endpoint) — Auth enforced upstream by #2015 (lineage stitching) — Walk: 20 hops,
#2011 + #2014 composition — Caveats (none blocking, all suitable for follow-ups)
Verdict: SHIP-WITH-CAVEATSNo NEEDS-FIX issues. Compression block correctly sequences profile stamping → session-id rotation → anchor metadata → single |
# Conflicts: # CHANGELOG.md
nesquena
left a comment
nesquena
left a comment
There was a problem hiding this comment.
Review — end-to-end ✅ (clean approve, 6-PR session-lifecycle release train all verified)
What this ships
Release M / v0.51.37 — release-train PR aggregating 6 contributor PRs from 4 authors. All on session lifecycle: compression boundary persistence, profile stamping, lineage collapse, fork-source marker, lineage report endpoint, continuation-transcript stitching. +800/-12 LOC across 14 files.
| PR | Author | Concern | LOC |
|---|---|---|---|
| #2004 | @franksong2702 | Persist compression boundary summary metadata | +160/-3 |
| #2006 | @qxxaa | Stamp s.profile on continuation session after compression |
+34/-1 |
| #2011 | @ai-ag2026 | Sidebar lineage collapse prefers latest compressed segment | +49/-1 |
| #2012 | @dso2ng | Read-only /api/session/lineage/report diagnostic endpoint |
+363/-0 |
| #2014 | @ai-ag2026 | session_source="fork" marker; keep forks out of compression-lineage collapse |
+28/-0 |
| #2015 | @Jellypowered | Stitch continuation-lineage transcripts in get_cli_session_messages() |
+139/-7 |
Closes #1833.
Cross-tool trace (verified against fresh hermes-agent tarball)
All 6 PRs touch WebUI session-lifecycle surfaces. The agent doesn't read compression_anchor_* fields — they're WebUI-internal display state. parent_session_id and session_source ARE shared with the agent's state.db.sessions table; the agent writes these through its own continuation/branching code paths. The PR's READS of state.db (lineage report, transcript stitching) are read-only and parameterized.
Cross-tool surface: read-only against state.db (which is canonically owned by hermes-agent). No new write paths introduced. ✓
End-to-end traces
#2004 — compression anchor metadata: New field compression_anchor_summary added to Session.__init__ (api/models.py:332), _PERSISTED_FIELDS (api/models.py:417), and compact() (api/models.py:578). Round-trips through save/load. Auto-compression block at api/streaming.py:3115-3128 sets compression_anchor_visible_idx, compression_anchor_message_key, compression_anchor_summary AFTER _compressed=True; manual compression endpoint at api/routes.py:7757-7765 sets them and calls s.save() directly. Both paths converge on a single save point. ✓
#2006 — profile stamping: _resolved_profile_name captured at api/streaming.py:2103-2118, immediately after _profile_home resolution. Used at api/streaming.py:3082-3088 inside the compression migration block:
s.session_id = new_sid
if not s.profile and _resolved_profile_name:
s.profile = _resolved_profile_name
logger.info("Stamped profile=%r on continuation session %s after compression", ...)
with LOCK:
if old_sid in SESSIONS:
SESSIONS[new_sid] = SESSIONS.pop(old_sid)The if not s.profile guard ensures existing-profile sessions aren't overwritten. CPython string assignment is atomic — concurrent readers don't tear. The capture happens on the streaming thread (which doesn't inherit TLS from the HTTP handler thread, per the PR's inline comment), so the fallback get_active_profile_name() path falls back to the process-global. For multi-tab scenarios where s.profile is None, this is bounded — see follow-up #4. ✓
#2012 — lineage report endpoint:
- Auth:
check_auth(self, parsed)runs at server.py:133 BEFORE dispatching tohandle_get. The new endpoint is auth-gated. - Path:
/api/session/lineage/report?session_id=<sid>— query param, not path component. No path-traversal vector. - SQL:
cur.execute("... WHERE s.id = ?", (row_id,))— parameterized. - Profile isolation:
_active_state_db_path()resolves to active profile'sstate.db. - Walk:
max_hops=20bound,seenset withparent_id in seencycle guard, uses_is_continuation_session(parent, current)filter. - Output: read-only structure with
mutation: False, no destructive proposals. ✓
#2015 — lineage transcript stitching: get_cli_session_messages at api/models.py:1666-1755 walks parent_session_id for up to 20 hops with seen cycle guard, gates on _is_continuation_session(parent_dict, current). SQL: parameterized via placeholders = ', '.join('?' for _ in session_chain) with positional binding. The merged dedup at api/routes.py:3033-3055 keys on (role, content, timestamp, tool_call_id, tool_name) — pathological collision possible for two distinct same-tuple messages but bounded since timestamp typically has sub-second precision. ✓
#2011 + #2014 sidebar composition: _sessionLineageKey at static/sessions.js:1979 returns null for s.session_source === 'fork' BEFORE any group lookup. Strict-equality so legacy forks (no session_source) still group with parent — backward-compatible. _collapseSessionLineageForSidebar at static/sessions.js:2102-2114 sorts by _compression_segment_count first, falls back to timestamp. ✓
Other audit — things that are correct already
- ✅ Profile stamping atomicity: capture at request entry → stamp at compression boundary → both on the same streaming thread. CPython string assignment is atomic. No tearing.
- ✅ Compression block ordering (PR body's tagged concern): #2006 stamps profile FIRST (line 3082), then #2004 sets anchor metadata (line 3115+), then single
s.save()at line 3185. Both writes finalize before persistence. No interleaving issue. - ✅ Lineage walk cycle protection: BOTH
read_session_lineage_reportandget_cli_session_messagesuse the same pattern:seen = {sid},if parent_id in seen: break, max 20 hops viarange(max_hops). Identical shape. - ✅ Lineage report endpoint auth: confirmed via
check_auth()at server.py:133 BEFOREhandle_getdispatch. All GET endpoints are auth-gated by the same path. - ✅ SQL parameterization: every cur.execute uses
?placeholders with positional binding. No string interpolation of user input. - ✅
_is_continuation_sessionconsistency: both lineage walkers (#2012 reader, #2015 stitcher) use the same predicate. Forks (session_source="fork") don't satisfy the continuation predicate, so they correctly don't get stitched into the lineage transcript even with #2014's marker. - ✅ Backward compat for legacy forks: pre-#2014 fork sessions lack
session_source="fork". The strict-equalityif (s.session_source==='fork')at sessions.js:1980 doesn't matchundefined/missing, so legacy forks still group with parent. No retroactive migration needed. - ✅ Merged-messages dedup: pathological tuple collision possible but bounded; sub-second timestamps prevent same-instant collisions in practice.
- ✅
compression_anchor_summarytruncation:_compact_summary_textat api/streaming.py:1539 caps at 320 chars with…ellipsis. Bounded UI string. - ✅ Helper duplication caveat (Opus advisor):
_visible_messages_for_anchor(nested in_handle_session_compress) vs_visible_messages_for_compression_anchor(module-level in streaming.py). Two near-identical implementations — worth consolidating in follow-up. - ✅
session_source="fork"is set ONLY on/api/session/branchcalls at api/routes.py:4267. Other session creation paths (compression continuation, /btw, gateway) don't set this field, so the strict-equality check correctly distinguishes forks from continuations. - ✅
_lineage_report_rowoutput shape:mutation: False, found: bool, segments: [...], children: [...], manual_review: bool. Pure read-only structure. Future UI can consume without server-state risk. - ✅ No XSS surface: lineage report returns JSON; the only string fields are
title,source,end_reasonfrom state.db — server-controlled (the agent owns state.db). - ✅
.gitignoreadds.venv/— minor housekeeping. Doesn't affect runtime.
Edge-case trace
| Scenario | Expected | Actual |
|---|---|---|
| Auto-compression on default profile | s.profile already set; stamp guard skips |
✅ |
| Auto-compression on non-default profile | continuation session inherits s.profile via stamp |
✅ |
Auto-compression on legacy session with s.profile=None |
falls back to get_active_profile_name(), may be process-global (multi-tab race) |
⚠ documented limitation |
| Compression card after page reload | compression_anchor_summary rendered from persisted state |
✅ |
Manual compression via /api/session/compress |
sets all 3 anchor fields + saves | ✅ |
| Lineage report with cycle in parent chain | seen set breaks the loop, manual_review: True |
✅ |
| Lineage report with > 20 hops | range(20) bound exits, manual_review: True |
✅ |
| Lineage report on missing session | _empty_lineage_report with found: False → 404 |
✅ |
| Lineage report SQL injection attempt | parameterized ? binding rejects |
✅ |
| Lineage report unauth | check_auth 401s before dispatch |
✅ |
| Continuation transcript stitching for messaging session | walks parent chain, dedups merged | ✅ |
| Continuation transcript on session with no parent | returns own messages only | ✅ |
| Sidebar collapse with touched parent vs newer compressed tip | newer (higher _compression_segment_count) wins |
✅ |
| Sidebar collapse with explicit fork in same lineage_root | fork excluded via session_source==='fork' early return |
✅ |
Legacy fork (no session_source) |
groups with parent (backward-compatible) | ✅ |
Branch endpoint sets session_source="fork" |
yes, at routes.py:4267 | ✅ |
Cross-tool: agent reads compression_anchor_* fields |
n/a — WebUI-internal only | ✅ |
Tests
tests/test_session_lineage_collapse.py: pass — new test for highest-segment-count vs touched-parent.tests/test_session_lineage_full_transcript.py(new): pass — sidecar + lineage merge integration test.tests/test_session_lineage_report.py(new): 196 LOC of behavioural tests against a real sqlite state.db fixture, covering bounded read-only output, cycle protection, hop limit, branch detection.tests/test_465_session_branching.py: pass — fork-marker + sidebar guard tests.tests/test_auto_compression_card.py: pass —sessionCompressionSummaryfallback in renderMessages.tests/test_sprint46.py: pass — pre-existing tests still green.- Full suite: 4952 passed, 59 skipped, 3 xpassed. 9 pre-existing macOS-shell failures unchanged from baseline (this branch predates stage-330's bash 3.2 fix).
Minor observations (non-blocking — Opus advisor follow-ups)
- Legacy fork classification — pre-#2014 fork sessions lack
session_source="fork"and remain grouped with parent lineage. If retroactive exclusion is intended, a one-time migration is needed. Likely acceptable since the strict-equality check is intentional backward-compat. - Merged-messages dedup partial key at api/routes.py:3033-3055 could collapse two genuinely distinct messages sharing
(role, content, timestamp, tool_call_id, tool_name). Pathological but possible. Would need richer dedup key (e.g., message_id) for full safety. - Helper duplication —
_visible_messages_for_anchor(nested in_handle_session_compress) vs_visible_messages_for_compression_anchor(module-level) — worth consolidating. - #2006 fallback race — for
s.profile=Nonelegacy sessions, fallback toget_active_profile_name()may pick up process-global from another tab. Bounded by the fix (most sessions haves.profileset), not eliminated. Documented in inline comment. get_cli_session_messagesSQL widening — went fromWHERE session_id = ?toWHERE session_id IN (?, ?, ...). Larger result set for stitched transcripts. Bounded by the 20-hop walk.- Lineage report endpoint URL shape: PR body says
/api/session/lineage-report/<sid>but actual code is/api/session/lineage/report?session_id=<sid>— minor inconsistency between docs and impl. Code is correct.
Recommendation
✅ Approved. Highest-coupling release of the batch — every PR touches session machinery — and stage merge clean across api/routes.py (4 PRs), api/streaming.py (2 PRs), api/models.py (2 PRs). All hunks at distinct anchors, no conflicts.
The two most critical fixes are independent and correct:
- #2006 profile stamping prevents memory writes from silently bleeding into the default profile's
MEMORY.mdafter auto-compression. Verified via the inline comment trace and atomicity reasoning. - #2012 lineage report endpoint correctly auth-gated, parameterized, profile-isolated, with bounded walks and cycle protection — matches the same shape as the existing #2015 stitcher.
Cross-tool safe (read-only against state.db, which the agent owns). No agent-side regression. The 4 Opus advisor caveats are all follow-up items, not blockers.
Parked at approval — ready for the release agent's merge/tag pipeline.
# Conflicts: # CHANGELOG.md
Release M — v0.51.37 — Compression / lineage backend (6 PRs)
Release M — v0.51.37 — Compression / lineage backend (6 PRs)
6 participants
Release M — v0.51.37 — Compression / lineage backend
Six contributor PRs from four authors. All on session lifecycle: compression boundary persistence, profile stamping, lineage collapse, fork-source marker, lineage report endpoint, and continuation-transcript stitching.
Constituent PRs
s.profileon continuation session after compression/api/session/lineage-report/<sid>diagnostic endpointsession_source="fork"marker; keep forks out of compression-lineage collapseget_cli_session_messages()Cross-PR interaction surface
This is the highest-coupling stage of the batch — every PR touches session machinery:
api/models.py: Persist compression boundary summary for reload UI #2004, Stitch continued session transcripts in WebUI #2015api/routes.py: Persist compression boundary summary for reload UI #2004, feat: add read-only session lineage report endpoint #2012, fix: keep explicit fork sessions out of compression lineage #2014, Stitch continued session transcripts in WebUI #2015api/streaming.py: Persist compression boundary summary for reload UI #2004, fix: stamp profile on continuation session after context compression #2006static/sessions.js: fix: prefer latest compressed session segment #2011, fix: keep explicit fork sessions out of compression lineage #2014api/agent_sessions.py: feat: add read-only session lineage report endpoint #2012 (+157 LOC reader)All hunks at distinct anchors; stage merge clean with no conflicts.
Tests
test_session_lineage_collapse.py,test_session_lineage_full_transcript.py,test_session_lineage_report.pyEspecially keen for your eyes on
/api/session/lineage-report/<sid>is a new surface. Auth gating + SID validation worth a look.Closes
#1833
Opus advisor
Running on stage diff. Will update with verdict before merge.