Skip to content

fix(api): serve full compression lineage in session messages endpoints (#79565) - #79785

Closed
thatssoheil wants to merge 2 commits into
NousResearch:mainfrom
thatssoheil:fix/session-messages-lineage
Closed

thatssoheil wants to merge 2 commits into
NousResearch:mainfrom
thatssoheil:fix/session-messages-lineage

Conversation

@thatssoheil

Copy link
Copy Markdown
Contributor

Fix: serve full compression lineage in session messages endpoints (#79565)

What

The desktop session-switch view showed only the latest compression
segment
of a long conversation — earlier segments appeared lost. Two
endpoints shared the same child-only fetch bug:

  1. gateway/platforms/api_server.py _handle_session_messages
    resolved to the latest compression child and loaded only that child's
    rows via get_messages.
  2. hermes_cli/web_routers/sessions.py get_session_messages (the
    dashboard endpoint) — same child-only resolution.

Fix

Both endpoints now serve the full compression lineage:

  • api_server.py: resolve forward to the compression tip, then reuse the
    existing, tested get_resume_conversations(tip) display_history — which
    already walks the whole lineage root→tip and reconciles replayed-user
    messages (the exact overlap handling the issue asks for).
  • sessions.py: use get_messages_as_conversation(include_ancestors=True)
    and page over the full lineage (limit/offset contract preserved).

No new lineage architecture — the projection already existed; this is a
surgical endpoint change. _message_response shape compatibility preserved.

Tests

  • tests/gateway/test_session_messages_lineage.py (new): gateway endpoint
    returns the full root→tip lineage; uncompressed sessions unchanged.
  • tests/hermes_cli/test_web_server.py (extended): dashboard endpoint
    returns full lineage, paginates across segments, guard tests for
    negative/oversized limits.

All fail on the old child-only code for the right reason ('early question' in ['latest question', 'latest answer']), pass with the fix. 155 tests
green across the three neighboring suites.

Prior art

This was previously attempted (#71857) with custom lineage projection and
caused duplicates; reusing get_resume_conversations display_history
avoids re-introducing that class of bug.

Copilot AI lite review requested due to automatic review settings August 6, 2026 00:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery comp/cli CLI entry point, hermes_cli/, setup wizard comp/desktop Electron desktop app (apps/desktop/*) area/sessions Session lifecycle, resume, persistence, history area/compression Context compression and continuation sessions sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Aug 6, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Related: #59661 addresses the same session-message endpoints with ancestor-inclusive reads. This PR uses the existing display-history projection to preserve replay-overlap handling, so these are competing approaches rather than duplicates.

@thatssoheil

Copy link
Copy Markdown
Contributor Author

Thanks for the pointer. I checked #59661 — it's the same bug class (#51058: compression rotation hides the parent transcript) across the same three surfaces (api_server messages, web_routers/sessions.py REST, hermes_state lineage read). Noted, and I see it's being actively shepherded (rebased on current main, sweeper keep_open).

The approaches differ in one material way worth flagging for the maintainers:

  • fix(rest): include compression-ancestor messages in /api/sessions/{id}/messages (#51058) #59661 adds include_ancestors to get_messages() at the SQL level — raw rows from the whole lineage, no replay-overlap handling.
  • This PR reads the lineage through get_messages_as_conversation(include_ancestors=True) / get_resume_conversations, which applies the existing replayed-user dedup + _rows_to_conversation shape normalization (same projection session.resume uses). For the desktop prefetch this matters: a compression rotation replays the parent's tail user turn into the child, and the raw SQL path can show it twice while the resume path dedups it — the prefetch and the resume would disagree.

So they're genuinely competing approaches, not duplicates. Happy to close this PR if maintainers prefer #59661's simpler SQL-level read; if the dedup-consistency argument wins, this one is the safer merge. Either way, no hard feelings — flagging for the triage decision.

…NousResearch#79565)

Desktop session switching prefetches GET /api/sessions/{id}/messages. The
endpoint resolved the id to the latest compression child and loaded only
that child's rows, so a compressed conversation showed only the latest
continuation segment while older dialogue stayed invisible (still intact
in the parent sessions of state.db).

Fix: resolve to the live tip (forward, resolve_resume_session_id), then
serve the FULL lineage display history (backward, ancestors -> tip) via
get_resume_conversations() - the same projection the CLI/TUI resume path
uses, which already reconciles overlap copied into continuation children.
The model-fed tip history is unchanged; only the display transcript is
now complete.

RED-GREEN: lineage test fails on old code (child-only rows), passes with
fix. Uncompressed sessions unchanged.
NousResearch#79565)

Both session-messages endpoints resolved to the latest compression child
and loaded only that child's rows, so after a compression fork the desktop
showed only the most recent segment - earlier dialogue silently vanished.

- gateway/platforms/api_server.py _handle_session_messages: resolve to the
  tip (forward), then serve the FULL lineage display history via
  get_resume_conversations() (ancestors -> tip, replayed-user dedup).
- hermes_cli/web_routers/sessions.py get_session_messages: page over the
  full lineage projection (get_messages_as_conversation include_ancestors)
  instead of get_messages(sid); pagination contract preserved.

RED-GREEN: lineage tests fail on old code (child-only rows), pass with the
fix, for both endpoints.
@thatssoheil
thatssoheil force-pushed the fix/session-messages-lineage branch from a1d6fc2 to 3d1d872 Compare August 29, 2026 03:39
@thatssoheil

Copy link
Copy Markdown
Contributor Author

Following @friendfish's triage below (thank you - the live trace and the #59661 status check are exactly the state-of-record this thread needed): #59661 (opened 2026-07-06, a month before this issue) already implements the same fix class on the same endpoints, it is green and mergeable, and the remaining findings on it (@ehz0ah's API-fork walk + the rotation-clone residue) are tracked in that thread. I am closing this PR in its favor instead of keeping a parallel implementation open - see my post on #59661 for the two findings my branch carried that are worth folding in there (cross-segment rotation-clone dedupe for non-user roles, and branch gating on the REST path).

Closing as duplicate; will delete the branch after this lands on the record. If #59661's shape changes in review, this thread's 5-segment lineage repro (326 raw rows -> 67 shown) remains the reference test case.

@thatssoheil

Copy link
Copy Markdown
Contributor Author

Duplicate of #59661 (predates the issue; see close-comment above for the two findings worth folding in).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/compression Context compression and continuation sessions area/sessions Session lifecycle, resume, persistence, history comp/cli CLI entry point, hermes_cli/, setup wizard comp/desktop Electron desktop app (apps/desktop/*) comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants