Skip to content

fix(api): add pagination to GET /api/sessions/{id}/messages - #60347

Closed
mahdiwafy wants to merge 1 commit into
NousResearch:mainfrom
mahdiwafy:fix/session-messages-pagination
Closed

fix(api): add pagination to GET /api/sessions/{id}/messages#60347
mahdiwafy wants to merge 1 commit into
NousResearch:mainfrom
mahdiwafy:fix/session-messages-pagination

Conversation

@mahdiwafy

Copy link
Copy Markdown
Contributor

Summary

Adds optional limit and offset query parameters to the session messages endpoint. Previously returned ALL messages in a single response — sessions with 500+ messages produced 1.2–1.6 MB JSON payloads causing GIL starvation and WebSocket timeouts on Desktop (#60155).

Changes

  • hermes_state.py: SessionDB.get_messages() accepts optional limit and offset params. Uses SQL LIMIT ? OFFSET ? when provided. No-op when omitted (backward compatible).
  • hermes_cli/web_server.py: GET /api/sessions/{id}/messages accepts limit and offset query params. Limit clamped to 500 max. Response includes pagination: {limit, offset, returned}.

Backward Compatibility

Callers that omit limit get the old behavior (all messages). No breaking change to existing API consumers.

Example

GET /api/sessions/20260707_102815_c4d80b/messages?limit=50&offset=0
→ {"session_id": "...", "messages": [...50 items...], "pagination": {"limit": 50, "offset": 0, "returned": 50}}

Closes #60155

The session messages endpoint returned ALL messages in a single
response with no limit/offset. Sessions with 500+ messages produced
1.2-1.6 MB JSON payloads, causing GIL starvation and WebSocket
timeouts on the Desktop client (NousResearch#60155).

Add optional limit/offset query params to both the API endpoint and
SessionDB.get_messages(). Limit clamped to 500 max per page. Response
now includes a pagination object with limit/offset/returned count.

Backward compatible: callers that omit limit get the old behavior
(all messages).

Closes NousResearch#60155
@alt-glitch alt-glitch added type/perf Performance improvement or optimization comp/dashboard Web dashboard / control panel UI (dashboard/, landing) P3 Low — cosmetic, nice to have labels Jul 7, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: #60155 (the bug this closes), #38370 (the pagination feature request), and #38942 (an open competing PR that adds limit/offset to this same endpoint plus /api/jobs). This is a competing/overlapping implementation of the same session-messages pagination — flagging the cluster so a maintainer can pick between this and #38942.

@mahdiwafy

Copy link
Copy Markdown
Contributor Author

Superseded by #60883 which includes this pagination work plus additional improvements (compact_rows SQL projection, system_prompt stripping, offset-without-limit fix). Authorship preserved via cherry-pick. Closing in favor of the salvage PR.

@mahdiwafy mahdiwafy closed this Jul 8, 2026
kshitijk4poor added a commit to kshitijk4poor/hermes-agent that referenced this pull request Jul 8, 2026
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
kshitijk4poor added a commit that referenced this pull request Jul 8, 2026
- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original #47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Merged via #60883 (rebase) — your pagination commit landed on main as 0d5549a with your authorship preserved. One follow-up on top: offset-without-limit was silently ignored (SQLite requires LIMIT for OFFSET), now emitted as LIMIT -1, plus regression tests for paging order and the soft-deleted-row interaction. Thanks!

santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
justemu pushed a commit to justemu/hermes-agent that referenced this pull request Jul 18, 2026
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/dashboard Web dashboard / control panel UI (dashboard/, landing) P3 Low — cosmetic, nice to have type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: GET /api/sessions/{id}/messages has no pagination — loads entire transcript at once, freezes Desktop on large sessions

3 participants