fix(api): add pagination to GET /api/sessions/{id}/messages - #60347
fix(api): add pagination to GET /api/sessions/{id}/messages#60347mahdiwafy wants to merge 1 commit into
Conversation
The session messages endpoint returned ALL messages in a single response with no limit/offset. Sessions with 500+ messages produced 1.2-1.6 MB JSON payloads, causing GIL starvation and WebSocket timeouts on the Desktop client (NousResearch#60155). Add optional limit/offset query params to both the API endpoint and SessionDB.get_messages(). Limit clamped to 500 max per page. Response now includes a pagination object with limit/offset/returned count. Backward compatible: callers that omit limit get the old behavior (all messages). Closes NousResearch#60155
Related: #60155 (the bug this closes), #38370 (the pagination feature request), and #38942 (an open competing PR that adds |
|
Superseded by #60883 which includes this pagination work plus additional improvements (compact_rows SQL projection, system_prompt stripping, offset-without-limit fix). Authorship preserved via cherry-pick. Closing in favor of the salvage PR. |
…sResearch#47437 - derive the compact_rows projection from SCHEMA_SQL (parse once, cache) instead of a hardcoded column list: the original NousResearch#47437 list was cut against a June schema and silently dropped session_key/chat_id/chat_type/ thread_id/display_name/origin_json/expiry_finalized/git_branch/ git_repo_root/compression_failure_* — including desktop sidebar fields. Schema-derived means declaratively reconciled new columns are included automatically; only system_prompt is excluded. - guard test pinning the schema<->projection contract (mutation-verified: dropping a column from the projection fails it) - wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions so the SQL projection pairs with the API-level field strip (?full=1 still returns complete rows end-to-end) - pass compact_rows at the remaining hot list callers: /api/status active count, _session_latest_descendant fallback, /api/sessions/stats by-source - thread compact_rows through the compression-tip projection (_get_session_rich_row) so projected tips can't reintroduce the blob - add pagination tests for get_messages (NousResearch#60347 shipped none): paging order, offset-past-end, active-flag interaction; add tip-projection compact test - AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
- derive the compact_rows projection from SCHEMA_SQL (parse once, cache) instead of a hardcoded column list: the original #47437 list was cut against a June schema and silently dropped session_key/chat_id/chat_type/ thread_id/display_name/origin_json/expiry_finalized/git_branch/ git_repo_root/compression_failure_* — including desktop sidebar fields. Schema-derived means declaratively reconciled new columns are included automatically; only system_prompt is excluded. - guard test pinning the schema<->projection contract (mutation-verified: dropping a column from the projection fails it) - wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions so the SQL projection pairs with the API-level field strip (?full=1 still returns complete rows end-to-end) - pass compact_rows at the remaining hot list callers: /api/status active count, _session_latest_descendant fallback, /api/sessions/stats by-source - thread compact_rows through the compression-tip projection (_get_session_rich_row) so projected tips can't reintroduce the blob - add pagination tests for get_messages (#60347 shipped none): paging order, offset-past-end, active-flag interaction; add tip-projection compact test - AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
|
Merged via #60883 (rebase) — your pagination commit landed on main as 0d5549a with your authorship preserved. One follow-up on top: offset-without-limit was silently ignored (SQLite requires LIMIT for OFFSET), now emitted as LIMIT -1, plus regression tests for paging order and the soft-deleted-row interaction. Thanks! |
…sResearch#47437 - derive the compact_rows projection from SCHEMA_SQL (parse once, cache) instead of a hardcoded column list: the original NousResearch#47437 list was cut against a June schema and silently dropped session_key/chat_id/chat_type/ thread_id/display_name/origin_json/expiry_finalized/git_branch/ git_repo_root/compression_failure_* — including desktop sidebar fields. Schema-derived means declaratively reconciled new columns are included automatically; only system_prompt is excluded. - guard test pinning the schema<->projection contract (mutation-verified: dropping a column from the projection fails it) - wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions so the SQL projection pairs with the API-level field strip (?full=1 still returns complete rows end-to-end) - pass compact_rows at the remaining hot list callers: /api/status active count, _session_latest_descendant fallback, /api/sessions/stats by-source - thread compact_rows through the compression-tip projection (_get_session_rich_row) so projected tips can't reintroduce the blob - add pagination tests for get_messages (NousResearch#60347 shipped none): paging order, offset-past-end, active-flag interaction; add tip-projection compact test - AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
…sResearch#47437 - derive the compact_rows projection from SCHEMA_SQL (parse once, cache) instead of a hardcoded column list: the original NousResearch#47437 list was cut against a June schema and silently dropped session_key/chat_id/chat_type/ thread_id/display_name/origin_json/expiry_finalized/git_branch/ git_repo_root/compression_failure_* — including desktop sidebar fields. Schema-derived means declaratively reconciled new columns are included automatically; only system_prompt is excluded. - guard test pinning the schema<->projection contract (mutation-verified: dropping a column from the projection fails it) - wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions so the SQL projection pairs with the API-level field strip (?full=1 still returns complete rows end-to-end) - pass compact_rows at the remaining hot list callers: /api/status active count, _session_latest_descendant fallback, /api/sessions/stats by-source - thread compact_rows through the compression-tip projection (_get_session_rich_row) so projected tips can't reintroduce the blob - add pagination tests for get_messages (NousResearch#60347 shipped none): paging order, offset-past-end, active-flag interaction; add tip-projection compact test - AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
…sResearch#47437 - derive the compact_rows projection from SCHEMA_SQL (parse once, cache) instead of a hardcoded column list: the original NousResearch#47437 list was cut against a June schema and silently dropped session_key/chat_id/chat_type/ thread_id/display_name/origin_json/expiry_finalized/git_branch/ git_repo_root/compression_failure_* — including desktop sidebar fields. Schema-derived means declaratively reconciled new columns are included automatically; only system_prompt is excluded. - guard test pinning the schema<->projection contract (mutation-verified: dropping a column from the projection fails it) - wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions so the SQL projection pairs with the API-level field strip (?full=1 still returns complete rows end-to-end) - pass compact_rows at the remaining hot list callers: /api/status active count, _session_latest_descendant fallback, /api/sessions/stats by-source - thread compact_rows through the compression-tip projection (_get_session_rich_row) so projected tips can't reintroduce the blob - add pagination tests for get_messages (NousResearch#60347 shipped none): paging order, offset-past-end, active-flag interaction; add tip-projection compact test - AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
…sResearch#47437 - derive the compact_rows projection from SCHEMA_SQL (parse once, cache) instead of a hardcoded column list: the original NousResearch#47437 list was cut against a June schema and silently dropped session_key/chat_id/chat_type/ thread_id/display_name/origin_json/expiry_finalized/git_branch/ git_repo_root/compression_failure_* — including desktop sidebar fields. Schema-derived means declaratively reconciled new columns are included automatically; only system_prompt is excluded. - guard test pinning the schema<->projection contract (mutation-verified: dropping a column from the projection fails it) - wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions so the SQL projection pairs with the API-level field strip (?full=1 still returns complete rows end-to-end) - pass compact_rows at the remaining hot list callers: /api/status active count, _session_latest_descendant fallback, /api/sessions/stats by-source - thread compact_rows through the compression-tip projection (_get_session_rich_row) so projected tips can't reintroduce the blob - add pagination tests for get_messages (NousResearch#60347 shipped none): paging order, offset-past-end, active-flag interaction; add tip-projection compact test - AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
…sResearch#47437 - derive the compact_rows projection from SCHEMA_SQL (parse once, cache) instead of a hardcoded column list: the original NousResearch#47437 list was cut against a June schema and silently dropped session_key/chat_id/chat_type/ thread_id/display_name/origin_json/expiry_finalized/git_branch/ git_repo_root/compression_failure_* — including desktop sidebar fields. Schema-derived means declaratively reconciled new columns are included automatically; only system_prompt is excluded. - guard test pinning the schema<->projection contract (mutation-verified: dropping a column from the projection fails it) - wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions so the SQL projection pairs with the API-level field strip (?full=1 still returns complete rows end-to-end) - pass compact_rows at the remaining hot list callers: /api/status active count, _session_latest_descendant fallback, /api/sessions/stats by-source - thread compact_rows through the compression-tip projection (_get_session_rich_row) so projected tips can't reintroduce the blob - add pagination tests for get_messages (NousResearch#60347 shipped none): paging order, offset-past-end, active-flag interaction; add tip-projection compact test - AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
Summary
Adds optional
limitandoffsetquery parameters to the session messages endpoint. Previously returned ALL messages in a single response — sessions with 500+ messages produced 1.2–1.6 MB JSON payloads causing GIL starvation and WebSocket timeouts on Desktop (#60155).Changes
SessionDB.get_messages()accepts optionallimitandoffsetparams. Uses SQLLIMIT ? OFFSET ?when provided. No-op when omitted (backward compatible).GET /api/sessions/{id}/messagesacceptslimitandoffsetquery params. Limit clamped to 500 max. Response includespagination: {limit, offset, returned}.Backward Compatibility
Callers that omit
limitget the old behavior (all messages). No breaking change to existing API consumers.Example
Closes #60155