Skip to content

fix(api): strip system_prompt/model_config from session list rows - #43653

Closed
OmarB97 wants to merge 1 commit into
NousResearch:mainfrom
OmarB97:fix/session-list-strip-heavy-fields-upstream
Closed

fix(api): strip system_prompt/model_config from session list rows#43653
OmarB97 wants to merge 1 commit into
NousResearch:mainfrom
OmarB97:fix/session-list-strip-heavy-fields-upstream

Conversation

@OmarB97

@OmarB97 OmarB97 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Why

GET /api/sessions and GET /api/profiles/sessions (the dashboard/desktop list endpoints) return every row with the fully rendered system_prompt — tens of KB per row (~34KB average across 5,965 rows on a real install; 201MB total in that DB). A 21-row sidebar page weighed 524KB, 96% of it system_prompt that no list consumer reads:

  • the desktop's SessionInfo type doesn't declare system_prompt/model_config, and grep finds zero references in apps/desktop/src and web/src;
  • the desktop sidebar re-fetches the list on every message.complete;
  • the desktop command palette fetches 200 rows per open (~6.8MB of dead weight).

The gateway's client API already solves this exact problem with an allowlist projection (gateway/platforms/api_server.py::_session_response — "Avoid exposing full system prompts/model_config through the client API", surfacing has_system_prompt/has_model_config booleans). The dashboard list endpoints just never got the same treatment.

What changed

A list projection at the two dashboard list endpoints: rows omit system_prompt and model_config unless the caller passes ?full=1 (escape hatch for scripts that want complete rows). One module-level helper (_strip_session_list_rows), applied after row assembly in get_sessions and to the merged window in get_profiles_sessions.

Deliberately scoped to the HTTP list layer:

  • GET /api/sessions/{id} and the messages endpoint are untouched (detail reads stay complete).
  • Direct list_sessions_rich() callers (CLI session browse, TUI, tips, tools) are untouched.
  • /api/sessions/search builds its own result shape and is unaffected.
  • The gateway client API already projects these fields out (see above).

Measurements

Same live install, 50-row sidebar fetch (limit=50&min_messages=1&order=recent):

default (lean): HTTP 200 0.031s  20,395 bytes   ← 25.7× smaller
full=1:         HTTP 200 0.033s  524,162 bytes  ← byte-identical to pre-patch

Testing

  • 3 new tests in tests/hermes_cli/test_web_server.py: default strip on both endpoints (light fields survive), ?full=1 round-trips system_prompt/model_config.
  • Full file on this branch: 265/265 pass. ruff check clean.

Risks

  • External scripts that read system_prompt from LIST responses (rather than the detail endpoint or the client API) would need ?full=1. No in-repo consumer does, and the client API never exposed these fields at all.

@alt-glitch alt-glitch added type/perf Performance improvement or optimization P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard labels Jun 10, 2026
@OmarB97
OmarB97 force-pushed the fix/session-list-strip-heavy-fields-upstream branch from 7643d0a to 4d5fbaa Compare June 15, 2026 01:10
@alt-glitch alt-glitch added the comp/dashboard Web dashboard / control panel UI (dashboard/, landing) label Jun 26, 2026
Omar Baradei @claude
Refresh on upstream/main: resolve conflicts (no behavior change)
1e70eaa 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@OmarB97
OmarB97 force-pushed the fix/session-list-strip-heavy-fields-upstream branch from 4d5fbaa to 1e70eaa Compare July 5, 2026 18:47
This was referenced Jul 8, 2026
Closed
Merged
kshitijk4poor added a commit to kshitijk4poor/hermes-agent that referenced this pull request Jul 8, 2026
@kshitijk4poor
fix: follow-up for salvaged NousResearch#60347/NousResearch#43653/Nou…
075337a 
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
kshitijk4poor added a commit that referenced this pull request Jul 8, 2026
@kshitijk4poor
fix: follow-up for salvaged #60347/#43653/#47437
4f220fc 
- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original #47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
@kshitijk4poor

kshitijk4poor commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Merged via #60883 (rebase) — your commit landed on main as 4df16c4 with your authorship preserved. The API-layer strip + ?full=1 escape hatch is live, paired with the SQL-level compact_rows projection from #47437. Thanks!

@kshitijk4poor kshitijk4poor mentioned this pull request Jul 8, 2026
Closed
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
@kshitijk4poor
fix: follow-up for salvaged NousResearch#60347/NousResearch#43653/Nou…
61b9eb6 
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
justemu pushed a commit to justemu/hermes-agent that referenced this pull request Jul 18, 2026
@kshitijk4poor @justemu
fix: follow-up for salvaged NousResearch#60347/NousResearch#43653/Nou…
8ad2eab 
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
@kshitijk4poor
fix: follow-up for salvaged NousResearch#60347/NousResearch#43653/Nou…
9a2189a 
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
@kshitijk4poor
fix: follow-up for salvaged NousResearch#60347/NousResearch#43653/Nou…
017194e 
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
@kshitijk4poor
fix: follow-up for salvaged NousResearch#60347/NousResearch#43653/Nou…
6d6054c 
…sResearch#47437

- derive the compact_rows projection from SCHEMA_SQL (parse once, cache)
  instead of a hardcoded column list: the original NousResearch#47437 list was cut
  against a June schema and silently dropped session_key/chat_id/chat_type/
  thread_id/display_name/origin_json/expiry_finalized/git_branch/
  git_repo_root/compression_failure_* — including desktop sidebar fields.
  Schema-derived means declaratively reconciled new columns are included
  automatically; only system_prompt is excluded.
- guard test pinning the schema<->projection contract (mutation-verified:
  dropping a column from the projection fails it)
- wire compact_rows=(not full) into /api/sessions and /api/profiles/sessions
  so the SQL projection pairs with the API-level field strip (?full=1 still
  returns complete rows end-to-end)
- pass compact_rows at the remaining hot list callers: /api/status active
  count, _session_latest_descendant fallback, /api/sessions/stats by-source
- thread compact_rows through the compression-tip projection
  (_get_session_rich_row) so projected tips can't reintroduce the blob
- add pagination tests for get_messages (NousResearch#60347 shipped none): paging order,
  offset-past-end, active-flag interaction; add tip-projection compact test
- AUTHOR_MAP entries for mahdiwafy + CodeForgeNet (plain emails)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/dashboard Web dashboard / control panel UI (dashboard/, landing) P3 Low — cosmetic, nice to have type/perf Performance improvement or optimization

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@OmarB97 @kshitijk4poor @alt-glitch