Skip to content

fix(dashboard): keep status responsive when session db locks - #58238

Closed
tianma-if wants to merge 1 commit into
NousResearch:mainfrom
tianma-if:codex/fix-dashboard-status-sessiondb-timeout
Closed

tianma-if wants to merge 1 commit into
NousResearch:mainfrom
tianma-if:codex/fix-dashboard-status-sessiondb-timeout

Conversation

@tianma-if

Copy link
Copy Markdown
Contributor

Summary

  • move /api/status active-session counting onto a best-effort read-only SessionDB path
  • cap the active-session count with a short async timeout so a locked state.db cannot hold the status response
  • degrade the non-critical active_sessions field to 0 if the DB is locked or unavailable

Context

Addresses the SessionDB lock path reported in #57203. The existing open PRs for that issue focus on the gateway health probe timeout, but the issue follow-up narrowed the observed 21s delay to SessionDB().list_sessions_rich(limit=50) after read_runtime_status(). This PR targets that separate blocking path.

Tests

  • .venv/bin/python -m pytest tests/hermes_cli/test_web_server.py -q -k 'status_active_session_count or get_status_degrades or test_get_status'
  • .venv/bin/python -m pytest tests/hermes_cli/test_web_server.py -q
  • .venv/bin/python -m py_compile hermes_cli/web_server.py tests/hermes_cli/test_web_server.py
  • .venv/bin/python -m ruff check hermes_cli/web_server.py tests/hermes_cli/test_web_server.py
  • git diff --check

@alt-glitch alt-glitch added type/perf Performance improvement or optimization comp/dashboard Web dashboard / control panel UI (dashboard/, landing) P3 Low — cosmetic, nice to have labels Jul 4, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Comment

Summary

Keeps dashboard status responsive when session db locks by adding a read-only SessionDB connection with its own timeout (_STATUS_ACTIVE_SESSIONS_TIMEOUT = 0.75s) for counting active sessions on /api/status.

Looks Good

  • Read-only db connection prevents write contention
  • Independent timeout from main gateway health probe
  • Best-effort pattern: on timeout, returns 0 gracefully
  • Proper executor pattern for async-safe db access
  • No security concerns

Reviewed by Hermes Agent

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Comment (LGTM — token read-only, formal approval deferred to maintainer)

Dashboard status endpoint robustness fix: refactors active session counting into a dedicated async function with a 0.75s timeout and read-only database connection. This prevents /api/status from blocking or competing with write operations during session DB lock contention. The timeout degrades gracefully to 0 with debug logging.

Looks Good

  • Read-only SessionDB connection avoids write contention
  • Proper timeout on run_in_executor prevents blocking the dashboard response
  • Degrades to 0 on timeout or exception (not a hard failure)
  • Tests verify read-only flag is passed and lock failures return 0 gracefully

Reviewed by Hermes Agent

liuhao1024 pushed a commit to liuhao1024/hermes-agent that referenced this pull request Jul 8, 2026
Flip the handler from async def to sync def so FastAPI executes it in
its threadpool: the SessionDB open + list_sessions_rich query no longer
block the single uvicorn event loop.

Residual hunk from PR NousResearch#53966 — that PR's get_profiles_sessions flip
already landed via NousResearch#54523/1bb7b59c5, and its get_status offload is
superseded by NousResearch#58238's read_only + timeout variant in this branch.

(cherry picked from commit 414c12a)
kshitijk4poor added a commit to kshitijk4poor/hermes-agent that referenced this pull request Jul 8, 2026
Rebase reconciliation with NousResearch#60884: _count_status_active_sessions (from
NousResearch#58238) now passes compact_rows=True (this branch's NousResearch#47437 projection),
so the fake asserts both.
kshitijk4poor added a commit that referenced this pull request Jul 8, 2026
Rebase reconciliation with #60884: _count_status_active_sessions (from
#58238) now passes compact_rows=True (this branch's #47437 projection),
so the fake asserts both.
@kshitijk4poor

Copy link
Copy Markdown
Contributor

Merged via #60884 (rebase) — your commit landed on main as 9a4341a with your authorship preserved. Follow-up commits on top: a fresh-install guard (read-only opens require state.db to exist) and compact_rows=True in the count query. Thanks!

santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
Flip the handler from async def to sync def so FastAPI executes it in
its threadpool: the SessionDB open + list_sessions_rich query no longer
block the single uvicorn event loop.

Residual hunk from PR NousResearch#53966 — that PR's get_profiles_sessions flip
already landed via NousResearch#54523/6f78e9cd2, and its get_status offload is
superseded by NousResearch#58238's read_only + timeout variant in this branch.

(cherry picked from commit 414c12a)
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
Rebase reconciliation with NousResearch#60884: _count_status_active_sessions (from
NousResearch#58238) now passes compact_rows=True (this branch's NousResearch#47437 projection),
so the fake asserts both.
justemu pushed a commit to justemu/hermes-agent that referenced this pull request Jul 18, 2026
Flip the handler from async def to sync def so FastAPI executes it in
its threadpool: the SessionDB open + list_sessions_rich query no longer
block the single uvicorn event loop.

Residual hunk from PR NousResearch#53966 — that PR's get_profiles_sessions flip
already landed via NousResearch#54523/1bb7b59c5, and its get_status offload is
superseded by NousResearch#58238's read_only + timeout variant in this branch.

(cherry picked from commit 414c12a)
justemu pushed a commit to justemu/hermes-agent that referenced this pull request Jul 18, 2026
Rebase reconciliation with NousResearch#60884: _count_status_active_sessions (from
NousResearch#58238) now passes compact_rows=True (this branch's NousResearch#47437 projection),
so the fake asserts both.
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
Flip the handler from async def to sync def so FastAPI executes it in
its threadpool: the SessionDB open + list_sessions_rich query no longer
block the single uvicorn event loop.

Residual hunk from PR NousResearch#53966 — that PR's get_profiles_sessions flip
already landed via NousResearch#54523/1bb7b59c5, and its get_status offload is
superseded by NousResearch#58238's read_only + timeout variant in this branch.

(cherry picked from commit 414c12a)
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
Rebase reconciliation with NousResearch#60884: _count_status_active_sessions (from
NousResearch#58238) now passes compact_rows=True (this branch's NousResearch#47437 projection),
so the fake asserts both.
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
Flip the handler from async def to sync def so FastAPI executes it in
its threadpool: the SessionDB open + list_sessions_rich query no longer
block the single uvicorn event loop.

Residual hunk from PR NousResearch#53966 — that PR's get_profiles_sessions flip
already landed via NousResearch#54523/1bb7b59c5, and its get_status offload is
superseded by NousResearch#58238's read_only + timeout variant in this branch.

(cherry picked from commit 414c12a)
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
Rebase reconciliation with NousResearch#60884: _count_status_active_sessions (from
NousResearch#58238) now passes compact_rows=True (this branch's NousResearch#47437 projection),
so the fake asserts both.
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
Flip the handler from async def to sync def so FastAPI executes it in
its threadpool: the SessionDB open + list_sessions_rich query no longer
block the single uvicorn event loop.

Residual hunk from PR NousResearch#53966 — that PR's get_profiles_sessions flip
already landed via NousResearch#54523/2229a295a, and its get_status offload is
superseded by NousResearch#58238's read_only + timeout variant in this branch.

(cherry picked from commit 414c12a)
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
Rebase reconciliation with NousResearch#60884: _count_status_active_sessions (from
NousResearch#58238) now passes compact_rows=True (this branch's NousResearch#47437 projection),
so the fake asserts both.
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
Flip the handler from async def to sync def so FastAPI executes it in
its threadpool: the SessionDB open + list_sessions_rich query no longer
block the single uvicorn event loop.

Residual hunk from PR NousResearch#53966 — that PR's get_profiles_sessions flip
already landed via NousResearch#54523/1bb7b59c5, and its get_status offload is
superseded by NousResearch#58238's read_only + timeout variant in this branch.

(cherry picked from commit 414c12a)
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
Rebase reconciliation with NousResearch#60884: _count_status_active_sessions (from
NousResearch#58238) now passes compact_rows=True (this branch's NousResearch#47437 projection),
so the fake asserts both.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/dashboard Web dashboard / control panel UI (dashboard/, landing) P3 Low — cosmetic, nice to have type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants