Skip to content

fix(agent): rehydrate session accumulators from SQLite on restart (#67762) - #67796

Open
JonthanaHanh wants to merge 1 commit into
NousResearch:mainfrom
JonthanaHanh:fix/session-cost-rehydrate-on-restart
Open

fix(agent): rehydrate session accumulators from SQLite on restart (#67762)#67796
JonthanaHanh wants to merge 1 commit into
NousResearch:mainfrom
JonthanaHanh:fix/session-cost-rehydrate-on-restart

Conversation

@JonthanaHanh

Copy link
Copy Markdown
Contributor

Problem

When the gateway restarts mid-session, init_agent() resets all session accumulators (session_estimated_cost_usd, token counts, session_api_calls) to zero. The persisted data in SQLite (sessions.estimated_cost_usd, session_model_usage rows) is correct, but nothing copies it back into the agent's in-memory state.

This means:

  • The live cost counter shows only post-restart costs
  • Token totals reset to zero
  • /insights dashboard shows the correct total, but the agent's runtime state lies

Verified against main (commit 26480e6c5).

Reproduction

  1. Start a session, run several turns (cost accumulates to, say, $4.27).
  2. Restart the gateway process.
  3. Resume the session by sending a new message.
  4. Inspect agent.session_estimated_cost_usd -- shows $0.0 (or only post-restart cost).
  5. Check sessions.estimated_cost_usd in ~/.hermes/state.db -- correctly shows the full total.

Fix

After the accumulator reset block in agent/agent_init.py, read the existing session row from SQLite (if present) and restore the persisted totals:

  • session_estimated_cost_usd
  • session_cost_status / session_cost_source
  • All token counters (input, output, cache, reasoning)
  • session_api_calls

For new sessions, get_session() returns None and the block is a no-op. DB errors are caught and swallowed (fail-open).

Files Changed

  • agent/agent_init.py -- add rehydration block after accumulator reset

Fixes #67762

When the gateway restarts mid-session, init_agent() resets all session
accumulators (cost, tokens, API call count) to zero. The persisted data
in SQLite is correct, but nothing copies it back into the agent state.

After resetting accumulators, read the existing session row from SQLite
(if present) and restore the persisted totals. No-op for new sessions.

Fixes NousResearch#67762
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/billing Account usage, credit usage, billing (cross-cutting) area/usage-cost Token accounting, usage reporting, billing, cost tracking sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state needs-decision Awaiting maintainer decision before any implementation labels Jul 20, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to #67770 and #67790. This PR restores additional direct session-row accumulators, while #67770 uses tested per-model cost aggregation and #67790 addresses sticky status semantics; please choose or consolidate the overlapping rehydration work.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for targeting the live-accounting reset; current main still resets these fields at agent/agent_init.py:2522-2533.

Problems

  • The proposed get_session() source is incomplete for this purpose. record_auxiliary_usage() intentionally writes only to session_model_usage, not sessions (hermes_state.py:4920-4930), so this restores an undercount after any auxiliary call.
  • agent/conversation_loop.py:2863-2871 maintains prompt/completion/total separately from canonical input/output/cache/reasoning totals. Mapping persisted input_tokens and output_tokens into both counter families changes their meaning after restart.
  • Commit 1a3d88d changes no tests, so the restart and source-divergence contract is unpinned.

Suggested changes

  • Aggregate the durable session_model_usage rows for the rehydrated fields, and preserve the distinction between canonical and display token counters.
  • Add a restart regression test using the same temporary state database, including an auxiliary-usage row.

The reset hunk is still present on current main, but the accounting source and counter mapping need rework. This is an automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform area/sessions Session lifecycle, resume, persistence, history labels Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/billing Account usage, credit usage, billing (cross-cutting) area/sessions Session lifecycle, resume, persistence, history area/usage-cost Token accounting, usage reporting, billing, cost tracking comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: agent.session_estimated_cost_usd resets to $0 on gateway restart

3 participants