Skip to content

fix(profiles): migrate session/routing identity on profile rename - #111927

Closed
xielevi wants to merge 2 commits into
NousResearch:mainfrom
xielevi:fix/profile-rename-migrate-identity
Closed

xielevi wants to merge 2 commits into
NousResearch:mainfrom
xielevi:fix/profile-rename-migrate-identity

Conversation

@xielevi

@xielevi xielevi commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Bug Description

Renaming a multiplexed profile leaves the old profile name baked into persisted session/routing
state, so the gateway keeps routing to a profile that no longer exists. With a bot still connected
to the renamed profile's chats, errors.log fills with Profile 'foo' does not exist ... falling back to global HERMES_HOME every few seconds, and renamed sessions drop out of the Desktop sidebar
/ break their @session: deep links (sessions.profile_name still names the old profile).

Fixes #111926

Root Cause

rename_profile moves profiles/<old>/ to profiles/<new>/ (row data travels with the directory)
and already migrates the alias, Honcho blocks, active_profile, and the live multiplexer's adapters.
But the profile name is also baked into keys/values the move does not touch, none of which were
migrated:

  • session-key namespace agent:<old>:* — the routing index (gateway_routing in the root
    state.db + sessions.json mirror) and the renamed profile's own sessions.session_key rows.
  • sessions.profile_name — read by the fail-closed owner ladder, Desktop sidebar scope, and
    @session:<profile>/<id> deep links.
  • gateway_heartbeats.profile and delivery_obligations (session_key + adapter_profile).

The routing index is load-bearing: a live multiplexer holds it in memory (SessionStore._entries)
and writes it back periodically, so a direct DB rewrite alone is clobbered on the next save — the
old namespace resurfaces until the gateway restarts.

This is the session-store sibling of the earlier ghost-profile rename fix (that covered the
filesystem/adapter layer; this covers the persisted identity layer).

Fix

Migrate profile-name-keyed state, split by who owns the store:

  • SessionDB.rekey_profile_state(old, new) — durable rewrite of all four state.db tables
    (session-key namespaces, profile_name, heartbeats, delivery rows, and the routing index key +
    the profile embedded in its JSON payload). Idempotent; skips delivery_obligations when the
    ledger has not created it yet.
  • SessionStore.rekey_profile_routing(old, new) — rekey the in-memory routing index (keys +
    origin.profile) and persist, the half a DB write cannot reach. Leaves a pre-existing new-name
    key untouched rather than merging.
  • Control-socket verb migrate-profile-identity (carrying {old,new}) so a live gateway does both.
    The socket protocol now passes params only to handlers that declare a params argument; bare
    handlers (identify/status/rescan/pause) are called with no args exactly as before.
  • rename_profile calls the verb when a multiplexer is live; otherwise it performs the durable DB
    rewrite itself (safe — nothing else holds the store open). Never fatal to the rename.
  • hermes profile migrate-identity <old> <new> — standalone retry for the case where the live verb
    could not migrate: the rename cannot simply be repeated (profiles/<old> is gone) and the CLI must
    not rewrite a routing DB a live gateway holds in memory. Delegates to the verb while a multiplexer
    is live, performs the durable rewrite itself when none is, is idempotent, and exits non-zero naming
    the database on a collision, a lock, or a partial failure. A gateway that is running but does not
    implement the verb (identify answers, the migrate verb does not) is reported as such.
    _migrate_profile_identity returns that success/failure result so the command can set its exit code,
    and the rename warning now names the exact invocation.

Checkpoints keyed by the profile's workdir path are a known related gap, called out in the issue and
left for a separate change.

How to Verify

  1. Multiplex gateway (gateway.multiplex_profiles: true) serving a secondary foo with a chat.
  2. hermes profile rename foo bar.
  3. errors.log no longer logs Profile 'foo' does not exist; state.db/sessions.json contain
    agent:bar:* keys and profile_name = 'bar', none under foo. No gateway restart required.
  4. Failure path: with a gateway that cannot answer the verb, the rename warns and prints
    hermes profile migrate-identity foo bar. That command exits non-zero while the gateway still
    declines, exits 0 once it has been restarted (which reloads the routing index, so the live rekey
    lands) or stopped (the durable rewrite is then safe), and running it twice in a row rekeys nothing
    the second time.

Test Plan

  • tests/hermes_state/test_rekey_profile_state.py — all four tables, routing JSON key +
    embedded profile, idempotent, no-op on equal/empty names.
  • tests/gateway/test_rekey_profile_routing.py — in-memory namespace + origin.profile
    rewrite, no-op, no-overwrite of an existing target key.
  • tests/gateway/test_control_socket.py::test_verb_handler_receives_params — params reach a
    declaring handler; bare handlers unaffected.
  • tests/hermes_cli/test_profiles.py — rename end-to-end for both the live-gateway (delegates
    to the verb, no direct DB write) and no-gateway (durable rewrite lands) paths.
  • tests/hermes_cli/test_profiles.py — review follow-up: the retry command repairs the
    failed-live-migration end state in both stores and is idempotent; a live gateway answering with an
    unusable payload exits non-zero, quotes the raw answer, names the retry command, and still performs
    no direct DB rewrite.
  • All 10 new tests proven red on this branch's base (implementation stashed) and green with it.
    tests/hermes_cli/ and tests/gateway/test_control_socket.py show no new failures — the 13
    failures in the test_update_* files are pre-existing on this branch's base (identical with the
    implementation stashed). ruff clean.

Risk Assessment

Low–Medium. New code paths only run during hermes profile rename; the control verb is additive and
the protocol change is backward-compatible (bare handlers keep their zero-arg signature). The
in-memory rekey reuses SessionStore's own lock and persist path. Blast radius is confined to the
rename flow and the new rekey_* methods.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery comp/cli CLI entry point, hermes_cli/, setup wizard sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Sep 15, 2026
@xielevi
xielevi force-pushed the fix/profile-rename-migrate-identity branch from b0dd2c0 to ccb72e4 Compare September 15, 2026 14:37
Renaming a profile moved profiles/<old>/ to profiles/<new>/, so the row DATA
travelled with the directory, but the profile name is also baked into
keys/values the move left untouched: session keys (agent:<old>:* namespace),
sessions.profile_name (fail-closed owner ladder / Desktop sidebar scope /
@session: deep links), sessions.origin_json.profile,
gateway_heartbeats.profile, delivery_obligations (session_key +
adapter_profile), telegram_dm_topic_* profile_name bindings, and the
gateway_routing index. Left stale, every inbound event on a chat keyed to the
old name resolved to a profile that no longer exists — flooding errors.log
with "Profile <old> does not exist ... falling back to global HERMES_HOME"
every few seconds — and renamed sessions dropped out of the sidebar / broke
their deep links.

The routing index is held in memory by a live multiplexer and written back
periodically, so a CLI-side DB rewrite alone is clobbered. Fix in layers:

- SessionDB.rekey_profile_state: atomic durable rewrite of the state.db
  tables, matching the agent:<name>: namespace by exact prefix (substr, not
  LIKE — '_' is a legal profile-name character and a LIKE wildcard), rewriting
  the profile inside routing/origin JSON, and REFUSING on a target collision
  (routing rows or telegram bindings) instead of silently merging.
- SessionStore.rekey_profile_routing: rekey the in-memory routing index
  (keys + origin.profile) then persist — the half a DB write cannot reach.
  Raises on a target-key collision before mutating.
- Control verb migrate-profile-identity (params-carrying; the socket passes
  params only to handlers that declare them, bare handlers unchanged) so a
  live gateway rekeys its in-memory copy AND both durable stores (routing home
  + the renamed profile's own state.db).
- rename_profile calls the verb when a multiplexer is live and, if it fails,
  does NOT fall back to a racing CLI-side write: it prints a warning telling
  the operator to restart the gateway and retry. With no live gateway it
  performs the durable rewrite itself (safe: nothing else holds the store
  open).

Checkpoints keyed by the profile's workdir path are a known related gap,
tracked separately, not addressed here.

Tests: rekey_profile_state (all tables, routing/origin JSON, collisions,
idempotent, no-op), rekey_profile_routing (namespace + origin, no-op, no
overwrite), control verb param passing, and rename end-to-end for both the
live-gateway (delegates, refuses unsafe fallback) and no-gateway (durable
rewrite) paths.
@xielevi
xielevi force-pushed the fix/profile-rename-migrate-identity branch from ccb72e4 to 9143428 Compare September 15, 2026 14:39
@kyssta-exe

Copy link
Copy Markdown
Contributor

Review: fix(profiles): migrate session/routing identity on profile rename

Summary
Renaming a multiplexed profile now also rekeys the profile name baked into session/routing state (agent:<old>:* routing keys, sessions.profile_name, heartbeats, delivery/routing index). Without this the gateway routed to a nonexistent profile on every inbound event and sessions fell out of the sidebar. Fixes #111926.

What changed

  • hermes_cli/profiles.py — rename_profile calls new _migrate_profile_identity before hot-serving the renamed profile.
  • Live gateway: migration via gateway/control_socket.migrate_gateway_profile_identity (avoids racing the in-memory routing index); offline: rekey_profile_state on root + profile state.db.
  • gateway/session.py, gateway/run.py, hermes_state_gateway.py + three test files covering rekey and control-socket paths.

Strengths

  • Correctly distinguishes the live vs. offline migration paths instead of writing the DB under a running gateway.
  • Per-DB try/except in the offline loop with release_or_close in finally — one bad DB doesn't abort the other.

Findings

  • hermes_cli/profiles.py (_migrate_profile_identity): when live_mux is true and the control-socket migration fails, the function warns and returns — it never attempts the offline rekey, and the rename itself has already completed. The warning says "restart the gateway, then retry the identity migration," but there is no standalone retry command; the operator's only recourse is unclear. Either fall through to the offline path when the gateway is about to be restarted anyway, or document/ship the exact retry invocation.
  • Minor: a non-dict truthy answer with ok is not True yields failure = None and a generic warning — include the raw answer for diagnosability.

Verdict
Needs minor polish (failed-live-migration retry path).

Reviewed using Hermes-Agent

A rename under a live multiplexer that could not reach the control verb warned and
stopped there, leaving the operator with no way to finish: the rename cannot be
repeated (profiles/<old> is gone) and the CLI deliberately never rewrites the
routing DB a live gateway holds in memory.

- `hermes profile migrate-identity <old> <new>`: retries the migration —
  delegates to the gateway control verb while a multiplexer is live, performs the
  durable rewrite of both state DBs when none is. Idempotent, and exits non-zero
  naming the offending database on a collision, a lock, or a partial failure. Only
  the name format and the existence of the new profile are checked; the old profile
  directory is expected to be gone.
- An older gateway that does not implement the verb is reported as such (`identify`
  answers while the migrate verb does not), not as "no gateway".
- `_migrate_profile_identity` returns an explicit success/failure result so the
  command can set its exit code; the rename warning now names the exact invocation.
- A failed control answer keeps the raw payload when it carries no reason field.
- The offline failure branch called `click.echo` in a module that never imports
  `click`: a failed second database raised NameError instead of printing its warning.
@xielevi

xielevi commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

Both findings addressed — thanks, the failed-live-migration path was a real gap.

Which option, and why not fall-through

We kept the ownership rule rather than falling through to the durable rewrite under a live gateway: the
CLI writing the routing DB while the gateway still holds SessionStore._entries has a race in either
order (the old process can _save() the old keys back before it actually exits), and
test_live_gateway_failure_does_not_rewrite_db_directly pins that invariant. So the CLI never touches
the store while a multiplexer is live — but it now ships the retry invocation instead of a dead end.

hermes profile migrate-identity <old> <new>

  • Live multiplexer → delegates to the migrate-profile-identity control verb (memory + root DB + profile DB).
  • No live multiplexer (restarted or stopped) → the CLI performs the durable rewrite of both databases itself.
  • The migration is driven by the rows that still name <old>, so profiles/<old> need not exist; only the
    name format and the existence of <new> are checked.
  • Idempotent: re-running a completed migration succeeds with nothing left to rekey (row counts 0).
  • Exits non-zero, naming the database and error, on a routing/session-identity collision, a lock, or one
    database failing after the other succeeded.
  • A failure caused by an older gateway process is reported as such: when the migrate verb answers nothing we
    identify the gateway, so "no gateway" and "gateway too old to implement the verb" produce different text.

_migrate_profile_identity now returns an explicit success/failure result, which is what lets the command set
its exit code.

The rename warning now ends with the exact invocation:

⚠ Profile was renamed, but the live gateway could not migrate session identity (<reason>).
Restart the gateway, then run:
    hermes profile migrate-identity <old> <new>

Raw answer (second finding)

The reason is now extracted as error/message/detail when the payload has one, and otherwise falls back to
repr(answer); a non-dict payload and a None answer both stay diagnosable.

Same pass, one adjacent bug: the offline failure branch called click.echo(..., err=True) but hermes_cli/profiles.py
never imports click — the warning would have raised NameError instead of printing, so a failed second database
turned into a traceback. It uses print(..., file=sys.stderr) now.

Tests — two new invariants, both proven red on this branch's base (implementation stashed) and green with it:

  • the failed-live-migration end state is repairable by the retry command (both stores rekeyed, second run idempotent);
  • a live gateway that answers with an unusable payload exits non-zero, quotes the raw answer, names the retry
    command, and still performs no direct DB rewrite.

@teknium1

Copy link
Copy Markdown
Collaborator

Salvaged into #112653 with your commit cherry-picked (authorship preserved). That PR lands #111927 (rename identity migration) together with the #112592 atomic-writer cluster (#112594 first-in, #112601 caller sites, #112596 sweep) on current main; overlapping same-idiom hunks were resolved to the first-landed version. Merge is gated on the maintainer; this PR will be closed with credit + SHA once that lands.

@teknium1

Copy link
Copy Markdown
Collaborator

Landed on main in #112653 (9085ef967c) with your commits cherry-picked and authorship intact — thank you, @xielevi. Closing here since the merge went through the consolidated carrier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: profile rename leaves stale profile name in session/routing state → gateway floods 'Profile does not exist'

4 participants