feat(gateway): add gateway.persist_model_override flag (opt-out for sticky session /model overrides)

[hilmanraz]

Summary

Adds an opt-out for the per-session /model override persistence that silently desyncs config.yaml-managed models from active gateway sessions (#68826).

When a user runs /model <name> inside a gateway chat (Telegram/Discord/etc — even once, even weeks prior), Hermes persists {model, provider, base_url} to the session store via SessionStore.set_model_override(). That override is rehydrated on every gateway restart (_rehydrate_session_model_override in gateway/run.py) and always outranks config.yaml's model.default / model.provider. There was previously no way to opt out — operators who manage models exclusively via hermes model (SSH) / config.yaml edits never get their config change to take effect on any session that has ever used /model in-chat, with no user-facing signal that the session is pinned away from config.

This PR adds gateway.persist_model_override (default true, preserves existing behavior). When set to false:

• /model still switches the model for the current process lifetime (the in-memory _session_model_overrides map is updated by the slash-command caller — the feature only gates the write-through to the session store and the rehydrate path).
• The switch is never written to disk, so it is never rehydrated after a restart. Sessions always fall back to model.default / model.provider from config.yaml on the next gateway boot.
• Clearing (set_model_override(None)) still works with the flag off, so an operator can wipe a previously-persisted override after flipping the flag without needing a Python one-liner.

Motivation

#68826 reported the symptom; this is the concrete fix. Users who switch models 2–3× a day via SSH (hermes config set model.default … + gateway restart) and occasionally use /model in the bot get permanently pinned to the stale in-chat override, with the only diagnostic being a grep for "Rehydrated persisted /model override" in ~/.hermes/logs/gateway.log. After this change, setting gateway.persist_model_override: false makes config.yaml the permanent source of truth — no more silent pinning.

Changes

• gateway/config.py: new persist_model_override field on GatewayConfig, threaded through to_dict / from_dict / the top-level-vs-nested gateway.* precedence block (mirrors the existing write_sessions_json pattern).
• gateway/session.py: SessionStore caches _persist_model_override from the config; set_model_override skips non-clearing writes when the flag is off. Clearing (None) is always honored so a pre-existing persisted override can still be wiped.
• gateway/run.py: _rehydrate_session_model_override early-returns when the store has persistence disabled, so a pre-existing persisted override (from before the flag was flipped) does not resurrect on restart.
• hermes_cli/config.py: DEFAULT_CONFIG['gateway']['persist_model_override'] = True so the flag shows up in generated config.yaml.
• tests/gateway/test_session_model_override_persist_flag.py: 7 new tests — flag-off no-persist, restart survival, clear-after-flip wipes prior persisted override, runner rehydrate skip, default-on sanity (existing behavior unchanged), live in-memory override still applies during process lifetime, and config round-trip. All pass locally.

Backward compatibility

Default is true → existing behavior is unchanged. Existing persisted overrides remain valid and continue to rehydrate. Operators who want the new contract opt in by setting gateway.persist_model_override: false.

Testing

PYTHONPATH=. python -m pytest tests/gateway/test_session_model_override_persist_flag.py -v

tests/gateway/test_session_model_override_persist_flag.py::test_flag_off_set_override_is_not_persisted PASSED
tests/gateway/test_session_model_override_persist_flag.py::test_flag_off_override_does_not_survive_restart PASSED
tests/gateway/test_session_model_override_persist_flag.py::test_flag_off_clear_still_works_to_wipe_prior_persisted PASSED
tests/gateway/test_session_model_override_persist_flag.py::test_flag_off_runner_does_not_rehydrate_persisted_override PASSED
tests/gateway/test_session_model_override_persist_flag.py::test_flag_on_default_still_persists_and_rehydrates PASSED
tests/gateway/test_session_model_override_persist_flag.py::test_flag_off_does_not_block_live_in_memory_override PASSED
tests/gateway/test_session_model_override_persist_flag.py::test_gateway_config_round_trips_flag PASSED

Closes #68826.

Checklist

• Default preserves existing behavior (no migration needed)
• Followed the existing write_sessions_json config-field pattern (field + to_dict + from_dict + top-level-vs-nested precedence)
• Clearing path is not gated by the flag (so flipping the flag off does not trap a pre-existing override)
• Tests cover the new flag, the runner rehydrate guard, the default-on sanity case, and config round-trip
• No new dependencies
• hermes config set gateway.persist_model_override false works out of the box once this lands

[alt-glitch]

This was generated by AI during triage.

Related to #68826 and merged #67113. This proposes a gateway-wide persistence-policy opt-out rather than the one-turn override mechanism, so it needs a maintainer precedence decision.

[hilmanraz]

Thanks for the triage note and for flagging #67113. They're complementary, not competing — different layers and different use cases:

#67113 (/model --once) is a per-invocation, user-explicit escape hatch: the user types --once when they want a one-turn switch with guaranteed no persistence. Great for "let me try this model for one prompt without pinning my session."

This PR (gateway.persist_model_override: false) is an operator-level policy: the operator sets it once in config.yaml, and from then on every in-chat /model is session-only (current process lifetime) and never rehydrated after a restart — config.yaml's model.default/model.provider stays the permanent source of truth. No per-invocation flag needed.

The two solve the same underlying desync symptom (#68826) but for different operators:

• --once suits an operator who is fine with sticky /model by default and just wants an opt-out for specific one-off switches.
• persist_model_override: false suits an operator who manages models exclusively via hermes model (SSH) / config.yaml edits (e.g. switching 2–3× a day) and never wants an accidental /model in the bot to silently pin a session away from config. With --once alone, that operator still has to remember --once on every in-chat switch or accept the sticky default; the config flag makes the whole gateway non-sticky by policy.

Happy to adjust if the maintainers prefer only one of the two to ship. A couple of notes on the design choice in case it helps the decision:

• Default is true → zero behavior change for existing users; purely opt-in.
• The clear path (set_model_override(None)) is deliberately NOT gated by the flag, so an operator who flips it off can still wipe a pre-existing persisted override without a Python one-liner (documented in the tests).
• The runner rehydrate guard means a persisted override from before the flag was flipped won't resurrect after the flag is on — otherwise flipping the flag wouldn't actually fix sessions that were already pinned.

[teknium1]

Thanks for separating the operator-wide policy from the merged /model --once path. The stale-session premise is still present on current main: gateway/run.py:4478 rehydrates before resolution, and gateway/session.py:2523 persists the override.

Problems

• gateway/session.py:2185 returns before sanitize_model_override(). Existing method semantics say a dict with no persistable values clears the override (gateway/session.py:2526-2532 on current main); with the new flag, {} instead leaves a prior persisted override intact. Normalize first and gate only a non-None cleaned value.
• The default-config hunk is stale: current main moved DEFAULT_CONFIG to hermes_cli/config_defaults.py in 1fe06115d1. GitHub reports this PR as dirty, so a salvage must relocate that hunk rather than apply it to hermes_cli/config.py.
• The added round-trip test does not exercise load_gateway_config() YAML loading and its top-level-vs-gateway.* precedence convention (tests/gateway/test_config.py:1059-1076).

Suggested changes

• Add loader-level tests for gateway.persist_model_override: false and precedence, plus the empty-dict clearing case.

Automated hermes-sweeper review.

[teknium1]

This gates before sanitize_model_override(), so {} cannot clear a previously persisted override when the flag is off, despite the existing method contract accepting a dict with no persistable values as a clear. Compute cleaned first and skip only when cleaned is not None.

[GottZ]

This was generated by AI during triage.

Summary

One open PR addresses #68826. #68861 adds an operator-wide configuration flag that gates both persistence and restart rehydration of session-scoped /model overrides, directly targeting the reported precedence of stale session state over config.yaml defaults.

Related pull requests

• feat(gateway): add gateway.persist_model_override flag (opt-out for sticky session /model overrides) #68861 best fix — (+276/-0) — n/a: The diff adds gateway.persist_model_override, skips non-clearing persistence and rehydration when disabled, and tests process-lifetime switching and restart behavior. Consistent with the contributor keep_open review, it needs revision because the gate precedes sanitize_model_override() and breaks {} clearing, the DEFAULT_CONFIG hunk targets the stale hermes_cli/config.py location rather than hermes_cli/config_defaults.py after 1fe0611, and loader-level YAML precedence is not tested; the separate operator-wide policy also still requires the requested maintainer precedence decision relative to merged feat: /model --once one-turn model override (salvages #29923) #67113's per-invocation /model --once path.

Suggested consolidation

Author action: rebase #68861 onto main, or split out the part that can merge. Preserve the operator-wide persistence policy as the salvage path, but first normalize before gating so {} still clears persisted state, relocate the default-config change, add load_gateway_config() tests for top-level-versus-gateway.* precedence, and obtain the documented maintainer precedence decision; keep the PR open while those contributor review findings remain unresolved.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I68826(["issue #68826 (open)"])
    P68861["PR #68861 (open)"]
    P68861 -->|best fix| I68826
    class I68826 open
    class P68861 open
    class P68861 best
    class P68861 target
    click I68826 "https://github.com/NousResearch/hermes-agent/issues/68826"
    click P68861 "https://github.com/NousResearch/hermes-agent/pull/68861"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 1 pull request and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 16 kB of PR diffs, 10 kB of issue/PR text, 4 kB of discussion (4 comments), 3 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.