fix(session): persist provider on model switch and use it on resume - #79604
Closed
webtecnica wants to merge 2 commits into
Closed
fix(session): persist provider on model switch and use it on resume#79604webtecnica wants to merge 2 commits into
webtecnica wants to merge 2 commits into
Conversation
Contributor
Author
|
Addressed the merge note from the #79811 review (thanks @teknium1 for flagging it): |
5 tasks
kshitijk4poor
pushed a commit
that referenced
this pull request
Aug 14, 2026
Salvage of #79604 (webtecnica) + #85721 (pierrenode), combined and rebased onto current main with simplify-code findings folded in. #79604: update_session_model() wrote the model name to sessions.model but never persisted the provider into model_config. On resume, the runtime recombined the persisted model with the config.yaml primary provider (which may not serve that model), producing auth errors. Fix: add optional provider parameter to update_session_model, merged into model_config via the shared _merge_model_config_json helper (not hand-rolled SQL). Wire both gateway /model call sites to pass result.target_provider. #85721: session_gateway_runtime() had no billing_provider fallback. A CLI session that never ran /model has no gateway_runtime or top-level provider in model_config — billing_provider (written on every session's first accounted API call) is the only durable record. Fix: add billing_provider as the last-resort fallback in session_gateway_runtime(), filtering bare billing buckets (auto/custom) that are not routable identities. Simplify-code findings addressed: - Use _merge_model_config_json instead of 40 lines of branched SQL - Share _BARE_BILLING_PROVIDERS from hermes_state.py (was duplicated as a set in tui_gateway/server.py) - Merge None-filtering from #85920 with the billing_provider fallback into one coherent return path Co-authored-by: pierrenode <298902573+pierrenode@users.noreply.github.com>
Collaborator
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a session falls back from the primary model to a fallback on a different provider, and the user then switches to that fallback model with
/model, the session persisted only the model name instate.db— never its provider. On resume, the runtime recombined the persisted model with the primary provider fromconfig.yaml(which doesn't serve that model), producing a guaranteed 404 on every turn.Root Cause
update_session_model()(inhermes_state.py) wrote the model name to thesessions.modelcolumn but never wrotemodel/providerkeys into themodel_configJSON blob — the provider/model pair was lost across restarts.Change
update_session_model(session_id, model, provider=None)now merges{"model": ..., "provider": ...}into themodel_configJSON when a provider is passed (json_set/json_removeSQL preserving lineage markers and thebrowser_model_lockbehavior). Both/modelcall sites ingateway/slash_commands.pypass the resolvedtarget_provider.tui_gateway/server.py::_stored_session_runtime_overrides, whenmodel_confighas no top-levelprovider, fall back tomodel_config.gateway_runtime.provider(written by_sync_session_model_from_agenton the last turn) instead of theconfig.yamlprimary provider. Precedence: top-level →gateway_runtime→billing_provider→ default.Verification
tests/test_hermes_state.py+tests/test_tui_gateway_server.py: 698 passed (pre-existing + new coverage for provider persistence and defensive resume).Closes #79536