Skip to content

fix: clear stale global context length on model switch - #26900

Open
hubdev4 wants to merge 1 commit into
NousResearch:mainfrom
hubdev4:fix/custom-provider-context-length-leak
Open

hubdev4 wants to merge 1 commit into
NousResearch:mainfrom
hubdev4:fix/custom-provider-context-length-leak

Conversation

@hubdev4

@hubdev4 hubdev4 commented May 16, 2026

Copy link
Copy Markdown

Summary

  • Clear stale model.context_length global overrides when switching models/providers.
  • Prevent custom endpoint context lengths from leaking into later provider selections.
  • Keep custom endpoint context lengths scoped to custom provider model metadata.
  • Add regression coverage for the deepsproxy-style custom provider flow.

Credit: this fix was implemented with Hermes Agent.

Why

model.context_length is a global runtime override. If a custom endpoint setup left context_length: 64000 in the global model: block, every provider selected afterward could inherit that 64k limit, including Codex, OpenAI, Anthropic, and other non-custom providers.

The custom provider wizard should preserve context length only as provider/model metadata, not as a global model override.

Testing

  • scripts/run_tests.sh tests/cli/test_cli_provider_resolution.py tests/hermes_cli/test_custom_provider_model_switch.py tests/hermes_cli/test_model_switch_custom_providers.py tests/hermes_cli/test_user_providers_model_switch.py -q
  • Result: 79 passed

Co-authored-by: Hermes Agent hermes-agent@users.noreply.github.com

Clear stale model.context_length overrides when switching models or providers so custom endpoint context values do not leak globally.

Co-authored-by: Hermes Agent <hermes-agent@users.noreply.github.com>
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 16, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

Note: PR #24724 (merged 2026-05-13) is the authoritative fix for clearing stale _config_context_length on model switch. This PR addresses a related but distinct path — global model.context_length leaking from custom endpoint setup into subsequent provider selections. Please verify this isn't already covered by #24724. Related: #24079, #11438.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the persisted-config path; the underlying issue remains on current main.

Problems

  • The implementation needs a conflict-aware salvage: the changed _model_flow_custom, _model_flow_named_custom, and _model_flow_api_key_provider functions were extracted from hermes_cli/main.py into hermes_cli/model_setup_flows.py by a77efada5. Current custom setup still preserves the stale value at hermes_cli/model_setup_flows.py:947-960, so the old main.py hunks do not affect the live flow.
  • The new regression test stubs _save_model_choice at tests/cli/test_cli_provider_resolution.py:564, but the PR changes that helper at hermes_cli/auth.py:5148. It does not cover that central mutation.

Suggested changes

  • Port the cleanup to the extracted flow and add a test using the real save path. Assert the active model.context_length is cleared while custom_providers[].models.<model>.context_length remains intact; runtime precedence is visible in agent/agent_init.py:1660-1708.

Automated hermes-sweeper review.

"hermes_cli.config.get_env_value",
lambda key: "" if key in {"OPENAI_BASE_URL", "OPENAI_API_KEY"} else "",
)
monkeypatch.setattr("hermes_cli.auth._save_model_choice", lambda model: None)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This stubs the helper changed in hermes_cli/auth.py:5148, so the regression does not exercise that central cleanup. Keep this flow test if it validates the explicit flow cleanup, but add a separate real _save_model_choice assertion for the helper behavior.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants