Skip to content

fix(agent): preserve config context_length when fallback/switch uses same model - #32452

Closed
xianshishan wants to merge 1 commit into
NousResearch:mainfrom
xianshishan:fix/context-length-fallback-256k
Closed

fix(agent): preserve config context_length when fallback/switch uses same model#32452
xianshishan wants to merge 1 commit into
NousResearch:mainfrom
xianshishan:fix/context-length-fallback-256k

Conversation

@xianshishan

Copy link
Copy Markdown

Summary

When a fallback provider is activated or /model switch is triggered, _config_context_length is unconditionally cleared (set to None). The subsequent get_model_context_length() call then resolves the context window from scratch. If the model isn't in any static table (DEFAULT_CONTEXT_LENGTHS) and live probing fails, the resolution falls through all 10 steps to DEFAULT_FALLBACK_CONTEXT (256K).

This causes the status bar to show 97.9K/256K even though the user configured context_length: 1000000 in config.yaml — most visible after a compaction interruption + resume cycle.

Root Cause

In try_activate_fallback() (chat_completion_helpers.py:1013) and switch_model() (agent_runtime_helpers.py:1367):

agent._config_context_length = None  # ← cleared unconditionally

Then get_model_context_length(config_context_length=None, ...) is called. Without the explicit config override, resolution falls through to 256K for custom endpoints that don't advertise context_length via /models.

Fix

Save the original _config_context_length before clearing it. When the fallback/switched model is the same as the primary (common for custom providers where the fallback is the same endpoint accessed via a different credential), restore the saved override so the resolution chain receives it at Step 0:

_effective_config_ctx = getattr(agent, "_config_context_length", None)
if _saved_config_ctx and fb_model == old_model:
    _effective_config_ctx = _saved_config_ctx

When the fallback model is different from the primary, the override is NOT restored — the new model should resolve its own context window.

Changes

File Change
agent/chat_completion_helpers.py Save _config_context_length before fallback activation; restore it for same-model fallback
agent/agent_runtime_helpers.py Same fix for the switch_model() path

Test plan

  • 401 agent tests pass
  • Same-model fallback preserves user's model.context_length
  • Different-model fallback resolves fresh context_length (doesn't inherit override)
  • restore_primary_runtime() continues to work correctly (restores from snapshot)

Closes #32423

…same model

When fallback activation or model switch cleared _config_context_length,
the subsequent get_model_context_length() call could fall through to
DEFAULT_FALLBACK_CONTEXT (256K) if the model wasn't in any static table
and probing failed. This caused the status bar to show 256K after
compaction interruption + resume, even though the user had configured
context_length: 1000000 in config.yaml.

Save the original _config_context_length before clearing it, and restore
it when the fallback/switched model is the same as the primary — the
common case for custom providers where the fallback is the same endpoint
accessed via a different credential.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have labels Jun 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Context window changes to 256K after interrupted compaction and resume

2 participants