Skip to content

fix(compression): honor custom provider context lengths - #21953

Closed
Bartok9 wants to merge 4 commits into
NousResearch:mainfrom
Bartok9:fix/21947-compression-custom-context
Closed

fix(compression): honor custom provider context lengths#21953
Bartok9 wants to merge 4 commits into
NousResearch:mainfrom
Bartok9:fix/21947-compression-custom-context

Conversation

@Bartok9

@Bartok9 Bartok9 commented May 8, 2026

Copy link
Copy Markdown
Contributor

Problem

Compression models configured with provider: auto and an empty model inherit the main model name, but the startup feasibility check did not inherit the custom_providers[].models.<model>.context_length metadata for that model. Users with custom OpenAI-compatible providers could see Hermes auto-lower the compression threshold to the 256K fallback even when config.yaml declared a larger context window.

Closes #21947

Root cause

get_model_context_length() already supports custom_providers, and the main compressor path already resolves custom-provider context into model.context_length before construction. The auxiliary compression feasibility probe, however, only passed auxiliary.compression.context_length and did not have access to the resolved custom provider list.

Fix

Cache the resolved custom provider metadata on the agent and pass it into _check_compression_model_feasibility() when resolving the inherited auxiliary compression model's context length.

Testing

  • python3 -m py_compile run_agent.py tests/run_agent/test_compression_feasibility.py: clean
  • python3 -m ruff check tests/run_agent/test_compression_feasibility.py --select F: clean
  • python3 -m pytest -o addopts='' tests/run_agent/test_compression_feasibility.py -q: 17 passed

Note: full-file ruff --select E,W,F on run_agent.py still reports pre-existing unrelated issues; the repo's CI uses the ruff diff workflow for PRs.

@Bartok9

Bartok9 commented May 8, 2026

Copy link
Copy Markdown
Contributor Author

The test CI failure is pre-existing on main — verified at main SHA 1997b3ba. Not introduced by this PR.

@Bartok9

Bartok9 commented May 9, 2026

Copy link
Copy Markdown
Contributor Author

The test CI failure is pre-existing on main — verified at main SHA 524cbabd. Not introduced by this PR.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 9, 2026
@Bartok9

Bartok9 commented May 12, 2026

Copy link
Copy Markdown
Contributor Author

CI note: The failing checks (test, e2e, Windows footguns) are pre-existing failures on main at 3c23b15f815e — they fail without this PR's changes. Running pytest locally against main directly reproduces the same 12 failures. None of the failing tests touch the code changed in this PR. This PR is CI-clean with respect to its own scope.

Bartok9 and others added 4 commits May 15, 2026 01:37
Pass resolved custom provider metadata into the compressor context resolver so auxiliary auto-compression inherits per-model context_length values from config.yaml.

Closes NousResearch#21947

Co-authored-by: Cursor <cursoragent@cursor.com>
Keep the new custom_providers passthrough compatible with dynamic test kwargs while preserving runtime behavior.

Co-authored-by: Cursor <cursoragent@cursor.com>
Avoid widening ContextCompressor's public initializer; the auxiliary feasibility check is the only path that needs custom provider metadata.

Co-authored-by: Cursor <cursoragent@cursor.com>
Keep the custom provider context fix line-stable in run_agent.py so pre-existing ty diagnostics do not appear as new diff warnings.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Bartok9
Bartok9 force-pushed the fix/21947-compression-custom-context branch from e8c12fd to 1593ef5 Compare May 15, 2026 05:39
@Bartok9

Bartok9 commented May 15, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main.

What I did:

  • Cherry-picked the 4 fix commits (e74b5f7, 88ebe31, 5e3f521, 1593ef5) onto a fresh branch off upstream/main
  • Dropped the merge commit (e8c12fdf) — main has moved on, the merge artifact was no longer relevant
  • Resolved two small conflicts:
    • tests/run_agent/test_compression_feasibility.py: agent._custom_providers = [] (accepted main's value — [] is correct, None was my older draft)
    • run_agent.py: deduped a self._custom_providers = _custom_providers assignment (main had it once with a long comment; the fix commit's intent was the one-liner version which is what landed)
  • Also caught + fixed 3 duplicate custom_providers= kwargs that the cherry-picks produced (test assertions had custom_providers=None, provider=..., custom_providers=[] after the merge — both kwargs collapsed correctly)

Verified: ast.parse clean on all 3 touched files (run_agent.py, agent/context_compressor.py, tests/run_agent/test_compression_feasibility.py).

Final diff: 2 files, +39 / -7

Closes #21947 (custom provider context length feasibility check).

@Bartok9

Bartok9 commented May 27, 2026

Copy link
Copy Markdown
Contributor Author

Closing to stay within contributor PR limit. Will resubmit with fresh rebase if the issue remains open in main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Auxiliary compression model does not inherit context_length from custom_providers

2 participants