Skip to content

fix: use custom provider context length for compression model - #15210

Closed
Hubedge wants to merge 1 commit into
NousResearch:mainfrom
Hubedge:fix/custom-provider-compression-context
Closed

fix: use custom provider context length for compression model#15210
Hubedge wants to merge 1 commit into
NousResearch:mainfrom
Hubedge:fix/custom-provider-compression-context

Conversation

@Hubedge

@Hubedge Hubedge commented Apr 24, 2026

Copy link
Copy Markdown

Summary

  • Resolve custom provider model-level context_length for the auxiliary compression model during feasibility checks.
  • Pass the resolved value into get_model_context_length(..., config_context_length=...) so OpenAI-compatible custom endpoints that omit context metadata do not fall back to 128K when the config declares a larger window.
  • Add a regression test covering custom_providers[].models[<model>].context_length with no auxiliary.compression.context_length override.

Tests

  • ./venv/bin/python -m pytest tests/run_agent/test_compression_feasibility.py -q -o 'addopts=' → 17 passed
  • ./venv/bin/python -m py_compile run_agent.py tests/run_agent/test_compression_feasibility.py && git diff --check → passed
  • Checked existing baseline failure on main: tests/hermes_cli/test_custom_provider_model_switch.py::TestCustomProviderModelSwitch::test_saved_model_still_probes_endpoint fails because the test expects no api_mode kwarg while current code calls fetch_api_models(..., api_mode=None). This failure is unrelated to this PR’s changed files.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Apr 24, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Addresses #12977 (custom_providers context_length not propagated to compression feasibility check). Related/competing PR: #13052 which fixes broader context_length resolution issues.

@Hubedge

Hubedge commented Apr 24, 2026

Copy link
Copy Markdown
Author

Thanks for flagging the relationship with #13052.

This PR is complementary, not competing:

Even after #13052 merges, the compression feasibility check remains a blind spot for custom endpoints whose model IDs do not match DEFAULT_CONTEXT_LENGTHS (e.g. proxy-renamed slugs). The helper introduced here (_custom_provider_context_length_for_model) also centralizes the custom-provider lookup logic so future auxiliary callers don't duplicate the same inline resolution.

Merge dependency: none. If #13052 lands first, I'm happy to rebase this to avoid trivial conflicts in run_agent.py.

Ready for review.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused report and regression test.

This is an automated hermes-sweeper review. The requested behavior is already implemented on current main:

  • agent/agent_init.py:1653-1665 resolves and stores compatible custom-provider metadata on the agent for auxiliary compression resolution.
  • agent/conversation_compression.py:236-245 passes that list as custom_providers while resolving the auxiliary compression model's context window.
  • agent/model_metadata.py:2083-2096 applies the matching per-model custom-provider context_length override before any endpoint probe or fallback.
  • The implementation landed in 7becb19ea00c13bdff6f78b71aa3ddfb0bdb5378 (fix(auxiliary): forward custom_providers to compression model context-length detection) and is contained in v2026.5.16.

This also matches the later linked reports and competing implementation discussion around #19539 and #19564.

@teknium1 teknium1 closed this Jul 12, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:implemented-on-main Sweeper: behavior already present on current main type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants