Skip to content

fix(agent): respect model.context_length config - #37548

Closed
vivganes wants to merge 1 commit into
NousResearch:mainfrom
vivganes:patch-fix-8430
Closed

fix(agent): respect model.context_length config#37548
vivganes wants to merge 1 commit into
NousResearch:mainfrom
vivganes:patch-fix-8430

Conversation

@vivganes

@vivganes vivganes commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

This PR fixes #8430 in a exhaustive way, introducing proper context length checking in case the user has provided a model.context_length config.

Related Issue

#8430

Fixes #8430

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  1. Show the existing error message only when context_length is None
  2. Show a new actionable error message if the model's context is less than the context_length value set in the config.

How to Test

  1. Run hermes agent with any model with context < 64000
  2. Set model.context_length in config to be less than 64000
  3. Send a 'hi' to hermes and see if you get a response

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate -- The existing PRs are not as exhaustive (adding a new error message and check) as this
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform:

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

@rodriguez46p-ui

Copy link
Copy Markdown

Hermes CI triage

CI is currently failing for this PR in two places:

  1. tests/agent/test_agent_init.py::{test_init_agent_rejects_model_below_user_config_override,test_init_agent_rejects_model_below_minimum_without_override} fail because the lightweight SimpleNamespace test double is missing agent._is_direct_openai_url. Add it in make_agent() alongside _is_azure_openai_url.

  2. tests/run_agent/test_plugin_context_engine_init.py::test_plugin_engine_update_model_args now raises for openrouter/auto when detected context length is 0. The previous guard was if _ctx and _ctx < MINIMUM_CONTEXT_LENGTH, so unknown/undetectable context (0) did not fail startup. The new minimum check should likely preserve that behavior, e.g. only enforce when _ctx is truthy/known:

if _config_context_length is not None and _ctx and _ctx < _config_context_length:
    ...
if _config_context_length is None and _ctx and _ctx < MINIMUM_CONTEXT_LENGTH:
    ...

Also small style nit: if( should be if ( / preferably no extra parens, and the raise ValueError(...) closing paren is over-indented.

Evidence: failing jobs test (2) and test (3) in workflow run 26838174166.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Jun 2, 2026
@liuhao1024

Copy link
Copy Markdown
Contributor

The split logic removes the _ctx and guard that the original code relied on to skip the check when context length is unknown (0).

Original code (line 1476 on main):

_ctx = getattr(agent.context_compressor, "context_length", 0)
if _ctx and _ctx < MINIMUM_CONTEXT_LENGTH:

The _ctx and short-circuit means: "if the compressor couldn't determine context length (0), skip the check entirely."

PR code:

if(_config_context_length is not None and _ctx < _config_context_length):

When _ctx is 0 (unknown) and model.context_length is configured (e.g., 60000), this evaluates to True and 0 < 60000 → True, raising a spurious ValueError. The same problem applies to the second condition: 0 < MINIMUM_CONTEXT_LENGTH would also be True.

Suggested fix — restore the _ctx truthiness check on both branches:

if _ctx and _config_context_length is not None and _ctx < _config_context_length:
    raise ValueError(...)
if _ctx and _config_context_length is None and _ctx < MINIMUM_CONTEXT_LENGTH:
    raise ValueError(...)

This preserves the original "skip when unknown" behavior while adding the user-configured override path. A test for the _ctx == 0 case would also be valuable.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the PR. Closing — this is working as intended, not a bug.

The 64K context-window floor is deliberate: Hermes' tool schemas + system prompt consume a large fixed prefix, and sub-64K windows aren't reliable for agentic tool-calling. We don't want anyone running below 64K, so there is intentionally no escape hatch to override the floor with a smaller value.

The real defect here was the error message — it advertised "or set model.context_length in config.yaml to override", which falsely implied a sub-64K override existed and generated a cluster of well-meaning PRs (this one included) trying to wire it up. We've corrected the message in #53569 so it no longer promises an override that was never meant to exist.

Note: model.context_length is still honored for the legitimate case where a local server under-reports its true window — but the declared value must itself be ≥64K. It is not a way to run below the floor.

Appreciate the clean, minimal diff — the reasoning just points the other way on this one.

@teknium1 teknium1 closed this Jun 27, 2026
teknium1 added a commit that referenced this pull request Jun 27, 2026
…age (#53569)

The error raised when a model's context window is below the 64K minimum
advertised "or set model.context_length in config.yaml to override" — but
the guard intentionally has no sub-64K escape hatch. Sub-64K models are
rejected by design (tool schemas + system prompt need the headroom).

The misleading clause invited a cluster of dup PRs (#11097, #11110, #8962,
#9142, #37548) all trying to wire an override that we don't want. Reword to
state the real options: pick a >=64K model, or — if your local server
under-reports its true window — declare the real value (which must itself
be >=64K). Guard behavior is unchanged.
pai-scaffolde pushed a commit to pai-scaffolde/hermes-agent that referenced this pull request Jun 28, 2026
…age (NousResearch#53569)

The error raised when a model's context window is below the 64K minimum
advertised "or set model.context_length in config.yaml to override" — but
the guard intentionally has no sub-64K escape hatch. Sub-64K models are
rejected by design (tool schemas + system prompt need the headroom).

The misleading clause invited a cluster of dup PRs (NousResearch#11097, NousResearch#11110, NousResearch#8962,
NousResearch#9142, NousResearch#37548) all trying to wire an override that we don't want. Reword to
state the real options: pick a >=64K model, or — if your local server
under-reports its true window — declare the real value (which must itself
be >=64K). Guard behavior is unchanged.
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…age (NousResearch#53569)

The error raised when a model's context window is below the 64K minimum
advertised "or set model.context_length in config.yaml to override" — but
the guard intentionally has no sub-64K escape hatch. Sub-64K models are
rejected by design (tool schemas + system prompt need the headroom).

The misleading clause invited a cluster of dup PRs (NousResearch#11097, NousResearch#11110, NousResearch#8962,
NousResearch#9142, NousResearch#37548) all trying to wire an override that we don't want. Reword to
state the real options: pick a >=64K model, or — if your local server
under-reports its true window — declare the real value (which must itself
be >=64K). Guard behavior is unchanged.
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
…age (NousResearch#53569)

The error raised when a model's context window is below the 64K minimum
advertised "or set model.context_length in config.yaml to override" — but
the guard intentionally has no sub-64K escape hatch. Sub-64K models are
rejected by design (tool schemas + system prompt need the headroom).

The misleading clause invited a cluster of dup PRs (NousResearch#11097, NousResearch#11110, NousResearch#8962,
NousResearch#9142, NousResearch#37548) all trying to wire an override that we don't want. Reword to
state the real options: pick a >=64K model, or — if your local server
under-reports its true window — declare the real value (which must itself
be >=64K). Guard behavior is unchanged.
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…age (NousResearch#53569)

The error raised when a model's context window is below the 64K minimum
advertised "or set model.context_length in config.yaml to override" — but
the guard intentionally has no sub-64K escape hatch. Sub-64K models are
rejected by design (tool schemas + system prompt need the headroom).

The misleading clause invited a cluster of dup PRs (NousResearch#11097, NousResearch#11110, NousResearch#8962,
NousResearch#9142, NousResearch#37548) all trying to wire an override that we don't want. Reword to
state the real options: pick a >=64K model, or — if your local server
under-reports its true window — declare the real value (which must itself
be >=64K). Guard behavior is unchanged.
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…age (NousResearch#53569)

The error raised when a model's context window is below the 64K minimum
advertised "or set model.context_length in config.yaml to override" — but
the guard intentionally has no sub-64K escape hatch. Sub-64K models are
rejected by design (tool schemas + system prompt need the headroom).

The misleading clause invited a cluster of dup PRs (NousResearch#11097, NousResearch#11110, NousResearch#8962,
NousResearch#9142, NousResearch#37548) all trying to wire an override that we don't want. Reword to
state the real options: pick a >=64K model, or — if your local server
under-reports its true window — declare the real value (which must itself
be >=64K). Guard behavior is unchanged.
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…age (NousResearch#53569)

The error raised when a model's context window is below the 64K minimum
advertised "or set model.context_length in config.yaml to override" — but
the guard intentionally has no sub-64K escape hatch. Sub-64K models are
rejected by design (tool schemas + system prompt need the headroom).

The misleading clause invited a cluster of dup PRs (NousResearch#11097, NousResearch#11110, NousResearch#8962,
NousResearch#9142, NousResearch#37548) all trying to wire an override that we don't want. Reword to
state the real options: pick a >=64K model, or — if your local server
under-reports its true window — declare the real value (which must itself
be >=64K). Guard behavior is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

5 participants