Skip to content

feat(agent): make MINIMUM_CONTEXT_LENGTH configurable via HERMES_MIN_CONTEXT_LENGTH - #44932

Closed
divnjl2 wants to merge 1 commit into
NousResearch:mainfrom
divnjl2:feat/configurable-min-context-length
Closed

feat(agent): make MINIMUM_CONTEXT_LENGTH configurable via HERMES_MIN_CONTEXT_LENGTH#44932
divnjl2 wants to merge 1 commit into
NousResearch:mainfrom
divnjl2:feat/configurable-min-context-length

Conversation

@divnjl2

@divnjl2 divnjl2 commented Jun 12, 2026

Copy link
Copy Markdown

Problem

agent/agent_init.py hard-rejects any model whose context window is below MINIMUM_CONTEXT_LENGTH = 64_000:

Model X has a context window of 40,000 tokens, which is below the minimum
64,000 required by Hermes Agent. ... or set model.context_length in config.yaml to override.

But the advertised override does not work: an explicit model.context_length below 64K is still rejected (the same constant is also the context_compressor floor). So there is no real escape hatch.

Who this hurts

Operators running local GGUF backends (llama.cpp / llama-swap / ik_llama) where the effective per-request context = -c / --parallel. A 256K model split into 6 slots is a 43K slot; that is a perfectly usable agent context, but it is hard-rejected at init, bricking the whole deployment.

Change

Read the floor from HERMES_MIN_CONTEXT_LENGTH (default 64000behaviour unchanged for everyone who doesn't set it). One line, backward-compatible. Lets small-slot operators lower the floor to match their real slot size.

Test

Local llama-swap fleet (35B/9B/4B split across two GPUs, slots 8-43K). Before: every cell dies at init with the 64K refusal. After (HERMES_MIN_CONTEXT_LENGTH=8000): 35B and 9B cells initialize and complete tool-calling turns normally.

@divnjl2
divnjl2 force-pushed the feat/configurable-min-context-length branch from 9fcc481 to 112e3c7 Compare June 12, 2026 13:44
The agent_init hard-rejects any model whose context window is below 64K, and
the compression floor (context_compressor) also pins to the same constant. The
rejection message advises 'set model.context_length in config.yaml to override',
but an explicit context_length below 64K is still rejected, so there is no real
override.

Operators running local GGUF backends (llama.cpp / llama-swap) where the
effective per-request context is -c / --parallel can legitimately have 8-43K
slots. Hard-coding 64K bricks every such deployment at init.

Make the floor read HERMES_MIN_CONTEXT_LENGTH (default 64000, behaviour
unchanged) so these operators can lower it to match their real slot size.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@divnjl2
divnjl2 force-pushed the feat/configurable-min-context-length branch from 112e3c7 to f25a5f0 Compare June 12, 2026 13:45
@alt-glitch alt-glitch added type/feature New feature or request P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Jun 12, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for identifying the real lower-context startup constraint and for keeping the patch narrow.

This automated hermes-sweeper review is closing the PR under the project's standing configuration policy:

  • AGENTS.md:102-107 reserves .env / HERMES_* variables for secrets and requires non-secret behavioral settings to live in config.yaml.
  • This patch's substantive mechanism is the new HERMES_MIN_CONTEXT_LENGTH environment variable in agent/model_metadata.py (f25a5f084cf9).
  • The current guard is real (agent/agent_init.py:1872-1880), so a future proposal can re-scope this as a validated, documented config.yaml setting with coverage for the startup, compression, and local-runtime paths.

The issue is not the local-model use case; it is the user-facing environment-variable configuration surface.


Closed as not-planned per standing maintainer policy (env-var-for-config). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.

@teknium1 teknium1 closed this Jul 14, 2026
@teknium1 teknium1 added the sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) label Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants