feat(gateway): hot-reload memory_char_limit and user_char_limit - #39454
Open
Gabreil-Lu wants to merge 1 commit into
Open
feat(gateway): hot-reload memory_char_limit and user_char_limit#39454Gabreil-Lu wants to merge 1 commit into
Gabreil-Lu wants to merge 1 commit into
Conversation
Previously, edits to memory.memory_char_limit and memory.user_char_limit in config.yaml only took effect after a full gateway restart, even though sister keys (model.context_length, compression.*, memory.provider) are already picked up transparently on the next message via the agent-config signature. Add the two keys to _CACHE_BUSTING_CONFIG_KEYS so the cached AIAgent is rebuilt when either cap changes. Tests pin the behavior: - test_reads_memory_char_limit_subkeys - test_memory_char_limit_change_busts_signature User-facing docs note the new behavior in the hot-reload tip.
alt-glitch
added
type/feature
Contributor
|
Thanks for the focused cache-invalidation fix. Current Automated hermes-sweeper review. |
teknium1
added
sweeper:risk-caching
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds
memory.memory_char_limitandmemory.user_char_limitto the gateway's cache-busting config keys, so edits to either cap inconfig.yamltake effect on the next message — no gateway restart required.Why
Sister keys (
model.context_length,compression.*,memory.provider) are already picked up transparently.memory_char_limit/user_char_limitwere the only memory-related config values still requiring a restart, which is inconsistent and surprising — operators hit it the first time they tried to bump the cap and saw no effect until they restarted the gateway.The fix is two entries in
_CACHE_BUSTING_CONFIG_KEYS(the same mechanism used for the other hot-reloadable keys).Related Issue
No specific issue filed; this was discovered while investigating why an in-place cap bump did not take effect.
Type of Change
Changes Made
gateway/run.py— add 2 entries to_CACHE_BUSTING_CONFIG_KEYSwith an inline comment explaining whytests/gateway/test_agent_cache.py— 2 new tests:test_reads_memory_char_limit_subkeys— caps are extracted from configtest_memory_char_limit_change_busts_signature— changing the cap changes the signature (so cache is invalidated)website/docs/user-guide/configuration.md— extend the hot-reload tip to mention the new keysHow to Test
memory: { memory_char_limit: 2200, user_char_limit: 1375 }in~/.hermes/config.yaml~/.hermes/config.yamltomemory_char_limit: 5000(no restart)Checklist
Code
Documentation & Housekeeping