Skip to content

fix(gateway): bust agent cache when model.default or model.provider changes - #37880

Closed
cmhaoren-sudo wants to merge 2 commits into
NousResearch:mainfrom
cmhaoren-sudo:fix/cache-busting-model-default
Closed

fix(gateway): bust agent cache when model.default or model.provider changes#37880
cmhaoren-sudo wants to merge 2 commits into
NousResearch:mainfrom
cmhaoren-sudo:fix/cache-busting-model-default

Conversation

@cmhaoren-sudo

Copy link
Copy Markdown

Problem

When model.default or model.provider is updated via /model or hermes config set, the Gateway's agent-pool cache signature did not include these keys. The cached (old-model) agent was reused for new gateway messages, requiring a manual Gateway restart to pick up the change.

Closes Issue #20628 — Dynamic Model Switching for Gateway Platforms.

Fix

Added ("model", "default") and ("model", "provider") to _CACHE_BUSTING_CONFIG_KEYS in gateway/run.py. Now when either config key changes, the cache signature is invalidated and a new agent is created with the updated model/provider.

Testing

  • Verified /model via Feishu now takes effect immediately without Gateway restart
  • Existing cache keys (context_length, max_tokens, compression, etc.) unchanged

Zhang Yinqi added 2 commits June 2, 2026 18:42
… rendering

- Replace _MARKDOWN_TABLE_RE (force-text) with _SHOULD_USE_CARD_RE
- Tables and fenced code blocks now render as schema 2.0 interactive cards
- Add _build_card_payload() with schema 2.0 for proper table/code rendering
- Add fallback chain: interactive card -> post -> plain text
- Align with OpenClaw feishu plugin's shouldUseCard pattern
…hanges

When model.default or model.provider is updated via /model or
hermes config set, the Gateway's agent-pool cache signature
did not include these keys, so the cached (old-model) agent was
reused for new gateway messages.

Add ("model", "default") and ("model", "provider") to
_CACHE_BUSTING_CONFIG_KEYS so a model/provider change triggers
agent recreation.

Fixes: Feature Request NousResearch#20628 — Dynamic Model Switching for Gateway Platforms
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery platform/feishu Feishu / Lark adapter labels Jun 3, 2026
@liuhao1024

Copy link
Copy Markdown
Contributor

The cache-busting fix (("model", "default"), ("model", "provider") in _CACHE_BUSTING_CONFIG_KEYS) looks correct and well-scoped — it follows the existing pattern for other config keys.

However, the diff also includes ~80 lines of unrelated Feishu card rendering changes in gateway/platforms/feishu.py that are not mentioned in the PR body:

  1. _MARKDOWN_HINT_RE regex change — adds ^\s*\| for table pipe detection
  2. _SHOULD_USE_CARD_RE — new regex replacing _MARKDOWN_TABLE_RE for interactive card triggers
  3. _build_card_payload() — new function building schema 2.0 interactive cards
  4. _build_outbound_payload() rewrite — routes tables/code blocks through interactive cards instead of forcing plain text
  5. Interactive card → post fallback — new error handling in send() for API rejection of interactive cards

These Feishu rendering changes are a separate concern from the model cache fix. They should be in their own PR so they can be reviewed and tested independently. Bundling unrelated changes makes it harder to bisect regressions and obscures the scope of each change.

Suggestion: split into two PRs — one for the _CACHE_BUSTING_CONFIG_KEYS fix and one for the Feishu interactive card rendering improvements.

@kuangmi-bit kuangmi-bit left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this fix! The 2-line cache-busting change in gateway/run.py is exactly right and should merge quickly.

However, the gateway/platforms/feishu.py changes (+53/−11) introducing _SHOULD_USE_CARD_RE, _build_card_payload, and the interactive card fallback logic are completely unrelated to model cache busting. This makes the PR hard to review and likely explains why it has been sitting since June 3rd with zero reviews.

Request: Split this into two PRs:

  1. PR A: gateway/run.py — the 2-line _CACHE_BUSTING_CONFIG_KEYS addition. This is safe, small, clearly closes #20628, and can merge immediately.
  2. PR B: gateway/platforms/feishu.py — the interactive card rendering refactor. This deserves its own review and testing cycle.

Also worth noting: this fix handles model.default and model.provider, but config changes to tools, skills, platforms, toolsets, and other sections still require a gateway restart. That broader config hot-reload problem is tracked in #47058 and would benefit from a similar approach — expanding _CACHE_BUSTING_CONFIG_KEYS to cover more config sections.

Once split, happy to approve the cache-busting PR.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused cache analysis. This is an automated hermes-sweeper review; current main already provides the requested gateway model-switch behavior.

  • gateway/run.py:17041 reloads gateway config per agent turn, and gateway/run.py:18015 resolves the effective model and provider from that fresh config.
  • gateway/run.py:18158 computes the cached-agent signature from that effective route; gateway/run.py:15886 includes both the effective model and runtime provider in the hash.
  • gateway/slash_commands.py:1893 explicitly evicts the cached agent after /model stores its session override. Persisted session model overrides shipped in 30e947e0a05ef535e4b25a183d8bbe34fd68d1d5 (released in v2026.7.7).
  • tests/gateway/test_agent_cache.py:449 already covers a model change producing a cache-signature miss.

The earlier review correctly identified the unrelated Feishu rendering commit; that code targeted the former gateway/platforms/feishu.py location, while the current adapter is plugins/platforms/feishu/adapter.py.

@teknium1 teknium1 closed this Jul 13, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/feishu Feishu / Lark adapter sweeper:implemented-on-main Sweeper: behavior already present on current main type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants