Skip to content

fix(gateway): bust cache on auxiliary compression changes - #39284

Open
Glucksberg wants to merge 1 commit into
NousResearch:mainfrom
Glucksberg:fix/aux-compression-cache-bust
Open

fix(gateway): bust cache on auxiliary compression changes#39284
Glucksberg wants to merge 1 commit into
NousResearch:mainfrom
Glucksberg:fix/aux-compression-cache-bust

Conversation

@Glucksberg

@Glucksberg Glucksberg commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • include auxiliary compression settings in the gateway agent cache signature, including a sanitized fallback_chain fingerprint
  • support nested configuration paths while preserving the existing cache-key behavior
  • invalidate a cached gateway agent when construction-time auxiliary compression feasibility can change
  • add regression coverage for fallback-chain and nested missing-path invalidation

Why

Gateway agent construction resolves auxiliary compression feasibility and can cache an unavailable-client warning or a reduced compression threshold. Changing the provider, model, endpoint, context length, API mode, or fallback chain must construct a fresh agent instead of retaining that stale state.

The earlier Honcho memoization change was deliberately removed because it overlaps upstream PR #46385; this contribution is now limited to auxiliary compression cache invalidation.

Validation

  • rebased onto 4281151ae859241351ba14d8c7682dc67ff4c126 on 2026-07-11; current head: 4ce5cd577eac58ce1f5d913b879bc9e2ede57367
  • .venv/bin/python -m pytest -q tests/gateway/test_agent_cache.py - 81 passed
  • .venv/bin/ruff check gateway/run.py tests/gateway/test_agent_cache.py and git diff --check upstream/main...HEAD passed

@Glucksberg Glucksberg closed this Jun 4, 2026
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch from d295b78 to acce1a2 Compare June 4, 2026 19:10
@Glucksberg Glucksberg reopened this Jun 4, 2026
@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists labels Jun 4, 2026
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch 2 times, most recently from a19e70f to 04a58a0 Compare June 4, 2026 20:42
@Glucksberg
Glucksberg marked this pull request as ready for review June 4, 2026 21:05
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch from 04a58a0 to 7a5f45c Compare July 11, 2026 08:31
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch from 7a5f45c to 4ce5cd5 Compare July 11, 2026 20:32
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch 2 times, most recently from a0f2f59 to 10fae52 Compare July 13, 2026 22:38
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch from 10fae52 to 87f19b2 Compare July 14, 2026 00:43
Glucksberg added a commit to Glucksberg/hermes-agent that referenced this pull request Jul 14, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for narrowing this to the gateway cache signature; the current-main premise is real: gateway/run.py:15743-15854 has no auxiliary compression fields, while initialization stores auxiliary context state (agent/agent_init.py:1618-1630) and feasibility is memoized per agent (agent/conversation_compression.py:490-497).

Problems

  • gateway/run.py:15935-15940 omits auxiliary.compression.api_key. That is a supported config field (hermes_cli/config.py:1594-1600) and _resolve_task_provider_model() reads it for compression clients (agent/auxiliary_client.py:5895-5901). A key-only edit therefore still reuses an agent whose feasibility probe has already been marked complete. Fingerprint this field like fallback_chain, rather than retaining its raw value.

Suggested changes

  • Add a signature-regression test where only auxiliary.compression.api_key changes.

Automated hermes-sweeper review.

Comment thread gateway/run.py
("auxiliary", "compression", "base_url"),
("auxiliary", "compression", "api_mode"),
("auxiliary", "compression", "context_length"),
("auxiliary", "compression", "fallback_chain"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include auxiliary.compression.api_key too, using a stable digest. It is a supported compression setting and is read by _resolve_task_provider_model; without it, a key-only repair still reuses an agent whose feasibility probe was already memoized.

@teknium1 teknium1 added sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
Glucksberg added a commit to Glucksberg/hermes-agent that referenced this pull request Jul 14, 2026
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch from 87f19b2 to ea7ed67 Compare July 16, 2026 16:15
@teknium1 teknium1 added the area/compression Context compression and continuation sessions label Jul 19, 2026
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch 4 times, most recently from 6377496 to 8933a9c Compare August 9, 2026 21:30
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch 3 times, most recently from 48baa1c to 5dcd341 Compare August 9, 2026 22:10
@Glucksberg
Glucksberg force-pushed the fix/aux-compression-cache-bust branch from 5dcd341 to 04e9260 Compare August 10, 2026 03:58
Glucksberg added a commit to Glucksberg/hermes-agent that referenced this pull request Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/compression Context compression and continuation sessions comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants