Skip to content

fix(context): revalidate compression state under session lock - #64511

Closed
ljy-2000 wants to merge 1 commit into
NousResearch:mainfrom
ljy-2000:codex/refresh-compaction-breaker-state
Closed

fix(context): revalidate compression state under session lock#64511
ljy-2000 wants to merge 1 commit into
NousResearch:mainfrom
ljy-2000:codex/refresh-compaction-breaker-state

Conversation

@ljy-2000

Copy link
Copy Markdown
Contributor

Summary

Revalidates durable compaction guard state while automatic compression is serialized, and prevents a delayed stale agent from rotating an already-compressed parent session a second time.

This is a concurrency hardening follow-up to #63965 / #63008. It does not reopen or claim to address the broader R1-R6 issue scope.

Root cause

The persisted fallback breaker and provider cooldown were loaded when a compressor was bound, then checked before the per-session compression lock. Another agent could update or clear those durable guards before this agent reached compress(), leaving the decision based on stale in-memory state.

The compression lock also serialized overlapping work but did not prove that the old session was still a live rotation owner. A delayed contender could acquire the parent lock after the winner released it, then rotate the already-ended parent into a second child.

Changes

  • refresh persisted cooldown and fallback-streak state before the initial automatic gate
  • refresh both guards again after acquiring the session compression lock
  • make forced cooldown refresh authoritative when another agent has cleared the durable row
  • after lock acquisition, reject a parent already ended with end_reason='compression'
  • preserve manual /compress policy bypass for breaker/cooldown while keeping session ownership mandatory
  • add production-seam regressions for guard escalation, guard clearing, cooldown clearing, forced retry, and stale-parent rotation

No config, schema, provider, tool, or plugin API changes are included.

Validation

  • 610 passed across compressor, rotation, concurrent fork, host-contract, Codex app-server, gateway synchronization, command, and state suites
  • original test_concurrent_compression_does_not_fork_session regression passes
  • Ruff passed
  • py_compile passed
  • Windows footgun scan passed for all 3 changed files
  • git diff --check passed
  • added-line secret and conflict-marker scans were clean

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Jul 14, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused concurrency hardening. The premise still holds on current main: ContextCompressor.bind_session_state() reads durable guard state at agent/context_compressor.py:856-865, while compress_context() evaluates the automatic breaker before lock acquisition at agent/conversation_compression.py:507-520 and acquires the session lock later at :557-701. Legacy rotation marks the parent end_reason='compression' at :858-902, but current main has no post-lock check of that parent before compress().

The proposed refreshes and parent-ownership check directly cover those verified windows, preserve forced /compress behavior, and add production-seam regressions for the two race classes.

Automated hermes-sweeper review.

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Salvaged in #67363 with your commit cherry-picked to preserve authorship — thanks @ljy-2000! The revalidation logic and rotated-parent ownership check land as-is (verified with real-SessionDB probes including two-thread races and the legacy no-lock-API path). Two follow-up commits on top close the review findings: (1) the durable-guard refresh now also runs at the should_compress() pre-gates via _automatic_compression_blocked (check-refresh-recheck, hot path pays no DB reads) so a cleared durable row can unblock a prebound agent whose stale fallback streak has no expiry timer; (2) a refresh finding no durable cooldown row no longer clears a live local cooldown whose DB persist failed. #67363 has auto-merge armed and will land once CI completes.

kshitijk4poor added a commit that referenced this pull request Jul 19, 2026
Follow-up on the salvaged #64511 commit:

- _automatic_compression_blocked() now refreshes durable guard state
  (cooldown + fallback streak) when — and only when — the in-memory
  snapshot says blocked, then re-evaluates. The should_compress()
  pre-gates (preflight/turn paths) consult this before ever reaching
  compress_context, and a stale fallback streak has no expiry timer, so
  without a gate-level refresh a cleared durable row could never unblock
  a prebound agent. The unblocked hot path pays no DB reads.
- A refresh that finds no durable cooldown row no longer clears a live
  local cooldown whose DB persist FAILED (_cooldown_persist_failed):
  an empty row is not evidence another agent cleared it, and honouring
  it would reopen the #11529 thrash window. A successful durable
  round-trip (record or read) makes the DB authoritative again.
- Guard tests for both directions (red on the pre-fix code), including
  a hot-path test asserting the unblocked gate never touches the DB.
@teknium1 teknium1 added area/sessions Session lifecycle, resume, persistence, history area/compression Context compression and continuation sessions labels Jul 19, 2026
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
Follow-up on the salvaged NousResearch#64511 commit:

- _automatic_compression_blocked() now refreshes durable guard state
  (cooldown + fallback streak) when — and only when — the in-memory
  snapshot says blocked, then re-evaluates. The should_compress()
  pre-gates (preflight/turn paths) consult this before ever reaching
  compress_context, and a stale fallback streak has no expiry timer, so
  without a gate-level refresh a cleared durable row could never unblock
  a prebound agent. The unblocked hot path pays no DB reads.
- A refresh that finds no durable cooldown row no longer clears a live
  local cooldown whose DB persist FAILED (_cooldown_persist_failed):
  an empty row is not evidence another agent cleared it, and honouring
  it would reopen the NousResearch#11529 thrash window. A successful durable
  round-trip (record or read) makes the DB authoritative again.
- Guard tests for both directions (red on the pre-fix code), including
  a hot-path test asserting the unblocked gate never touches the DB.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/compression Context compression and continuation sessions area/sessions Session lifecycle, resume, persistence, history comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants