fix(autocompact): retry circuit breaker after cooldown

[chioarub]

Summary

Refs #1373; partially addresses #1373 by fixing the auto-compact cooldown breaker. Remaining long-session OOM guards are tracked in #1379.

• Change the auto-compact failure circuit breaker from permanent suppression to a cooldown-based breaker.
• Add half-open retry behavior: after cooldown, one compaction attempt is allowed; success resets failures, failure re-trips the breaker.
• Do not count user-aborted compactions as automatic compaction failures.
• Surface a clear warning/blocking message when automatic compaction is paused and the live context is still above the safe threshold.
• Preserve the original protection against repeated doomed compaction attempts.

Why

The existing breaker stops auto-compact after 3 consecutive failures to avoid retry storms. However, once tripped, it can suppress future auto-compaction for the rest of a long-running query loop. If the conversation continues accumulating large tool results/history, memory pressure can grow until Node/V8 crashes with heap OOM.

This PR keeps the retry-storm protection but makes it recoverable.

Behavior

• Before 3 failures: unchanged.
• On the 3rd non-user failure: auto-compact enters cooldown and the user is warned.
• During cooldown: OpenClaude does not hammer the compact endpoint.
• After cooldown: one half-open compaction attempt is allowed.
• On success: failure state resets.
• On failure: cooldown is renewed.
• If the conversation is already over the safe threshold while compaction is cooling down, OpenClaude stops before sending another oversized request and shows clear recovery guidance.

Out of scope

• Does not raise Node/V8 heap limits; launcher heap behavior is tracked separately.
• Does not add an absolute state.messages cap because arbitrary message slicing can corrupt tool_use/tool_result pairing and compact-boundary semantics; follow-up work is tracked in Track remaining long-session OOM guards after auto-compact cooldown #1379.
• Does not broadly refactor in-process teammates; bounding teammate allMessages retention is tracked in Track remaining long-session OOM guards after auto-compact cooldown #1379.

Testing

• bun test src/services/compact/autoCompact.test.ts
• bun test src/query/autoCompactCooldown.test.ts
• bun run smoke
• bun run --cwd web typecheck && bun run --cwd web build
• python -m pytest -q python/tests
• bun run security:pr-scan -- --base upstream/main
• bun run test:provider
• npm run test:provider-recommendation
• bun run build
• git diff --check

Fork CI also passed on this branch before opening the upstream PR: smoke-and-tests and web.

Local full-suite note: bun test --max-concurrency=1 was run in a sanitized environment and had one unrelated baseline failure in src/utils/conversationRecovery.hooks.test.ts (deserializeMessagesWithInterruptDetection strips thinking blocks only for OpenAI-compatible providers). The focused and CI-relevant suites above pass.

Notes

The breaker remains intentionally conservative. Cooldown retry is half-open rather than a full reset, so an unrecoverable compaction failure causes one retry per cooldown window rather than another burst of repeated attempts.

[jatmn]

Thanks for the contribution. The cooldown breaker change looks focused, but I think the PR should avoid closing the whole linked issue as written.

Findings

• [P2] Keep #1373 open until the remaining OOM paths are tracked or fixed
  src/utils/swarm/inProcessRunner.ts:1004
  #1373 reports the auto-compact breaker as the primary cause, but it also asks for an absolute state.messages cap and calls out the in-process teammate allMessages buffer as another unbounded OOM path. This PR fixes the main auto-compact breaker path, but it explicitly leaves the cap and teammate-retention work out of scope. With Fixes #1373, merging this would close the issue while those reported memory-growth protections are still unresolved. Please change the closing keyword to Refs #1373 / Partially addresses #1373 and link a follow-up for the remaining OOM work, or include those pieces before closing the issue.

[chioarub]

Updated the PR description to avoid closing #1373.

It now uses Refs #1373; partially addresses #1373, and the remaining state.messages cap plus in-process teammate allMessages retention work is tracked in #1379. No code changes were needed for this review finding.

[jatmn]

Thanks for the update. I rechecked the previously discussed issue-linking concern and took another pass through the auto-compact cooldown paths. I found one small issue worth tightening before this lands.

Findings

• [P3] Either use or remove the cooldown duration in breaker resolution
  src/services/compact/autoCompact.ts:97
resolveAutoCompactCircuitBreakerState accepts cooldownMs, and the resolver tests pass different cooldown values, but the helper never reads that parameter. The current runtime path still works when nextRetryAtMs was set by the trip path, but the helper API and tests make it look like the cooldown duration participates in the decision when it does not. Please either remove cooldownMs from this helper/tests, or have the resolver derive the active cooldown from lastFailureAtMs + cooldownMs when appropriate so the duration is actually covered by the unit tests.

[chioarub]

Pushed follow-up commit a017b52 addressing the cooldown resolver finding.

Changes:

• resolveAutoCompactCircuitBreakerState now uses cooldownMs by deriving retry time from lastFailureAtMs + cooldownMs when nextRetryAtMs is absent or invalid, while preserving explicit nextRetryAtMs precedence.
• Added resolver coverage for active and expired derived cooldowns.
• Added regressions for fresh breaker tracking metadata and SDK manual /compact clearing stale auto-compact cooldown state.

Local validation:

• bun run smoke
• bun test --max-concurrency=1 in a CI-like sanitized provider environment: 2956 pass, 0 fail
• python -m pytest -q python/tests: 44 passed
• bun run security:pr-scan -- --base upstream/main
• bun run test:provider
• npm run test:provider-recommendation
• bun run --cwd web typecheck
• bun run --cwd web build
• git diff --check

[jatmn]

Thanks for the update. I rechecked the previously discussed cooldown resolver path and ran through the focused auto-compact changes again. I found one portability issue in the new regression coverage.

Findings

• [P2] Use a cross-platform cwd in the SDK cooldown fixture
  src/test/fixtures/queryEngineManualCompactCooldown.fixture.ts:116
  The new QueryEngine.autoCompactCooldown.test.ts fixture constructs QueryEngine with cwd: '/tmp'. On Windows that path does not normally exist, and QueryEngine.submitMessage() calls setCwd(cwd), which rejects the fixture before the manual /compact regression can run. This makes bun test src/QueryEngine.autoCompactCooldown.test.ts fail locally on Windows with Path "/tmp" does not exist. Please use a real cross-platform temporary directory, such as tmpdir() from node:os, for both the fixture cwd and the mocked init message.

[chioarub]

Pushed follow-up commit 04dac1d19ac7 addressing the Windows portability finding.

Changes:

• Replaced the SDK cooldown fixture's hard-coded /tmp cwd with a real temporary directory created from tmpdir().
• Reused that directory for both the QueryEngine cwd and the mocked init message cwd, then cleaned it up in finally.
• Added resolver regression coverage for invalid nextRetryAtMs fallback and finite explicit retry-time precedence.

Review and validation:

• Completed two surgical review cycles; the second cycle was clean from contract, test-verification, and surgicality reviewers.
• bun install --frozen-lockfile
• bun run smoke
• bun test --max-concurrency=1 (2959 pass, 0 fail; run with sanitized local provider env)
• python -m pip install -r python/requirements.txt
• python -m pytest -q python/tests (44 passed)
• bun run security:pr-scan -- --base upstream/main
• bun run test:provider (609 pass, 0 fail)
• npm run test:provider-recommendation (78 pass, 0 fail)
• bun install --cwd web --frozen-lockfile
• bun run --cwd web typecheck
• bun run --cwd web build
• git diff --check

[jatmn]

Thanks for the update. I rechecked the earlier issue-linking, cooldown resolver, and Windows portability follow-ups and took another pass through the final auto-compact changes. I found one remaining issue before this is ready.

Findings

• [P2] Give the SDK cooldown fixture test more than Bun's default 5s timeout
  src/QueryEngine.autoCompactCooldown.test.ts:8
  This regression test now spawns a second Bun process to load QueryEngine from a cold TypeScript graph. On a fresh Windows checkout, bun test src/QueryEngine.autoCompactCooldown.test.ts timed out at Bun's default 5 second per-test limit before the fixture finished, even though rerunning with a higher timeout passes. That makes the new SDK coverage flaky exactly on the platform this PR just fixed. Please set an explicit longer timeout for this test or avoid the extra-process path so the regression stays reliable on cold runs.

[chioarub]

Pushed follow-up commit 0210238 addressing the SDK cooldown fixture timeout finding.

Changes:

• Added an explicit per-test timeout for QueryEngine.autoCompactCooldown.test.ts so Bun no longer uses the default 5s cap.
• Added a 60s timeout around the spawned fixture process, with the Bun test timeout set slightly higher, so cold Windows TypeScript graph loads have headroom while true fixture hangs still fail deterministically.
• Kept the out-of-process fixture path to preserve isolation from Bun's shared module/mock cache.

Validation:

• bun test src/QueryEngine.autoCompactCooldown.test.ts
• bun test src/QueryEngine.autoCompactCooldown.test.ts src/query/autoCompactCooldown.test.ts src/services/compact/autoCompact.test.ts --max-concurrency=1
• bun run smoke
• python -m pytest -q python/tests
• bun run security:pr-scan -- --base upstream/main
• bun run test:provider
• npm run test:provider-recommendation
• bun run --cwd web typecheck
• bun run --cwd web build
• git diff --check

GitHub Actions on this commit are green: smoke-and-tests and web both passed.

[jatmn]

Thanks for the update. I rechecked the previously discussed issue-linking, cooldown resolver, Windows fixture, and fixture-timeout paths, then took another pass through the current auto-compact cooldown changes. I do not see any remaining actionable issues from my side.