fix(agent): preserve memory-provider context during compression

[GottZ]

Summary

Fixes #7192 by carrying the string returned from MemoryManager.on_pre_compress() into Hermes-managed context compression instead of discarding it.

• forwards provider context through the orchestrator without dropping focus_topic or force
• includes non-empty provider context in both initial and iterative summary prompts
• preserves it across auxiliary-to-main summary-model retries
• strictly redacts and bounds provider context before plugin-engine and summary-model egress
• frames provider text as an escaped JSON string so payload markers cannot close the host-owned data block
• keeps older context-engine plugins compatible by filtering optional kwargs from the callable signature
• propagates internal engine TypeErrors after one call and releases compression locks even when capability inspection fails

Lineage and related work

This consolidates the useful parts and review findings from #7195, #7424, #11236, #22576, #29598, and #43567, plus the broader hardening work in Tranquil-Flow#2.

The original #7195 implementation is preserved as the first commit with @Tranquil-Flow's authorship. Follow-up commits port that behavior onto current main, replace source-text assertions with executed behavior contracts, and incorporate the compatibility, retry, lock-lifecycle, redaction, and prompt-framing findings from the related reviews.

Compatibility and scope

ContextEngine.compress() now documents optional force and memory_context arguments. The host inspects plugin signatures before calling them, so existing strict-signature engines continue to receive only the arguments they support. This replaces the former broad except TypeError retry, which could execute a stateful engine twice when the TypeError came from inside the implementation.

Codex app-server-native compaction remains intentionally outside this handoff. Its hidden thread is compacted by the app server and Hermes has no truthful summary-prompt injection point for provider-returned text. Existing Codex compaction behavior is unchanged and covered by its regression suite.

No provider-specific core code, new persistence file, configuration key, or prompt-cache mutation is introduced.

Verification

• 849/849 focused and adjacent tests across 54 live-discovered files
• 853/853 in the same scope on a conflict-free synthetic merge with current upstream/main at 46e87b14f
• provider-context prompt/orchestrator/retry contracts, including both retry branches
• real SQLite lock/lease release after refresher construction/start, engine, signature-inspection, post-dispatch state, no-op prompt rebuild, and refresher-stop failures
• strict provider-context redaction, URL-credential masking at non-navigation egress, 6,000-character bound, and delimiter-collision regression
• default actionable-URL redaction compatibility, context-engine ABC, strict-signature compatibility, summary continuity, auto-focus, anti-thrash, and Codex app-server regressions
• Ruff, bytecode compilation, diff check, and added-line security scan
• independent fresh-context re-review: PASS with no findings on exact head e013a6bfcf3bcbb1c969e694fba4f70a05265d51

[Copilot]

Pull request overview

This PR fixes the long-standing gap where MemoryManager.on_pre_compress() provider-returned text was computed but not reliably preserved through Hermes-managed context compression. It threads provider context into the compression pipeline (agent → context engine → summarizer), while hardening egress redaction/framing and improving context-engine compatibility and lock/refresher cleanup behavior.

Changes:

• Capture and sanitize on_pre_compress() return text, then forward it through the compression orchestrator without dropping focus_topic / force.
• Inject bounded, strictly redacted provider context into both initial and iterative summary prompts (and preserve it across auxiliary→main retry paths) using an escaped JSON string framing.
• Replace the prior broad TypeError retry behavior with signature-based kwarg filtering to avoid double-invoking stateful engines, and strengthen compression lock lease cleanup across failure modes.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
agent/conversation_compression.py	Captures provider context, filters engine kwargs via signature inspection, and hardens lock/refresher release across error paths.
agent/context_engine.py	Adds sanitize_memory_context() and documents new optional compress() params (force, memory_context).
agent/context_compressor.py	Injects sanitized provider context into summarization prompts (initial + iterative) and preserves it across retry fallbacks.
agent/redact.py	Adds opt-in strict URL credential redaction for non-navigation egress boundaries.
tests/run_agent/test_pre_compress_memory_context.py	New end-to-end behavior contracts for agent→engine provider-context forwarding, sanitization/bounds, and TypeError propagation.
tests/run_agent/test_413_compression.py	Updates a compression test stub to match the expanded engine call signature.
tests/agent/test_pre_compress_memory_context.py	New prompt-level behavior contracts for provider-context injection, framing, bounding, and retry survival.
tests/agent/test_context_compressor_summary_continuity.py	Updates summary continuity test stub to accept the new optional kwarg.
tests/agent/test_compression_concurrent_fork.py	Adds/updates tests asserting lock/refresher release on signature/refresher/no-op prompt and engine error failure modes.
tests/agent/test_redact.py	Adds regression tests for the new opt-in strict URL credential redaction behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[teknium1]

Thanks for consolidating the provider-context handoff work. The premise remains valid on current origin/main: agent/conversation_compression.py:713-721 invokes on_pre_compress() but discards its return before calling the compressor, despite the return-value contract in agent/memory_manager.py:883-900 and agent/memory_provider.py:220-230.

The PR threads that value through the compression dispatch and summary retry paths, sanitizes it before egress, and replaces the broad TypeError retry with signature-based optional-kwarg filtering for legacy context engines. No blocking defect was identified in this read-only review.

Automated hermes-sweeper review.

[teknium1]

Merged via PR #67938 — all 9 of your commits were cherry-picked onto current main with authorship preserved in git log (rebase-merge, merge commit 766c617). Thanks for the thorough consolidation: the sanitization boundary, signature-based kwarg filtering, and lock-scope hardening all landed intact, along with your full behavior-contract test suites.

One tiny fix-up during salvage: a conflict-resolution slip on our side briefly dropped the if memory_context: guard in _supported_compression_kwargs — restored to your exact semantics and folded back into your commit.

Also credited: @Tranquil-Flow for the foundational capture commit carried in this PR.