Skip to content

fix(security): harden compaction secret redaction - #49556

Closed
AndrewMoryakov wants to merge 1 commit into
NousResearch:mainfrom
AndrewMoryakov:fix/compaction-token-redaction
Closed

fix(security): harden compaction secret redaction#49556
AndrewMoryakov wants to merge 1 commit into
NousResearch:mainfrom
AndrewMoryakov:fix/compaction-token-redaction

Conversation

@AndrewMoryakov

@AndrewMoryakov AndrewMoryakov commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • force secret redaction across all context compaction summary boundaries
  • add opt-in strict URL redaction for safety-boundary callers without changing default URL passthrough behavior
  • redact manual focus topics, auto focus hints, legacy previous summaries, summarizer input, fallback summaries, and summarizer output

Security notes

This preserves the existing default behavior where web URLs pass through unchanged for browsing/OAuth workflows. Only compaction uses redact_urls=True, because compaction sends text to an auxiliary summarizer and persists the resulting summary.

Tests

  • python -m pytest tests/agent/test_redact.py tests/agent/test_context_compressor.py tests/agent/test_compress_focus.py tests/agent/test_context_compressor_summary_continuity.py -q
  • git diff --check

@alt-glitch alt-glitch added type/security Security vulnerability or hardening comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists labels Jun 20, 2026
@alt-glitch

alt-glitch commented Jun 20, 2026

Copy link
Copy Markdown
Collaborator

Related: #13228 (merged base — redacts secrets from compaction input/output), #20998 (open — force-redact compression summaries), #43666 / #43940 (persistence-boundary redaction gaps).

This PR extends the merged baseline by threading a force=True, redact_urls=True helper through all compaction summary boundaries (focus topic, previous summary, summarizer input/output, fallback) and broadens URL userinfo redaction to catch token-only userinfo (scheme://token@host). Verified against main: _redact_compaction_text and the redact_urls parameter are not yet present, so this is a genuine gap-fill rather than a duplicate of the merged fix.

@AndrewMoryakov
AndrewMoryakov force-pushed the fix/compaction-token-redaction branch from ac32fdf to e495b6d Compare June 20, 2026 09:27
@sdyckjq-lab sdyckjq-lab mentioned this pull request Jul 1, 2026
Open
19 tasks
@teknium1

teknium1 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Thanks for closing the compaction-specific redaction gap. The premise remains valid on current main: compaction calls redact_sensitive_text() without force=True for serialized input and output (agent/context_compressor.py:1525, :1558, :2063), and reinjects _previous_summary directly into the auxiliary summarizer prompt (:1963). The redactor intentionally preserves opaque web URL query values in its normal mode (agent/redact.py:655-667), so current behavior does not provide this PR's strict persistence-boundary guarantee.

The existing merged baseline, 3368814a3dfe6f61709ae11fa16615969391785c, covers ordinary secret-pattern redaction but not forced redaction plus URL-query redaction. The changed paths have moved since the PR base and GitHub reports the branch as dirty, but the implementation is narrowly salvageable by reapplying the helper and its call sites onto current agent/context_compressor.py and agent/redact.py.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
@teknium1 teknium1 mentioned this pull request Jul 22, 2026
Merged
teknium1 added a commit that referenced this pull request Jul 22, 2026
@teknium1 @AndrewMoryakov
fix(compression): apply strict redaction at every compaction text bou…
8a0a319 
…ndary

Compaction summaries persist across sessions and re-enter every subsequent
summarizer prompt, but every redact_sensitive_text() call in
context_compressor.py used default mode: a no-op under
security.redact_secrets:false, and opaque OAuth-callback / URL-userinfo
credentials passed through even when enabled. The stored _previous_summary
also re-entered the iterative-update prompt unredacted.

Add _redact_compaction_text() — redact_sensitive_text(force=True,
redact_url_credentials=True) — and thread it through all compaction text
boundaries: serializer input (content + tool args), deterministic fallback
summary, summarizer LLM output, manual + auto focus topics, the latest-user
task snapshot, and _previous_summary re-entry.

Note: force=True at this boundary intentionally overrides
security.redact_secrets:false — that opt-out targets live tool output, not
persisted summaries.

Salvages the compaction half of #49556 (the redact.py strict-URL half
landed independently via 75af6dc/62a00a739). Addresses #43666 item 2.

Co-authored-by: AndrewMoryakov <topazd2@gmail.com>
teknium1 added a commit that referenced this pull request Jul 22, 2026
@teknium1 @AndrewMoryakov
fix(compression): apply strict redaction at every compaction text bou…
0acdf1d 
…ndary (#69294)

Compaction summaries persist across sessions and re-enter every subsequent
summarizer prompt, but every redact_sensitive_text() call in
context_compressor.py used default mode: a no-op under
security.redact_secrets:false, and opaque OAuth-callback / URL-userinfo
credentials passed through even when enabled. The stored _previous_summary
also re-entered the iterative-update prompt unredacted.

Add _redact_compaction_text() — redact_sensitive_text(force=True,
redact_url_credentials=True) — and thread it through all compaction text
boundaries: serializer input (content + tool args), deterministic fallback
summary, summarizer LLM output, manual + auto focus topics, the latest-user
task snapshot, and _previous_summary re-entry.

Note: force=True at this boundary intentionally overrides
security.redact_secrets:false — that opt-out targets live tool output, not
persisted summaries.

Salvages the compaction half of #49556 (the redact.py strict-URL half
landed independently via 75af6dc/62a00a739). Addresses #43666 item 2.

Co-authored-by: AndrewMoryakov <topazd2@gmail.com>
@teknium1

teknium1 commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Merged via #69294 (commit 0acdf1d). The compaction half of your PR was rebuilt against main's landed strict-URL redaction (your redact.py half had been superseded by 75af6dc/62a00a739) and merged with Co-authored-by credit to you — strict redaction now applies at all nine compaction text boundaries. Thanks!

@teknium1 teknium1 closed this Jul 22, 2026
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
@teknium1 @AndrewMoryakov
fix(compression): apply strict redaction at every compaction text bou…
5b95388 
…ndary (NousResearch#69294)

Compaction summaries persist across sessions and re-enter every subsequent
summarizer prompt, but every redact_sensitive_text() call in
context_compressor.py used default mode: a no-op under
security.redact_secrets:false, and opaque OAuth-callback / URL-userinfo
credentials passed through even when enabled. The stored _previous_summary
also re-entered the iterative-update prompt unredacted.

Add _redact_compaction_text() — redact_sensitive_text(force=True,
redact_url_credentials=True) — and thread it through all compaction text
boundaries: serializer input (content + tool args), deterministic fallback
summary, summarizer LLM output, manual + auto focus topics, the latest-user
task snapshot, and _previous_summary re-entry.

Note: force=True at this boundary intentionally overrides
security.redact_secrets:false — that opt-out targets live tool output, not
persisted summaries.

Salvages the compaction half of NousResearch#49556 (the redact.py strict-URL half
landed independently via 9179c57/04577d1bf). Addresses NousResearch#43666 item 2.

Co-authored-by: AndrewMoryakov <topazd2@gmail.com>
33hodl pushed a commit to 33hodl/hermes-agent that referenced this pull request Aug 12, 2026
@teknium1 @AndrewMoryakov
fix(compression): apply strict redaction at every compaction text bou…
4c6beeb 
…ndary (NousResearch#69294)

Compaction summaries persist across sessions and re-enter every subsequent
summarizer prompt, but every redact_sensitive_text() call in
context_compressor.py used default mode: a no-op under
security.redact_secrets:false, and opaque OAuth-callback / URL-userinfo
credentials passed through even when enabled. The stored _previous_summary
also re-entered the iterative-update prompt unredacted.

Add _redact_compaction_text() — redact_sensitive_text(force=True,
redact_url_credentials=True) — and thread it through all compaction text
boundaries: serializer input (content + tool args), deterministic fallback
summary, summarizer LLM output, manual + auto focus topics, the latest-user
task snapshot, and _previous_summary re-entry.

Note: force=True at this boundary intentionally overrides
security.redact_secrets:false — that opt-out targets live tool output, not
persisted summaries.

Salvages the compaction half of NousResearch#49556 (the redact.py strict-URL half
landed independently via 75af6dc/62a00a739). Addresses NousResearch#43666 item 2.

Co-authored-by: AndrewMoryakov <topazd2@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/security Security vulnerability or hardening

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@AndrewMoryakov @alt-glitch @teknium1