feat(redaction): exact-value secret redaction from user pattern file

[hermes-node8]

Problem

Hermes redacts shapes: any KEY=value whose key matches the built-in keyword families (PASS, TOKEN, SECRET, API_KEY, ...) is masked by redact_sensitive_text() at every message boundary. But exact values — secrets under non-obvious keys (PIN, x_passphrase), or bare values in prose — are only masked when a provider-specific rule happens to recognize their shape. The gaps are tracked in #77162 / #77165 / #77465 (plain-.env secrets under non-suffixed names, profile ContextVar scope, encoding transforms) and #72778 (file-read call-site wiring).

Solution

A fail-safe, file-driven exact-value pass inside redact_sensitive_text():

• Loads an optional user pattern file — plain JSON, hand-written or generated by companion tooling; no external tooling required:

  {
    "mask": {"head": 2, "tail": 2, "floor": 12},
    "literals": ["exact-value", {"value": "top-tier", "mask": "full"}],
    "key_patterns": {"PIN": true, "x_passphrase": true}
  }

• Masks exact registered literals and registered KEY=value / KEY: value / "KEY": "value" forms regardless of value length — the class the keyword families miss (PIN=1234).

• Path resolution: security.redact_patterns (config.yaml, bridged at startup) → HERMES_REDACT_PATTERNS env → default $HERMES_HOME/state/redaction/redact_patterns.json (~/.hermes fallback — profiles/relocated installs work).

• Mask styles: length-driven default (2+2 visible chars, floor 12 — preserves the built-in ≥8-hidden invariant while phones and short emails stay partially visible for troubleshooting); per-entry "full" or custom {"head", "tail", "floor"}; unknown styles fall back to the safe default.

Safety properties

• Fail-safe: missing/unreadable file = no-op; the built-in families always run. A broken file keeps the last-good pattern set active (no unmasked gap) and auto-recovers when repaired.
• Sentinel on file reads: file_read=True content uses the non-reusable sentinel ([Bug]: redact_sensitive_text corrupts API keys in config files when read via read_file/search_files, causing 401 #35519 semantics) — a masked value can never be written back over the real file.
• Mtime-cached: pattern rotations are picked up without restart.
• Display-only: never modifies .env, config, or vault sources.
• Precision: exact-value matching only (no substring patterns) — over-redaction is bounded to the file's own contents.

Reviewer feedback addressed

• Env-var convention — user-facing configuration follows config.yaml (security.redact_patterns), bridged to the env var at all three entry points, mirroring the existing security.redact_secrets bridge; the env var remains for programmatic overrides.
• Alternation first-match — literals are sorted longest-first before alternation compilation, so overlapping values mask deterministically.
• Short-value early return — removed; short values under registered keys (PIN=1234) are masked by the key-pattern pass.
• Cache on parse failure — a broken file keeps the last-good compiled patterns and pins the cache mtime to the broken file (auto-recovers when valid again). Resetting the cache on failure would drop the pattern set and unmask known secrets — strictly worse.

Related work

• feat(plugins): additive-only redaction pattern registry #84927 (merged) covers plugin-author formats via ctx.register_redaction_patterns() — complementary: a developer/plugin API for vendor token shapes; this PR is the end-user file-driven exact-value layer. No overlap.
• At-rest residue (state.db, request dumps — Credential redaction not applied to state.db session store (only logs + optional JSON snapshots) #54027 / Redaction gaps at the persistence boundary: tool output file dumps, compaction blocks, DB URIs (split from #43083) #43666 / PII (email/SSN) persisted unredacted in the session JSON transcript export #57740) is out of scope: this pass masks display/egress; it does not scrub history.

Validation

• New tests: tests/agent/test_redact_registry_patterns.py (mask styles incl. unknown-style fallback, longest-first alternation, short values, mtime rotation pickup, broken-file fail-safe with auto-recovery, file-read sentinel, write-back invariant, missing-file no-op, HERMES_HOME default-path resolution, JSON key forms, quoted multi-word values, separator-whitespace preservation, empty-value passthrough, path-keyed cache switching) and tests/hermes_cli/test_redact_patterns_bridge.py (env wins, config fallback).
• 15/15 passing on current main (includes pre-publication external-security-audit regressions).
• Exercised live in the reference deployment since 2026-08-13; apply-checked and full suite against v0.20.0, v0.20.1, v0.20.2, and current main.

Follow-ups

• Secret redaction is bypassed when reading config files via the file_read tool #72778 (file_read call-site wiring) — the sentinel semantics are in place; call sites get wired in a separate PR.
• A "patterns" (regex) section plus named shape styles (email keeps domain, phone last-4) — natural v2 on the same file.

[hermes-node8]

Friendly bump — this PR has been open since Aug 16 and CI hasn't run on the branch yet (fork PRs appear to need a maintainer to approve the workflow run; no checks have reported). Could someone approve the run and take a look? Happy to adjust anything that comes up in review.

[hermes-node8]

Superseded — this branch now carries the updated implementation, aligned with Hermes' current redaction direction (#34029 URL pass-through, #43666 compaction-boundary semantics). The replacement PR is opened from this branch; link to follow.

[hermes-node8]

Superseded by #97383 — the updated implementation, aligned with upstream's current redaction direction (#34029 URL pass-through, #43666 compaction-boundary semantics). This PR is closed as superseded.