fix(redact): stop masking prose words that embed a secret keyword (port of ironclaw#6129) - #67776
Merged
Merged
Conversation
…cretary, tokenizer, author=) Port from nearai/ironclaw#6129: their sensitive-marker scrubber matched markers as bare substrings, so tool results containing 'Secretary of the Treasury' were scrubbed as 'secret' on replay, evicting legitimate content and forcing the model into a re-fetch loop. Hermes' lowercase/dotted/YAML config-key redaction patterns (_CFG_DOTTED_RE, _CFG_ANCHORED_RE, _YAML_ASSIGN_RE) had the same false-positive class: their key classes allow arbitrary alphanumeric affixes around the keyword, so ordinary document text like 'Secretary: J.Smith', 'tokenizer: cl100k_base' (HF model cards), and BibTeX 'author=Smith' got value-masked on the surfaces that run these passes (browser snapshots, log lines, kanban summaries, CLI-echoed output). Fix: post-match word-boundary validation of the keyword occurrence inside the matched key. Boundaries: key edges, non-letters (_ - . digits), camelCase transitions (clientSecret, secretKey, APIToken), plural 's' (secrets:, tokens:). Concatenated real-world compounds keep matching via explicit alternatives (authtoken, authkey, secretkey, accesstoken). ALL-CAPS keys keep legacy embedded matching (MYTOKEN=...) — all-caps is almost never prose, same rationale as _ENV_ASSIGN_RE. Same discipline the file already applies to exact-match body/query keys (ported from ironclaw#2529) and the deliberate 'auth' exclusion that keeps 'author:' from matching.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prose/document words that merely embed a secret keyword —
Secretary:(secret),tokenizer:(token), BibTeXauthor=(auth) — no longer get value-masked by the lowercase/dotted/YAML config-key redaction passes. Real credential key shapes (separators, camelCase, acronyms, plurals, common concatenated compounds, all-caps env style) redact exactly as before.Port of nearai/ironclaw#6129, where the same bug class (sensitive markers matched as bare substrings) scrubbed tool results containing "Secretary of the Treasury" as
secreton transcript replay, evicting legitimate content and sending the model into a re-fetch loop.Root cause in hermes:
_CFG_DOTTED_RE/_CFG_ANCHORED_RE/_YAML_ASSIGN_REkey classes allow arbitrary alphanumeric affixes around the keyword ([A-Za-z0-9_.\-]*secret[A-Za-z0-9_.\-]*), soSecretary,Undersecretary,secretariat,tokenizer,authored,credentialingall matched as "keys". Confirmed live on main:These passes run on model- and user-visible surfaces: browser snapshots (
force=True), kanban summaries, TUI verbose text, CLI-echoed command output, and log lines. (File reads and ordinary terminal output are unaffected — they usecode_file=True, which skips these passes.)Changes
agent/redact.py:_key_has_secret_keyword()post-match validator wired into the_redact_env/_redact_yamlcallbacks. A keyword occurrence counts only at a word boundary within the key:_ - .digits) —client_secret,db.password,oauth2_tokenclientSecret,secretKey,APIToken(acronym-run rule)s—secrets:,tokens:authtoken(ngrok),authkey(tailscale),secretkey(minio),accesstoken,apikeyMYTOKEN=…) — all-caps is almost never prose, same rationale as_ENV_ASSIGN_RE(which is untouched)tests/agent/test_redact.py: newTestKeywordWordBoundaryclass — 8 preserved-prose cases + 7 still-masked shape groups.This is the same discipline
redact.pyalready applies elsewhere: the body/query key sets are exact-match "NOT substring" (ported from ironclaw#2529), and bareauthwas already excluded from the YAML key set soauthor:wouldn't match — this closes the remaining affix hole.Adaptation notes
IronClaw's fix is a pure alnum-boundary check on lowercased text in their transcript-replay scrubber. Hermes' equivalent surface is the config-key redaction patterns, and hermes keys are not pre-lowercased, so the port adds camelCase/acronym boundary awareness (their lowercased haystack can't distinguish
clientSecretfromsecretary) plus plural and compound handling to avoid false negatives on real key shapes IronClaw's marker list doesn't have.Validation
Secretary: JanetYellen…tokenizer: cl100k_base…author=Smith2020…(BibTeX)press.secretary=…client_secret:/clientSecret:/APIToken:authtoken:/authkey=/secretkey:/secrets:MYTOKEN=…(all-caps)force=True): all passtests/agent/test_redact.py: 163 passed (148 existing + 15 new)test_terminal_output_transform_hook,test_kanban_redaction,test_browser_type_redaction,test_approval_prompt_redaction,test_redact_config_bridge,test_browser_secret_exfil,test_debug,test_trace_upload,test_signal,test_tui_gateway_server, +4 more): 1,092 passedInfographic