Repository navigation
fix(hooks): close two residual secret-disclosure channels to the model - #841
Merged
Merged
Conversation
PR #819 sanitized the shell-hook callback __name__ but left two paths that still render operator-supplied text into MODEL-facing refusals. Both reproduced on the real paths before fixing (qa-output/round13). P1-a agent/shell_hooks.py — an unparseable command (the shape that still HOLDS an inline credential: `sh -c 'export TOK=...` with the quote left open) put the raw command into result["error"], which _evaluate_result hands to _fail_closed_block as the model-facing reason. Measured before: hook sh#8ece5304 failed closed: command "/bin/sh -c 'export TOK=<planted>; echo hi" cannot be parsed: No closing quotation Now reports the exception CLASS only. Swept the two sibling `str(exc)` producers (spawn / communicate) for the same reason: exception text routinely embeds argv. The raw command stays on the log channel, unchanged, for operators. P1-b hermes_cli/plugins.py — getattr(cb, "__name__", repr(cb)) fell through to repr() for any callback without a __name__, and functools.partial's repr renders every bound argument. Measured before: pre_tool_call plugin callback functools.partial(<function _slow at 0x...>, '<planted>') timed out after 0.407s One _callback_label() helper now covers all three sites (:5709 model-facing, :5955/:6141 log): __qualname__/__name__, else the WRAPPED function's name for partials, else <type>@id — never repr. Labels stay diagnostic: `partial(_policy)`, `hook sh#8ece5304`. P1-c two regressions, driven through the real consumer (resolve_pre_tool_block / _evaluate_result), asserting the sentinel is absent from BOTH fail-closed paths while the block itself is preserved and the hook stays identifiable. Verified: 314/314 across 10 plugin/hook suites; ruff clean; gitleaks clean with .gitleaks.toml byte-unchanged. Mutation of either fix turns the matching regression RED by name.
… attempts — infra outage shape, not a verdict); tree unchanged
Collaborator
Author
FleetReviewReviewed with 2 of 3 model families — openai unavailable. Confidence: 3/5 Findings
FleetReview provenance · models: C=claude-code-opus-5, D=grok-4.6, G=grok-4.6 · cost: $6.50 · duration: 12m 42s · rounds: 1 · files examined: 3 |
…ror channels FleetReview P1 on #841: a webhook target's URL reached the MODEL through the fail-closed refusal messages. WebhookTarget.label falls back to self.url when the optional `name:` is omitted, _make_callback synthesized __name__ from it, and _callback_label trusts __name__ — so a Slack/Discord bearer URL was interpolated into both the timeout and suppression blocks. Third sibling of the shell-hook/partial pair #841 already closed. Fixed at the PRODUCER, mirroring shell_hooks.hook_display_name: WebhookTarget.display_label returns the operator's `name:` or `webhook#<sha256 [:8]>`, never the URL. `label` still yields the URL for the log channel. _callback_label's docstring claimed __name__/__qualname__ "are developer-chosen identifiers and safe" — false in this repo (two producers synthesize them from operator config; register_hook accepts any plugin value). Corrected to state that each producer sanitizes at the source and this is the last line of defence. P2, same pass: - shell_hooks now carries TWO error channels. `error` stays redacted (exception class only) for the model; new `error_detail` keeps the full diagnostic for the log and `hermes hooks test` / `doctor`, which had been blinded — shlex "No closing quotation" and spawn EACCES/ENOEXEC were indistinguishable. error_detail never reaches a model-facing string. - The no-name fallback used `id(callback)`, a CPython heap address that is per-process and GC-recyclable, in a model-facing string that the docstring called "stable". Now a sha256 digest of the fully-qualified type. Verified: P1 reproduced on the real path (repro LEAKS -> CLEAN) before/after; 205/205 across 6 focused suites; ruff clean.
Collaborator
Author
FleetReviewReviewed with 2 of 3 model families — openai unavailable. Confidence: 3/5 Findings
FleetReview provenance · models: C=claude-code-opus-5, D=grok-4.6, G=grok-4.6 · cost: $10.77 · duration: 1h 29m 37s · rounds: 4 · files examined: 6 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #819 (merged as 846ba49). Closes the three residual P1s
Apollo named on t_59c0886e after the FleetReview record for
hermes-agent-pr-819-7945f423came back withunresolvedP0P1 = 3.#819 sanitized the shell-hook callback
__name__; these are the twopaths it did not cover, both of which render operator-supplied text into
a model-facing refusal.
Reproduced before fixing
Both on the real paths, with
__file__asserted against the worktree(the shared venv resolves the LIVE tree — the round-2 measurement trap).
Probe:
qa-output/round13/repro_p1a_p1b.py.hook sh#8ece5304 failed closed: command "/bin/sh -c 'export TOK=<planted>; echo hi" cannot be parsed: No closing quotationhook sh#8ece5304 failed closed: command cannot be parsed (ValueError)pre_tool_call plugin callback functools.partial(<function _slow at 0x…>, '<planted>') timed out after 0.407spre_tool_call plugin callback partial(_slow) timed out after 0.402sP1-a —
agent/shell_hooks.pyAn unparseable command is exactly the shape that still holds an
inline credential (
sh -c 'export TOK=…with the quote left open)._spawnput the raw command intoresult["error"], and_evaluate_resulthands that to_fail_closed_blockas the model-facingreason — so
hook_display_namesanitized the hook's name while the rawcommand walked in through the reason. Argus's 15 adversarial shapes in
round 8 were all parseable, so this path was never exercised.
Now reports the exception class only. Swept the two sibling
str(exc)producers (spawn / communicate) for the same reason —exception text routinely embeds argv. All six
result["error"]producers are now secret-free; the raw command stays on the log channel
(6 sites, unchanged) for operators.
P1-b —
hermes_cli/plugins.pygetattr(cb, "__name__", repr(cb))fell through torepr()for anycallback without a
__name__, andfunctools.partial's repr rendersevery bound argument. One
_callback_label()helper now covers allthree sites (
:5709model-facing via both fail-closed paths,:5955/:6141log):__qualname__/__name__, else the wrappedfunction's name for partials, else
<type>@id— neverrepr.Labels stay diagnostically useful:
partial(_policy),hook sh#8ece5304.P1-c — regressions
Two tests driven through the real consumers
(
resolve_pre_tool_block/_evaluate_result), each asserting thesentinel is absent from both fail-closed paths while the block itself
is preserved and the hook stays identifiable.
Sentinel local is named
planted(notsecret) per the round-7 gitleaksconvention;
.gitleaks.tomlis byte-unchanged.Verification
test_plugins,test_shell_hooks,test_shell_hooks_fail_closed,_consent,_tree_kill,test_plugin_event_bus,test_hooks_cli,test_subagent_stop_hook,test_plugin_dev,test_agent_plugins).test_unparseable_hook_command_is_not_disclosedRED by name while theP1-b test stays green, and vice versa for
test_refusals_do_not_disclose_a_callback_without_a_name. Treerestored byte-exact after each (
git status --porcelainclean).ruff checkclean;gitleaks protect --staged— no leaks.Fail-closed policy is unchanged: a hook that cannot be parsed, or a
callback that times out, still blocks — only the wording changed.
Base:
fork/main@ a3e3c16, with #819's merge 846ba49 confirmed asan ancestor.
Card: t_59c0886e. Not enqueued and not marked ready — Apollo lands this
via
fleet-merge.shagainst the FleetReview record.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.