fix(plugins): rank pre_tool_call block above approve - #87449
Conversation
get_pre_tool_call_directive() returned on the first valid directive of
either kind, so hook registration order decided the outcome: a plugin
returning {"action": "approve"} shadowed a later plugin's
{"action": "block"}. A security plugin's veto was silently lost, and
under approvals.mode: off an approve directive means no prompt at all,
so the vetoed tool call ran unchallenged.
Hold the first approve back until the whole result list has been scanned
for a block, so precedence is block > approve > no directive regardless
of registration order. Within a single action the first valid directive
still wins, and an invalid (message-less) block still falls through to a
later approve rather than swallowing it.
Fixes NousResearch#87420
5440732 to
31e1e8b
Compare
|
Rebased onto current Both conflicts were in the function this PR touches, resolved on merit rather than by taking a side:
The docs merge keeps main's "(Python plugins registered first, then shell hooks)" and observer-only clauses alongside the new precedence sentence. 70 tests pass in |
fix(plugins): rank pre_tool_call block above approve
|
… rule Review asked for an upgrade note, since ranking `block` above `approve` changes runtime semantics for existing plugin pairs without any API change to signal it. Adds a warning admonition naming exactly what changes (an earlier `approve` no longer suppresses a later `block`) and what does not (single directive hooks, observer-only hooks, `modify`). Also states the shallow-merge rule the accumulator has always had: a `modify` directive replaces a top-level key rather than merging into the value under it, so hooks should return a new value instead of mutating a nested object they received, which is shared with the original arguments. The same review asked whether the deferred-approve snapshot should deep-copy. It should not, and the reasoning is now recorded at the snapshot: a later `modify` reaches the accumulator only via `update()`, which rebinds top-level keys rather than mutating the values behind them, so it cannot reach through the snapshot's shared references. Deep-copying would diverge from the documented shallow-merge contract and break hooks passing unpicklable or identity-significant objects through args. Comments and documentation only; no behaviour change.
|
Thanks, useful pass. Acted on two of the four, and one does not reproduce. Shallow copy at the deferred-approve snapshot: documented rather than deepenedYou are right that modified_args.update(partial)
I took your second option rather than the deep copy. Deep-copying would diverge from the shallow-merge contract If you can construct a hook sequence that does defeat the snapshot, I would rather see it than argue from the code, and I will take the deep copy. Upgrade note: addedAgreed that this changes runtime semantics with no API change to signal it. There is no CHANGELOG in the repo, so it went to
|
What does this PR do?
_get_pre_tool_call_directive_details()aggregated pluginpre_tool_callresults with first-valid-wins semantics, so hook registration order — not severity — decided the outcome. A plugin returning{"action": "approve"}registered ahead of a security plugin returning{"action": "block"}made the block unreachable: the loop returned on the first valid directive of either kind.That is backwards for the one case the directive exists to serve. An
approveonly escalates to the human-approval gate, but underapprovals.mode: off(yolo-equivalent) an approve directive means no prompt at all — so a Safe Mode / startup-integrity / policy plugin's veto was silently discarded and the tool call ran unchallenged.The fix holds the first
approveback until the whole result list has been scanned for a veto, making precedenceblock>approve> no directive regardless of registration order. This mirrors how the approval layer already treats a deny from any source as decisive, and how the thread tool-whitelist block short-circuits ahead of the hooks.Deliberately narrow — no change to hook invocation, to the directive shape, or to what counts as a valid directive:
rule_keyforapprove.blockis still invalid and still ignored — and now correctly falls through to a laterapproverather than swallowing it.Related Issue
Fixes #87420
Type of Change
(Also a bug fix; filing it under security because the failure mode is a silently-dropped plugin veto on a tool call.)
Changes Made
hermes_cli/plugins.py—_get_pre_tool_call_directive_details(): return immediately on the first validblock; buffer the first validapproveand return it only after the full result list has been scanned. Docstring updated to state the precedence rule instead of "the first valid directive wins".tests/hermes_cli/test_plugins.py— four regression tests inTestPreToolCallDirective:test_later_block_outranks_earlier_approve— the issue's exact repro (fails before this change, passes after)test_earlier_block_still_wins— the reverse order is unchangedtest_message_less_block_does_not_suppress_approve— an invalid block does not swallow a valid approvetest_first_approve_wins_among_approves— precedence only reorders block vs approve; among approves the first still wins,rule_keyincludedwebsite/docs/user-guide/features/hooks.md— documented the precedence rule in thepre_tool_calldirective section, which previously stated only "The first valid directive wins."How to Test
Before the change, the reporter's repro returns
('approve', 'earlier plugin approves'); after it,('block', 'later security plugin blocks'):Regression tests:
Checklist
Code
fix(scope):,feat(scope):, etc.)get_pre_tool_call_directive,pre_tool_callprecedence); the only prior work is feat(plugins): pre_tool_call approve action escalates to human gate (closes #51221) #58698 (merged, added theapproveaction) and feat(plugins): add pre_tool_call approve action and plugin mode #11816 (open, unrelated to precedence)pytest tests/ -qand all tests pass — ran the relevant suites (tests/hermes_cli/,tests/tools/test_approval.py) rather than the full tree. One pre-existing, unrelated collection error is present on Windows:tests/hermes_cli/test_doctor_journal_modes.pyfails at import withAttributeError: module 'os' has no attribute 'geteuid'. That file is untouched by this PR and fails identically on a cleanupstream/main.Documentation & Housekeeping
docs/, docstrings) — docstring +website/docs/user-guide/features/hooks.mdcheck-windows-footguns.py --allis cleanCredit to @yangc222 for the report, the precise root-cause location, and the repro — the semantics implemented here are the ones proposed in the issue.