Plugin hook policy: hung or raising guards fail closed without wedging, block outranks approve, shell approve reaches the gate, async pre_gateway_dispatch awaited (#105223 #109624 #87420 #92553 #105827 #110241)

[teknium1]

Guard/policy plugins can no longer be silently defeated on the hook path: a raising or hung pre_tool_call guard fails closed without wedging the agent until restart, a plugin's veto outranks another plugin's approve, shell hooks can escalate to the approval gate, execution middleware fires on cold surfaces, and async pre_gateway_dispatch callbacks run on the gateway loop.

• pre_tool_call: a hung guard callback permanently fails closed — all subsequent tool calls blocked until process restart ('still running' key has no expiry) #105223 plugins_dispatch.py::_run_hook_callback_bounded: after the 60s suppression window a fresh call id may respawn a worker for a hung callback, capped at _HOOK_MAX_ABANDONED_WORKERS = 3 live abandoned workers per callback; at the cap it keeps failing closed with a WARNING naming callback + module (the Concurrent observer-hook invocations are dropped as if a callback had timed out #98382 "no unbounded workers" constraint holds). Supersedes fix(plugins): recover tool calls after a hung policy guard #105241 (@fangliquanflq, co-authored), fix(plugins): per-session hook single-flight + matcher gate before the flight window #109441, fix(plugins): serialize overlapping hook callback fires instead of dropping #104763.
• [Bug]: pre_tool_call hook callbacks fail OPEN on exception but fail CLOSED on timeout — inconsistent policy-hook contract #109624 a pre_tool_call callback that raises now appends a block directive naming the callback and error — the same veto a timeout already produced, so exception and timeout no longer disagree. Salvages fix(plugins): fail closed on policy hook exceptions #109632 (@KoNit-K).
• [Bug]: pre_tool_call directive aggregation is first-valid-wins — a plugin block is shadowed by an earlier plugin's approve #87420 plugins.py::_get_pre_tool_call_directive_details: block > approve > none regardless of registration order; first valid approve wins among approves; modify accumulation unchanged. Supersedes fix(plugins): give block directives precedence #68644 (@synscott, earliest, co-authored), fix(plugins): rank pre_tool_call block above approve #87449 (@jackulau, co-authored).
• pre_tool_call shell hooks silently discard the documented "approve" action, and hooks doctor reports them healthy #92553 shell_hooks.py::_parse_pre_tool_call translates {"action":"approve","message"?,"rule_key"?} to the plugin-side directive; the decision: approve dialect (auto-allow) deliberately unmapped. Supersedes fix(hooks): stop shell hook approval requests from bypassing confirmation #92562 (@fangliquanflq, co-authored), feat(hooks): support approval escalation from shell hooks #110325 (@Mauridios).
• fix(plugins): tool_execution/llm_execution middleware silently fail-open on query/cron surfaces — _run_execution_chain missing lazy discovery #105827 middleware.py::_run_execution_chain uses _delivery_manager() so tool/llm execution middleware fires in query/cron processes where discovery has not run. Salvages fix(plugins): route execution middleware through lazy discovery (#105827) #105832 (@liuhao1024).
• [Bug]: pre_gateway_dispatch async callbacks run through the thread bridge and block the gateway event loop #110241 new ainvoke_hook (plugins_dispatch → plugins → lifecycle); _hm_pre_gateway_dispatch_hook is async and awaited on the gateway loop; pre_gateway_dispatch stays unbounded; the sync invoke_hook is untouched for every other caller. Salvages fix(gateway): await async pre_gateway_dispatch callbacks on the event loop #110265 (@twidtwid); supersedes feat(plugins): await async pre-gateway dispatch hooks #102485 (@dmspark, earliest, co-authored), fix(gateway): await async pre-dispatch hooks #110253 (@KoNit-K, co-authored).
• Docs (hooks.md): precedence wording, timeout/raise/recovery paragraph, shell approve stdout shape, async pre_gateway_dispatch note.

Root cause in one sentence: the dispatcher treated an abandoned worker as "running forever", treated a raising guard as "no opinion", picked the first approve/block in registration order, and only resolved async callbacks through a helper thread.

Validation

Issue	Live repro before (real PluginManager / shell hook script / GatewayRunner)	After	Test (red on base → green)
#105223	hung guard: 8 calls → 1 worker, every call blocked forever	8 calls → 3 workers (cap), still blocked with a named WARNING; 2nd call after the window allowed	test_hung_worker_does_not_fail_closed_forever, test_hung_worker_caps_new_call_identities_after_suppression
#109624	results=[] — tool ran (fail-open)	[{'action':'block','message':'pre_tool_call plugin callback boom raised RuntimeError: …'}]	test_pre_tool_call_callback_exception_fails_closed[2]
#87420	('approve', 'earlier plugin approves')	('block', 'later security plugin blocks')	test_later_block_outranks_earlier_approve, test_first_approve_wins_among_approves_and_keeps_later_modify
#92553	shell approve → None, tool ran unprompted	{'action':'approve','message':…} → human gate	test_approve_is_parsed_like_the_plugin_directive[4], test_approve_reaches_the_human_gate_through_plugin_manager
#105827	middleware skipped, terminal ran	gate fired, terminal not run	test_execution_chain_lazily_discovers
#110241	sync bridge, 2.00s thread timeout, skip lost	in-loop await, 0.05s, skip honoured	test_async_hook_callback_is_awaited_on_the_gateway_loop, test_narrow_legacy_signature_still_gets_only_its_fields

Existing-test changes (justified in commit bodies): test_hung_worker_blocks_new_call_identity_after_suppression re-pinned at the new bound (#105223); #109632's two tests merged into one parametrized test; three sync hook stubs in tests/gateway/test_bot_loop_guard.py made async (#110241). No other existing assertion changed.

Lane run scripts/run_tests.sh tests/hermes_cli tests/agent/test_shell_hooks.py tests/gateway: 22775 passed, 13 failed — all 13 in untouched files, 9 pass in isolation (40-worker load), 4 test_dashboard_auth_gate SystemExit: 75 fail identically on a pristine base worktree (port held on the host). Post-rebase on ee8a919fd27: touched test files 163 passed, 0 failed.

Prompt-cache invariant: nothing here mutates past context or the system prompt mid-conversation.

Fixes #105223. Fixes #109624. Fixes #87420. Fixes #92553. Fixes #105827. Fixes #110241.
Salvages #105832, #109632, #110265. Supersedes #105241, #109441, #104763, #68644, #87449, #92562, #110325, #102485, #110253.

Infographic

[github-actions]

૮ >ﻌ< ა ci review

_{ran on 6505462 — fix(plugins): ainvoke_hook shares the sync path's failure co}

debug info

CI timings

CI timings · View report · View job

Wall time 5m47s vs 6m24s (-9.6%). 6 job(s) slower, 4 faster, 3 unchanged.

• Docs Site / docs-site-checks: +62.0s
• Python tests / Run tests: -57.0s
• OS-specific tests / Windows-only tests: -44.0s
• Python tests / e2e: -14.0s
• Check no committed infographics / check-no-committed-infographics: -11.0s