fix(engine): enforce tool use for stop/pause/cancel commands - #2814
Conversation
The LLM was narrating about calling mission_pause/mission_list instead of actually executing them because neither the tool-intent nudge nor the execution obligation recognized stop/pause/cancel as action commands. - Add stop/pause/cancel/halt/disable to signals_tool_intent ACTION_VERBS so the nudge fires when the LLM says "I'll pause the mission" - Add stop/pause/cancel phrases to signals_execution_intent EXEC_PHRASES so the obligation system forces tool calls for "stop it", "pause the X" - Add bare imperative detection (startswith) for "stop", "stop pinging", "pause", "cancel" — avoids false positives like "I can't stop" - Add 5 regression tests covering true positives and false negatives Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request adds support for 'stop', 'pause', 'cancel', 'halt', and 'disable' intents in the orchestrator, including updates to tool and execution intent logic and new unit tests. Feedback suggests expanding the execution phrase lists for consistency and refactoring the bare command detection to handle punctuation by using word-based splitting.
| "stop it", "stop that", "stop this", "stop the ", | ||
| "pause it", "pause that", "pause this", "pause the ", | ||
| "cancel it", "cancel that", "cancel this", "cancel the ", |
There was a problem hiding this comment.
For consistency with the verbs added to signals_tool_intent, please include the "halt" and "disable" variants in EXEC_PHRASES.
| "stop it", "stop that", "stop this", "stop the ", | |
| "pause it", "pause that", "pause this", "pause the ", | |
| "cancel it", "cancel that", "cancel this", "cancel the ", | |
| "stop it", "stop that", "stop this", "stop the ", | |
| "pause it", "pause that", "pause this", "pause the ", | |
| "cancel it", "cancel that", "cancel this", "cancel the ", | |
| "halt it", "halt that", "halt this", "halt the ", | |
| "disable it", "disable that", "disable this", "disable the ", |
| "cancel it", "cancel that", "cancel this", "cancel the ", | ||
| "please run ", "please execute ", "please fetch ", | ||
| "please send ", "please deploy ", | ||
| "please stop ", "please pause ", "please cancel ", |
| trimmed = lower.strip() | ||
| IMPERATIVE_STARTS = ["stop ", "pause ", "cancel ", "halt "] | ||
| BARE_COMMANDS = ["stop", "pause", "cancel", "halt"] | ||
| if trimmed in BARE_COMMANDS: | ||
| return True | ||
| if any(trimmed.startswith(s) for s in IMPERATIVE_STARTS): | ||
| return True |
There was a problem hiding this comment.
The current bare imperative detection is too strict and misses commands with trailing punctuation (e.g., "stop.") or commands on the first line of a multi-line message. Using a word-based check on the first word is more robust and handles punctuation correctly.
| trimmed = lower.strip() | |
| IMPERATIVE_STARTS = ["stop ", "pause ", "cancel ", "halt "] | |
| BARE_COMMANDS = ["stop", "pause", "cancel", "halt"] | |
| if trimmed in BARE_COMMANDS: | |
| return True | |
| if any(trimmed.startswith(s) for s in IMPERATIVE_STARTS): | |
| return True | |
| # Bare imperative commands at the start of the message. | |
| # "stop", "pause", "cancel", "halt", "disable" are unambiguous commands | |
| # that don't match the "verb + pronoun/article" pattern above. | |
| # Checking the first word avoids false positives like "I can't stop". | |
| words = lower.split() | |
| BARE_COMMANDS = ["stop", "pause", "cancel", "halt", "disable"] | |
| if words and words[0].rstrip(".,!?") in BARE_COMMANDS: | |
| return True |
References
- When detecting commands or keywords in a string, use token-based or word-boundary checks instead of simple substring containment to avoid false positives (e.g., 'sync' matching 'nc', 'ghost' matching 'host').
- Add missing "please halt " to EXEC_PHRASES for consistency with please stop/pause/cancel - Strip trailing punctuation from bare commands so "Stop." and "cancel!" are detected - Add 2 regression tests covering both fixes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ency (#2814) - Add "halt it/that/this/the" and "disable it/that/this/the" to EXEC_PHRASES for consistency with signals_tool_intent - Add "please disable " to polite execution phrases - Add "disable" to BARE_COMMANDS and IMPERATIVE_STARTS - Add regression test for halt/disable execution intent phrases Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Addressed all review feedback from gemini-code-assist:
All 468 engine tests pass, fmt clean. |
…2814) * fix(engine): enforce tool use for stop/pause/cancel commands (nearai#2808) The LLM was narrating about calling mission_pause/mission_list instead of actually executing them because neither the tool-intent nudge nor the execution obligation recognized stop/pause/cancel as action commands. - Add stop/pause/cancel/halt/disable to signals_tool_intent ACTION_VERBS so the nudge fires when the LLM says "I'll pause the mission" - Add stop/pause/cancel phrases to signals_execution_intent EXEC_PHRASES so the obligation system forces tool calls for "stop it", "pause the X" - Add bare imperative detection (startswith) for "stop", "stop pinging", "pause", "cancel" — avoids false positives like "I can't stop" - Add 5 regression tests covering true positives and false negatives Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings (iteration 1) - Add missing "please halt " to EXEC_PHRASES for consistency with please stop/pause/cancel - Strip trailing punctuation from bare commands so "Stop." and "cancel!" are detected - Add 2 regression tests covering both fixes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): address gemini-code-assist review — halt/disable consistency (nearai#2814) - Add "halt it/that/this/the" and "disable it/that/this/the" to EXEC_PHRASES for consistency with signals_tool_intent - Add "please disable " to polite execution phrases - Add "disable" to BARE_COMMANDS and IMPERATIVE_STARTS - Add regression test for halt/disable execution intent phrases Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
mission_pause/mission_listinstead of narrating about themstartswithguard to avoid false positives like "I can't stop"Closes #2808
Test plan
cargo fmt— cleancargo clippy --all --benches --tests --examples --all-features— zero warningscargo test -p ironclaw_engine— 466/466 passsignals_tool_intent_stop_pause_cancel,signals_tool_intent_no_false_positive_stop_discussion,signals_execution_intent_stop_pause_cancel,signals_execution_intent_bare_stop,signals_execution_intent_no_false_positive_stop_in_sentence🤖 Generated with Claude Code