Repository navigation
Add AUTO mode denial observability and caps #4476
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
f58e32e
dc25b0b
9338f8b
f74f0bb
b1d2647
e6f02fb
07107ba
b8b86ea
b235d58
e666485
415dc8b
5eedfaa
90df3c0
d51bf52
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -479,6 +479,126 @@ async function waitForStatus( | |
| } | ||
|
|
||
| describe('CoreToolScheduler', () => { | ||
| type SchedulerDenialTrackingInternals = { | ||
| toolCalls: ToolCall[]; | ||
| autoModeFallbackCallIds: Set<string>; | ||
| _handleConfirmationResponseInner: ( | ||
| callId: string, | ||
| toolCall: ToolCall, | ||
| originalOnConfirm: ( | ||
| outcome: ToolConfirmationOutcome, | ||
| payload?: ToolConfirmationPayload, | ||
| ) => Promise<void>, | ||
| outcome: ToolConfirmationOutcome, | ||
| signal: AbortSignal, | ||
| payload?: ToolConfirmationPayload, | ||
| ) => Promise<void>; | ||
| }; | ||
|
|
||
| function createSchedulerForDenialTrackingApprovalTest() { | ||
| const denialState = { | ||
| consecutiveBlock: 0, | ||
| consecutiveUnavailable: 0, | ||
| totalBlock: 20, | ||
| totalUnavailable: 0, | ||
| }; | ||
| const setAutoModeDenialState = vi.fn(); | ||
| const scheduler = new CoreToolScheduler({ | ||
| config: { | ||
| getSessionId: () => 'test-session-id', | ||
| getApprovalMode: () => ApprovalMode.AUTO, | ||
| getAutoModeDenialState: () => denialState, | ||
| setAutoModeDenialState, | ||
| getToolRegistry: () => | ||
| ({ | ||
| getTool: () => undefined, | ||
| }) as unknown as ToolRegistry, | ||
| getUsageStatisticsEnabled: () => false, | ||
| getDebugMode: () => false, | ||
| getChatRecordingService: () => undefined, | ||
| } as unknown as Config, | ||
| onAllToolCallsComplete: vi.fn(), | ||
| onToolCallsUpdate: vi.fn(), | ||
| getPreferredEditor: () => 'vscode', | ||
| onEditorClose: vi.fn(), | ||
| }); | ||
| const confirmationDetails: ToolCallConfirmationDetails = { | ||
| type: 'exec', | ||
| title: 'Run command', | ||
| command: 'python', | ||
| rootCommand: 'python', | ||
| onConfirm: vi.fn().mockResolvedValue(undefined), | ||
| }; | ||
| const toolCall = { | ||
| status: 'awaiting_approval', | ||
| request: { | ||
| callId: 'call-1', | ||
| name: ToolNames.SHELL, | ||
| args: {}, | ||
| isClientInitiated: false, | ||
| prompt_id: 'prompt-1', | ||
| }, | ||
| tool: {}, | ||
| confirmationDetails, | ||
| } as unknown as ToolCall; | ||
| const internals = | ||
| scheduler as unknown as SchedulerDenialTrackingInternals; | ||
| internals.toolCalls = [toolCall]; | ||
| return { internals, toolCall, setAutoModeDenialState }; | ||
| } | ||
|
|
||
| it('does not reset total denial counters for unrelated AUTO approvals', async () => { | ||
| const { internals, toolCall, setAutoModeDenialState } = | ||
| createSchedulerForDenialTrackingApprovalTest(); | ||
|
|
||
| await internals._handleConfirmationResponseInner( | ||
| 'call-1', | ||
| toolCall, | ||
| vi.fn().mockResolvedValue(undefined), | ||
| ToolConfirmationOutcome.ProceedOnce, | ||
| new AbortController().signal, | ||
| ); | ||
|
|
||
| expect(setAutoModeDenialState).not.toHaveBeenCalled(); | ||
| }); | ||
|
|
||
| it('resets denial counters after approving a denialTracking fallback prompt', async () => { | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [Suggestion] The two new tests cover (1) no fallback callId + approve → no reset, and (2) fallback callId + approve → reset. But there's no negative test for (3) fallback callId + Cancel → no reset. The production code correctly guards this with it('does not reset denial counters when user cancels a denialTracking fallback prompt', async () => {
const { internals, toolCall, setAutoModeDenialState } =
createSchedulerForDenialTrackingApprovalTest();
internals.autoModeFallbackCallIds.add('call-1');
await internals._handleConfirmationResponseInner(
'call-1',
toolCall,
vi.fn().mockResolvedValue(undefined),
ToolConfirmationOutcome.Cancel,
new AbortController().signal,
);
expect(setAutoModeDenialState).not.toHaveBeenCalled();
});— qwen3.7-max via Qwen Code /review |
||
| const { internals, toolCall, setAutoModeDenialState } = | ||
| createSchedulerForDenialTrackingApprovalTest(); | ||
| internals.autoModeFallbackCallIds.add('call-1'); | ||
|
|
||
| await internals._handleConfirmationResponseInner( | ||
| 'call-1', | ||
| toolCall, | ||
| vi.fn().mockResolvedValue(undefined), | ||
| ToolConfirmationOutcome.ProceedOnce, | ||
| new AbortController().signal, | ||
| ); | ||
|
|
||
| expect(setAutoModeDenialState).toHaveBeenCalledWith({ | ||
| consecutiveBlock: 0, | ||
| consecutiveUnavailable: 0, | ||
| totalBlock: 0, | ||
| totalUnavailable: 0, | ||
| }); | ||
| }); | ||
|
|
||
| it('does not reset denial counters after cancelling a denialTracking fallback prompt', async () => { | ||
| const { internals, toolCall, setAutoModeDenialState } = | ||
| createSchedulerForDenialTrackingApprovalTest(); | ||
| internals.autoModeFallbackCallIds.add('call-1'); | ||
|
|
||
| await internals._handleConfirmationResponseInner( | ||
| 'call-1', | ||
| toolCall, | ||
| vi.fn().mockResolvedValue(undefined), | ||
| ToolConfirmationOutcome.Cancel, | ||
| new AbortController().signal, | ||
| ); | ||
|
|
||
| expect(setAutoModeDenialState).not.toHaveBeenCalled(); | ||
| }); | ||
|
|
||
| function createSchedulerForLegacyToolTests(options: { | ||
| toolsByName: Map<string, MockTool>; | ||
| approvalMode?: ApprovalMode; | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Suggestion] Unlike the equivalent
case 'fallback'branch incoreToolScheduler.ts(lines ~1755-1767), this path setswasAutoModeDenialFallbackbut emits no debug log with the fallback reason or denial counter state. When the ACP/daemon path triggers adenial_capfallback, operators cannot see why it was triggered or what the counter values were.Consider mirroring the coreToolScheduler logging for parity:
— qwen3.7-max via Qwen Code /review