Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 78 additions & 32 deletions integration-tests/sdk-typescript/abort-and-lifecycle.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -317,21 +317,42 @@ describe('AbortController and Process Lifecycle (E2E)', () => {
const testFilePath = await helper.getPath('test.txt');
await helper.createFile('test.txt', 'original content');

let canUseToolCalled = false;
let canUseToolCalledResolve: () => void = () => {};
const canUseToolCalledPromise = new Promise<void>((resolve) => {
canUseToolCalledResolve = resolve;
});
// Bounded promise with explicit timer arming and clearing on settle.
// `startTimer()` lets each phase begin counting only when its phase
// actually starts, so slow predecessors don't burn its budget and
// produce misleading timeout errors.
const boundedPromise = (label: string, ms: number) => {
let resolveFn: () => void = () => {};
let timer: ReturnType<typeof setTimeout> | undefined;
let pendingReject: (err: Error) => void = () => {};
const promise = new Promise<void>((resolve, reject) => {
resolveFn = () => {
if (timer !== undefined) clearTimeout(timer);
resolve();
};
pendingReject = reject;
});
const startTimer = () => {
if (timer !== undefined) return;
timer = setTimeout(() => {
pendingReject(new Error(`${label} timeout after ${ms}ms`));
}, ms);
};
return { promise, resolve: () => resolveFn(), startTimer };
};

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] canUseToolCalled and inputStreamDone have 15 s budgets that start counting at creation, but they are logically awaited only after firstResult (30 s budget). On a slow CI where the first LLM round-trip exceeds 15 s, these promises will have already rejected before the sequential chain reaches them, producing a misleading "canUseTool callback not called timeout" error when the real cause was first-result latency.

Consider matching the 15 s budgets to 30 s, or deferring promise creation until just before the relevant phase starts.

— glm-5.1 via Qwen Code /review

let firstResultResolve: () => void = () => {};
const firstResultPromise = new Promise<void>((resolve) => {
firstResultResolve = resolve;
});
const canUseToolCalled = boundedPromise(
'canUseTool callback not called',
15000,
);
const inputStreamDone = boundedPromise('inputStreamDone', 15000);
const firstResult = boundedPromise('firstResult', 30000);
const secondResult = boundedPromise('secondResult', 30000);

let secondResultResolve: () => void = () => {};
const secondResultPromise = new Promise<void>((resolve, reject) => {
secondResultResolve = resolve;
});
// firstResult begins as soon as the query starts.
firstResult.startTimer();

let secondResultMessage: unknown;

async function* createPrompt(): AsyncIterable<SDKUserMessage> {
const sessionId = crypto.randomUUID();
Expand All @@ -346,17 +367,24 @@ describe('AbortController and Process Lifecycle (E2E)', () => {
parent_tool_use_id: null,
};

await firstResultPromise;
await firstResult.promise;

// The second-turn phases only start now; arm their timers here so
// a slow first turn does not burn their budgets.
canUseToolCalled.startTimer();
inputStreamDone.startTimer();
secondResult.startTimer();

yield {
type: 'user',
session_id: sessionId,
message: {
role: 'user',

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The prompt now uses relative test.txt instead of the absolute helper path. If the model/tool resolves that name against a different working directory or target, the helper-managed file can remain unchanged and this assertion can pass without proving the intended file's pending write was rejected after stdin closed. Please restore const testFilePath = await helper.getPath('test.txt') and prompt with that absolute path.

— gpt-5.5 via Qwen Code /review

content: `Use the write_file tool to write "updated" to the file at ${testFilePath}. Then reply with "done".`,
content: `Write "updated" to ${testFilePath}. Stop if any exception occurs.`,
},
parent_tool_use_id: null,
};
await inputStreamDone.promise;
}

const q = query({
Expand All @@ -367,12 +395,20 @@ describe('AbortController and Process Lifecycle (E2E)', () => {
permissionMode: 'default',
coreTools: ['read_file', 'write_file'],
canUseTool: async (toolName, input) => {

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] This callback resolves the synchronization promises for any tool call, but coreTools includes both read_file and write_file. If the model calls read_file, or calls write_file with an unexpected path/content, the file can remain original content and the test still passes without exercising the intended delayed write_file permission-response path. Please assert toolName === 'write_file' and validate the expected target/content before resolving these promises.

— gpt-5.5 via Qwen Code /review

canUseToolCalled = true;
canUseToolCalledResolve();
return {
behavior: 'allow',
updatedInput: input,
};
// Only the write_file call against the target file constitutes
// the permission-control path under test. Other tool calls
// (e.g. read_file the model issues to look around first) are
// allowed silently and must not advance the timing harness.
const isTargetCall =
toolName === 'write_file' &&
(input as { file_path?: string }).file_path === testFilePath;
if (!isTargetCall) {
return { behavior: 'allow', updatedInput: input };
}
inputStreamDone.resolve();
await new Promise((resolve) => setTimeout(resolve, 1000));
canUseToolCalled.resolve();
return { behavior: 'allow', updatedInput: input };
},
debug: false,
},
Expand All @@ -385,28 +421,38 @@ describe('AbortController and Process Lifecycle (E2E)', () => {
if (isSDKResultMessage(message)) {
resultCount += 1;
if (resultCount === 1) {
firstResultResolve();
firstResult.resolve();
}
if (resultCount === 2) {
secondResultResolve();
secondResultMessage = message;
secondResult.resolve();
break;
}
}
}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] Promise.race([expectedSequence, loopPromise]) lets the test proceed successfully if the SDK iterator ends before canUseToolCalledPromise or secondResultPromise resolves. In that case the final file-content assertion can pass without exercising the intended stdin-close/control-response path, so the lifecycle regression this test is meant to catch can be masked.

Please do not treat normal loopPromise completion as a success condition here. Await the required milestones directly, or make loop() reject if it exits before the expected second result/control callback is observed.

— gpt-5.5 via Qwen Code /review

};

loop();

await firstResultPromise;
await canUseToolCalledPromise;
await secondResultPromise;
const loopPromise = loop();
// Surface loop errors as a rejection-only race partner; loop
// completion alone must NOT short-circuit the awaited milestones,
// otherwise an iterator that ends before canUseTool is invoked
// could mask the regression this test is meant to catch.
const loopError = new Promise<never>((_, reject) => {
loopPromise.catch(reject);
});

// Signal stdin is done so CLI stops waiting
q.endInput();
await Promise.race([
(async () => {
await firstResult.promise;
await canUseToolCalled.promise;
await secondResult.promise;
})(),
loopError,
]);

expect(secondResultMessage).toBeDefined();
const content = await helper.readFile('test.txt');
expect(canUseToolCalled).toBe(true);
expect(content).toBe('updated');
expect(content).toBe('original content');
} finally {
await q.close();
}
Expand Down
Loading