Skip to content

test(e2e): skip two model-flaky SDK E2E cases (#8256) #8259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wenshao merged 3 commits into from
Jul 31, 2026
Merged

test(e2e): skip two model-flaky SDK E2E cases (#8256) #8259

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
886d43f
test(e2e): skip two model-flaky SDK E2E cases (#8256)
qwen-code-dev-bot Jul 31, 2026
c35732c
test(e2e): reference tracking issue in skipped-test FIXMEs (#8256)
qwen-code-dev-bot Jul 31, 2026
878da93
test(e2e): correct FIXME failure description for sdk-mcp-server skip …
qwen-code-dev-bot Jul 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion integration-tests/sdk-typescript/sdk-mcp-server.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,14 @@ describe('SDK MCP Server Integration (E2E)', () => {
});

describe('Async Tool Handlers', () => {
it('should handle async tool handlers with delays', async () => {
// FIXME: Flaky on the live model: the model sometimes answers without
// calling the delayed_response tool, so the foundToolUse / tool-result
// assertions fail intermittently on main. Already survived asserting on the
// deterministic tool result, a forced prompt, retry: 2, and the 5-minute
// suite timeout. Skipped like the other model-nondeterministic cases
// (permission-control.test.ts); the durable fix is driving it with the fake
// OpenAI server harness the interactive tests use.
it.skip('should handle async tool handlers with delays', async () => {
// Define a tool with async delay using Zod schema
const delayedTool = tool(
'delayed_response',
Expand Down
9 changes: 8 additions & 1 deletion integration-tests/sdk-typescript/subagents.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,14 @@ describe('Subagents (E2E)', () => {
});

describe('Subagent Execution', () => {
it('should delegate task to subagent when appropriate', async () => {
// FIXME: Flaky on the live model: the main agent delegates, but the
// file-reader subagent sometimes replies without calling read_file, so
// foundSubagentToolCall fails (reproduced locally: "expected false to be
// true" at that assertion). Already survived the forced-delegation prompt,
// the suite-timeout fix, and retry: 2. Skipped like the other
// model-nondeterministic cases (permission-control.test.ts); the durable fix
// is the fake OpenAI server harness the interactive tests use.
it.skip('should delegate task to subagent when appropriate', async () => {
const fileReaderAgent: SubagentConfig = {
name: 'file-reader',
description: 'Reads a requested file and reports its exact contents.',
Expand Down
Loading