Support test.fixme, test.fixmeIf, and test.todoIf

[Electroid]

What does this PR do?

This adds the same API as playwright's test.fixme() to the test runner. This allows developers to mark a test as flaky or broken, but does not fail the overall test suite.

We will use this internally to get our Github CI to green so we can turn on the merge queue. Others may find it useful.

• Documentation or TypeScript types (it's okay to leave the rest blank in this case)
• Code changes

How did you verify your code works?

I wrote automated tests

[github-actions]

❌ @Electroid 7 files with test failures on bun-darwin-aarch64:

• test/integration/next/default-pages-dir/test/dev-server.test.ts 3 failing
• test/js/bun/http/bun-server.test.ts 1 failing
• test/js/third_party/jsonwebtoken/claim-exp.test.js 1 failing
• test/js/third_party/jsonwebtoken/claim-iat.test.js 1 failing
• test/js/third_party/jsonwebtoken/claim-nbf.test.js 1 failing
• test/js/third_party/jsonwebtoken/option-maxAge.test.js 1 failing
• test/js/third_party/jsonwebtoken/verify.test.js 1 failing

View test output

^{#e9c0d3dbd57bdf66010d68e01917b84bb279ab68}

[github-actions]

❌ @Electroid 6 files with test failures on linux-x64:

• test/js/bun/shell/leak.test.ts 2 failing
• test/js/third_party/jsonwebtoken/claim-exp.test.js 1 failing
• test/js/third_party/jsonwebtoken/claim-iat.test.js 1 failing
• test/js/third_party/jsonwebtoken/claim-nbf.test.js 1 failing
• test/js/third_party/jsonwebtoken/option-maxAge.test.js 1 failing
• test/js/third_party/jsonwebtoken/verify.test.js 1 failing

View test output

^{#e9c0d3dbd57bdf66010d68e01917b84bb279ab68}

[github-actions]

❌ @Electroid 7 files with test failures on linux-x64-baseline:

• test/js/bun/shell/leak.test.ts 2 failing
• test/js/node/process/process.test.js 1 failing
• test/js/third_party/jsonwebtoken/claim-exp.test.js 1 failing
• test/js/third_party/jsonwebtoken/claim-iat.test.js 1 failing
• test/js/third_party/jsonwebtoken/claim-nbf.test.js 1 failing
• test/js/third_party/jsonwebtoken/option-maxAge.test.js 1 failing
• test/js/third_party/jsonwebtoken/verify.test.js 1 failing

View test output

^{#e9c0d3dbd57bdf66010d68e01917b84bb279ab68}

[github-actions]

❌🪟 @Electroid, there are 36 test regressions on Windows x86_64

• test\bundler\bundler_edgecase.test.ts
• test\cli\hot\hot.test.ts
• test\cli\run\require-cache.test.ts
• test\cli\run\transpiler-cache.test.ts
• test\js\bun\dns\resolve-dns.test.ts
• test\js\bun\http\fetch-file-upload.test.ts
• test\js\bun\shell\bunshell-file.test.ts
• test\js\bun\shell\shelloutput.test.ts
• test\js\bun\shell\throw.test.ts
• test\js\deno\fetch\response.test.ts
• test\js\bun\http\bun-server.test.ts
• test\js\node\env-windows.test.ts
• test\js\node\dns\node-dns.test.js
• test\js\node\process\process-args.test.js
• test\js\node\process\process.test.js
• test\js\node\util\node-inspect-tests\parallel\util-inspect.test.js
• test\js\node\worker_threads\worker_threads.test.ts
• test\js\third_party\body-parser\express-body-parser-test.test.ts
• test\js\third_party\socket.io\socket.io-close.test.ts
• test\js\third_party\socket.io\socket.io-connection-state-recovery.test.ts
• test\js\third_party\socket.io\socket.io-handshake.test.ts
• test\js\third_party\socket.io\socket.io-messaging-many.test.ts
• test\js\third_party\socket.io\socket.io-middleware.test.ts
• test\js\third_party\socket.io\socket.io-namespaces.test.ts
• test\js\third_party\socket.io\socket.io-server-attachment.test.ts
• test\js\third_party\socket.io\socket.io-socket-middleware.test.ts
• test\js\third_party\socket.io\socket.io-socket-timeout.test.ts
• test\js\third_party\socket.io\socket.io-utility-methods.test.ts
• test\js\third_party\socket.io\socket.io.test.ts
• test\js\web\fetch\body.test.ts
• test\js\web\fetch\body-stream.test.ts
• test\js\web\fetch\fetch.test.ts
• test\js\web\timers\setTimeout.test.js
• test\js\web\websocket\websocket.test.js
• test\regression\issue\08095.test.ts
• test\js\web\workers\worker.test.ts

Full Test Output

[github-actions]

❌ @Electroid 5 files with test failures on bun-darwin-x64:

• test/js/third_party/jsonwebtoken/claim-exp.test.js 1 failing
• test/js/third_party/jsonwebtoken/claim-iat.test.js 1 failing
• test/js/third_party/jsonwebtoken/claim-nbf.test.js 1 failing
• test/js/third_party/jsonwebtoken/option-maxAge.test.js 1 failing
• test/js/third_party/jsonwebtoken/verify.test.js 1 failing

View test output

^{#e9c0d3dbd57bdf66010d68e01917b84bb279ab68}

[Jarred-Sumner]

Per IRL discussion, I don't think we should do this. Maybe todoIf. But we should fix our own test suite instead of making it easier to write flaky tests.