Rerunning vitest with browser mode no longer works since vitest 1.0.0

[nicojs]

Describe the bug

StrykerJS uses the programmatic API. For example:

process.env.NODE_ENV = "test";
import { createVitest } from "vitest/node";
const vitest = await createVitest("test", { watch: false });
await vitest.start();
vitest.state.filesMap.clear();
await vitest.start();
await vitest.close();

This works without browser mode and did work in browser mode until [email protected], but no longer in [email protected]

Reproduction

Using this script:

import { createVitest } from "vitest/node";
process.env.NODE_ENV = "test";

const vitest = await createVitest("test", { watch: false });
await vitest.start();
vitest.state.filesMap.clear();
await vitest.start();
await vitest.close();

With this config:

import { defineConfig } from 'vitest/config';
export default defineConfig({
  test: {
    globals: true,
    browser: {
      enabled: true,
      name: 'chromium',
      provider: 'playwright',
      headless: true,
    },
  },
});

I expect (v1.0.0-beta.6):

 RUN  v1.0.0-beta.6 /home/nicojs/github/vitest-browser
      Browser runner started at http://localhost:5173

 ✓ src/math.component.spec.ts (1)
   ✓ my-math (1)
     ✓ should add A + B

 Test Files  1 passed (1)
      Tests  1 passed (1)
   Start at  10:51:10
   Duration  970ms (transform 0ms, setup 0ms, collect 18ms, tests 11ms, environment 0ms, prepare 0ms)


 RUN  v1.0.0-beta.6 /home/nicojs/github/vitest-browser
      Browser runner started at http://localhost:5173

 ✓ src/math.component.spec.ts (1)
   ✓ my-math (1)
     ✓ should add A + B

But I get (v1.0.0 and v1.0.2):

 RUN  v1.0.2 /home/nicojs/github/vitest-browser
      Browser runner started at http://localhost:5173

 ✓ src/math.component.spec.ts (1)
   ✓ my-math (1)
     ✓ should add A + B

 Test Files  1 passed (1)
      Tests  1 passed (1)
   Start at  10:55:29
   Duration  901ms (transform 0ms, setup 0ms, collect 11ms, tests 8ms, environment 0ms, prepare 0ms)


 RUN  v1.0.2 /home/nicojs/github/vitest-browser
      Browser runner started at http://localhost:5173


⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: page.goto: Target page, context or browser has been closed
Call log:
  - navigating to "http://localhost:5173/?path=src%2Fmath.component.spec.ts&id=src%2Fmath.component.spec.ts", waiting until "load"

 ❯ PlaywrightBrowserProvider.openPage node_modules/@vitest/browser/dist/providers.js:41:23
 ❯ runTests node_modules/vitest/dist/vendor/node.hPXN4l9e.js:3577:9
 ❯ Object.runWorkspaceTests node_modules/vitest/dist/vendor/node.hPXN4l9e.js:3596:7

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

 Test Files  no tests
      Tests  no tests
     Errors  1 error
   Start at  10:55:29
   Duration  12ms (transform 0ms, setup 0ms, collect 0ms, tests 0ms, environment 0ms, prepare 0ms)

See this zip for reproduction: vitest-browser.zip

System Info

System:
    OS: Linux 5.15 Ubuntu 20.04.6 LTS (Focal Fossa)
    CPU: (16) x64 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz
    Memory: 3.56 GB / 15.49 GB
    Container: Yes
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 18.18.0 - ~/.nvm/versions/node/v18.18.0/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v18.18.0/bin/yarn
    npm: 9.8.1 - ~/.nvm/versions/node/v18.18.0/bin/npm
    pnpm: 8.9.2 - ~/.nvm/versions/node/v18.18.0/bin/pnpm
  Browsers:
    Chrome: 118.0.5993.117
  npmPackages:
    @vitest/browser: ^1.0.2 => 1.0.2 
    vitest: ^1.0.2 => 1.0.2

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[sheremet-va]

Interesting! I am not able to reproduce the error with your reproduction (M1 Air). Maybe some kind of race condition? 🤔

[sheremet-va]

Oh, wait, I didn't update Vitest there 🤦🏻

[sheremet-va]

Looks like this is the PR that caused it: #4623

But the actual error is that inside the provider we close the page, but we don't remove the reference to it. But when we do that, we get another bug 🤷🏻

[AriPerkkio]

Oops, didn't see this coming up while implementing #4623.

@nicojs it might be good idea to add stryker-mutator/stryker-js into Vitest Ecosystem CI so that we would see this kind of issues before releasing. There are not yet that many JS API utilizers out there yet.