vitest fails to load typescript worker code in node (but does so correctly in browser mode)

[andreashuber69]

Describe the bug

As already asked here, I'm looking for a way to execute tests in node while using node:worker_threads. Equivalent code works just fine in browser mode.

Reproduction

git clone --branch original https://github.com/andreashuber69/vitest-repro.git
cd vitest-repro
npm install

(EDIT: modified to check out original code, to check out the workaround discussed below please use git clone https://github.com/andreashuber69/vitest-repro.git)

The following commands work as expected:

• npm run test-browser
• npm run test-node-js

The following doesn't work:

• npm run test-node-ts

Apparently under node, test code isn't compiled to js first (as in browser mode), which is why the above produces the following error:

Error: Cannot find module '.../src/nodeWorker.js'

Attempts at importing tsx (see commented out code in ./src/nodeWorker.spec.ts) have failed with the following error:

TypeError: Unknown file extension ".ts" for .../src/nodeWorker.ts

(presented as a solution in this issue but apparently doesn't work any longer)

How can I use vitest with typescript code that uses node:worker_threads?

System Info

System:
    OS: Linux 6.8 Pop!_OS 22.04 LTS
    CPU: (8) x64 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
    Memory: 24.06 GB / 31.20 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 20.12.1 - /usr/local/bin/node
    npm: 10.5.0 - /usr/local/bin/npm
  Browsers:
    Brave Browser: 125.1.66.110
    Chrome: 125.0.6422.60
  npmPackages:
    @vitest/browser: ^1.5.0 => 1.5.0 
    @vitest/coverage-istanbul: ^1.5.0 => 1.5.0 
    vitest: ^1.5.0 => 1.5.0

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[hi-ogawa]

Browser mode is quite different and it just happens that Vite's web worker support makes it work, but I don't think the same technique naturally transfers to node worker threads. It's unlikely to have some magical feature on Vitest out-of-the-box as noted in #3234 (comment)

Regarding tsx loader failing on Worker constructor, this seems like a well known issue/limitation of current Node loader:

• New --import API breaks worker threads privatenumber/tsx#354
• ESM loaders cannot be defined via Worker option execArgv in v20 nodejs/node#47747

The tsx issue suggests one workaround privatenumber/tsx#354 (comment) and that would probably work.

[sheremet-va]

The equivalent cannot work just fine in the browser mode because the browser mode doesn't support node:worker_threads module.

If you are testing web worker, the recommendation is to use @vitest/web-worker package.

[andreashuber69]

The equivalent cannot work just fine in the browser mode because the browser mode doesn't support node:worker_threads module.

With equivalent code I meant code implemented in terms of web worker, see test-browser script.

[andreashuber69]

For anyone landing here looking for a solution: I didn't get it to work. I tried with the following lines

        const tsx = new URL("../node_modules/tsx/dist/cli.mjs", import.meta.url);
        const worker = new Worker(tsx, { env: SHARE_ENV, argv: ["./src/nodeWorker.ts"] });

Both cli.mjs and nodeWorker.ts are found, but apparently the message posted on the main thread never makes it to the worker.

[hi-ogawa]

Oh right, tsx probably spawns extra process in between, so Node's out-of-the-box worker communication doesn't work.

Just reading back the original Node issue nodejs/node#47747 (comment), it might be that the fix is near in Node 22, but I just double checked it still doesn't work (and it actually breaks more for some use cases).

For the workaround on Node 20, I think you can use tsx/esm/api like this to wrap your worker. I just tested this approach here https://github.com/hi-ogawa/reproductions/blob/main/vitest-5757-ts-worker/src/ts-worker/index.mjs

import { Worker } from "node:worker_threads";

class TsWorker extends Worker {
  constructor(filename, options = {}) {
    options.workerData ??= {};
    options.workerData.__ts_worker_filename = filename.toString();
    super(new URL("./worker.mjs", import.meta.url), options);
  }
}

const worker = new TsWorker(new URL("./your-actual-worker.ts", import.meta.url));

// worker.mjs
import { tsImport } from "tsx/esm/api";
import { workerData } from "node:worker_threads";

tsImport(workerData.__ts_worker_filename, import.meta.url);

[andreashuber69]

For the workaround on Node 20, I think you can use tsx/esm/api like this to wrap your worker.

Indeed, that works. I've modified vitest-repro accordingly and adapted the instructions for the reproduction to check out the original code.

Thank you very much!

[daniel-nagy]

If you don't want the stain of a non-typescript file on your filesystem (or just want to avoid the extra file)

import { type WorkerOptions, Worker } from "node:worker_threads";

const worker = /* JavaScript */ `
  import { createRequire } from "node:module";
  import { workerData } from "node:worker_threads";

  const filename = "${import.meta.url}";
  const require = createRequire(filename);
  const { tsImport } = require("tsx/esm/api");
  
  tsImport(workerData.__ts_worker_filename, filename);
`;

export class TsWorker extends Worker {
  constructor(filename: string | URL, options: WorkerOptions = {}) {
    options.workerData ??= {};
    options.workerData.__ts_worker_filename = filename.toString();
    super(new URL(`data:text/javascript,${worker}`), options);
  }
}