Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Allow using tsx instead of ts-node for loading TS config files #13143

Open
MasterOdin opened this issue Aug 16, 2022 · 24 comments
Open

Comments

@MasterOdin
Copy link
Contributor

🚀 Feature Proposal

I would like to be able to use the tsx compiler for handling the jest.config.ts file, as opposed to relying on ts-node.

Motivation

tsx is a quickly rising alternative to ts-node as it runs much faster as it runs on the back of esbuild, and doesn't do typechecking. While parsing the jest configuration file should already be generally quick, it could be made quicker to using tsx. Additionally, for teams that have moved to tsx, it would allow having just one runtime compiler in their dependency for everything, vs having tsx for most things, and ts-node just for reading jest.config.ts files.

Example

It would be used implicitly within jest-config library where when it detects a file that ends with .ts, it checks if ts-node is available and uses that, else it'll check for tsx and use that. If neither are available output an error message that neither could be found.

All the user would need to do would be to have tsx installed via their package manager and available via node_modules.

Pitch

Looking at https://github.com/facebook/jest/blob/main/packages/jest-config/src/readConfigFileAndSetRootDir.ts, there didn't seem to be an obvious way to modify the behavior of how it loads config files, where even if it was possible to get tsx registered before triggering jest-config, it would still attempt to use ts-node for .ts extension.

@github-actions
Copy link

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

@github-actions github-actions bot added the Stale label Sep 15, 2022
@MasterOdin
Copy link
Contributor Author

I'm still interested in this. jest is the only thing pulling in ts-node in our stack, and it would be nice to be able to eliminate it for tsx.

@github-actions github-actions bot removed the Stale label Sep 16, 2022
@SimenB
Copy link
Member

SimenB commented Sep 16, 2022

Making this pluggable would be nice - we had a PR at some point for an SWC-based loader as well.

Thoughts on adding some sort of loader docblock at the top of the file, similar to e.g. /** @jest-environment jsdom */?

@MasterOdin
Copy link
Contributor Author

MasterOdin commented Sep 16, 2022

Sure, I think that adding a brief loader docblock at the top of the file would make sense, in the concept that within your dependency tree you end up with multiple TS loaders for whatever reason, and you want to make sure that you end up using a specific one, vs the default ordering of jest. Adding support for that should be pretty straight-forward, as compared to actually implementing these other loaders.

we had a PR at some point for an SWC-based loader as well.

Do you know what it was named? I did find a PR for esbuild-loader (#12041) that has stalled, but not swc.

@jsardev
Copy link

jsardev commented Sep 29, 2022

I think we could use something like a require option where you could put whatever transpiler out there. This is how ava does it. This way you can use whatever you want: @swc-node/register, ts-node/register, esbuild-register.

Edit: My bad, this is working on running the tests, not loading the config file

@LinusU
Copy link
Contributor

LinusU commented Oct 26, 2022

I was trying to do this today and thought it would be as easy as adding --loader @esbuild-kit/esm-loader to NODE_OPTIONS:

NODE_OPTIONS='--experimental-vm-modules --loader @esbuild-kit/esm-loader' jest

However, when I do that I get an interesting error:

TypeError: Unexpected response from worker: undefined
    at ChildProcessWorker._onMessage (/Users/linus/my-project/node_modules/jest-worker/build/workers/ChildProcessWorker.js:289:15)
    at ChildProcess.emit (node:events:527:28)
    at emit (node:internal/child_process:938:14)
    at processTicksAndRejections (node:internal/process/task_queues:84:21)

Adding some logging to node_modules/jest-worker/build/workers/ChildProcessWorker.js reveals that this is the message being sent from the child process:

{
  "type": "dependency",
  "path": "file:///Users/linus/my-project/node_modules/jest-worker/build/workers/processChild.js"
}

It seems like ChildProcessWorker is expecting an array where the first item is the message type, not an object though...

I'm not really sure how to continue debugging this, but would love some input!

Generally, I think it would be amazing if Jest could work with the --loader argument to Node.js!

@LinusU
Copy link
Contributor

LinusU commented Oct 26, 2022

Okay, turns out the mysterious object above wasn't emitted from Jest at all, it was the loader.

Filed an issue about that here: esbuild-kit/esm-loader#43

Knowing that I simply added this code to ignore the messages:

  }
  _onMessage(response) {
+   // Ignore messages emitted by @esbuild-kit/esm-loader
+   // ref: https://github.com/esbuild-kit/esm-loader/issues/43
+   if (typeof response === 'object' && typeof response.type === 'string' && response.type === 'dependency') {
+     return
+   }
    // TODO: Add appropriate type check
    let error;
    switch (response[0]) {

This led me to the next problem.

SyntaxError: Unexpected token, expected "{"

Well, this stack trace was in Babel, and I didn't want to use that. So I added "transform": {} to my Jest config, and tried again:

SyntaxError: Unexpected token ':'

This time the stack trace only shows Runtime.loadEsmModule.

Going in to this function it seems like it always loads the file from disk and passes it thru a function that calls the transformers. I tried replacing it with just an import of the file instead, something like:

        return core;
      }
-     const transformedCode = await this.transformFileAsync(modulePath, {
-       isInternalModule: false,
-       supportsDynamicImport: true,
-       supportsExportNamespaceFrom: true,
-       supportsStaticESM: true,
-       supportsTopLevelAwait: true
-     });
+     const transformedCode = `import '${modulePath}'`;
      try {
        const module = new (_vm().SourceTextModule)(transformedCode, {
          context,

This results in the following error though:

● Test suite failed to run

Your test suite must contain at least one test.

Hmm, alright, I'm not sure why it gives that specific error, but SourceTextModule won't be able to route import calls thru the specified loader as far as I can tell. (actually, I'm not sure about this anymore, I guessed that since it looked like imports were routed via the importModuleDynamically callback, but I see now that that is just when calling import(...))

Lets just try calling my loader here directly and see if it works:

        return core;
      }
-     const transformedCode = await this.transformFileAsync(modulePath, {
-       isInternalModule: false,
-       supportsDynamicImport: true,
-       supportsExportNamespaceFrom: true,
-       supportsStaticESM: true,
-       supportsTopLevelAwait: true
-     });
+     const esmLoader = await import('@esbuild-kit/esm-loader')
+     const transformedCode = (await esmLoader.load(`file://${modulePath}`, { format: 'esm' }, (url) => {
+       return { source: fs().readFileSync(new URL(url), 'utf8') }
+     })).source
      try {
        const module = new (_vm().SourceTextModule)(transformedCode, {
          context,

Test Suites: 15 passed, 15 total
Tests: 53 passed, 53 total

Success!

Okay, if there is a way to get ahold of the loader some way I think using it to load files would be viable.

@SimenB would there be any interest in maybe adding a flag for Jest that makes it use the loader that Node.js uses to load test files? I think that this would be awesome since we can run the tests in the same way that we run the normal code!


edit: actually, adding a log of transformedCode shows that it has import statements in it, that in my case refers to TypeScript files. So I'm actually only transforming the first file myself withe the call to esmLoader.load, any imported files seems to be transformed by Node.js using the loader specified with --loader.

edit2: hmm, no every file is passed thru here I think, via the link function of the module...

@SimenB
Copy link
Member

SimenB commented Oct 26, 2022

I'd be interested in a PR that shows the changes needed. 👍

@LinusU
Copy link
Contributor

LinusU commented Oct 26, 2022

Here it is! 🎉

#13521

@MasterOdin
Copy link
Contributor Author

#13521 wouldn't address the issue of how jest loads the config file though, just how it transforms and runs the test files right?

@LinusU
Copy link
Contributor

LinusU commented Oct 27, 2022

@MasterOdin it seems like you are right, I will make a small update to address that 👍

In fact, I didn't notice that this issue was talking about config file specifically at all 😅

@github-actions
Copy link

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

@github-actions github-actions bot added the Stale label Nov 26, 2022
@LinusU
Copy link
Contributor

LinusU commented Nov 30, 2022

(not stale)

@github-actions github-actions bot removed the Stale label Nov 30, 2022
@github-actions
Copy link

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

@github-actions github-actions bot added the Stale label Dec 30, 2022
@LinusU
Copy link
Contributor

LinusU commented Jan 4, 2023

(not stale)

@github-actions github-actions bot removed the Stale label Jan 4, 2023
@SimenB
Copy link
Member

SimenB commented Jan 4, 2023

Still happy to take a PR adding support for some sort of docblock to the config file if anyone's up for it in the new year 😀

@MasterOdin
Copy link
Contributor Author

@SimenB Trying to find time to do this now, but to be clear on your comments on this thread as well as in #12041 and #11989 is that if the user does not specify a docblock, then jest will only try to use ts-node, throwing the error (plus perhaps linking to the docs about the new docblock) if that's not installed. The only way to use an alternative loader is by specifying the docblock, e.g. /* @jest-config-loader tsx */, or whatever. Of course, if that loader is not installed, then it'll throw the same error as missing ts-node, just mentioning the specified package.

@SimenB
Copy link
Member

SimenB commented Jan 7, 2023

Yeah, that sounds about right. 👍 We can probably remove built-in ts-node in next major and have that also be opt-in via a docblock.

@SimenB
Copy link
Member

SimenB commented Jan 7, 2023

Also, not sure if we should define som "config loader" interface (just path-to-config.* -> Config.InitialOptions) rather than us instantiating e.g. tsx and having to provide it options. Us just doing const config = import(moduleNameFromDocblock).then(m => m.default(pathToConfig)) seems better than having to pass a bunch of different options depending on what the "config loader" does. Then we could link to modules providing this interface for whatever module you wanna use.

@LinusU
Copy link
Contributor

LinusU commented Feb 1, 2023

@SimenB have you considered the approach in #13521? Instead of having anything specific to Jest, that would then work with any Node.js compatible loader. It would also work with all code, instead of just the config files.

@Kurt-von-Laven
Copy link

ts-node hasn't seen a release in over a year at this point and still doesn't support TypeScript 5.0's multiple inheritance feature for tsconfig.json files. tsx, swc-node, or the native Node.js ESM loader are looking increasingly like the way of the future to me.

we had a PR at some point for an SWC-based loader as well.

Do you know what it was named? I did find a PR for esbuild-loader (#12041) that has stalled, but not swc.

@MasterOdin, I believe it was #13779.

@adanski
Copy link

adanski commented Dec 5, 2023

It's worth to mention ts-node does not work with the newest Node 18.x and 20.x anymore (works only as a loader but not a standalone command) which might make it pretty much useless for anything other than Jest config file.

TypeStrong/ts-node#1997
TypeStrong/ts-node#2094

benelan added a commit to Esri/calcite-design-system that referenced this issue Dec 8, 2023
Offroaders123 added a commit to Offroaders123/jsmediatags that referenced this issue Feb 22, 2024
Heck yeah, man!!! Figured out how to get all tests passing, just by instead running them on the built version of the TS code, rather than the transpiled version of it. It's not as ideal as having it just work, but I can do that later, once things are migrated properly.

Now the built TS source is put to the `./dist` folder as well, which is where the tests/mocks run from now instead. Yoo!!!

jestjs/jest#13143 (Kind of helped a bit, but still ran into other issues, so I went with this route instead)
Offroaders123 added a commit to Offroaders123/jsmediatags that referenced this issue Feb 22, 2024
Heck yeah, man!!! Figured out how to get all tests passing, just by instead running them on the built version of the TS code, rather than the transpiled version of it. It's not as ideal as having it just work, but I can do that later, once things are migrated properly.

Now the built TS source is put to the `./dist` folder as well, which is where the tests/mocks run from now instead. Yoo!!!

jestjs/jest#13143 (Kind of helped a bit, but still ran into other issues, so I went with this route instead)
@peterbe
Copy link

peterbe commented Apr 10, 2024

Any news on this? Or workarounds?

As far as I understand, the problem is that ts-node can't handle what tsx can handle.

Also, is this problem with jest or with ts-jest?

@axmad386
Copy link

Any news on this? Or workarounds?

As far as I understand, the problem is that ts-node can't handle what tsx can handle.

Also, is this problem with jest or with ts-jest?

Sadly, I give up with jest to handle typescript and esm. Migrated to vitest solve all the problems.

Offroaders123 added a commit to Offroaders123/gamecontroller.js that referenced this issue Jun 17, 2024
Stil trying to see if I can move over to the built-in Node test runner instead of Jest, it didn't work the first time I tried moving to it. It doesn't have code coverage support, at least that I'm aware of, so I'm removing it in the Jest one first to help prevent accidental difference concerns.

I started moving everything to TypeScript, but I really don't like how complex Jest is to use with either TS or plain ESM, so I want to use `tsx` with the Node test runner instead, that's a very nice and simple setup, I really like it.

jestjs/jest#13143
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants