-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test runner: recursive or globbed loading of non-JS test files #44023
Comments
@nodejs/test_runner |
Honestly, I would just add |
probably for starters, as @benjamingr support |
I don’t think supporting ts (or jsx) by default makes any sense if TS/JSX content isn’t auto loaded. |
Perhaps we should implement a solution for #43895 (that will allow passing a custom function to filter files) I am willing to implement some |
Maybe a better solution would be to have an option for the loader to have an option to tell which file patten it supports for tests? /cc @nodejs/loaders |
why will it be better? I think a configuration file will cover more use-cases (real usecases from here), for example:
loaders should process test files regardless of this discussion, there is a seperate issue about that |
Running the test runner on a |
I completely agree that that’s the only way to handle it. Nothing should be implicitly included unless node understands it;cams adding a loader should also implicitly include anything the loader handles. Users should only have to configure file extensions if they’re deviating from their loaders’ defaults. |
I might not fully understand how loaders work, so correct me in case I am wrong - a loader does not expose what file extensions it supports, |
Personally, I see a huge advantage in letting users specify the file pattern precisely: I often find myself putting test fixtures, shared code etc. in the |
I see mts and cts file extensions were not mentioned above. This is evidence in favor of allowing users and customization hooks to specify. |
I think probably just allowing an RE or glob of test files then as an argument for |
core doesn't currently support glob and regex can be hard to distinguish from paths |
Currently |
it currently accepts paths |
That's incorrect, Line 153 in d29e78a
Lines 529 to 531 in d29e78a
We could potentially change it to also accept a string. (i.e. |
@aduh95 see node/test/parallel/test-runner-cli.js Line 11 in 2fe4e94
node/test/parallel/test-runner-cli.js Line 23 in 2fe4e94
node/test/parallel/test-runner-cli.js Line 55 in 2fe4e94
https://github.com/nodejs/node/blob/2fe4e9473fb50630326754886dee1d301a1a661e/test/parallel/test-runner-exit-code.js#L47 etc |
I think a glob would an ideal solution:
If node does not understand the file it was explicitly pointed to, user-error. I can't imagine a user suddenly complaining that node didn't support typescript before and still doesn't when fed a glob on typescript. I think this will be even less of an issue when we add the more detailed error message to |
assuming that adding glob to core has a bigger scope than a small fix to the test runner, |
Perhaps import { glob } from 'node:fs/promises';
const [
file1,
file2,
...files
] = await glob('./src/**/*.spec.mjs'); As for the team, I imagine it depends on where it lives. Yes, probably a good idea to reach out for lessons learnt 🙂 |
I see that this was already raised here #40731 (comment) and #40954 (comment) with no real objections |
Something else to consider: will node support a single flavor of globbing, or will it support multiple behaviors via options, similar to bash's various |
I’m growing concerned by the number of dependencies that are becoming part of core as part of shipping our own test runner. Like are we going to vendor https://github.com/isaacs/node-glob or https://github.com/isaacs/minimatch now too, or create our own slimmed-down versions or alternatives? This feels a lot like the path we went down with |
What dependencies has core already taken on because of the test runner?
It looks like that may already be in the works independent of the test runner. If they do make it into core, there is no reason for the test runner not to use them. |
We should also consider if an equivalent to this pseudo-code makes sense: Equivalent to the behavior of the |
@cspotcode yes that's a suggestion in #44023 (comment) :) Also making sure you don't miss the meeting doodle at #44023 (comment) |
That's done!
There have been discussions to include it in require('node:test').run({
concurrency: true,
timeout: 10_000, // whatever
files: require('node:fs').globSync("**/*.test.ts"),
}); IMO there should be a way to a file which name would be Other than that, I think that's a very neat idea, I like both options. |
I am +1 on However, |
I assume |
that is pretty much what I suggested, but instead of using |
One more ping to @nodejs/test_runner @cjihrig @aduh95 for the meeting doodle to make sure you saw it https://doodle.com/meeting/participate/id/dwjGRJmb |
I feel like we should solve the overall question of how to configure Node from a config file, before solving specific use cases like |
@benjamingr is there a verdict for the meeting time? |
Hey, due to a combination of bereavement leave and food poisoning / tummy flu I've totally dropped the ball on this and won't be able to engage in the next 3-4 days (at least until the Shiv'ah is after us). I'll send another doodle next week (hopefully) or if someone else wants to pick it up - go tor it. |
PR-URL: #44241 Fixes: #44023 Fixes: #43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs#44241 Fixes: nodejs#44023 Fixes: nodejs#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs#44241 Backport-PR-URL: nodejs#TBD Fixes: nodejs#44023 Fixes: nodejs#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs#44241 Backport-PR-URL: nodejs#44873 Fixes: nodejs#44023 Fixes: nodejs#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs#44241 Backport-PR-URL: nodejs#44873 Fixes: nodejs#44023 Fixes: nodejs#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs#44241 Backport-PR-URL: nodejs#44873 Fixes: nodejs#44023 Fixes: nodejs#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs#44241 Backport-PR-URL: nodejs#44873 Fixes: nodejs#44023 Fixes: nodejs#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs#44241 Backport-PR-URL: nodejs#44873 Fixes: nodejs#44023 Fixes: nodejs#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs#44241 Backport-PR-URL: nodejs#44873 Fixes: nodejs#44023 Fixes: nodejs#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: #44241 Backport-PR-URL: #44873 Fixes: #44023 Fixes: #43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs/node#44241 Backport-PR-URL: nodejs/node#44873 Fixes: nodejs/node#44023 Fixes: nodejs/node#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs/node#44241 Backport-PR-URL: nodejs/node#44873 Fixes: nodejs/node#44023 Fixes: nodejs/node#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
PR-URL: nodejs/node#44241 Fixes: nodejs/node#44023 Fixes: nodejs/node#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]> (cherry picked from commit 59527de13d39327eb3dfa8dedc92241eb40066d5)
PR-URL: nodejs/node#44241 Fixes: nodejs/node#44023 Fixes: nodejs/node#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]> (cherry picked from commit 59527de13d39327eb3dfa8dedc92241eb40066d5)
PR-URL: nodejs/node#44241 Fixes: nodejs/node#44023 Fixes: nodejs/node#43675 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]> (cherry picked from commit 59527de13d39327eb3dfa8dedc92241eb40066d5)
What is the problem this feature will solve?
As per https://nodejs.org/dist/latest-v18.x/docs/api/test.html#test-runner-execution-model, the test runner can be started as
node --test dirname/
to recursively run all test files (ending in.js
,.cjs
or.mjs
) contained in the directorydirname
. This unfortunately means that non-JS test files, specifically test files written in TypeScript in my case, will not be found since they end in.ts
instead of.js
. The docs say to provide each test file separately, e.g.node --loader=ts-node/esm --test dirname/foo.test.ts dirname/bar.test.ts
but this gets messy very quickly.What is the feature you are proposing to solve the problem?
I see a few options (in order of my preference, starting with most preferred):
node --loader ts-node/esm --test "dirname/**/*.ts"
). Note that this does work on Linux, to some extent, since the shell will expand asterisks. This doesn't work on Windows, though, and doesn't work when the string is quoted. Tools like mocha will perform globbing independently of the operating system or shell.node --loader ts-node/esm --test --test-extensions=.ts,.cts,.mts dirname/
).node --loader ts-node/esm --test dirname/
and ts-node could hook into the directory traversal).What alternatives have you considered?
node --test
pointed to an index file that performs the globbing and dynamically imports all actual test files (very ugly; loses process separation between test files)I also opened an issue at ts-node (TypeStrong/ts-node#1853) specifically for TypeScript, to see if they are interested in better test runner support.
The text was updated successfully, but these errors were encountered: