test_runner: support programmatically running --test

[MoLow]

Fixes: #44023
FIxes: #43675

The meeting discussed in #44023 did not end up taking place, but there was consunsus on adding a programmatic (and configurable) way to run --test.

this does not yet support configuring inside the test files themselves, that will wait for the discussions regarding global node/test runner configuration (#43973, #43895)

todo:

• add tests

[MoLow]

CC @nodejs/test_runner

[benjamingr]

I'm in favor (though still not in full capacity to engage) - this is also still missing tests, would be useful to test concurrent invocations

[benjamingr]

Looks good, wondering if the API of "provide a stream for output and return a promise for undefined" is what users would want/expect.

Additionally there should be a way to get more than one failure probably?

[MoLow]

Looks good, wondering if the API of "provide a stream for output and return a promise for undefined" is what users would want/expect.

what other alternatives are there? we can return a stream synchronously but then test failures won't throw/reject - they will simply be reported in the stream

Additionally there should be a way to get more than one failure probably?

in such a case - the failure describes the failed subtests

[MoLow]

recapping a conversation I had with @benjamingr:
instead of returning a promise resolving undefined - we can synchronously return a TapStream instance, and make it work similarly to fs.FSWatcher - where it both streams TAP outputs and emits events (i.e subtest, test etc events) - this has a few benefits:

1. you don't have to wait for completion for feedback, you have a stream (that can be parsed) and events
2. you can catch failures of more than a single subtest using events
3. if you do prefer a promise - you can achieve that either using stream.toArray() or using events.once()
4. it makes sense that a test runner will not throw when a test fails, but will report that in another way

@nodejs/test_runner WDYT?

[ljharb]

LGTM!

[juanarbol]

This needs to be backported to v16.x