Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: always default test runner to use spec reporter #54540

Closed
cjihrig opened this issue Aug 24, 2024 · 3 comments
Closed

Proposal: always default test runner to use spec reporter #54540

cjihrig opened this issue Aug 24, 2024 · 3 comments
Labels
feature request Issues that request new features to be added to Node.js. test_runner Issues and PRs related to the test runner subsystem.

Comments

@cjihrig
Copy link
Contributor

cjihrig commented Aug 24, 2024

What is the problem this feature will solve?

The test runner currently uses the more human friendly spec reporter when a TTY output is detected. Otherwise, it defaults to generating TAP output. I think the cons of this behavior outweigh the pros.

In Node core, this would remove occurrences of nested TAP output which can cause issues like #54535. We also have a number of places throughout the codebase, such as the GitHub Actions config, where we force the spec reporter anyway.

In userland, it is a DX improvement (no more TAP output in CI runs). TAP output would still be available via the --test-reporter=tap

What is the feature you are proposing to solve the problem?

Make the default test runner reporter be spec.

Notes:

  • This would be a semver major change.
  • We should try to avoid churn in snapshot output by forcing the output to whatever is currently being used. This will help avoid conflicts in the future and make backporting other changes simpler.

What alternatives have you considered?

No response

@cjihrig cjihrig added the feature request Issues that request new features to be added to Node.js. label Aug 24, 2024
@github-project-automation github-project-automation bot moved this to Awaiting Triage in Node.js feature requests Aug 24, 2024
@targos
Copy link
Member

targos commented Aug 24, 2024

I'm +1, but I've never had a TAP parser in my pipeline so I don't really know if the impact is big.

@RedYetiDev RedYetiDev added the test_runner Issues and PRs related to the test runner subsystem. label Aug 24, 2024
@RedYetiDev RedYetiDev moved this from Awaiting Triage to Triaged in Node.js feature requests Aug 24, 2024
@cjihrig
Copy link
Contributor Author

cjihrig commented Aug 24, 2024

The impact would be for people who want TAP output to pass --test-reporter=tap when starting Node. I think that is pretty reasonable for a semver major change.

EDIT: It's worth noting that while this changes a default, passing --test-reporter=tap would be backward compatible to all versions of Node since test reporters were introduced.

cjihrig added a commit to cjihrig/node that referenced this issue Aug 24, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: nodejs#54540
cjihrig added a commit to cjihrig/node that referenced this issue Aug 24, 2024
This is a breaking change. Prior to this commit, the test_runner
defaulted to the spec reporter if using a TTY, and the TAP
reporter otherwise. This commit makes spec the default reporter
unconditionally. TAP output is still available via the
--test-reporter=tap CLI flag.

Fixes: nodejs#54540
@RedYetiDev RedYetiDev moved this from Triaged to In Progress in Node.js feature requests Aug 24, 2024
@RedYetiDev
Copy link
Member

Fixed by #54548

cjihrig added a commit to cjihrig/node that referenced this issue Aug 25, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: nodejs#54540
cjihrig added a commit to cjihrig/node that referenced this issue Aug 26, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: nodejs#54540
nodejs-github-bot pushed a commit that referenced this issue Aug 27, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: #54540
PR-URL: #54547
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Jake Yuesong Li <[email protected]>
cjihrig added a commit to cjihrig/node that referenced this issue Aug 27, 2024
This is a breaking change. Prior to this commit, the test_runner
defaulted to the spec reporter if using a TTY, and the TAP
reporter otherwise. This commit makes spec the default reporter
unconditionally. TAP output is still available via the
--test-reporter=tap CLI flag.

Fixes: nodejs#54540
RafaelGSS pushed a commit that referenced this issue Aug 30, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: #54540
PR-URL: #54547
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Jake Yuesong Li <[email protected]>
RafaelGSS pushed a commit that referenced this issue Aug 30, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: #54540
PR-URL: #54547
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Jake Yuesong Li <[email protected]>
louwers pushed a commit to louwers/node that referenced this issue Nov 2, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: nodejs#54540
PR-URL: nodejs#54547
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Jake Yuesong Li <[email protected]>
louwers pushed a commit to louwers/node that referenced this issue Nov 2, 2024
This is a breaking change. Prior to this commit, the test_runner
defaulted to the spec reporter if using a TTY, and the TAP
reporter otherwise. This commit makes spec the default reporter
unconditionally. TAP output is still available via the
--test-reporter=tap CLI flag.

Fixes: nodejs#54540
PR-URL: nodejs#54548
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Jake Yuesong Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Issues that request new features to be added to Node.js. test_runner Issues and PRs related to the test runner subsystem.
Projects
None yet
Development

No branches or pull requests

3 participants