[v20.x backport] backport multiple test runner related prs #49225

rluvaton · 2023-08-18T09:59:42Z

Backports:

Original Landed at	PR	Original landed commit
Jul 26	- test: relocate test files fixtures for better understanding #48787	`bcaf198`
Jul 24	- test_runner: fix global before not called when no global test exists #48877	`a0f3ed8`
Jul 31	- test_runner: cleanup test timeout abort listener #48915	`7a33f2b`
Aug 1	- test_runner: fix timeout in *Each hook failing further tests #48925	`88094bb`
Aug 2	- test_runner: fix global after not failing the tests #48913	`c58e8fc`
Aug 3	- test: remove `--no-warnings` flag in `test_runner` fixtures #48989	1353681
Aug 11	- test_runner: expose location of tests #48975	`4e61c22`
Aug 14	- test_runner: reland run global after() hook earlier #49116	b574075

Notes:

Created 1 PR for all those commits as suggested by @RafaelGSS
after each commit I run some tests that are relevant and made sure they are passing:
a. test-runner-cli.js
b. test-runner-output.mjs
c. test-runner-run.mjs
The order of the rows is the same order as I cherry-picked
I added bcaf198 before a0f3ed8 even though in real life a0f3ed8 was added before bcaf198 because of failing tests that were added in a0f3ed8 that match the test pattern that was used in test-runner-cli.js test (this is the whole reason of creating test: relocate test files fixtures for better understanding #48787 in the first place

nodejs-github-bot · 2023-08-18T09:59:46Z

Review requested:

@nodejs/test_runner

nodejs-github-bot · 2023-08-18T10:08:59Z

CI: https://ci.nodejs.org/job/node-test-pull-request/53422/

nodejs-github-bot · 2023-08-18T13:12:53Z

CI: https://ci.nodejs.org/job/node-test-pull-request/53423/

cjihrig

RSLGTM

PR-URL: nodejs#48787 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: nodejs#48877 Reviewed-By: Chemi Atlow <[email protected]> Reviewed-By: Moshe Atlow <[email protected]>

fix nodejs#48475 PR-URL: nodejs#48915 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: nodejs#48925 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: nodejs#48913 Fixes: nodejs#48867 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

no longer needed after nodejs#48915 fix PR-URL: nodejs#48989 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

This commit adds each test's line and column number to the reporter output. This will aid in debugging test suite failures when error stacks are not helpful, test suites are large, or tests have the same name. This data is also exposed on the spec reporter. This commit also replaces the filename that was previously being reported, with the filename where the test actually exists. These are normally correct, but could be wrong if tests were run from a file other than the user's entrypoint. PR-URL: nodejs#48975 Fixes: nodejs#48457 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Chemi Atlow <[email protected]> Reviewed-By: Moshe Atlow <[email protected]>

This commit reverts the revert in bb52656. It also includes the fix for the issue that required the revert (nodejs#49059 (comment)) and an additional common.mustCall() in the added test. Refs: nodejs#49059 Refs: nodejs#49110 PR-URL: nodejs#49116 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Chemi Atlow <[email protected]> Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>

rluvaton · 2023-09-04T19:52:09Z

Rebased with v20.x-staging after the 20.6.0 release

nodejs-github-bot · 2023-09-04T19:59:32Z

CI: https://ci.nodejs.org/job/node-test-pull-request/53727/

RafaelGSS

LGTM with green ci. Thanks for the hard work!

rluvaton · 2023-09-05T06:51:01Z

Thank you! Appreciate it!

PR-URL: #48787 Backport-PR-URL: #49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: #48877 Backport-PR-URL: #49225 Reviewed-By: Chemi Atlow <[email protected]> Reviewed-By: Moshe Atlow <[email protected]>

fix #48475 PR-URL: #48915 Backport-PR-URL: #49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: #48925 Backport-PR-URL: #49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: #48913 Backport-PR-URL: #49225 Fixes: #48867 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

no longer needed after #48915 fix PR-URL: #48989 Backport-PR-URL: #49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

This commit adds each test's line and column number to the reporter output. This will aid in debugging test suite failures when error stacks are not helpful, test suites are large, or tests have the same name. This data is also exposed on the spec reporter. This commit also replaces the filename that was previously being reported, with the filename where the test actually exists. These are normally correct, but could be wrong if tests were run from a file other than the user's entrypoint. PR-URL: #48975 Backport-PR-URL: #49225 Fixes: #48457 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Chemi Atlow <[email protected]> Reviewed-By: Moshe Atlow <[email protected]>

This commit reverts the revert in bb52656. It also includes the fix for the issue that required the revert (#49059 (comment)) and an additional common.mustCall() in the added test. Refs: #49059 Refs: #49110 PR-URL: #49116 Backport-PR-URL: #49225 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Chemi Atlow <[email protected]> Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>

UlisesGascon · 2023-09-10T18:04:32Z

Landed in 3dbc61c...546ad5f

PR-URL: #48787 Backport-PR-URL: #49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: #48913 Backport-PR-URL: #49225 Fixes: #48867 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

no longer needed after #48915 fix PR-URL: #48989 Backport-PR-URL: #49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: nodejs/node#48787 Backport-PR-URL: nodejs/node#49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: nodejs/node#48913 Backport-PR-URL: nodejs/node#49225 Fixes: nodejs/node#48867 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

no longer needed after #48915 fix PR-URL: nodejs/node#48989 Backport-PR-URL: nodejs/node#49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: nodejs/node#48787 Backport-PR-URL: nodejs/node#49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

PR-URL: nodejs/node#48913 Backport-PR-URL: nodejs/node#49225 Fixes: nodejs/node#48867 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

no longer needed after #48915 fix PR-URL: nodejs/node#48989 Backport-PR-URL: nodejs/node#49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. v20.x v20.x Issues that can be reproduced on v20.x or PRs targeting the v20.x-staging branch. labels Aug 18, 2023

rluvaton marked this pull request as ready for review August 18, 2023 10:05

rluvaton added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 18, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 18, 2023

rluvaton mentioned this pull request Aug 18, 2023

test_runner: reland run global after() hook earlier #49116

Merged

cjihrig approved these changes Aug 18, 2023

View reviewed changes

This was referenced Aug 21, 2023

CI Reliability 2023-08-21 nodejs/reliability#641

Open

CI Reliability 2023-08-22 nodejs/reliability#642

Open

CI Reliability 2023-08-23 nodejs/reliability#643

Open

rluvaton and others added 8 commits September 4, 2023 22:51

test: reorder test files fixtures for better understanding

2e0df36

PR-URL: nodejs#48787 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

test_runner: fix global before not called when no global test exists

b84b2bf

PR-URL: nodejs#48877 Reviewed-By: Chemi Atlow <[email protected]> Reviewed-By: Moshe Atlow <[email protected]>

test_runner: cleanup test timeout abort listener

b91c906

fix nodejs#48475 PR-URL: nodejs#48915 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

test_runner: fix timeout in *Each hook failing further tests

684137b

PR-URL: nodejs#48925 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

test_runner: fix global after not failing the tests

19771a8

PR-URL: nodejs#48913 Fixes: nodejs#48867 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

test: remove --no-warnings flag in test_runner fixtures

dbc3124

no longer needed after nodejs#48915 fix PR-URL: nodejs#48989 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

rluvaton force-pushed the backport-prs-to-v20.x branch from 5f893ac to 2ad1bbc Compare September 4, 2023 19:51

rluvaton added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 4, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 4, 2023

RafaelGSS approved these changes Sep 4, 2023

View reviewed changes

mcollina mentioned this pull request Sep 7, 2023

v20.6.1 proposal #49528

Merged

UlisesGascon approved these changes Sep 10, 2023

View reviewed changes

UlisesGascon pushed a commit that referenced this pull request Sep 10, 2023

test: reorder test files fixtures for better understanding

25e967a

PR-URL: #48787 Backport-PR-URL: #49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

UlisesGascon pushed a commit that referenced this pull request Sep 10, 2023

test_runner: fix global before not called when no global test exists

75333f3

PR-URL: #48877 Backport-PR-URL: #49225 Reviewed-By: Chemi Atlow <[email protected]> Reviewed-By: Moshe Atlow <[email protected]>

UlisesGascon pushed a commit that referenced this pull request Sep 10, 2023

test_runner: fix timeout in *Each hook failing further tests

08738b2

PR-URL: #48925 Backport-PR-URL: #49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

UlisesGascon closed this Sep 10, 2023

rluvaton deleted the backport-prs-to-v20.x branch September 10, 2023 18:05

targos pushed a commit that referenced this pull request Nov 27, 2023

test: reorder test files fixtures for better understanding

0fea550

PR-URL: #48787 Backport-PR-URL: #49225 Reviewed-By: Moshe Atlow <[email protected]> Reviewed-By: Chemi Atlow <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[v20.x backport] backport multiple test runner related prs #49225

[v20.x backport] backport multiple test runner related prs #49225

rluvaton commented Aug 18, 2023 •

edited

Loading

nodejs-github-bot commented Aug 18, 2023

nodejs-github-bot commented Aug 18, 2023

nodejs-github-bot commented Aug 18, 2023

cjihrig left a comment

rluvaton commented Sep 4, 2023 •

edited

Loading

nodejs-github-bot commented Sep 4, 2023

RafaelGSS left a comment

rluvaton commented Sep 5, 2023

UlisesGascon commented Sep 10, 2023

[v20.x backport] backport multiple test runner related prs #49225

[v20.x backport] backport multiple test runner related prs #49225

Conversation

rluvaton commented Aug 18, 2023 • edited Loading

nodejs-github-bot commented Aug 18, 2023

nodejs-github-bot commented Aug 18, 2023

nodejs-github-bot commented Aug 18, 2023

cjihrig left a comment

Choose a reason for hiding this comment

rluvaton commented Sep 4, 2023 • edited Loading

nodejs-github-bot commented Sep 4, 2023

RafaelGSS left a comment

Choose a reason for hiding this comment

rluvaton commented Sep 5, 2023

UlisesGascon commented Sep 10, 2023

rluvaton commented Aug 18, 2023 •

edited

Loading

rluvaton commented Sep 4, 2023 •

edited

Loading