Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: mark empty udp tests flaky on OS X #31936

Closed
wants to merge 1 commit into from
Closed

test: mark empty udp tests flaky on OS X #31936

wants to merge 1 commit into from

Conversation

sam-github
Copy link
Contributor

@sam-github sam-github commented Feb 24, 2020 •
edited
Loading

They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

Note: CI doesn't test on Catalina, yet (though I assume nodejs developers do). Test failures will indirectly block fixing of #29216, which requires running notarization on Catalina, which requires tests passing on Catalina.

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@sam-github
test: mark empty udp tests flaky on OS X
e8667bb 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: nodejs#30030
Refs: nodejs/build#2189 (comment)
@nodejs-github-bot nodejs-github-bot added the test Issues and PRs related to the tests. label Feb 24, 2020
@sam-github sam-github mentioned this pull request Feb 24, 2020
Merged
@sam-github sam-github added the flaky-test Issues and PRs related to the tests with unstable failures on the CI. label Feb 24, 2020
@sam-github
Copy link
Contributor Author

I don't have access to a machine I can repro this on, but I asked @AshCripps to try it (nodejs/build#2189 (comment)). @lpinca , perhaps you can review, too?

@lpinca
Copy link
Member

lpinca commented Feb 24, 2020 •
edited
Loading

I've run make -j8 test with the patch applied but the process still exits with 2 not sure if expected. All the tested listed here timeout as expected.

AshCripps
AshCripps approved these changes Feb 25, 2020
View reviewed changes
Copy link
Member

@AshCripps AshCripps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me. @lpinca you have to run the tests manually using the python script to skip flaky-tests like the ci does:

bash-3.2$ tools/test.py --flaky-tests=skip
[07:30|% 100|+ 2790|-   0]: Done
bash-3.2$

@lpinca
Copy link
Member

lpinca commented Feb 25, 2020

Oh, I forgot to use the --flaky-tests flag.

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/29350/

@sam-github
Copy link
Contributor Author

@AshCripps would it be helpful to get this fast-tracked? I kicked of a full CI on it, but it still needs 25 hours, but we can fast-track given its triviality and impact on CI stability if its blocking bringing the OS X 10.15/Catalina machines into CI.

@AshCripps
Copy link
Member

@sam-github Fast track would be useful, pretty sure every release line will fail these tests when 10.15 is added to CI

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/29351/

@sam-github
Copy link
Contributor Author

@nodejs/build @nodejs/testing Can we fast-track marking these tests flaky?

@sam-github sam-github added the fast-track PRs that do not need to wait for 48 hours to land. label Feb 25, 2020
@sam-github
Copy link
Contributor Author

Btw, the build status reported is incorrect - PR passed on everything but arm, as would be expected. I kicked a resume on the PR, though.

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/29367/

@sam-github
Copy link
Contributor Author

Landed in 24a4f76

@sam-github sam-github closed this Feb 26, 2020
sam-github added a commit that referenced this pull request Feb 26, 2020
@sam-github
test: mark empty udp tests flaky on OS X
24a4f76 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

PR-URL: #31936
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Beth Griggs <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
BethGriggs pushed a commit that referenced this pull request Feb 27, 2020
@sam-github @BethGriggs
test: mark empty udp tests flaky on OS X
9bd1317 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

PR-URL: #31936
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Beth Griggs <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
codebytere pushed a commit that referenced this pull request Feb 27, 2020
@sam-github @codebytere
test: mark empty udp tests flaky on OS X
f89fb27 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

PR-URL: #31936
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Beth Griggs <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
@codebytere codebytere mentioned this pull request Feb 29, 2020
Merged
@sam-github sam-github deleted the catalina-udp-flakes branch March 6, 2020 19:04
@richardlau richardlau mentioned this pull request Mar 7, 2020
Closed
4 tasks
puzpuzpuz pushed a commit to puzpuzpuz/node that referenced this pull request Mar 8, 2020
@sam-github @puzpuzpuz
test: mark empty udp tests flaky on OS X
0b5fda0 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: nodejs#30030
Refs: nodejs/build#2189 (comment)

PR-URL: nodejs#31936
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Beth Griggs <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>
@puzpuzpuz puzpuzpuz mentioned this pull request Mar 8, 2020
Closed
3 tasks
@BethGriggs BethGriggs mentioned this pull request Mar 12, 2020
Merged
codebytere pushed a commit that referenced this pull request Mar 14, 2020
@sam-github @codebytere
test: mark empty udp tests flaky on OS X
5a537ba 
They fail on OS X 10.15 (aka "Catalina"), but pass on earlier OS X.

Refs: #30030
Refs: nodejs/build#2189 (comment)

PR-URL: #31936
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Beth Griggs <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Rod Vagg <[email protected]>

PR-URL: #32146
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Vladimir de Turckheim <[email protected]>
Reviewed-By: Stephen Belanger <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Shelley Vohr <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jasnell jasnell jasnell approved these changes

@rvagg rvagg rvagg approved these changes

@lpinca lpinca lpinca approved these changes

@BethGriggs BethGriggs BethGriggs approved these changes

@AshCripps AshCripps AshCripps approved these changes

Assignees
No one assigned
Labels
fast-track PRs that do not need to wait for 48 hours to land. flaky-test Issues and PRs related to the tests with unstable failures on the CI. test Issues and PRs related to the tests.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@sam-github @lpinca @nodejs-github-bot @AshCripps @jasnell @rvagg @BethGriggs @targos