Skip to content
This repository has been archived by the owner on Apr 22, 2023. It is now read-only.

Support for regarding flaky tests failures as non-fatal #8689

Merged
merged 2 commits into from
Dec 4, 2014
Merged

Support for regarding flaky tests failures as non-fatal #8689

merged 2 commits into from
Dec 4, 2014

Conversation

orangemocha
Copy link
Contributor

In this PR:

  1. Support for flaky tests in test runner
  2. Mark current v0.12 flaky tests as flaky

This is needed in preparation of some upcoming CI changes that will allow Jenkins to validate PRs more effectively.

@tjfontaine
Copy link

the crypto test is non-deterministic, and can be fixed with a merge of v0.10 into v0.12

The dns test needs to be improved to allow for both error codes

But I do support in principle the premise of marking tests as potentially flakey

@orangemocha
Copy link
Contributor Author

Thanks for the clarification. Of course, the idea is to update the status file once the tests get fixed.

Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.

Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.
@orangemocha orangemocha merged commit 0674cba into nodejs:v0.12 Dec 4, 2014
@orangemocha
Copy link
Contributor Author

Thank you, landed in 0674cba

rvagg added a commit to rvagg/io.js that referenced this pull request Feb 12, 2015
rvagg added a commit to nodejs/node that referenced this pull request Feb 16, 2015
Reverts nodejs/node-v0.x-archive#8689

PR-URL: #812
Reviewed-By: Jeremiah Senkpiel <[email protected]>
Reviewed-By: Johan Bergström <[email protected]>
Reviewed-By: Ben Noordhuis <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants