Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests should not fail fast #909

Merged
merged 2 commits into from
Mar 8, 2022

Conversation

ikedam
Copy link
Contributor

@ikedam ikedam commented Jul 24, 2021

test.sh stops running as soon as it finds a test failure in a module (so-called fail fast).

Generally speaking, it's not a good idea:

  • Developers have to fix tests one by one. They fix a test, run tests again, find a next test failure, fix that, run tests again, find a next test failure...
  • Developers cannot see all test results if they don't have complete testing environment. (e.g. consider one without Docker in his/her environment)
  • CI doesn't show whether a pull request works correct if some existing test breaks for itself (e.g. Can't pass golang unit tests #882 )

for d in $(go list ./... | grep -v vendor); do
go test -race -coverprofile=profile.out -covermode=atomic $d
go test -race -coverprofile=profile.out -covermode=atomic $d && true
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

&& true is a trick to let script continue even a command failed with set -e.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It preserves exit codes of commands ($?).

@ajvb
Copy link
Contributor

ajvb commented Mar 1, 2022

@ikedam would you mind merging develop into your branch so it will re-trigger the CI jobs?

@ikedam
Copy link
Contributor Author

ikedam commented Mar 5, 2022

Merged latest develop.

@ajvb ajvb merged commit e801ca3 into getsops:develop Mar 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants