Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Require should support parallel execution, and should not abort all tests in case of failure #1623

Open
priyankshah217 opened this issue Jul 10, 2024 · 2 comments

Comments

@priyankshah217
Copy link

Description

I am testing a workflow, so if one test step fails, the test should not proceed. For example, if the login fails, there is no point in adding items to the cart. I am using assertions from the 'require' package. It works fine for a handful of tests, but as the number of tests grows, parallel execution seems to be the only option. However, when we made these tests run in parallel, they failed at very basic checks.
I understand that in "require," if an assertion fails, we call the FailNow method, which stops execution. Ideally, it should only stop the test/goroutine that failed this assertion and not affect other running tests/goroutines.

Here is basic code snippet with testify
https://go.dev/play/p/7a8mh9eVO6r

Without testify (Using native go testing framework)
https://go.dev/play/p/9NxHqKpmPam

Proposed solution

When tests are run in parallel, it's crucial that a required assertion failing does not disrupt other ongoing runs.

Use case

I'm phasing out parallel runs and considering switching to a native test framework. We're thinking of using Testify, and I may create a PR if time allows.

@jhoppenstedt
Copy link

jhoppenstedt commented Aug 1, 2024

Ideally, it should only stop the test/goroutine that failed this assertion and not affect other running tests/goroutines.

I observed a similar behaviour when calling suite.T().Skip("") which also fails the whole suite..
Code snippet: https://go.dev/play/p/p_Q71r_hP2S

@Antonboom
Copy link

Suite doesn't support t.Parallel()

Covered by testifylint#suite-broken-parallel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants