JUNIT reporter doesn't mark test as failed if the beforeAll throws error

[jonatan-zint-tfs]

Describe the bug

When a beforeAll hook throws an error, the generated JUNIT report will report all tests in that describe block as without failiure, ergo succeeded.

That doesn't seem right to me.

Reproduction

I set up a minimal example in this stackblitz:

https://stackblitz.com/edit/vitest-dev-vitest-brpfrz

System Info

See the stackblitz example

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[sheremet-va]

What is your expectation here? Do you expect all tests to have the same error? I am not actually sure what JUNIT spec defines this situation. Does it support a single failure inside a suite?

[jonatan-zint-tfs]

Reading this it seems junit allows only failed testcases, not suites.

However not reporting the failiures at all does seem worse to me instead of reporting the same failiure to each testcase affecte by the bad beforeAll. That is also the behaviour if beforeEach throws an error.

[sheremet-va]

However not reporting the failiures at all does seem worse to me instead of reporting the same failiure to each testcase affecte by the bad beforeAll. That is also the behaviour if beforeEach throws an error.

I do agree that we need to show them, I am just not sure how. Maybe jest has an answer.

[torsten-ueckert]

With Jest the individual test cases are not run and every test case is reported as failing with the same error in beforeAll

[hi-ogawa]

It looks like a failure in afterAll doesn't make relevant test cases as failed. I made a repro to compare all four cases of
afterAll/afterEach/beforeAll/beforeEach.

https://stackblitz.com/edit/vitest-dev-vitest-m7bw4w?file=junit.xml