test(wpt): mark timed out tests as 'failed' #2644
Conversation
test/wpt/runner/runner.mjs
Outdated
| this.#stats.testsFailed += 1 | ||
| console.log(colors(`[${finishedFiles}/${total}] FAILED - ${test}`, 'red')) |
There was a problem hiding this comment.
Tests can have expected failures, marking the entire file as failing isn't the correct behavior here.
There was a problem hiding this comment.
Hi @KhafraDev and thank you for your review! Sorry, I hadn't considered this scenario. I'll try and fix that 😁
There was a problem hiding this comment.
Hi @KhafraDev, I changed the logic for marking a Test File as failed. Now, a Test File fails if one of its tests fails without being an expected failure. Let me know if it is acceptable or if further changes are needed.
Thank you in advance!
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #2644 +/- ##
==========================================
- Coverage 85.54% 85.28% -0.27%
==========================================
Files 76 84 +8
Lines 6858 7584 +726
==========================================
+ Hits 5867 6468 +601
- Misses 991 1116 +125 ☔ View full report in Codecov by Sentry. |
This relates to...
#2627
Changes
The
WPTRunnerclass insiderunner.mjshas been refactored to provide a more informative test statistics.Features
Example of File report
New log appearance
Log example for a timed out File
Status