-
Notifications
You must be signed in to change notification settings - Fork 163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: Fixes flaky app layout tests that depend on timeout #2666
Conversation
render(<BreadcrumbGroup items={defaultBreadcrumbs} />); | ||
await waitFor(() => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change breaks the intention of the tests. waitFor
runs the first check without any delay and when it passes it moves forward.
The point of this test was to assert there is still only one breadcrumb instance in a long term, even after a timeout.
We can talk about individual flaky tests and how to make them more stable, but applying waitFor for every test in this file is incorrect.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes total sense!
I added a delay before each wait for to that the first check is delayed as it was before the change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now the setup is even more far from original.
How about keep everything unchanged, and only add a couple waitFor
to potentially flaky tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The first one was different. This one is not. I am replacing this:
render()
delay()
assert()
with this:
render()
delay()
assertUntilItWorks()
The latter is equivalent of having a larger delay, but better efficient. I also run it 1500 times.
The change can be applied not to all tests but only to some. However, it looks like they all can fail. I reproduced failures in 2 or 3 different tests in that file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still do not understand what you disliked so much about renderAsync
, so you decided to remove it.
I think the correct fix would be to improve renderAsync
to make it more stable. It will happen here: #2676
I will close this PR as not needed. Thanks for taking your time
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2666 +/- ##
=======================================
Coverage 95.79% 95.79%
=======================================
Files 743 744 +1
Lines 20528 20563 +35
Branches 6992 7008 +16
=======================================
+ Hits 19664 19699 +35
Misses 856 856
Partials 8 8 ☔ View full report in Codecov by Sentry. |
Description
Use waitFor instead of delay for global-breadcrumbs.test.tsx tests.
See failing test: https://github.com/cloudscape-design/components/actions/runs/10718898212/job/29722075662
How has this been tested?
Review checklist
The following items are to be evaluated by the author(s) and the reviewer(s).
Correctness
CONTRIBUTING.md
.CONTRIBUTING.md
.Security
checkSafeUrl
function.Testing
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.