-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix Flaky Test - SetSectionTest#testLiterals #43820
Conversation
b7d96d2
to
c9ee926
Compare
Hi @mohitbadve, thanks for the report. Did you actually observe an intermittent failure with this test locally or elsewhere? I took a look at the test again and I don't think there's anything wrong. First of all, synchronous or asynchronous execution should not really matter because both methods call the same (async) logic under the hood. The |
Hello @mkouba, thanks for your response. You have raised valid points about the logic under the hood. I used NonDex to detect the flakiness of the test. It can be used to detect nondeterministic behavior of tests under certain conditions which usually don't get considered in the local environment. To verify the issue, you can run NonDex via Maven as follows: While figuring out the solution, I thought that the call to Looking forward to discussing this further. |
I've never heard of this tool but it seems that its main feature is to shuffle the ordering for the supported APIs. Unfortunately, the debug results do not seem to be very helpful:
Actually, I added some logging in the code and I think that I found the problem. And it's not in the test but in the way the params of the set section are evaluated. We expect the iterator of a I will send a follow-up pull request with a fix.
|
- iterators of a HashMap#entrySet() are not guaranteed to be consistent; even if no modifications are made to the map - this issue was discovered by @mohitbadve with the https://github.com/TestingResearchIllinois/NonDex project - see also quarkusio#43820
Superseded by #43875. |
Thank you so much for taking the time to thoroughly understand the issue and for considering my PR. I really appreciate the effort you put into analyzing the problem and coming up with a solution. Your changes make sense, and I’m glad that they have been merged. |
Thank you for the report! ;-) |
This PR addresses an issue with the test testLiterals() in the Qute engine that was intermittently failing due to non-deterministic evaluation order of template literals. The original test assumed synchronous execution of the template rendering, which led to an assertEquals failure because the variables alpha and baz were being rendered in reverse order (e.g., "1::4::false::Andy" instead of the expected "1::4::Andy::false").
Changes Made: