Fix non-deterministic behaviors in IgniteSQLBuilder and IgniteStoreMetadataAnalyzer #296
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
I encountered non-deterministic behavior while running the following tests using NonDex:
org.apache.gora.ignite.store.TestIgniteStoreIn gora-ignite/src/main/java/org/apache/gora/ignite/utils/IgniteSQLBuilder, method
createInsertQueryandfillInsertQueryconvert the entrySet of Map objectdatainto List objectlist:These two methods then iterate the list and pass its elements as other methods' parameters. The conversion from entrySet of Map to List causes the random order of List's element. This randomness further causes the non-deterministic behavior of
putmethod in gora-ignite/src/main/java/org/apache/gora/ignite/store/IgniteStore, which calls these two methods. All tests callingputmethod of classIgniteStoreare thus flaky.Similarly, in gora-ignite/src/main/java/org/apache/gora/ignite/store/IgnoreStoreMetadataAnalyzer, method
getTableNamesiterates an unsorted ResultSet objectexecuteQueryand appends its elements into List of table name Strings. The order of the table names in the list is random, but the list is then returned and compared with a list with fixed order in line 42 of gora-ignite/src/test/java/org/apache/gora/ignite/store/TestIgniteStoreThe randomness of the list returned by
getTableNamesthus may cause unexpected failure of TestIgniteStore.Steps to Reproduce
I used tool NonDex to detect the flaky tests.
NonDex: https://github.com/TestingResearchIllinois/NonDex
Run the tests with NonDex:
The error message shows:
Click to view
Potential Solution
To fix the non-deterministic behavior of method
createInsertQueryandfillInsertQueryof classIgniteSQLBuilder, we can pass a fix-ordered List instead of a random-ordered Map as parameters into these two functions:Meanwhile, in
putmethod of classIgniteStore, we maintain two lists storing Map objectdata's keys and values:Whenever we put a key-value pair into Map object
data, we update these two lists by adding the key and value. Therefore, the order of elements in these two lists will be deterministic, exactly matching the order in which we put key-value pairs into Map objectdata. We can then pass these two lists into methodcreateInsertQueryandfillInsertQueryand fix the non-deterministic behavior.To fix the non-deterministic behavior of method
getTableNamesin class IgnoreStoreMetadataAnalyzer, we can sort the Listtabbefore returning it:Then, the order of elements in List
tabwill be deterministic, following alphabetical order.Correspondingly, we need to change the list for comparison in line 42 of
TestIgniteStoreto alphabetical order: