KAFKA-14834: [2/N] Test coverage for out-of-order data in joins#13497
Conversation
| STREAMS_CONFIG.put(StreamsConfig.APPLICATION_ID_CONFIG, appID + "-multi-inner"); | ||
|
|
||
| final List<List<TestRecord<Long, String>>> expectedResult = Arrays.asList( | ||
| @SuppressWarnings("RedundantTypeArguments (explicit type arguments speedup compilation and analysis time)") final List<List<TestRecord<Long, String>>> expectedResult = Arrays.asList( |
There was a problem hiding this comment.
For my own education: why do we need this, and why do we add the generic types below?
There was a problem hiding this comment.
Hmm.. I'm not sure where that suppression came from. Probably an accident, I will remove it.
The reason the explicit types are added below is because I was getting an error that type inference slows down if there are too many variadic arguments passed to a method with a generic that requires type inference. I forget if it was just my IDE complaining or if spotbugs was failing too but I figure adding the types in doesn't introduce any harm.
| new Input<>(INPUT_TOPIC_RIGHT, null, 13), | ||
| new Input<>(INPUT_TOPIC_RIGHT, "d", 14), | ||
| new Input<>(INPUT_TOPIC_LEFT, "D", 15), | ||
| new Input<>(INPUT_TOPIC_LEFT, "E", 4), // out-of-order data |
There was a problem hiding this comment.
Seems we only need this for table-table join tests, but it only blows up stream-stream join results? I am ok to merge as is, but seems it's adding "some noise" the stream-stream-join case (not too bad for binary joins, but for 3-way join it's a lot).
Not sure if we could have different inputs for stream vs table case?
There was a problem hiding this comment.
Yeah, I noticed this as well. If you don't think having the out-of-order data in stream-stream join tests is valuable, I can open a follow-up PR to remove this (to reduce noise). Just LMK.
There was a problem hiding this comment.
Don't think it does really harm, but it makes it very hard to actually read the test.
2 participants
In preparation for updating DSL join processors to have updated semantics when versioned stores are used (cf KIP-914), this PR adds test coverage for out-of-order data in joins to the existing integration tests for stream-table joins and primary-key table-table joins. Follow-up PRs will build on top of this change by adding new tests for versioned stores, and the out-of-order data will produce different results in those settings.
Committer Checklist (excluded from commit message)