fix: Fix semi join result mismatch with filter and multi duplicated matched rows #13121
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
JkSelf
requested review from
kevinwilfong and
xiaoxmeng
and removed request for
xiaoxmeng
✅ Deploy Preview for meta-velox canceled.
|
JkSelf
force-pushed
the
semi-bug-fix
branch
2 times, most recently
from
4018c2d to
9370b0a
Compare
JkSelf
changed the title
Collaborator
Author
|
@rui-mo @jinchengchenghh @zhli1142015 Can you help to review this PR? Thanks. |
Collaborator
Author
|
@pedroerp @xiaoxmeng Can you help to review this PR? Thanks. |
Collaborator
Author
|
@xiaoxmeng Thanks for your review. Have resolved all your comments. Can you help to review again? Thanks. |
xiaoxmeng
reviewed
Jun 23, 2025
xiaoxmeng
reviewed
Jun 24, 2025
xiaoxmeng
reviewed
Jun 24, 2025
xiaoxmeng
approved these changes
Jun 24, 2025
Contributor
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Collaborator
Author
|
@xiaoxmeng Thank you for helping review and land this PR! |
Contributor
|
@xiaoxmeng merged this pull request in 27299ef. |
1 task
nimesh1601
pushed a commit
to nimesh1601/velox
that referenced
this pull request
Jul 7, 2025
…atched rows (facebookincubator#13121) Summary: In a semi join, only the first matching record is retained when there are multiple matching rows on the other side. [ PR#13096](facebookincubator#13096) addresses this issue by selecting only the last matched row for the final output, which resolves result mismatch issue when no filter expression is present. However, this approach can not result mismatches if a filter is applied. For example: ``` Left Record Right Record a b c d 2 5 2 4 2 5 ``` With the join condition `a == c `and the filter `b > d`, selecting only the last matched row results in right side `(2, 5)` which does not satisfy the filter, leading to an empty result. The correct result should be `(2, 5, 2, 4)`. This PR ensures that all matched rows are retained when a filter expression exists. During the filtering process, only the first matched row is kept. Pull Request resolved: facebookincubator#13121 Reviewed By: kevinwilfong Differential Revision: D77203870 Pulled By: xiaoxmeng fbshipit-source-id: 0dcbf57148309347770aa981dec27fd148525d44
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In a semi join, only the first matching record is retained when there are multiple matching rows on the other side. PR#13096 addresses this issue by selecting only the last matched row for the final output, which resolves result mismatch issue when no filter expression is present. However, this approach can not result mismatches if a filter is applied.
For example:
With the join condition
a == cand the filterb > d, selecting only the last matched row results in right side(2, 5)which does not satisfy the filter, leading to an empty result. The correct result should be(2, 5, 2, 4).This PR ensures that all matched rows are retained when a filter expression exists. During the filtering process, only the first matched row is kept.