fix: Fix the semi merge join with duplicate match vectors #13096
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D73397297 |
✅ Deploy Preview for meta-velox canceled.
|
kevinwilfong
approved these changes
Apr 21, 2025
Contributor
kevinwilfong
left a comment
kevinwilfong
left a comment
There was a problem hiding this comment.
Thanks for the fix!
…cubator#13096) Summary: When semi-join with multiple matched vectors, it produce redundant matched rows with one per each matched vector but we actually just need to produce exact one matched row. This PR fixes this and verified with unit test Reviewed By: kevinwilfong Differential Revision: D73397297
xiaoxmeng
force-pushed
the
export-D73397297
branch
from
905283b to
3a02b38
Compare
Contributor
|
This pull request was exported from Phabricator. Differential Revision: D73397297 |
Contributor
|
This pull request has been merged in e11e394. |
1 task
Collaborator
|
@xiaoxmeng @kevinwilfong This issue has fixed in #11771. And #11771 also fixed the anti and full outer join issues. Can you help to review if you have time? Thanks. |
JkSelf
added a commit
to JkSelf/velox
that referenced
this pull request
Apr 23, 2025
zhouyuan
added a commit
to oap-project/velox
that referenced
this pull request
Apr 23, 2025
facebook-github-bot
pushed a commit
that referenced
this pull request
Jun 24, 2025
…atched rows (#13121) Summary: In a semi join, only the first matching record is retained when there are multiple matching rows on the other side. [ PR#13096](#13096) addresses this issue by selecting only the last matched row for the final output, which resolves result mismatch issue when no filter expression is present. However, this approach can not result mismatches if a filter is applied. For example: ``` Left Record Right Record a b c d 2 5 2 4 2 5 ``` With the join condition `a == c `and the filter `b > d`, selecting only the last matched row results in right side `(2, 5)` which does not satisfy the filter, leading to an empty result. The correct result should be `(2, 5, 2, 4)`. This PR ensures that all matched rows are retained when a filter expression exists. During the filtering process, only the first matched row is kept. Pull Request resolved: #13121 Reviewed By: kevinwilfong Differential Revision: D77203870 Pulled By: xiaoxmeng fbshipit-source-id: 0dcbf57148309347770aa981dec27fd148525d44
nimesh1601
pushed a commit
to nimesh1601/velox
that referenced
this pull request
Jul 7, 2025
…atched rows (facebookincubator#13121) Summary: In a semi join, only the first matching record is retained when there are multiple matching rows on the other side. [ PR#13096](facebookincubator#13096) addresses this issue by selecting only the last matched row for the final output, which resolves result mismatch issue when no filter expression is present. However, this approach can not result mismatches if a filter is applied. For example: ``` Left Record Right Record a b c d 2 5 2 4 2 5 ``` With the join condition `a == c `and the filter `b > d`, selecting only the last matched row results in right side `(2, 5)` which does not satisfy the filter, leading to an empty result. The correct result should be `(2, 5, 2, 4)`. This PR ensures that all matched rows are retained when a filter expression exists. During the filtering process, only the first matched row is kept. Pull Request resolved: facebookincubator#13121 Reviewed By: kevinwilfong Differential Revision: D77203870 Pulled By: xiaoxmeng fbshipit-source-id: 0dcbf57148309347770aa981dec27fd148525d44
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
When semi-join with multiple matched vectors, it produce redundant matched rows with one per each matched vector but we actually just
need to produce exact one matched row. This PR fixes this and verified with unit test
Differential Revision: D73397297