fix: Update the filterInput when flatten the right projections

[wankunde]

What changes were proposed in this pull request?

Bug fix for MergeJoin which the join filter is not null. For example, in the tpcds 95, the subquery ws_wh contains a join filter ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk

SELECT
    ws1.ws_order_number,
    ws1.ws_warehouse_sk wh1,
    ws2.ws_warehouse_sk wh2
  FROM web_sales ws1, web_sales ws2
  WHERE ws1.ws_order_number = ws2.ws_order_number
    AND ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk

How to reproduce this issue:

In tpc95 query, if the right matches contains more than one batch, and these batches are DictionaryVector.
When we addOutputRow() from those batches, SMJ join will only copy the Dictionary indices.
And then we decode the incorrect value from the last DictionaryVector.

void copyRow(
    const RowVectorPtr& source,
    vector_size_t sourceIndex,
    const RowVectorPtr& target,
    vector_size_t targetIndex,
    const std::vector<IdentityProjection>& projections) {
  for (const auto& projection : projections) {
    const auto& sourceChild = source->childAt(projection.inputChannel);
    const auto& targetChild = target->childAt(projection.outputChannel);
    targetChild->copy(sourceChild.get(), targetIndex, sourceIndex, 1);
  }
}

Why are the changes needed?

Fix data quality issue.

How was this patch tested?

Test with TPCDS 95 query and local query

SELECT
    ws1.ws_order_number,
    ws1.ws_warehouse_sk wh1,
    ws2.ws_warehouse_sk wh2,
    ws2.ws_warehouse_sk - ws1.ws_warehouse_sk as diff
  FROM web_sales ws1, web_sales ws2
  WHERE ws1.ws_order_number = ws2.ws_order_number
    AND ws1.ws_order_number is not null
    AND ws2.ws_order_number is not null
    AND ws1.ws_warehouse_sk - ws2.ws_warehouse_sk = 0;

[netlify]

✅ Deploy Preview for meta-velox canceled.

Name	Link
🔨 Latest commit	c9c651a
🔍 Latest deploy log	https://app.netlify.com/sites/meta-velox/deploys/67728f55e95d5b00080ad29d

[JkSelf]

@wankunde Thank you for addressing this issue. It is a known problem and can be resolved in #11771

[wankunde]

@wankunde Thank you for addressing this issue. It is a known problem and can be resolved in #11771

Hi, @JkSelf , in #11771 only semi join and anti join are fixed with JoinTracker, but inner join with a filter has not been fixed.
In my case, the filter exprSet will eval on the incorrect input DictionaryVector.

[JkSelf]

@wankunde Thank you for addressing this issue. It is a known problem and can be resolved in #11771

Hi, @JkSelf , in #11771 only semi join and anti join are fixed with JoinTracker, but inner join with a filter has not been fixed. In my case, the filter exprSet will eval on the incorrect input DictionaryVector.

@wankunde Based on our offline discussion, can #11771 resolve your issue?

[wankunde]

@wankunde Thank you for addressing this issue. It is a known problem and can be resolved in #11771

Hi, @JkSelf , in #11771 only semi join and anti join are fixed with JoinTracker, but inner join with a filter has not been fixed. In my case, the filter exprSet will eval on the incorrect input DictionaryVector.

@wankunde Based on our offline discussion, can #11771 resolve your issue?

Thanks, @JkSelf , #11771 already fixed the Q95 issue in
https://github.com/facebookincubator/velox/pull/11771/files#diff-8279da91a596a4b97a73bc4d9aa635da1baaf254dcdb2f7f8049bd21e7a9551eR416

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. If you'd still like this PR merged, please comment on the PR, make sure you've addressed reviewer comments, and rebase on the latest main. Thank you for your contributions!