[SPARK-17673] [SQL] Incorrect exchange reuse with RowDataSourceScan

[ericl]

What changes were proposed in this pull request?

It seems the equality check for reuse of RowDataSourceScanExec nodes doesn't respect the output schema. This can cause self-joins or unions over the same underlying data source to return incorrect results if they select different fields.

How was this patch tested?

New unit test passes after the fix.

[SparkQA]

Test build #66014 has finished for PR 15273 at commit c0e08b8.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #66017 has finished for PR 15273 at commit f0df3a0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #66018 has finished for PR 15273 at commit e2dfb25.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

Did the unit test fail without the fix?

[ericl]

Yep, it returns 1 row only.

On Wed, Sep 28, 2016, 12:59 AM Reynold Xin [email protected] wrote:

Did the unit test fail without the fix?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#15273 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAA6Sjsfx6ug7bh3-hu4ItWjGx5mANC6ks5quh58gaJpZM4KIWY9
.

[rxin]

Thanks - merging in master/2.0.

I think the test is actually in a non-ideal location, but I'm going to merge first in order to make 2.0.1 rc4.