[WIP][SPARK-33098][SQL] Fix In expression casts

[peter-toth]

What changes were proposed in this pull request?

Fix InConversion rule by finding the common wider type of list expressions first and then the final common type for comparison between the value and the list expressions.

Before this PR the query SELECT * FROM t WHERE c IN ('2') where c is Int generated the following filter:

Filter cast(c#31 as string) IN (cast(2 as string))

After this PR it generates:

Filter cast(c#31 as int) IN (cast(2 as int))

that is identical to what a WHERE c = '2' would do.

Why are the changes needed?

To make In behave like = and avoid issues described in https://issues.apache.org/jira/browse/SPARK-33098

Does this PR introduce any user-facing change?

Yes, some queries will not fail.

How was this patch tested?

Existing UTs and I will add some new ones as well.

[tanelk]

Does it duplicate #22038?

[peter-toth]

Does it duplicate #22038?

Hmm, it might be the case. Unfortunately, I didn't notice that old PR. Let check how they relate to each other.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34192/

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34192/

[peter-toth]

Does it duplicate #22038?

Hmm, it might be the case. Unfortunately, I didn't notice that old PR. Let check how they relate to each other.

It is a duplicate indeed. I will close my PR.

[SparkQA]

Test build #129587 has finished for PR 29988 at commit 558c032.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.