[SPARK-7952][SPARK-7984][SQL] equality check between boolean type and numeric type is broken.

[cloud-fan]

The origin code has several problems:

• true <=> 1 will return false as we didn't set a rule to handle it.
• true = a where a is not Literal and its value is 1, will return false as we only handle literal values.

[cloud-fan]

boolean is already comparable, we don't need to cast it to byte.

[SparkQA]

Test build #33766 has finished for PR 6505 at commit a3f8e9c.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #33794 has finished for PR 6505 at commit 18211e7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

cc @yhuai

[rxin]

Note that this no longer merges cleanly.

[liancheng]

This test case can be simplified a bit, also please drop the temp table at last:

  val data = Seq(
    (1, true),
    (0, false),
    (2, true),
    (2, false),
    (null, true),
    (null, false),
    (0, null),
    (1, null),
    (null, null)
  ).map { case (i, b) =>
    (i.asInstanceOf[Integer], b.asInstanceOf[java.lang.Boolean])
  }

  data.toDF("i", "b").registerTempTable("t")

  try {
    // checkAnswer calls
  } finally {
    sqlCtx.dropTempTable("t")
  }

[SparkQA]

Test build #33806 has finished for PR 6505 at commit fc0d741.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

"typegd"?

[liancheng]

LGTM except for a few minor issues. Thanks for fixing this!

[cloud-fan]

Hi @liancheng , do we need to drop temp tables at last? If so, we need to change a lot of test cases in SQLQuerySuite.

[liancheng]

@cloud-fan Unlike persisted tables, existing temp tables can be overriden without triggering any error. But in general it's a better practice to clean up the scene. That's why we made SQLTestUtils and provide a bunch of withXxx methods to help automate these boilerplate code. For example, you may make SQLQuerySuite extend SQLTestUtils and use withTempTable("t") { ... } to wrap your test code in the newly added test case.

[liancheng]

Seems that we can replace this CaseKeyWhen with an If?

[SparkQA]

Test build #33809 has finished for PR 6505 at commit 9ba2130.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #33810 has finished for PR 6505 at commit 625973c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #33811 has finished for PR 6505 at commit ebc8c61.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

this test case is too confusing as it is.

Maybe you can have tuple4, where the 3rd element is the comparison value for =, and 4th element is the value for <=>. You can also just define the type in the Seq itself, i.e.

Seq[(java.lang.Integer, java.lang.Boolean, java.lang.Boolean)](
  (0, false, true, true)
  ...
)

[yhuai]

Can you give some examples to explain what is the problem?

[SparkQA]

Test build #33857 has finished for PR 6505 at commit b6401ba.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #33863 has finished for PR 6505 at commit 77f0f39.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[cloud-fan]

Is it ready to go? cc @rxin @liancheng @yhuai

[yhuai]

Can you update the description to explain what is the problem with more information (e.g. a case that we do not handle correctly)?

[cloud-fan]

@yhuai , updated.

[liancheng]

LGTM, thanks for working on this!

[rxin]

Merged in master.

[rxin]

@marmbrus if you have time would be great to take a look at this too.

[marmbrus]

Shouldn't this be a SQL internal Decimal type?

[marmbrus]

We should probably also add a test for this case since it seem broken ATM.

[rxin]

Good catch. This breaks for Decimal. We should probably compare Literal itself, instead of comparing the wrapped value for literals.

[rxin]

@cloud-fan can you fix this and add a test case? thanks.