[SPARK-25398] Minor bugs from comparing unrelated types #22384
Conversation
… have been trying to do
| // other places, so drop them to reduce the memory usage. | ||
| !acc.internal && (!acc.metadata.isDefined || | ||
| acc.metadata.get != Some(AccumulatorContext.SQL_ACCUM_IDENTIFIER)) | ||
| !acc.internal && acc.metadata != Some(AccumulatorContext.SQL_ACCUM_IDENTIFIER) |
| closure | ||
| .getClass | ||
| .getInterfaces.exists(_.getName.equals("scala.Serializable")) | ||
| .getInterfaces.exists(_.getName == "scala.Serializable") |
There was a problem hiding this comment.
See JIRA for explanation of this one
| .select('a) | ||
| .where('a > 1) | ||
| .where('a != 200) | ||
| .where('a =!= 200) |
There was a problem hiding this comment.
(!= isn't actually the Column operator for historical reasons having to do with precedence)
There was a problem hiding this comment.
Oh, thank you for fixing this.
| assert(unsafeDate.numElements == dateArray.length) | ||
| dateArray.zipWithIndex.map { case (e, i) => | ||
| assert(unsafeDate.get(i, DateType) == e) | ||
| assert(unsafeDate.get(i, DateType).asInstanceOf[Int] == e) |
There was a problem hiding this comment.
Issue here is that you can't really compare AnyRef to a primitive, not via boxing or unboxing; this may just be making some Scala coercion explicit but seemed worthwhile
|
Test build #95897 has finished for PR 22384 at commit
|
|
Thanks, @srowen . I also checked that this PR resolves all existing |
|
Test build #4334 has finished for PR 22384 at commit
|
srowen
changed the title
|
Looks fine. |
|
Merged to master/2.4 |
## What changes were proposed in this pull request? Correct some comparisons between unrelated types to what they seem to… have been trying to do ## How was this patch tested? Existing tests. Closes #22384 from srowen/SPARK-25398. Authored-by: Sean Owen <[email protected]> Signed-off-by: Sean Owen <[email protected]> (cherry picked from commit cfbdd6a) Signed-off-by: Sean Owen <[email protected]>
## What changes were proposed in this pull request? Correct some comparisons between unrelated types to what they seem to… have been trying to do ## How was this patch tested? Existing tests. Closes apache#22384 from srowen/SPARK-25398. Authored-by: Sean Owen <[email protected]> Signed-off-by: Sean Owen <[email protected]>
4 participants
What changes were proposed in this pull request?
Correct some comparisons between unrelated types to what they seem to… have been trying to do
How was this patch tested?
Existing tests.