[SPARK-11802][SQL] Kryo-based encoder for opaque types in Datasets

[rxin]

I also found a bug with self-joins returning incorrect results in the Dataset API. Two test cases attached and filed SPARK-11803.

[cloud-fan]

How about putting this method into ExpressionEncoder? I think Encoders is only used at java side as the lack of implicit magic.

[cloud-fan]

then we can change the return type to ExpressionEncoder[T] and make the tests less verbose with asInstanseOf

[rxin]

what if we want to use kryo in scala?

[cloud-fan]

oh, this can't be done by implicit, I was wrong, nvm.

[SparkQA]

Test build #46146 has finished for PR 9789 at commit 073f302.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):\n * case class SerializeWithKryo(child: Expression) extends UnaryExpression\n * case class DeserializeWithKryo[T](child: Expression, tag: ClassTag[T]) extends UnaryExpression\n

[SparkQA]

Test build #46179 has finished for PR 9789 at commit 765c7e6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):\n * case class SerializeWithKryo(child: Expression) extends UnaryExpression\n * case class DeserializeWithKryo[T](child: Expression, tag: ClassTag[T]) extends UnaryExpression\n * public final class UTF8String implements Comparable<UTF8String>, Externalizable, KryoSerializable\n

[SparkQA]

Test build #46175 has finished for PR 9789 at commit 7e39a17.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):\n * case class SerializeWithKryo(child: Expression) extends UnaryExpression\n * case class DeserializeWithKryo[T](child: Expression, tag: ClassTag[T]) extends UnaryExpression\n * public final class UTF8String implements Comparable<UTF8String>, Externalizable, KryoSerializable\n