[SPARK-22660][BUILD] Use position() and limit() to fix ambiguity issue in scala-2.12

[kellyzly]

…a-2.12 and JDK9

What changes were proposed in this pull request?

Some compile error after upgrading to scala-2.12

spark_source/core/src/main/scala/org/apache/spark/executor/Executor.scala:455: ambiguous reference to overloaded definition, method limit in class ByteBuffer of type (x$1: Int)java.nio.ByteBuffer
method limit in class Buffer of type ()Int
match expected type ?
     val resultSize = serializedDirectResult.limit
error  

The limit method was moved from ByteBuffer to the superclass Buffer and it can no longer be called without (). The same reason for position method.

/home/zly/prj/oss/jdk9_HOS_SOURCE/spark_source/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformationExec.scala:427: ambiguous reference to overloaded definition, [error] both method putAll in class Properties of type (x$1: java.util.Map[_, _])Unit [error] and  method putAll in class Hashtable of type (x$1: java.util.Map[_ <: Object, _ <: Object])Unit [error] match argument types (java.util.Map[String,String])
 [error]       props.putAll(outputSerdeProps.toMap.asJava)
 [error]             ^

This is because the key type is Object instead of String which is unsafe.

How was this patch tested?

running tests

Please review http://spark.apache.org/contributing.html before opening a pull request.

[srowen]

This kind of change is fine as it's backwards-compatible, if it resolves a real compiler issues later. But collect as many of this type of problem in this PR rather than across several.

[SparkQA]

Test build #4001 has finished for PR 19854 at commit fb30b90.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

ok to test

[SparkQA]

Test build #84344 has finished for PR 19854 at commit d3f0efe.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

Let's fix the title to [SPARK-22660][BUILD] Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9 and fill up the PR description BTW.

[HyukjinKwon]

This exceeds 100 character length limit. See the logs in Jenkins tests.

[srowen]

This change isn't valid and isn't related

[kellyzly]

Sure I can remove the comment like " sun.misc.Cleaner is not used in JDK9(see SPARK-22659), we can use CleanUtil in HADOOP-12760 to solve it"
sun.misc.Cleaner has moved to a new location in OpenJDK 9. If uncommenting code about Cleaner, compile will fail. Cleaner is used to release memory. If commenting code about cleaner, there maybe some function lost. Can you give me some suggestions?

[kellyzly]

@srowen ： have deleted changes about Platform.java. So current pr is only for solving problems met in scala-2.12 and can not fix the problem about JDK9.

[srowen]

Nit: space before brace

[viirya]

I think the direct reference should be scala/bug#10418.

[viirya]

And we should not leave commented code.

[viirya]

Is this related?

[viirya]

scala/bug#10418. I see.

[kellyzly]

yes, the change to the putAll is similar as the solution provided in scala/bug#10418

[viirya]

ditto. I think it is better to remove commented code and refer to scala/bug#10418.

[srowen]

Ping @kellyzly

[SparkQA]

Test build #84525 has finished for PR 19854 at commit dbe6b3f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

Nit: h{ -> h {

[SparkQA]

Test build #84540 has finished for PR 19854 at commit 83a09cb.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jiangxb1987]

Please fix the title and also fulfill the PR description.

[kellyzly]

thanks @HyukjinKwon ,@srowen, @viirya 's review

[srowen]

Merged to master