ZEPPELIN-1384. Spark interpreter binary compatibility to scala 2.10 / 2.11 broken

[zjffdu]

What is this PR for?

As described in ZEPPELIN-1384, we may hit the following error when running zeppelin on spark 1.6 if we build zeppelin this way

dev/change_scala_version.sh 2.11
mvn -DskipTests -Drat.skip=true -Pscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0 -Phadoop-2.6 -Pyarn -Ppyspark -Psparkr clean package

java.lang.NoSuchMethodError: scala.runtime.VolatileByteRef.create(B)Lscala/runtime/VolatileByteRef;
    at scala.xml.MetaData$.iterate$1(MetaData.scala:39)
    at scala.xml.MetaData$.normalize(MetaData.scala:45)
    at scala.xml.Elem.<init>(Elem.scala:99)
    at org.apache.spark.ui.jobs.StagePage$$anonfun$26.apply(StagePage.scala:57)
    at org.apache.spark.ui.jobs.StagePage$$anonfun$26.apply(StagePage.scala:55)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.immutable.List.foreach(List.scala:318)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at org.apache.spark.ui.jobs.StagePage.<init>(StagePage.scala:55)
    at org.apache.spark.ui.jobs.StagesTab.<init>(StagesTab.scala:34)
    at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:57)
    at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:195)
    at org.apache.spark.ui.SparkUI$.createLiveUI(SparkUI.scala:146)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:473)
    at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_1(SparkInterpreter.java:440)
    at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:354)
    at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:137)
    at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:743)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)

The root cause is that scala-xml is removed from scala 2.11 to a separate library, so here we have class conflict of scala-xml api. In this PR, I make the scope of scala-xml to be provided and also make the scope of scala-library to be provided although it will be override in ZEPPELIN_HOME/spark/pom.xml

What type of PR is it?

[Bug Fix]

Todos

• - Task

What is the Jira issue?

• https://issues.apache.org/jira/browse/ZEPPELIN-1384

How should this be tested?

Tested manually.
Use the following command to build zeppelin and then run it on spark-1.6.2

dev/change_scala_version.sh 2.11
mvn -DskipTests -Drat.skip=true -Pscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0 -Phadoop-2.6 -Pyarn -Ppyspark -Psparkr clean package

Screenshots (if appropriate)

Questions:

• Does the licenses files need update? No
• Is there breaking changes for older versions? No
• Does this needs documentation? No

[bzz]

LGTM. Thank you @zjffdu for digging into!

CI is green as well.

Merging to master and branch-0.6 in case there are no further discussion