[ZEPPELIN-127] Fix issue raised by ZEPPELIN-127 related to DisplayUtils

[doanduyhai]

Dean1.Wood ([email protected]) reported the problem. Following code is throwing exception.

val wbIPR=Array(8.40,      8.09,      7.56,      6.83,      5.88,      4.72,      3.36,      1.78,      0.0)
val pressureIPR=Array(0.00,10000000.00,20000000.00,30000000.00,40000000.00,50000000.00,60000000.00,70000000.00,80000000.00)
case class IPR (w:Array[Double],p:Array[Double])
val iprInit=IPR(wbIPR,pressureIPR)
val iprList=sc.parallelize((1.0 to 0.9 by -0.001).map(k => IPR(iprInit.w.map(_ * k), iprInit.p)).zipWithIndex,5)
iprList.count()

To fix the issue, simple trick: wrap the implicit value around an object and import it

intp.interpret("object MaxResult {" +
        "implicit val sparkMaxResult = new SparkMaxResult(" +
        Integer.parseInt(getProperty("zeppelin.spark.maxResult")) + ")" +
        "}");
intp.interpret("import MaxResult.sparkMaxResult");

[Leemoonsoo]

Thanks for submitting the fix. Let me try it.

[Leemoonsoo]

Tested with following case and Looks good to me.

val wbIPR=Array(8.40,      8.09,      7.56,      6.83,      5.88,      4.72,      3.36,      1.78,      0.0)
val pressureIPR=Array(0.00,10000000.00,20000000.00,30000000.00,40000000.00,50000000.00,60000000.00,70000000.00,80000000.00)
case class IPR (w:Array[Double],p:Array[Double])
val iprInit=IPR(wbIPR,pressureIPR)
val iprList=sc.parallelize((1.0 to 0.9 by -0.001).map(k => IPR(iprInit.w.map(_ * k), iprInit.p)).zipWithIndex,5)
iprList.count()

[doanduyhai]

Would this PR make it for 0.5.0 or delayed to 0.6.x ?

[Leemoonsoo]

I think it's better to have this PR on 0.6.x, because of this PR enabling new feature while 0.5.x are finalized and in a voting process.
Once it's on master branch, then I think we can have discussion about backporting it to 0.5.1 if we really want to this feature on 0.5.x branch.

[doanduyhai]

Ok, sounds reasonable. Let's release 0.5.0 with stable features then

[corneadoug]

@doanduyhai Do we have the still the need of that PR?

[doanduyhai]

@corneadoug Yes we need it, it's a very convenient helper class to display html/img ...

[corneadoug]

@doanduyhai Then can you create a Jira issue for it and maybe update the PR description?

[doanduyhai]

@corneadoug Comment updated with original issue description.

The JIRA is already created: https://issues.apache.org/jira/browse/ZEPPELIN-127

[corneadoug]

@doanduyhai Thanks, since it wasn't in the description, I didn't know
@Leemoonsoo Can I merge this?

[Leemoonsoo]

master branch runs

val wbIPR=Array(8.40,      8.09,      7.56,      6.83,      5.88,      4.72,      3.36,      1.78,      0.0)
val pressureIPR=Array(0.00,10000000.00,20000000.00,30000000.00,40000000.00,50000000.00,60000000.00,70000000.00,80000000.00)
case class IPR (w:Array[Double],p:Array[Double])
val iprInit=IPR(wbIPR,pressureIPR)
val iprList=sc.parallelize((1.0 to 0.9 by -0.001).map(k => IPR(iprInit.w.map(_ * k), iprInit.p)).zipWithIndex,5)
iprList.count()

with out this patch. Do we still need this patch?

[doanduyhai]

@Leemoonsoo

Yes we do need this patch because DisplayUtils has been disabled temporarily because of the issue it causes. This patch fixes the issue and re-enable DisplayUtils

[Leemoonsoo]

@doanduyhai I see. Thanks for the explanation.

Current master branch works with both scala-2.10 and scala-2.11 without rebuild, depending on which version of spark SPARK_HOME points. is displayUtils works in this way, too?

[doanduyhai]

@Leemoonsoo

The fix is using pretty standard Scala (object xxx extends Serializable) and not impacted by Scala 2.11 breaking changes