Move Spark specific things from pom.xml to spark/pom.xml #88
Conversation
Leemoonsoo
force-pushed
the
pom_refactor
branch
2 times, most recently
from
9415ade to
602d34a
Compare
|
I have moved spark related dependency configuration from pom.xml -> spark/pom.xml |
Leemoonsoo
force-pushed
the
pom_refactor
branch
7 times, most recently
from
4bfdcc3 to
64681ea
Compare
Leemoonsoo
force-pushed
the
pom_refactor
branch
3 times, most recently
from
dc4d9dd to
e0885a1
Compare
|
This PR is ready to merge. Here's summary of changes. Move spark specific dependency configuration from pom.xml -> spark/pom.xmlBecause of spark specific dependency configuration were inside of common parent pom.xml, all other interpreter were influenced by this. Flink is one of example that conflict dependency library version with the version spark uses. This PR moves spark specific configuration from pom.xml to spark/pom.xml Removal of huge dependencyManagement blockdependencyManagement block introduced to pom.xml for resolving dependency conflict between Zeppelin and Spark. However, since Zeppelin changed to run interpreter as separate process, we don't have massive dependency conflict. So most dependency management blocks are just removed. Automated test against standalone spark cluster.Incompatible version of dependency can raise error especially when spark runs against spark cluster. While this PR brings big changes to dependency, i've made our CI (travis) test against spark standalone cluster to make sure Zeppelin runs with spark cluster. Automated test multiple profile spark-1.3, spark-1.2, spark-1.1 |
|
Merging if there're not more discussions. |
|
+1 |
#### Interpreter for [Apache Flink](http://flink.apache.org/). Flink people helped a lot to write the interpreter. Thanks so much! Some codes are copied from Flink's development branch. Once Flink releases 0.9, copied code and snapshot repository configuration will be removed. #### Build if there're no options, by default it is building against flink 0.9.0-milestone-1. With combination of Zeppelin, it is good idea to use 0.9-SNAPSHOT, because of it support .collect() that helps really a lot to get results data and display it on Zeppelin. So, you might want to build in this way, ``` mvn package -Dflink.version=0.9-SNAPSHOT -DskipTests ``` #### Screenshot  Author: Lee moon soo <[email protected]> Closes #75 from Leemoonsoo/flink and squashes the following commits: f08bd25 [Lee moon soo] Update pom.xml after #88 460cf46 [Lee moon soo] jarr up -> jar up 501efb3 [Lee moon soo] Add scalastyle e69e5ba [Lee moon soo] Add license 7be1f90 [Lee moon soo] Add apache snapshot repo ebbd0da [Lee moon soo] Fix unittest and update comment 27fc306 [Lee moon soo] Cleaning up f2a66df [Lee moon soo] Initial implementation of interpreter for Apache Flink
https://issues.apache.org/jira/browse/ZEPPELIN-98 Part of the 109b080 AKA #88 - fix YARN artefacts dependencies versions Author: Alexander Bezzubov <[email protected]> Closes #97 from bzz/fix-yarn-dependencies and squashes the following commits: 9ddef5d [Alexander Bezzubov] ZEPPELIN-98: fix YARN artifacts version missmatch
Fixes S3 support for Hadoop >= 2.3 (ZEPPELIN-111 https://issues.apache.org/jira/browse/ZEPPELIN-111) Regression from PR #88. Author: Jérémy Subtil <[email protected]> Closes #105 from jersub/fix-jets3t and squashes the following commits: 0e75073 [Jérémy Subtil] Fix the jets3t dependency according to the Hadoop version
….fix to V_1.0.0 * commit 'e942a45378632793f12320640b09e4a1bbe6bd94': [ZP-000] fix flyway
2 participants
Move Spark specific dependencyManagement and properties from pom.xml to spark/pom.xml.
Which interfere other interpreter's dependency version.