Skip to content

Commit 9ab0c24

Browse files
committed
Fix the description about building SPARK with YARN
1 parent 199e59a commit 9ab0c24

File tree

1 file changed

+12
-3
lines changed

1 file changed

+12
-3
lines changed

docs/building-spark.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -60,20 +60,29 @@ mvn -Dhadoop.version=2.0.0-mr1-cdh4.2.0 -DskipTests clean package
6060
mvn -Phadoop-0.23 -Dhadoop.version=0.23.7 -DskipTests clean package
6161
{% endhighlight %}
6262

63-
For Apache Hadoop 2.x, 0.23.x, Cloudera CDH, and other Hadoop versions with YARN, you can enable the "yarn" profile and optionally set the "yarn.version" property if it is different from "hadoop.version". As of Spark 1.3, Spark only supports YARN versions 2.2.0 and later.
63+
For Apache Hadoop 2.2.0 and later and Cloudera CDH 5 with YARN, you can enable the "yarn" profile and optionally set the "yarn.version" property if it is different from "hadoop.version". As of Spark 1.3, Spark only supports YARN versions 2.2.0 and later.
6464

6565
Examples:
6666

6767
{% highlight bash %}
6868
# Apache Hadoop 2.2.X
69-
mvn -Pyarn -Phadoop-2.2 -Dhadoop.version=2.2.0 -DskipTests clean package
69+
mvn -Pyarn -Phadoop-2.2 -Dhadoop.version=2.2.X -DskipTests clean package
7070

7171
# Apache Hadoop 2.3.X
72-
mvn -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -DskipTests clean package
72+
mvn -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.X -DskipTests clean package
7373

7474
# Apache Hadoop 2.4.X or 2.5.X
7575
mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=VERSION -DskipTests clean package
7676

77+
# Cloudera CDH 5.0.X
78+
mvn -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0-cdh5.0.X -DskipTests clean package
79+
80+
# Cloudera CDH 5.1.X
81+
mvn -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0-cdh5.1.X -DskipTests clean package
82+
83+
# Cloudera CDEH 5.2.X or 5.3.X
84+
mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=VERSION -DskipTests clean package
85+
7786
Versions of Hadoop after 2.5.X may or may not work with the -Phadoop-2.4 profile (they were
7887
released after this version of Spark).
7988

0 commit comments

Comments
 (0)