You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SPARK-4642] Documents about running-on-YARN needs update
Added descriptions about these parameters.
- spark.yarn.report.interval
- spark.yarn.queue
- spark.yarn.user.classpath.first
- spark.yarn.scheduler.reporterThread.maxFailures
Modified description about the defalut value of this parameter.
- spark.yarn.submit.file.replication
HDFS replication level for the files uploaded into HDFS for the application. These include things like the Spark jar, the app jar, and any distributed cache files/archives.
36
36
</td>
@@ -42,13 +42,34 @@ Most of the configs are the same for Spark on YARN as for other deployment modes
42
42
Set to true to preserve the staged files (Spark jar, app jar, distributed cache files) at the end of the job rather than delete them.
@@ -91,6 +112,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes
91
112
The amount of off heap memory (in megabytes) to be allocated per driver. This is memory that accounts for things like VM overheads, interned strings, other native overheads, etc. This tends to grow with the container size (typically 6-10%).
92
113
</td>
93
114
</tr>
115
+
<tr>
116
+
<td><code>spark.yarn.queue</code></td>
117
+
<td>default</td>
118
+
<td>
119
+
The YARN queue name which the application is being submitted.
0 commit comments