Skip to content

Commit 1ae51f6

Browse files
aarondavpwendell
authored andcommitted
[SPARK-4183] Enable NettyBlockTransferService by default
Note that we're turning this on for at least the first part of the QA period as a trial. We want to enable this (and deprecate the NioBlockTransferService) as soon as possible in the hopes that NettyBlockTransferService will be more stable and easier to maintain. We will turn it off if we run into major issues. Author: Aaron Davidson <[email protected]> Closes #3049 from aarondav/enable-netty and squashes the following commits: bb981cc [Aaron Davidson] [SPARK-4183] Enable NettyBlockTransferService by default
1 parent ebd6480 commit 1ae51f6

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

docs/configuration.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -371,6 +371,16 @@ Apart from these, the following properties are also available, and may be useful
371371
map-side aggregation and there are at most this many reduce partitions.
372372
</td>
373373
</tr>
374+
<tr>
375+
<td><code>spark.shuffle.blockTransferService</code></td>
376+
<td>netty</td>
377+
<td>
378+
Implementation to use for transferring shuffle and cached blocks between executors. There
379+
are two implementations available: <code>netty</code> and <code>nio</code>. Netty-based
380+
block transfer is intended to be simpler but equally efficient and is the default option
381+
starting in 1.2.
382+
</td>
383+
</tr>
374384
</table>
375385

376386
#### Spark UI

0 commit comments

Comments
 (0)