@@ -866,33 +866,15 @@ Apart from these, the following properties are also available, and may be useful
866866 <td >1</td >
867867 <td >
868868 (Netty only) Connections between hosts are reused in order to reduce connection buildup for
869- large clusters. For small clusters with many hard disks, this may result in insufficient
869+ large clusters. For clusters with many hard disks and few hosts , this may result in insufficient
870870 concurrency to saturate all disks, and so users may consider increasing this value.
871871 </td >
872872</tr >
873- <tr >
874- <td ><code >spark.shuffle.io.serverThreads</code ></td >
875- <td >min(num-cores, 8)</td >
876- <td >
877- (Netty only) Size of fixed pool of threads to make requests. As each thread gets 16 MB of
878- off-heap buffer space initially, the default is limited to 8 cores to avoid excessive off-heap
879- allocation. It has been experimentally determined that this default should be sufficient to
880- saturate a 10Gb/s network, but in some cases this may have to be adjusted.
881- </td >
882- </tr >
883- <tr >
884- <td ><code >spark.shuffle.io.clientThreads</code ></td >
885- <td >min(num-cores, 8)</td >
886- <td >
887- (Netty only) Size of fixed pool of threads to make requests. This has the same properties as
888- <code>spark.shuffle.io.serverThreads</code>, above.
889- </td >
890- </tr >
891873<tr >
892874 <td ><code >spark.shuffle.io.maxRetries</code ></td >
893875 <td >3</td >
894876 <td >
895- (Netty only) Fetches that fail due to IO-related exceptions are automatically retried if this is
877+ (Netty only) Fetches that fail due to IO-related exceptions are automatically retried if this is
896878 set to a non-zero value. This retry logic helps stabilize large shuffles in the face of long GC
897879 pauses or transient network connectivity issues.
898880 </td >
0 commit comments