Skip to content

Commit bc46d27

Browse files
ulysses-youHyukjinKwon
authored andcommitted
[SPARK-33840][DOCS] Add spark.sql.files.minPartitionNum to performence tuning doc
### What changes were proposed in this pull request? Add `spark.sql.files.minPartitionNum` and it's description to sql-performence-tuning.md. ### Why are the changes needed? Help user to find it. ### Does this PR introduce _any_ user-facing change? Yes, it's the doc. ### How was this patch tested? Pass CI. Closes #30838 from ulysses-you/SPARK-33840. Authored-by: ulysses-you <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>
1 parent 0603913 commit bc46d27

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

docs/sql-performance-tuning.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,16 @@ that these options will be deprecated in future release as more optimizations ar
8585
</td>
8686
<td>2.0.0</td>
8787
</tr>
88+
<tr>
89+
<td><code>spark.sql.files.minPartitionNum</code></td>
90+
<td>Default Parallelism</td>
91+
<td>
92+
The suggested (not guaranteed) minimum number of split file partitions. If not set, the default
93+
value is `spark.default.parallelism`. This configuration is effective only when using file-based
94+
sources such as Parquet, JSON and ORC.
95+
</td>
96+
<td>3.1.0</td>
97+
</tr>
8898
<tr>
8999
<td><code>spark.sql.broadcastTimeout</code></td>
90100
<td>300</td>

0 commit comments

Comments
 (0)