Skip to content

[SPARK-40404][DOCS] Add precondition description for spark.shuffle.service.db.backend in running-on-yarn.md #37853

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
LuciferYang wants to merge 4 commits into from
Closed

[SPARK-40404][DOCS] Add precondition description for spark.shuffle.service.db.backend in running-on-yarn.md #37853

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d8fbfbf
fix
LuciferYang Sep 12, 2022
d8b39ef
fix
LuciferYang Sep 12, 2022
b92cf5c
revert change of docs/spark-standalone.md
LuciferYang Sep 14, 2022
88a253e
add more
LuciferYang Sep 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions docs/running-on-yarn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -852,9 +852,11 @@ The following extra configuration options are available when the shuffle service
<td><code>spark.shuffle.service.db.backend</code></td>
<td>LEVELDB</td>
<td>
To specify the kind of disk-base store used in shuffle service state store, supports `LEVELDB` and `ROCKSDB` now
and `LEVELDB` as default value.
The original data store in `LevelDB/RocksDB` will not be automatically convert to another kind of storage now.
When work-preserving restart is enabled in YARN, this is used to specify the disk-base store used
in shuffle service state store, supports `LEVELDB` and `ROCKSDB` with `LEVELDB` as default value.
The original data store in `LevelDB/RocksDB` will not be automatically converted to another kind
of storage now. The original data store will be retained and the new type data store will be
created when switching storage types.
</td>
<td>3.4.0</td>
</tr>
Expand Down