[SPARK-38562][K8S][DOCS] Add doc for Volcano scheduler
#35870
Conversation
holdenk
left a comment
holdenk
left a comment
There was a problem hiding this comment.
Overall LGTM pending the existing suggestions.Would be really good to get the documentation in along with the feature for 3.3.
|
@holdenk Thanks for your suggestion, will address soon! |
|
@dongjoon-hyun comments addressed, Thanks! |
|
@dongjoon-hyun @holdenk Would you mind take a look again on this? |
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
@Yikun . I saw that you added the following in Volcano website. However, Volcano website is out of Apache Spark's control. We cannot depend on it. Please give a specific instruction for Volcano 1.5.1 installation.
You can also replace master of above url with specific tag/branch (such as release-1.5 branch for latest v1.5.x version, v1.5.1 tag for v1.5.1 version) to install Volcano with specific version.
docs/running-on-kubernetes.md
Outdated
There was a problem hiding this comment.
As a side note, v1beta1 could be a blocker for the users until Volcano community makes it official. I mentioned this before to you, @Yikun . The multi-arch image support is also another TODO, IIRC.
There was a problem hiding this comment.
I mentioned this before to you, @Yikun . The multi-arch image support is also another TODO, IIRC.
also cc @william-wang are working on this, will ready soon.
There was a problem hiding this comment.
@dongjoon-hyun Thanks for the reminding, we are already working on the multi-arch and will make it ready as soon as possible. For the v1beta1, I understand your concern, we have stayed in v1beta1 for a long time, it is stable and there are 50+ enterprise users use Volcano in their production environment. We have planed to upgrade it to v1 version this year and v1 will also keep complete compatibility with v1beta1 :)
Yikun
force-pushed
the
SPARK-38562
branch
2 times, most recently
from
0e9f1b9 to
318efa3
Compare
|
@dongjoon-hyun Thanks for review, addressed. 75accf8 |
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Thank you for your patience and updates, @Yikun . And, thank you all!
Merged to master.
### What changes were proposed in this pull request? This is PR to doc for volcano scheduler capability for Spark on K8S. ### Why are the changes needed? Guide user how to use volcano as spark on kubernetes custom scheduler ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI passed Closes #35870 from Yikun/SPARK-38562. Authored-by: Yikun Jiang <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 80deb24) Signed-off-by: Dongjoon Hyun <[email protected]>
|
I backported this to branch-3.3, too. |
|
@dongjoon-hyun thank you for all your help! Thanks all! @holdenk @william-wang @martin-g |
5 participants
What changes were proposed in this pull request?
This is PR to doc for volcano scheduler capability for Spark on K8S.
Why are the changes needed?
Guide user how to use volcano as spark on kubernetes custom scheduler
Does this PR introduce any user-facing change?
No
How was this patch tested?
CI passed