[SPARK-38561][K8S][DOCS][3.3] Add doc for Customized Kubernetes Schedulers
#35955
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Mar 24, 2022
docs/running-on-kubernetes.md
Outdated
Member
There was a problem hiding this comment.
What does this refer in Volcano scheduler?
Member
Author
There was a problem hiding this comment.
Set scheduler hints according to configuration
create pg (contains scheduler hints) according to PodGroupTemplate
existing Pod info dynamically.
such as we can set priority according driver.priority, of course, it didn't use it in volcano current implementations
Member
Author
|
@dongjoon-hyun @holdenk Would you mind take a look again on this? the doc for Customized Kubernetes Schedulers for 3.3 branch. |
Member
Author
|
All comments addressed, also sync changes on master: #36001 |
### What changes were proposed in this pull request? This is PR to doc for basic framework capability for Customized Kubernetes Schedulers. ### Why are the changes needed? Guide user how to use spark on kubernetes custom scheduler ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI passed Closes apache#35869 from Yikun/SPARK-38561. Authored-by: Yikun Jiang <[email protected]> Signed-off-by: Holden Karau <[email protected]>
dongjoon-hyun
pushed a commit
that referenced
this pull request
Mar 29, 2022
…dulers` ### What changes were proposed in this pull request? This is PR to doc for basic framework capability for Customized Kubernetes Schedulers. ### Why are the changes needed? Guide user how to use spark on kubernetes custom scheduler ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI passed Closes #35955 from Yikun/SPARK-38561-3.3. Authored-by: Yikun Jiang <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
Member
Author
|
@dongjoon-hyun Thanks! |
srowen
pushed a commit
that referenced
this pull request
Apr 2, 2022
…nfiguration and fix typos ### What changes were proposed in this pull request? Add note for specific scheduler configuration and fix typos ### Why are the changes needed? We addressed some comments on #35955 when backport to 3.3 branch. ### Does this PR introduce _any_ user-facing change? No, doc only ### How was this patch tested?  SKIP_API=1 bundle exec jekyll serve --watch Closes #36001 from Yikun/SPARK-38561-FOLLOWUP. Authored-by: Yikun Jiang <[email protected]> Signed-off-by: Sean Owen <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This is PR to doc for basic framework capability for Customized Kubernetes Schedulers.
Why are the changes needed?
Guide user how to use spark on kubernetes custom scheduler
Does this PR introduce any user-facing change?
No
How was this patch tested?
CI passed