Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Use language independent worker initiators #17104

Closed
chenk008 opened this issue Jul 15, 2021 · 2 comments · Fixed by #17401
Closed

[Core] Use language independent worker initiators #17104

chenk008 opened this issue Jul 15, 2021 · 2 comments · Fixed by #17401
Labels
enhancement Request for new feature and/or capability

Comments

@chenk008
Copy link
Contributor

chenk008 commented Jul 15, 2021

Describe your feature request

Currently, ray supports Python/Java/C++ workers, raylet will fork one language worker on demand. Now runtime env can works with python worker, but don't support Java/C++ workers.

How about add an language independent worker initiators? I think we can use setup_worker.py as the initiator , and the initiator will setup worker enviroment (e.g. create a container).

When we want to create k8s native worker Pod , the initiator can create Pod.

The new workflow looks like:
image

@chenk008 chenk008 added the enhancement Request for new feature and/or capability label Jul 15, 2021
@chenk008 chenk008 changed the title Using language independent worker initiators [Core] Using language independent worker initiators Jul 15, 2021
@chenk008 chenk008 added this to the Containerized Workers milestone Jul 15, 2021
@chenk008 chenk008 changed the title [Core] Using language independent worker initiators [Core] Use language independent worker initiators Jul 15, 2021
@jovany-wang
Copy link
Contributor

It makes sense to me.

@raulchen
Copy link
Contributor

Sounds good. And we should move the language-specific code from the worker pool to the initiator. cc @SongGuyang

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Request for new feature and/or capability
Projects
None yet
3 participants