Skip to content

[rllib] Add async remote workers #4253

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ericl merged 15 commits into from
Mar 8, 2019
Merged

[rllib] Add async remote workers #4253

ericl merged 15 commits into from
Mar 8, 2019

Conversation

@ericl
Copy link
Contributor

@ericl ericl commented Mar 5, 2019

What do these changes do?

Add a variant of remote_worker_envs, async_remote_worker_envs, that does dynamic batching as envs complete. It also supports multi-agent.

Related issue number

Closes #4235

@ericl
Copy link
Contributor Author

ericl commented Mar 5, 2019

@bjg2 this unifies the vectorized and multi-agent code paths for remote envs, and adds support for asynchronous execution. Let me know if I missed anything.

@AmplabJenkins
Copy link

AmplabJenkins commented Mar 5, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12539/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Mar 5, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12540/
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Mar 5, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12557/
Test FAILed.

@bjg2
Copy link
Contributor

bjg2 commented Mar 5, 2019

@bjg2 this unifies the vectorized and multi-agent code paths for remote envs, and adds support for asynchronous execution. Let me know if I missed anything.

Looks good! I'm just not sure naming nailed it. Word Vector is omitted and Async is in there even though it might be sync (it is an argument). Maybe RemoveBaseEnv or something like that?

richardliaw
richardliaw reviewed Mar 6, 2019
View reviewed changes
richardliaw
richardliaw reviewed Mar 6, 2019
View reviewed changes
python/ray/rllib/env/async_remote_env.py
return obs


@ray.remote(num_cpus=0)
Copy link
Contributor

@richardliaw richardliaw Mar 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't be surprised if this requires GPU resources in the future (no need to address this rn)

Copy link
Contributor Author

@ericl ericl Mar 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think so too.

@AmplabJenkins
Copy link

AmplabJenkins commented Mar 7, 2019

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12652/
Test FAILed.

@ericl ericl merged commit c7f74db into ray-project:master Mar 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@richardliaw richardliaw richardliaw approved these changes

Assignees

@richardliaw richardliaw

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ericl @AmplabJenkins @bjg2 @richardliaw