[rllib] Remove need to pass around registry#2250
Conversation
|
Test FAILed. |
|
I'm very skeptical of this change. RLlib should stick to using Ray APIs. Relying on implementation details like this will make it hard to change the implementation later on. Also, using out-of-band mechanisms like this will compromise properties like fault tolerance. |
|
We're not tied to any particular implementation here -- the requirement is that we support a global registry. Right now it uses redis, like named actors, but we can easily switch it to use the object store or some other mechanism in the future. |
ericl
changed the title
|
I updated this to refactor both this and named actors to use an internal KV interface. |
|
Test FAILed. |
|
@ericl that seems like a good approach for now. Can you say why you introduced the |
|
The motivation for the hook is that we need register calls to work before
ray has initialized (most generally, on import or modules). Hence, we need
to queue puts and flush them post Ray init.
…On Mon, Jun 18, 2018, 10:16 AM Robert Nishihara ***@***.***> wrote:
@ericl <https://github.com/ericl> that seems like a good approach for now.
Can you say why you introduced the post_init_hooks? What is that needed
for and how were you solving that problem before?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2250 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAA6SufoOcjNZzp-mb2rUnM0tfl7jOi8ks5t9-CHgaJpZM4UnGwI>
.
|
robertnishihara
left a comment
robertnishihara
left a comment
There was a problem hiding this comment.
Looks fine to me assuming tests pass.
|
Test FAILed. |
|
Test FAILed. |
|
Test FAILed. |
python/ray/tune/registry.py
Outdated
|
|
||
| from types import FunctionType | ||
|
|
||
| import cloudpickle |
There was a problem hiding this comment.
cloudpickle technically isn't a dependency of Ray (e.g., it is not in our setup.py), so users may not have it installed.
We have our own cloudpickle which you can import with e.g., import ray.cloudpickle as pickle. The point of doing this is that it's very easy to have a cloudpickle version mismatch on different machines, but if we ship our own then we don't run into this problem.
|
Test PASSed. |
|
Test PASSed. |
|
Lint and a couple tests are failing on Travis... |
|
Test PASSed. |
* 'master' of https://github.com/ray-project/ray: (157 commits) Fix build failure while using make -j1. Issue 2257 (ray-project#2279) Cast locator with index type (ray-project#2274) fixing zero length partitions (ray-project#2237) Make actor handles work in Python mode. (ray-project#2283) [xray] Add error table and push error messages to driver through node manager. (ray-project#2256) addressing comments (ray-project#2210) Re-enable some actor tests. (ray-project#2276) Experimental: enable automatic GCS flushing with configurable policy. (ray-project#2266) [xray] Sets good object manager defaults. (ray-project#2255) [tune] Update Trainable doc to expose interface (ray-project#2272) [rllib] Add a simple REST policy server and client example (ray-project#2232) [asv] Pushing to s3 (ray-project#2246) [rllib] Remove need to pass around registry (ray-project#2250) Support multiple availability zones in AWS (fix ray-project#2177) (ray-project#2254) [rllib] Add squash_to_range model option (ray-project#2239) Mitigate randomly building failure: adding gen_local_scheduler_fbs to raylet lib. (ray-project#2271) [rllib] Refactor Multi-GPU for PPO (ray-project#1646) [rllib] Envs for vectorized execution, async execution, and policy serving (ray-project#2170) [Dataframe] Change pandas and ray.dataframe imports (ray-project#1942) [Java] Replace binary rewrite with Remote Lambda Cache (SerdeLambda) (ray-project#2245) ...
4 participants
What do these changes do?
Avoid the need to pass the dict of registered objects around by fetching them from global redis.