Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up import time #18

Open
AdamGleave opened this issue May 25, 2018 · 0 comments
Open

Speed up import time #18

AdamGleave opened this issue May 25, 2018 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@AdamGleave
Copy link
Member

It currently takes around 2 seconds to load the pirl module. Most of the time this isn't a big deal, but due to a combination of:

  1. We restart each Ray worker after a single call to work around TensorFlow/CUDA brokenness;
  2. A complete experiment can consist of 100s of calls;
    this adds up to a significant overhead when the results are mostly cached. (The overhead is still there for non-cached calls but is negligible compared to the cost of e.g. running an RL/IRL algorithm.)

I'm not sure how much we can realistically shave off. Just importing Ray takes ~0.5s, and we can't avoid that. Importing TensorFlow seems to be a big culprit; we actually can delay that, but we'd need to make imports in pirl.agents and pirl.irl be on-demand. (Currently they all get loaded by the config.)

@AdamGleave AdamGleave added enhancement New feature or request good first issue Good for newcomers labels May 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant