Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR] Faster parallelisation using joblib instead of mutliprocess #3

Open
Uddiptaatwork opened this issue Sep 14, 2023 · 2 comments
Open
Assignees
Labels
asap Short term extension/improvement/bugfix O(week) enhancement New feature or request

Comments

@Uddiptaatwork
Copy link
Member

In the current implementation of parallelised simulations, a mild lag is caused due to the communication between shell and python while executing the subprocesses in parallel.

  • Use joblib instead for parallelising.
@Uddiptaatwork Uddiptaatwork added the enhancement New feature or request label Sep 14, 2023
@Uddiptaatwork Uddiptaatwork self-assigned this Sep 14, 2023
@Uddiptaatwork Uddiptaatwork added the asap Short term extension/improvement/bugfix O(week) label Oct 13, 2023
@james-alvey-42 james-alvey-42 changed the title Faster parallelisation using joblib instead of mutliprocess [FR] Faster parallelisation using joblib instead of mutliprocess Mar 5, 2024
@james-alvey-42
Copy link
Collaborator

We can make this part of the new peregrine simulator class

@Uddiptaatwork
Copy link
Member Author

  • Also add ray functionality for parallelisation with intra and inter node options.

@Uddiptaatwork Uddiptaatwork added this to the v0.0.2 public release milestone Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
asap Short term extension/improvement/bugfix O(week) enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants