Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature request] parallel replay buffer for DQN #604

Closed
flodorner opened this issue Dec 6, 2019 · 0 comments · Fixed by #608
Closed

[feature request] parallel replay buffer for DQN #604

flodorner opened this issue Dec 6, 2019 · 0 comments · Fixed by #608
Labels
enhancement New feature or request

Comments

@flodorner
Copy link

When training small models with low dimensional state spaces, the replay buffer seems to be a bottleneck in DQN. Sampling and updating the buffer for multiple indexes can somewhat easily be vectorized using numpy and can lead to significant speedups of these functions for large batch sizes.
I have implemented the improvement here and wanted to create this issue before starting my pull request, in order to get feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
2 participants