-
Notifications
You must be signed in to change notification settings - Fork 290
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Performance] Accelerate GAE #1142
Merged
Merged
Commits on Apr 6, 2023
-
Configuration menu - View commit details
-
Copy full SHA for b40e9a9 - Browse repository at this point
Copy the full SHA b40e9a9View commit details
Commits on Apr 13, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 1669802 - Browse repository at this point
Copy the full SHA 1669802View commit details -
Configuration menu - View commit details
-
Copy full SHA for fe31219 - Browse repository at this point
Copy the full SHA fe31219View commit details
Commits on May 9, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 658380e - Browse repository at this point
Copy the full SHA 658380eView commit details -
Optimize vec_generalized_advantage_estimate
An optimized vecotrized version for the generalized advantage estimation is used in case gamma and lambda are scalars. When handling consecutive trajectories of the form ``` reward = [r00, r01, r02, r03, r10, r11] done = [False, False, False, True, False, False] ``` , `vec_generalized_advantage_estimate` needs to build a giant famma tensor of size [Batch, T, T] with a decayed gamma tensor that suits each trajectory. Thus it needs to allocate a big tensor `[B, T, T]` and do a heavy matrix multiplication. In case gamma and lambda are scalars, this can be optimized by building a single tensor of the form ``` r_transformed = [[r00, r01, r02, r03] [r10, r11, 0, 0]] ``` and applying the gamma filter `[r00 + gamma r01 + gamma ** 2 r02 + ..., ro1 + gamma r02 + gamma ** 2 r03 + ...,]` to calculate the GAE.
Configuration menu - View commit details
-
Copy full SHA for 5a585d0 - Browse repository at this point
Copy the full SHA 5a585d0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 70bd48d - Browse repository at this point
Copy the full SHA 70bd48dView commit details -
* move helper methods to util * reuse existing helper methods * remove wip file
Configuration menu - View commit details
-
Copy full SHA for 268056e - Browse repository at this point
Copy the full SHA 268056eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0c6dc55 - Browse repository at this point
Copy the full SHA 0c6dc55View commit details -
Add hard check that _fast_vec_gae is faster than original implementation
In case gamma and lmbda are scalars, `fast_vec_gae` should be always faster than `vec_generalized_advantage_estimate` if len(T) is large enough.
Configuration menu - View commit details
-
Copy full SHA for 245e68f - Browse repository at this point
Copy the full SHA 245e68fView commit details -
Revert "Add hard check that _fast_vec_gae is faster than original imp…
…lementation" This reverts commit 245e68f.
Configuration menu - View commit details
-
Copy full SHA for cd65976 - Browse repository at this point
Copy the full SHA cd65976View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9c32f01 - Browse repository at this point
Copy the full SHA 9c32f01View commit details -
Configuration menu - View commit details
-
Copy full SHA for 41945f5 - Browse repository at this point
Copy the full SHA 41945f5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 88b9788 - Browse repository at this point
Copy the full SHA 88b9788View commit details -
Improve perfomance of
_inv_pad_sequence
in case there is only one split, _inv_pad_sequence can skip its calculation.
Configuration menu - View commit details
-
Copy full SHA for 291386a - Browse repository at this point
Copy the full SHA 291386aView commit details
Commits on May 10, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 6d5da1b - Browse repository at this point
Copy the full SHA 6d5da1bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 4a634c6 - Browse repository at this point
Copy the full SHA 4a634c6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 914d44b - Browse repository at this point
Copy the full SHA 914d44bView commit details -
Configuration menu - View commit details
-
Copy full SHA for a921333 - Browse repository at this point
Copy the full SHA a921333View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.