Implement the grad and enhance the cache of norm_convolution fusion ops. #36168
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
JamesLim-sy
reviewed
Sep 29, 2021
| handle, args_.filter_desc.desc(), args_.out_desc.desc(), | ||
| args_.conv_desc.desc(), args_.in_desc.desc(), dgrad_algo_, | ||
| &workspace_size)); | ||
| return RoundUp(workspace_size, 512); |
There was a problem hiding this comment.
这里如果也cache住的话，wgrad_op->GetWorkspaceSizeInBytes(ctx.cudnn_handle())得到的dweight_workspace_size和这里得到的dgrad_workspace_size取最大值，这里的开销也可以减掉了。
There was a problem hiding this comment.
这部分开销不算大，而且cache的对象不一样。Fused方式是需要把整个CudnnFusionOp都cache下来，主要目的是cache住 FusedOpsPlan。后续可以再验证下这部分对性能是否有影响，若有则再进一步cache吧。
This was referenced Oct 9, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Performance optimization
PR changes
OPs
Describe
cudnnFusedOpsPlan_t的CudnnNormConvolution融合计算的写法，包括：CudnnNormConvolutionGradCudnnFusionOpCache，将生成的CudnnFusionOpcache起来，避免每次调用cudnnMakeFusedOpsPlan造成的巨大的CPU开销