Fix attn_bias_add bug. #37147
Merged
Fix attn_bias_add bug. #37147
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
xingfeng01
previously approved these changes
Nov 12, 2021
| @@ -11,10 +11,14 @@ limitations under the License. */ | |||
|
|
|||
| #pragma once | |||
|
|
|||
| #include "paddle/fluid/operators/fused/attn_bias_add.cu.h" | |||
| // #include "paddle/fluid/operators/fused/attn_bias_add.cu.h" | |||
There was a problem hiding this comment.
之前手动实现的bias_add是模仿已有op的实现写的。现在恢复成直接调用已有op实现，对性能没有影响。
| } | ||
| } | ||
|
|
||
| void ComputeBackward(const T* input, const T* weight, const T* d_output, | ||
| T* d_input, T* d_weight, T* d_bias) { | ||
| // void ComputeBackward(const T* input, const T* weight, const T* d_output, |
|
LGTM for LaunchElementwiseCudaKernel |
limin2021
force-pushed
the
fix-bias-add-bug
branch
from
a692a7f
to
c9e3439
Compare
xingfeng01
previously approved these changes
Nov 13, 2021
zkh2016
approved these changes
Nov 15, 2021
xingfeng01
approved these changes
Nov 15, 2021
TCChenlong
approved these changes
Nov 15, 2021
lanxianghit
approved these changes
Nov 16, 2021
zkh2016
pushed a commit
to zkh2016/Paddle
that referenced
this pull request
Nov 16, 2021
fused_attention_op的实现中，使用了bias_add，且其实现是通过使用kernel primitive来实现的，之后kernel primitive的WriteData api接口及函数内部实现发生了更改，将判断越界的逻辑移到了template的参数中，使得调用的分支有错误，产生了越界赋值操作，污染了别的显存空间的内容。具体表现为：test_fused_attention_op_api.py 单次执行基本上不会报错，多次循环执行不同shape的输入，结果计算不对，具有偶发性，bug不易察觉。
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Bug fixes
PR changes
OPs
Describe
问题描述：
fused_attention_op的实现中，使用了bias_add，且其实现是通过使用kernel primitive来实现的，之后kernel primitive的WriteData api接口及函数内部实现发生了更改，将判断越界的逻辑移到了template的参数中，使得调用的分支有错误，产生了越界赋值操作，污染了别的显存空间的内容。具体表现为：test_fused_attention_op_api.py 单次执行基本上不会报错，多次循环执行不同shape的输入，结果计算不对，具有偶发性，bug不易察觉。
解决：
调用处，由
kernel_primitives::WriteData<OutT, VecSize, 1, 1>(out + fix, result, num);
改成：
kernel_primitives::WriteData<OutT, VecSize, 1, 1, true>(out + fix, result, num);
为了避免此类问题的发生，直接将fused_attention_op中对bias_add和reduce的实现改成直接调用paddle已有op的最外层接口进行实现。
2.修改了fused_attention和fused_feedforward op的api参数的english doc（根据 #36972 中chenlong的review进行修改）。
预览（只展示修改的地方）如下：
functional.fused_multi_head_attention：
nn.FusedMultiHeadAttention:
