Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support bias is none for fused_attention op. #37411

Merged
merged 1 commit into from
Nov 23, 2021

Conversation

limin2021
Copy link
Contributor

@limin2021 limin2021 commented Nov 22, 2021

PR types

Bug fixes

PR changes

OPs

Describe

Fix bus of fused_attention op.
1.Add support for bias is none, as well as corresponding unittest.
2.Add assert for input shape limitations in case of API misuse, where
num_head * head_dim should equal to embed_dim.

Results of related unittests:
image

image

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

Copy link
Contributor

@Superjomn Superjomn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

请更新具体修改的 Op 定义,每个Op 是否会用到模型推理中
如:

修改 Op定义如下:

1. xx OP, 将 xxattr 改为,会作用到推理
2. xx Op,增加 xxx attr,不会作用到推理

整体结论:修改的Op [会/不会] 影响推理

@limin2021
Copy link
Contributor Author

limin2021 commented Nov 23, 2021

所做修改对推理的影响:

  1. fused_attention OP中, 将 QKVBias 和LinearBias 的Input attr 改成 .AsDispensable(),不会影响到推理
    整体结论:本次修改 不会影响推理。

训练侧新增的fuse op对推理带来的整体影响,会在后续提一个defusion的pr来专门处理。

Copy link
Contributor

@Superjomn Superjomn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Recently, due to the usage of fusion op in the dygraph program, the jit module will expose the fusion op directlly to inference model, and this will result in the chaotic in the operator granularity in inference model representation.

This requires some new functionality like defusion in jit module, and other members will continue working on this.

Considering this PR is just a bugfix, it is approved.

@limin2021 limin2021 merged commit 1a8786c into PaddlePaddle:develop Nov 23, 2021
limin2021 added a commit to limin2021/Paddle that referenced this pull request Nov 23, 2021
Add support for bias is none for fused_attention op.
lanxianghit pushed a commit that referenced this pull request Nov 24, 2021
…ion op. (#37411) (#37483)

Add support for bias is none for fused_attention op.
lanxianghit pushed a commit that referenced this pull request Nov 29, 2021
…37566) (#37608)

cherry-pick of PR #37566:

Based on #37411, this PR:

    Continue to fix the bugs when bias add is none in static graph for fused_attention op.
    Polish and improve the unittests in test_fused_attention_op_api.py.
Zjq9409 pushed a commit to Zjq9409/Paddle that referenced this pull request Dec 10, 2021
Add support for bias is none for fused_attention op.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants