Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AMP OP&Test] Append bf16/fp16 support 4 elementwise_max #51151

Merged
merged 11 commits into from
Mar 14, 2023

Conversation

piDack
Copy link
Contributor

@piDack piDack commented Mar 3, 2023

PR types

New features

PR changes

Others

Describe

  • Append bf16/fp16 ut support 4 elementwise_max
  • fp16/bf16相关支持和API Check在pr40590实现
  • fp16中max_relative_error=0.005参考了pr40590
  • 在bf16中,当x,y都为0-dim时,numeric解是不符合要求的,y_pos计算出来比fp32的时候大0.001,y_neg比fp32的时候小0.01,因此bf16的时候计算出来numeric值偏大,因此取delta为0.05

@paddle-bot
Copy link

paddle-bot bot commented Mar 3, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@@ -120,9 +120,13 @@ def test_check_output(self):

def test_check_grad_normal(self):
if hasattr(self, 'attrs'):
self.check_grad(['X', 'Y'], 'Out', check_eager=False)
self.check_grad(
['X', 'Y'], 'Out', numeric_grad_delta=0.01, check_eager=False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

为什么要指定numeric_grad_delta

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不指定数值解错误

)


class TestElementwiseFP16Op(TestElementwiseOp):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改一下类名,用TestElementwiseMaxFP16Op代替

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -131,6 +178,53 @@ def test_check_grad_ingore_y(self):
self.check_grad(['X'], 'Out', no_grad_set=set('Y'))


class TestElementwiseMaxBF16Op_ZeroDim1(TestElementwiseBF16Op):
def setUp(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

很多子类的setUp都复用了基类的一些代码,建议不同点用init_data等函数单独提取出来,减少重复代码

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@Vvsmile
Copy link
Contributor

Vvsmile commented Mar 13, 2023

PR的Describe里面请多补充FP16/BF16/API Check已在其他PR中完成,以及不同于规范的点及原因,如FP16中max_relative_error=0.005及BF16中delta=0.05

@piDack
Copy link
Contributor Author

piDack commented Mar 13, 2023

PR的Describe里面请多补充FP16/BF16/API Check已在其他PR中完成,以及不同于规范的点及原因,如FP16中max_relative_error=0.005及BF16中delta=0.05
👌

@Vvsmile
Copy link
Contributor

Vvsmile commented Mar 14, 2023

LGTM

Copy link
Contributor

@ZzSean ZzSean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants