[oneDNN] Initial bf16 amp integration

[arlesniak]

PR types

New features

PR changes

APIs

Describe

Initial bf16 amp integration

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[arlesniak]

I restarted PR-CI-musl a couple of times but every time it has the following error:
fatal: unable to access 'https://github.com/PaddlePaddle/Paddle.git/': Failed connect to 172.19.57.45:3128; Connection timed out

[luotao1]

I restart PR_CI_MUSL and it runs OK now.

[arogowie-intel]

I'm sorry I couldn't review your code thoroughly since it's too hard and time consuming for me without having more knowledge about actual implementation for this topic. Thus some of my comments may be inaccurate.

[arlesniak]

@arogowie-intel Thank you for the comments

[arlesniak]

@arogowie-intel Thank you for your comments.

[luotao1]

LGTM

[luotao1]

2021-03-20 00:31:12 ****************

2021-03-20 00:31:12 0. You must have one RD (XiaoguangHu01 or lanxianghit) and one TPM (saxon-zh or jzhang533 or swtkiwi or Heeenrrry or TCChenlong) approval for the api change for the management reason of API interface.

2021-03-20 00:31:12 1. You must have one TPM (saxon-zh or jzhang533 or swtkiwi or Heeenrrry or TCChenlong) approval for the api change for the management reason of API document.

2021-03-20 00:31:12 

2021-03-20 00:31:12 There are 2 approved errors.

2021-03-20 00:31:12 ****************

2021-03-20 00:31:12 API Difference is: 

2021-03-20 00:31:12 + paddle.fluid.contrib.mixed_precision.AutoMixedPrecisionListsBF16 (paddle.fluid.contrib.mixed_precision.bf16.amp_lists.AutoMixedPrecisionListsBF16, ('document', '1aea006c5da7d91bb0e8b7c6dba32c01'))

2021-03-20 00:31:12 + paddle.fluid.contrib.mixed_precision.AutoMixedPrecisionListsBF16.__init__ (ArgSpec(args=['self', 'custom_bf16_list', 'custom_fp32_list', 'custom_fp32_varnames'], varargs=None, keywords=None, defaults=(None, None, None)), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))

2021-03-20 00:31:12 + paddle.fluid.contrib.mixed_precision.bf16_guard (ArgSpec(args=[], varargs=None, keywords=None, defaults=None), ('document', 'd2a4daf8c41dabdf686dba448a4d23ff'))

2021-03-20 00:31:12 + paddle.fluid.contrib.mixed_precision.rewrite_program_bf16 (ArgSpec(args=['main_prog', 'amp_lists', 'use_bf16_guard'], varargs=None, keywords=None, defaults=(None, False)), ('document', '9849657dba7a5c8a2e2b98894183276d'))

2021-03-20 00:31:12 + paddle.fluid.contrib.mixed_precision.convert_float_to_uint16 (ArgSpec(args=['in_list'], varargs=None, keywords=None, defaults=None), ('document', '6adf97f83acf6453d4a6a4b1070f3754'))

[swtkiwi]

LGTM