[CPU BF16] Bfloat16 inference optimizations #2633
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alexey-varyzgin
requested review from
GlebKazantaev and
ilyachur
as code owners
alexey-varyzgin
requested review from
a team,
maxnick and
yury-intel
and removed request for
a team,
ilyachur and
GlebKazantaev
maxnick
force-pushed
the
feature/bf16_2021_3
branch
from
69e00f2 to
23a23f4
Compare
alexey-varyzgin
force-pushed
the
feature/bf16_2021_3
branch
from
ed07d19 to
532c2c8
Compare
maxnick
force-pushed
the
feature/bf16_2021_3
branch
4 times, most recently
from
aa1aeda to
3733b56
Compare
alexey-varyzgin
force-pushed
the
feature/bf16_2021_3
branch
from
edc714f to
17b8b24
Compare
maxnick
force-pushed
the
feature/bf16_2021_3
branch
from
17b8b24 to
e57e960
Compare
alexey-varyzgin
force-pushed
the
feature/bf16_2021_3
branch
from
344e3bc to
873a244
Compare
maxnick
force-pushed
the
feature/bf16_2021_3
branch
2 times, most recently
from
193f254 to
1f74934
Compare
maxnick
force-pushed
the
feature/bf16_2021_3
branch
from
1f74934 to
9cfd7cc
Compare
…tridedSlice operation.
…ring has been done.
…ferent mem orders.
…luginConfiguration function.
maxnick
force-pushed
the
feature/bf16_2021_3
branch
from
42daac5 to
9ebc520
Compare
maxnick
approved these changes
Nov 25, 2020
|
BTW I see that this PR still adds empty inference-engine/src/legacy_api/src/ngraph_ops/interp.cpp file. Lets remove it |
Actually the file itself is not empty. Github shows empty changes by some reason. Apparently because the file was changed, but the changes were removed in different commits. |
dmitry-gorokhov
approved these changes
Nov 27, 2020
evolosen
pushed a commit
to evolosen/openvino
that referenced
this pull request
Dec 3, 2020
* [CPU BF16] Greedy mode was added * [IE TESTS][BF16] Added support for operations with bf16 precision in the single layer tests. * Added cpu specific bfloat16 single layer tests for the jit_eltwise primitive. * [CPU TESTS] Activation and logical single layer tests fixes. * [IE TESTS] Fix activation single layer tests run. * [IE TESTS][CPU] CPUTestBase further refactoring. * [CPU BF16] Support for Bfloat16 type was added to the MVN layer. (#3) * [CPU BF16] MVN layer bfloat16 compatibility. * [CPU BF16] MVN bfloat16 minor fixes. * [CPU BF16] MVN node exception about BF16 support replaced with precision redefinition. * [CPU BF16] MVN layer bloat16 support fixed for quantization operations and blocking layout. * [CPU] Input and output precision checks were added to MVN layer. * [IE TESTS][CPU BF16] Most of the bloat16 tests have been fixed. * Bf16 crop layer (#4) * [IE TESTS][CPU] Cpu specific test for the Crop layer has been created. * [IE TESTS][CPU] Deprecated Crop single layer test removed. * [CPU BF16] Bfloat16 precision was added to the Crop layer. * [CPU BF16] Crop layer minor code improvements. * [IE TESTS][CPU] Crop layer test added 2D tensor tests. * [IE TESTS][CPU] Crop layer test, obsolete comment removed. * [IE TESTS][CPU] Fixed CropIE include path. * Crop test fix for older gcc compiler. * [CPU BF16] Reduce layer extended with bfloat16 support. * [IE TESTS][CPU] CPU specific single layer test for Reduce operation. * BF16 optimized layers * [CPU BF16] Bfloat16 custom type added to the MKLDNN plugin. * [CPU BF16] Mem alignment to 16 bytes added to bfloat16 class union. * [IE TESTS][CPU] Permute cpu specific single layer test and minor cpu tests fixes * MVN cpu single layer tests extended with nhwc ndhwc layouts. * Mod mode removed from Eltwise cpu single layer test. * Permute cpu specific single layer test. * Smoke keyword was added to the CPU single layer tests. * Normalize node was modified for BF16 support * [CPU BF16] The RegionYolo layer has been extended with the bfloat16 type support. * Resample node was extended with BF16 * Select layer was enabled with BF16 * psroi supports bf16 (#7) * reorders replaces converts (#9) * BF16 planar pooling was enabled * [CPU BF16] Cpu_convert added to the RegionYOLO node. * [IE TESTS][CPU] Crop single layer test has been rewritten using the StridedSlice operation. * [IE TESTS][CPU] Covert layer test extended with bf16 precision. * [CPU BF16] The bfloat16 class was renamed bfloat16_t and some refactoring has been done. * [CPU BF16] RegionYOLO and Softmax were aligned with the review. * [IE TESTS CPU] CPU single layer tests refactored according to the review suggestions. * [IE TESTS CPU] The Reduce CPU single layer test was extended with different mem orders. * [IE TESTS CPU] Minor fixes after the review. * [IE TESTS CPU] Common plugin configuration has been moved to PreparePluginConfiguration function. * Minor changes after review * StridedSlice, Select, ScaleShift notes were resolved * Fixes to the Reduce operation cpu test and minor fixes related to the review. * GPU eltwise tests fix. * psroi unrolled to the primary state; code clean (#12) * PSROIPooling layer with C++ optimizations * Minor fix for compatibility with CPUTestsBase for fuse_permute_reorder test. * Code clean & psroi rollbacked Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Yury Gaydaychuk <[email protected]>
mryzhov
pushed a commit
to mryzhov/openvino
that referenced
this pull request
Dec 11, 2020
* [CPU BF16] Greedy mode was added * [IE TESTS][BF16] Added support for operations with bf16 precision in the single layer tests. * Added cpu specific bfloat16 single layer tests for the jit_eltwise primitive. * [CPU TESTS] Activation and logical single layer tests fixes. * [IE TESTS] Fix activation single layer tests run. * [IE TESTS][CPU] CPUTestBase further refactoring. * [CPU BF16] Support for Bfloat16 type was added to the MVN layer. (#3) * [CPU BF16] MVN layer bfloat16 compatibility. * [CPU BF16] MVN bfloat16 minor fixes. * [CPU BF16] MVN node exception about BF16 support replaced with precision redefinition. * [CPU BF16] MVN layer bloat16 support fixed for quantization operations and blocking layout. * [CPU] Input and output precision checks were added to MVN layer. * [IE TESTS][CPU BF16] Most of the bloat16 tests have been fixed. * Bf16 crop layer (#4) * [IE TESTS][CPU] Cpu specific test for the Crop layer has been created. * [IE TESTS][CPU] Deprecated Crop single layer test removed. * [CPU BF16] Bfloat16 precision was added to the Crop layer. * [CPU BF16] Crop layer minor code improvements. * [IE TESTS][CPU] Crop layer test added 2D tensor tests. * [IE TESTS][CPU] Crop layer test, obsolete comment removed. * [IE TESTS][CPU] Fixed CropIE include path. * Crop test fix for older gcc compiler. * [CPU BF16] Reduce layer extended with bfloat16 support. * [IE TESTS][CPU] CPU specific single layer test for Reduce operation. * BF16 optimized layers * [CPU BF16] Bfloat16 custom type added to the MKLDNN plugin. * [CPU BF16] Mem alignment to 16 bytes added to bfloat16 class union. * [IE TESTS][CPU] Permute cpu specific single layer test and minor cpu tests fixes * MVN cpu single layer tests extended with nhwc ndhwc layouts. * Mod mode removed from Eltwise cpu single layer test. * Permute cpu specific single layer test. * Smoke keyword was added to the CPU single layer tests. * Normalize node was modified for BF16 support * [CPU BF16] The RegionYolo layer has been extended with the bfloat16 type support. * Resample node was extended with BF16 * Select layer was enabled with BF16 * psroi supports bf16 (#7) * reorders replaces converts (#9) * BF16 planar pooling was enabled * [CPU BF16] Cpu_convert added to the RegionYOLO node. * [IE TESTS][CPU] Crop single layer test has been rewritten using the StridedSlice operation. * [IE TESTS][CPU] Covert layer test extended with bf16 precision. * [CPU BF16] The bfloat16 class was renamed bfloat16_t and some refactoring has been done. * [CPU BF16] RegionYOLO and Softmax were aligned with the review. * [IE TESTS CPU] CPU single layer tests refactored according to the review suggestions. * [IE TESTS CPU] The Reduce CPU single layer test was extended with different mem orders. * [IE TESTS CPU] Minor fixes after the review. * [IE TESTS CPU] Common plugin configuration has been moved to PreparePluginConfiguration function. * Minor changes after review * StridedSlice, Select, ScaleShift notes were resolved * Fixes to the Reduce operation cpu test and minor fixes related to the review. * GPU eltwise tests fix. * psroi unrolled to the primary state; code clean (#12) * PSROIPooling layer with C++ optimizations * Minor fix for compatibility with CPUTestsBase for fuse_permute_reorder test. * Code clean & psroi rollbacked Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Yury Gaydaychuk <[email protected]>
mryzhov
pushed a commit
to mryzhov/openvino
that referenced
this pull request
Dec 16, 2020
* [CPU BF16] Greedy mode was added * [IE TESTS][BF16] Added support for operations with bf16 precision in the single layer tests. * Added cpu specific bfloat16 single layer tests for the jit_eltwise primitive. * [CPU TESTS] Activation and logical single layer tests fixes. * [IE TESTS] Fix activation single layer tests run. * [IE TESTS][CPU] CPUTestBase further refactoring. * [CPU BF16] Support for Bfloat16 type was added to the MVN layer. (#3) * [CPU BF16] MVN layer bfloat16 compatibility. * [CPU BF16] MVN bfloat16 minor fixes. * [CPU BF16] MVN node exception about BF16 support replaced with precision redefinition. * [CPU BF16] MVN layer bloat16 support fixed for quantization operations and blocking layout. * [CPU] Input and output precision checks were added to MVN layer. * [IE TESTS][CPU BF16] Most of the bloat16 tests have been fixed. * Bf16 crop layer (#4) * [IE TESTS][CPU] Cpu specific test for the Crop layer has been created. * [IE TESTS][CPU] Deprecated Crop single layer test removed. * [CPU BF16] Bfloat16 precision was added to the Crop layer. * [CPU BF16] Crop layer minor code improvements. * [IE TESTS][CPU] Crop layer test added 2D tensor tests. * [IE TESTS][CPU] Crop layer test, obsolete comment removed. * [IE TESTS][CPU] Fixed CropIE include path. * Crop test fix for older gcc compiler. * [CPU BF16] Reduce layer extended with bfloat16 support. * [IE TESTS][CPU] CPU specific single layer test for Reduce operation. * BF16 optimized layers * [CPU BF16] Bfloat16 custom type added to the MKLDNN plugin. * [CPU BF16] Mem alignment to 16 bytes added to bfloat16 class union. * [IE TESTS][CPU] Permute cpu specific single layer test and minor cpu tests fixes * MVN cpu single layer tests extended with nhwc ndhwc layouts. * Mod mode removed from Eltwise cpu single layer test. * Permute cpu specific single layer test. * Smoke keyword was added to the CPU single layer tests. * Normalize node was modified for BF16 support * [CPU BF16] The RegionYolo layer has been extended with the bfloat16 type support. * Resample node was extended with BF16 * Select layer was enabled with BF16 * psroi supports bf16 (#7) * reorders replaces converts (#9) * BF16 planar pooling was enabled * [CPU BF16] Cpu_convert added to the RegionYOLO node. * [IE TESTS][CPU] Crop single layer test has been rewritten using the StridedSlice operation. * [IE TESTS][CPU] Covert layer test extended with bf16 precision. * [CPU BF16] The bfloat16 class was renamed bfloat16_t and some refactoring has been done. * [CPU BF16] RegionYOLO and Softmax were aligned with the review. * [IE TESTS CPU] CPU single layer tests refactored according to the review suggestions. * [IE TESTS CPU] The Reduce CPU single layer test was extended with different mem orders. * [IE TESTS CPU] Minor fixes after the review. * [IE TESTS CPU] Common plugin configuration has been moved to PreparePluginConfiguration function. * Minor changes after review * StridedSlice, Select, ScaleShift notes were resolved * Fixes to the Reduce operation cpu test and minor fixes related to the review. * GPU eltwise tests fix. * psroi unrolled to the primary state; code clean (#12) * PSROIPooling layer with C++ optimizations * Minor fix for compatibility with CPUTestsBase for fuse_permute_reorder test. * Code clean & psroi rollbacked Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Yury Gaydaychuk <[email protected]>
mryzhov
pushed a commit
to mryzhov/openvino
that referenced
this pull request
Jan 14, 2021
* [CPU BF16] Greedy mode was added * [IE TESTS][BF16] Added support for operations with bf16 precision in the single layer tests. * Added cpu specific bfloat16 single layer tests for the jit_eltwise primitive. * [CPU TESTS] Activation and logical single layer tests fixes. * [IE TESTS] Fix activation single layer tests run. * [IE TESTS][CPU] CPUTestBase further refactoring. * [CPU BF16] Support for Bfloat16 type was added to the MVN layer. (#3) * [CPU BF16] MVN layer bfloat16 compatibility. * [CPU BF16] MVN bfloat16 minor fixes. * [CPU BF16] MVN node exception about BF16 support replaced with precision redefinition. * [CPU BF16] MVN layer bloat16 support fixed for quantization operations and blocking layout. * [CPU] Input and output precision checks were added to MVN layer. * [IE TESTS][CPU BF16] Most of the bloat16 tests have been fixed. * Bf16 crop layer (#4) * [IE TESTS][CPU] Cpu specific test for the Crop layer has been created. * [IE TESTS][CPU] Deprecated Crop single layer test removed. * [CPU BF16] Bfloat16 precision was added to the Crop layer. * [CPU BF16] Crop layer minor code improvements. * [IE TESTS][CPU] Crop layer test added 2D tensor tests. * [IE TESTS][CPU] Crop layer test, obsolete comment removed. * [IE TESTS][CPU] Fixed CropIE include path. * Crop test fix for older gcc compiler. * [CPU BF16] Reduce layer extended with bfloat16 support. * [IE TESTS][CPU] CPU specific single layer test for Reduce operation. * BF16 optimized layers * [CPU BF16] Bfloat16 custom type added to the MKLDNN plugin. * [CPU BF16] Mem alignment to 16 bytes added to bfloat16 class union. * [IE TESTS][CPU] Permute cpu specific single layer test and minor cpu tests fixes * MVN cpu single layer tests extended with nhwc ndhwc layouts. * Mod mode removed from Eltwise cpu single layer test. * Permute cpu specific single layer test. * Smoke keyword was added to the CPU single layer tests. * Normalize node was modified for BF16 support * [CPU BF16] The RegionYolo layer has been extended with the bfloat16 type support. * Resample node was extended with BF16 * Select layer was enabled with BF16 * psroi supports bf16 (#7) * reorders replaces converts (#9) * BF16 planar pooling was enabled * [CPU BF16] Cpu_convert added to the RegionYOLO node. * [IE TESTS][CPU] Crop single layer test has been rewritten using the StridedSlice operation. * [IE TESTS][CPU] Covert layer test extended with bf16 precision. * [CPU BF16] The bfloat16 class was renamed bfloat16_t and some refactoring has been done. * [CPU BF16] RegionYOLO and Softmax were aligned with the review. * [IE TESTS CPU] CPU single layer tests refactored according to the review suggestions. * [IE TESTS CPU] The Reduce CPU single layer test was extended with different mem orders. * [IE TESTS CPU] Minor fixes after the review. * [IE TESTS CPU] Common plugin configuration has been moved to PreparePluginConfiguration function. * Minor changes after review * StridedSlice, Select, ScaleShift notes were resolved * Fixes to the Reduce operation cpu test and minor fixes related to the review. * GPU eltwise tests fix. * psroi unrolled to the primary state; code clean (#12) * PSROIPooling layer with C++ optimizations * Minor fix for compatibility with CPUTestsBase for fuse_permute_reorder test. * Code clean & psroi rollbacked Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Yury Gaydaychuk <[email protected]>
jiwaszki
pushed a commit
to akuporos/openvino
that referenced
this pull request
Jan 15, 2021
* [CPU BF16] Greedy mode was added * [IE TESTS][BF16] Added support for operations with bf16 precision in the single layer tests. * Added cpu specific bfloat16 single layer tests for the jit_eltwise primitive. * [CPU TESTS] Activation and logical single layer tests fixes. * [IE TESTS] Fix activation single layer tests run. * [IE TESTS][CPU] CPUTestBase further refactoring. * [CPU BF16] Support for Bfloat16 type was added to the MVN layer. (#3) * [CPU BF16] MVN layer bfloat16 compatibility. * [CPU BF16] MVN bfloat16 minor fixes. * [CPU BF16] MVN node exception about BF16 support replaced with precision redefinition. * [CPU BF16] MVN layer bloat16 support fixed for quantization operations and blocking layout. * [CPU] Input and output precision checks were added to MVN layer. * [IE TESTS][CPU BF16] Most of the bloat16 tests have been fixed. * Bf16 crop layer (#4) * [IE TESTS][CPU] Cpu specific test for the Crop layer has been created. * [IE TESTS][CPU] Deprecated Crop single layer test removed. * [CPU BF16] Bfloat16 precision was added to the Crop layer. * [CPU BF16] Crop layer minor code improvements. * [IE TESTS][CPU] Crop layer test added 2D tensor tests. * [IE TESTS][CPU] Crop layer test, obsolete comment removed. * [IE TESTS][CPU] Fixed CropIE include path. * Crop test fix for older gcc compiler. * [CPU BF16] Reduce layer extended with bfloat16 support. * [IE TESTS][CPU] CPU specific single layer test for Reduce operation. * BF16 optimized layers * [CPU BF16] Bfloat16 custom type added to the MKLDNN plugin. * [CPU BF16] Mem alignment to 16 bytes added to bfloat16 class union. * [IE TESTS][CPU] Permute cpu specific single layer test and minor cpu tests fixes * MVN cpu single layer tests extended with nhwc ndhwc layouts. * Mod mode removed from Eltwise cpu single layer test. * Permute cpu specific single layer test. * Smoke keyword was added to the CPU single layer tests. * Normalize node was modified for BF16 support * [CPU BF16] The RegionYolo layer has been extended with the bfloat16 type support. * Resample node was extended with BF16 * Select layer was enabled with BF16 * psroi supports bf16 (#7) * reorders replaces converts (#9) * BF16 planar pooling was enabled * [CPU BF16] Cpu_convert added to the RegionYOLO node. * [IE TESTS][CPU] Crop single layer test has been rewritten using the StridedSlice operation. * [IE TESTS][CPU] Covert layer test extended with bf16 precision. * [CPU BF16] The bfloat16 class was renamed bfloat16_t and some refactoring has been done. * [CPU BF16] RegionYOLO and Softmax were aligned with the review. * [IE TESTS CPU] CPU single layer tests refactored according to the review suggestions. * [IE TESTS CPU] The Reduce CPU single layer test was extended with different mem orders. * [IE TESTS CPU] Minor fixes after the review. * [IE TESTS CPU] Common plugin configuration has been moved to PreparePluginConfiguration function. * Minor changes after review * StridedSlice, Select, ScaleShift notes were resolved * Fixes to the Reduce operation cpu test and minor fixes related to the review. * GPU eltwise tests fix. * psroi unrolled to the primary state; code clean (#12) * PSROIPooling layer with C++ optimizations * Minor fix for compatibility with CPUTestsBase for fuse_permute_reorder test. * Code clean & psroi rollbacked Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Maksim Kutakov <[email protected]> Co-authored-by: Yury Gaydaychuk <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.