support 1D and 3D arrays in MKLDNN.

[zheng-da]

MKLDNN currently only supports 2D and 4D arrays. As @safrooze reported that computation on 1D and 3D arrays is much slower than on 2D and 4D arrays. We can easily accelerate performance by adding a dummy dimension so that the computation can take advantage of MKLDNN. Right now users have to do it manually. We should improve the MKLDNN integration and accelerate 1D and 3D arrays automatically.
@pengzhao-intel @TaoLv

[TaoLv]

I think mkldnn already has support for 3D arrays now. You can find the formats definition (ncdhw or nCdhw16c) here: https://github.com/intel/mkl-dnn/blob/master/include/mkldnn_types.h#L134

[sandeep-krishnamurthy]

Please see more details by Sina here - #11909

[zheng-da]

I know mkldnn has 3D arrays, but not all mkldnn operators support 3D arrays. I believe the current implementation of mkldnn integration only allows 2D and 4D arrays.

For example, mkldnn convolution only supports 2D kernel on 4D arrays. Currently, 1D convolution actually calls the native implementation. But if we add a fake dim to turn 1D conv into 2D conv, we can make substantial speedup. I think this applies to many other operators.

[safrooze]

3D tensors are common in audio signals, e.g. WaveNet. I specifically ran into this issue with WaveNet inference on CPU.

[TaoLv]

@safrooze thanks for the information. mkldnn has already supported 3D convolution. The input of it may be 5D tensor if we take batch size and channel size into account. I think 1D convolution is still under development for mkldnn. As @zheng-da mentioned, currently we can add a fake dim to turn it into 2D convolution to benefit from mkldnn.
@zheng-da Right, we need spend some effort to integrate mkldnn 3D convolution into mxnet now.

[zheng-da]

@TaoLv I think the task should be more general. The integration of MKLDNN should support 1D and 3D input arrays.
Of course, it's also great to integrate mkldnn 3D convolution.

[safrooze]

@TaoLv By 3D tensor, I meant something with (batch_size, channels, width), which is a 1D convolution.

[pengzhao-intel]

@zheng-da @safrooze It's on the plan to integrate 1D conv into MXNet soon by @xinyu-intel

[pengzhao-intel]

The request features are implemented :)
Closing now. Feel free to reopen if there is another issue.