fail to fall back when sparse arrays are passed to MKLDNN-enabled operators.

[zheng-da]

Currently, the MKLDNN-enabled operators, such as convolution and pooling, can't handle sparse arrays correctly. The reason is that the storage inference of these operators doesn't return the right dispatch mode.
The MKLDNN-enabled operators include:

• Activation @azai91
• BatchNorm @zheng-da
• Convolution @ZhennanQin
• Deconvolution @ZhennanQin
• FullyConnected @haojin2
• LRN @azai91
• Pooling @luobao-intel
• Softmax @luobao-intel

We may also need to test the operators below:

• Concat
• Copy
• Sum
• Flatten

@haojin2 @azai91 @pengzhao-intel @TaoLv

[pengzhao-intel]

@zheng-da Could you help provide a test case?
Does cuDNN backend handle these cases?

[frankfliu]

Thank you for submitting the issue! @sandeep-krishnamurthy requesting this be labeled.

[zheng-da]

@eric-haibin-lin @haojin2 could you please provide a test case?

[eric-haibin-lin]

There’s an existing test batch norm training test disabled due to flakyness

[eric-haibin-lin]

concat is tested and fixed.

[haojin2]

I'll take FullyConnected if no one's working on it right now.

[zheng-da]

i created a PR for batchnorm.

[pengzhao-intel]

what else need to cover? @luobao-intel can help

[zheng-da]

@pengzhao-intel @zouluobao everything I listed needs to be covered. The fix for BatchNorm has been merged and @haojin2 is working on FullyConnected. We need to fix other operators.

[pengzhao-intel]

@luobao-intel will handle other OPs :)

[zheng-da]

@pengzhao-intel @luobao-intel we need to fix them and merge the PRs to the v1.3 release, which is probably before the end of next week.

[haojin2]

@zheng-da fullyconnected is done in #11498, if you want to add the MKLDNN backward storage type inference for it please go ahead and do so. Thanks!

[zheng-da]

@haojin2 I thought you have fixed its backward storage type inference.

[haojin2]

@zheng-da checkout https://github.com/apache/incubator-mxnet/pull/11498/files#diff-f2984a387afd7a02b74fe43b9ef50ee1R214

[pengzhao-intel]

@zheng-da related PR are merged. Could you verify if all issues are fixed and close the issue?