[FRONTEND][TENSORFLOW] Some bug fixes for tensorflow NCHW data_format

[vv1133]

update #3008

• fix tensorflow depthwise_conv2d nchw
• fix tensorflow biasadd nchw
• add tensorflow tests: depthwise_conv2d, biasadd

[vv1133]

Please review @srkreddy1238 @Laurawly @Huyuwei

[u99127]

Can you please explain further what the bug is and how you are proposing to fix it ?

[vv1133]

Can you please explain further what the bug is and how you are proposing to fix it ?

• TensorFlow bias_add with data_format NCHW should reshape inputs[1] to [c, 1, 1] first like below.

  if attr['data_format'] == 'NCHW':
      bias_shape = attr['_input_shapes'][inputs[1]]
      bias = _op.reshape(inputs[1], newshape=tuple([bias_shape, 1, 1]))
  else:
      bias = inputs[1]
  return _op.add(inputs[0], bias)

• The previous code of transposing weight with data_format NCHW in frotend/tensorflow.py function _conv:

    tmp_shape = [tmp_shape[ii] for ii in (3, 2, 0, 1)]
    inputs[1] = _sym.transpose(inputs[1], axes=(3, 2, 0, 1))

TensorFlow DepthwiseConv2d weights shape: [H, W, C, DEPTH_MULT]
But refer to "./nnvm/src/top/nn/convolution.cc: Conv2DInferShape" weights_shape should transpose to [C, DEPTH_MULT, H, W]. So the transpose shape should be [2, 3, 0, 1] when the opname is "depthwise".

[srkreddy1238]

Thanks @vv1133 .