src/operator/./bilinear_sampler-inl.h:105: Have not implemented the data req combinations! gdata_req=0 ggrid_req=1

[chowkamlee81]

iam in the path of semantic segmentation. I want to warp encoder layer results of previous frame to current frame using optical flow based CNN. Basically i had key_img, cur_img and feat_key to warp the optical flow results and pass to the decoder.

Problem with mx.sym.BilinearSampler symbol code. KIndly provide the inputs.

Below is my symbol.

def get_cur_train_symbol( num_classes):
data_cur = mx.sym.Variable(name="data_cur")
data_key = mx.sym.Variable(name="data_key")
conv_feat = mx.sym.Variable(name="feat_key")

shared convolutional layers

flow, scale_map = get_flownet(data_cur, data_key)
flow_grid = mx.sym.GridGenerator(data=flow, transform_type='warp', name='flow_grid')
conv_feat = mx.sym.BilinearSampler(data=conv_feat, grid=flow_grid, name='warping_feat')
conv_feat = conv_feat * scale_map
#conv_feats = mx.sym.SliceChannel(conv_feat, axis=1, num_outputs=2)

fc6_bias = mx.symbol.Variable('fc6_bias', lr_mult=2.0)
fc6_weight = mx.symbol.Variable('fc6_weight', lr_mult=1.0)
fc6 = mx.symbol.Convolution(data=conv_feat, kernel=(1, 1), pad=(0, 0), num_filter=1024, name="fc6", bias=fc6_bias, weight=fc6_weight,workspace=workspace)
relu_fc6 = mx.sym.Activation(data=fc6, act_type='relu', name='relu_fc6')
score_bias = mx.symbol.Variable('score_bias', lr_mult=2.0)
score_weight = mx.symbol.Variable('score_weight', lr_mult=1.0)
score = mx.symbol.Convolution(data=relu_fc6, kernel=(1, 1), pad=(0, 0), num_filter=num_classes, name="score", bias=score_bias,weight=score_weight, workspace=workspace)

upsampling = mx.symbol.Deconvolution(
    data=score, num_filter=num_classes, kernel=(32, 32), stride=(16, 16), num_group=num_classes, no_bias=True,
    name='upsampling', attr={'lr_mult': '0.0'}, workspace=workspace)

croped_score = mx.symbol.Crop(*[upsampling, data_cur], offset=(8, 8), name='croped_score')
softmax = mx.symbol.SoftmaxOutput(data=croped_score, normalization='valid', multi_output=True, use_ignore=True,ignore_label=255, name="softmaxoutput")
return softmax

[sxjscience]

@chowkamlee81 Have you solved the problem? This looks like a bug. Could you provide an example to reproduce the error? Thanks a lot!

[sxjscience]

@chowkamlee81 There is no need to provide an example. I've checked the code again. This is a bug. We need to fix the logic here https://github.com/apache/incubator-mxnet/blob/master/src/operator/bilinear_sampler-inl.h#L95-L108. @piiswrong

[cjolivier01]

Please provide an example to reproduce the issue

[cjolivier01]

I don't understand the problem description and am unfamiliar with this code. seems like @sxjscience knows what the fix should be, so reassigning.

[ghost]

any news? I had this bug too.

[eric-haibin-lin]

i can help take a look

[haojin2]

A fix delivered in #12386, but corresponding unit test currently blocked by the flakiness of test_bilinear_sampler