I dont quite understand how Upsampling works.

[ascust]

I am trying to implement a fully convolutional neural network which requires an upsampling step before softmaxout. The small score maps need to be upsampled to the same size as the ground-truth label. I tried to use Upsampling layer but it seems the layer requires weights parameters. I simply just want to use bilinear methods to scale the score maps, for example I have 21 score maps with size 64_64. I just need them to be resized to for example 128_128. I dont see the need of those weights. Could anyone explain to me how this Upsampling layer works? If this layer is not what I am looking for, what else can I use to achieve this?

[piiswrong]

Bilinear upsampling is implemented via convolution, therefore the weights. If you don't want them to be modified by training you need to set lr_scale to 0 for them.

[ascust]

So it works like standard bilinear interpolation if I dont do weights update? Could you explain if I do update the weights, what will change? I ask this because I think bilinear interpolation is somehow a fixed method.

[piiswrong]

If you update the weights it is trianed like a deconvolution layer, which
may be better than static bilinear upsampling.
On Feb 3, 2016 5:14 PM, "AsCu" [email protected] wrote:

So it works like standard bilinear interpolation if I dont do weights
update? Could you explain if I do update the weights, what will change? I
ask this because I think bilinear interpolation is somehow a fixed method.

—
Reply to this email directly or view it on GitHub
#1412 (comment).

[ascust]

So does it mean I need to create a convolution filter for it and use it to do bilinear interpolation? The problem is that the program complains that I didnot put the weights into net parameters.

[piiswrong]

You need a weight matrix. It can be initial ized with bilinear initializer
in mx. init
On Feb 3, 2016 7:05 PM, "AsCu" [email protected] wrote:

So does it mean I need to create a convolution filter for it and use it to
do bilinear interpolation? The problem is that the program complains that I
didnot put the weights into net parameters.

—
Reply to this email directly or view it on GitHub
#1412 (comment).

[ascust]

I did not find the corresponding documentation. I found Xavier, Normal, Uniform and so on. Would you tell me how to do it specifically. I suggest you could add this to the document.

[ascust]

import mxnet as mx
import numpy as np

from PIL import Image
#upsample filter
def upsample_filt(size):
    factor = (size + 1) // 2
    if size % 2 == 1:
        center = factor - 1
    else:
        center = factor - 0.5
    og = np.ogrid[:size, :size]
    return (1 - abs(og[0] - center) / factor) * \
           (1 - abs(og[1] - center) / factor)


#network
net = mx.symbol.Variable('data')
net = mx.symbol.UpSampling(net, scale=2, sample_type='bilinear', num_args = 2, name="up")

#image data
im  = Image.open("a.png")
im_data = np.array(im, dtype=np.float32)
im_data = np.swapaxes(im_data, 0, 2)
im_data = np.swapaxes(im_data, 1, 2)
im_data = np.expand_dims(im_data, 0)
im_data = mx.nd.array(im_data)
#upsampling weights
filt = upsample_filt(4)
initw = np.zeros((3,1,4,4), dtype=np.float32)
initw[range(3), 0, :, :] = filt
initw = mx.nd.array(initw)
#forward
c_exec = net.bind(ctx=mx.cpu(), args={'data': im_data, 'up_weight': initw})
c_exec.forward()

I ran into an error "Floating point exception: 8". The above is the minimum working code. I am not sure it is because of the UpSampling itself or I missed something. Could you check into this?

[piiswrong]

net = mx.symbol.UpSampling(net, num_filter=3, scale=2, sample_type='bilinear', num_args = 2, name="up")
You need to set num_filter to the number of channels. This is a nuisance due to a hacky solution. I'll try to find time to fix this.

[vchuravy]

Can we reopen this so that we have a reminder to fix the num_filter issue?

@pluskid The error for this on the Julia side was quite opaque ERROR: LoadError: DivideError: integer division error and just stalled the process.

[yxchng]

can i know what is num_args in the upsampling layer? I don't quite understand what i should put.

[stupidZZ]

Do I still need init the weight of upsampling with 'bilinear' mode?

[kmonachopoulos]

Each upsample layer consists of (1) Upsample (bilinear, zero padding, nearest- neighbor) and (2) convolution with a transposed filter. That is why you need the convolution operation in these layers. Convolution always reduce the input matrix to the output, to go to a bigger output, you have to upsample the input first and potentially do zero - padding to the output later on in order to fix the dimensions. Also, the filters are called learned filters, because the un-transposed version (filter) that we used to calculate the transposed filter learned the features during training.