Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add methods to propagate the sizes of the samples in the batch #6

Open
jpuigcerver opened this issue Sep 19, 2016 · 0 comments
Open

Comments

@jpuigcerver
Copy link
Owner

We should two methods to the layer modules (e.g. nn.* and cudnn.* modules). All these modules interact with our tools through two main methods: forward(input) and backward(input, gradOutput), that perform the forward and backpropagation passes through the module, respectively. It would be nice to have two similar methods that also receive the input and output sizes, in the case that the different batch elements have different sizes. For instance, consider the LSTM layer from the cudnn package:

layer:forward_with_sizes(input, inputSizes), this would return the output tensor AND a tensor with the output sizes. If the sizes do not change, it can just return inputSizes. For instance, in the Maxpool layer, the output sizes would need to be computed according to the stride/size parameters of the pooling.

layer:backward_with_sizes(input, gradOutput, outputSizes), this would return the gradInput and gradInputSizes tensors, to be passed to the previous layers during backpropagation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant