Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LSTM restoration #256

Closed
notimesea opened this issue Sep 5, 2016 · 2 comments
Closed

LSTM restoration #256

notimesea opened this issue Sep 5, 2016 · 2 comments

Comments

@notimesea
Copy link

notimesea commented Sep 5, 2016

Is there any reference how to restore weights of LSTM to serialize model? (e.g. to rnn SeqLSTM).

In other words how to understand which part of self.weight part corresponds to each LSTM weights part (W_i, R_i, bW_i, bR_i, W_f, R_f, bW_f, bR_f, W_o, R_o, bW_o, bR_o, W_c, R_c, bW_c, bR_c)
It is needed to run pre-learned model on CPU.

P.S. There is no cudnn.convert for this layer :(

@ngimel
Copy link
Collaborator

ngimel commented Sep 5, 2016

RNN:weights() and RNN:biases() return tables of tensors corresponding to each weight/bias part. Please look in the RNN.lua source and refer to cudnn manual for cudnnGetRNNLinLayerMatrixParams and cudnnGetRNNLinLayerBiasParams for information on what table member corresponds to what matrix. There is no cudnn.convert for this layer because RNN and LSTM are not part of nn package.

@JoostvDoorn
Copy link

For now you can use this to convert between SeqLSTM and cudnn.LSTM: https://gist.github.com/JoostvDoorn/bd43ce85a101c0aeee03f734ead688fb

It will be helpful if this can be done in the same manner as cudnn.convert.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants