Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark Update #10

Merged
merged 8 commits into from
May 17, 2016
Merged

Benchmark Update #10

merged 8 commits into from
May 17, 2016

Conversation

nicholas-leonard
Copy link
Contributor

@nicholas-leonard nicholas-leonard commented Apr 22, 2016

Hi @glample,

This PR provides the following updates:

  • Torch uses SeqLSTM . I give explicit credit to @jcjohnson for taking the time to implement such blazing fast code (see PR#207).
  • There is only one input and target batch. This allows the benchmarking of huge networks as otherwise the xValues tensor was so big that it would blow up the memory.
  • Input is now of shape seqlen x batchsize x hiddensize instead of batchsize x seqlen x hiddensize this gives a slight speedup to all implementations and is the natural way to present sequential data as it makes the step-wise tensors contiguous in memory.
  • hiddensize = inputsize. This is the usual use-case for RNN/LSTMs.
  • Benchmarks for hiddensize of 128, 512 and 1024, batchsize of 32 and 128 as mentioned in Benchmark hidden sizes that give better efficiency and bigger batches #9 .

@nicholas-leonard nicholas-leonard changed the title seqlstm Benchmark Update (work in progress) May 14, 2016
@nicholas-leonard nicholas-leonard changed the title Benchmark Update (work in progress) Benchmark Update May 15, 2016
@nicholas-leonard
Copy link
Contributor Author

nicholas-leonard commented May 15, 2016

@ma2rten Is there a way to store the input and target tensor inside a GPU tensor before sending it to TensorFlow? I think that would make your code considerably faster.

@nicholas-leonard
Copy link
Contributor Author

@glample I think this is ready to merge. I can incorporate any tweaks later.

@glample glample merged commit 7b737bf into glample:master May 17, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants