Element Research rnn benchmark ++

[nicholas-leonard]

Changes :

• simple rnn is faster (uses JoinTable + one Linear)
• uses torch.Timer() instead of os.clock() (more precise)
• calls cutorch.synchronize before measuring time. (Don't remember how to do this for theano, but it should be done as well).
• measures setup time (torch and theano)

It would be nice if the benchmarks included the setup time as well.
Please update the benchmarks.

[glample]

Thanks for the changes! This RNN is much faster.
The warming part in Theano doesn't seem to make a difference, but is clearly needed in torch (I guess it's because the sequencer needs to clone the recurrentModule when it receives the first batch?). I forgot to put it in the code, but in these benchmarks I was already running a forward call before starting the timer (otherwise the forward time was clearly wrong), so I basically still have the same running times for LSTM.

[nicholas-leonard]

@glample Yeah the warming part isn't necessary for theano. But since the "setup" time in torch implies a forward/backward, I wanted to include it in theano's setup time just to be fair. It would be really nice to include this setup time in the README as this may be theano's hidden weakness, although they have made great strides to reduce compile time since I last used it.