forward() doesn't support non-Variable inputs

[ezyang]

I originally thought that no one would actually do this, but I was wrong! In bnlstm, https://github.com/pytorch/benchmark/blob/master/benchmarks/models/bnlstm.py an integer "time" is passed in to SeparatedBatchNorm1d, which is used to determine which running_mean and running_var is used.

If there are many other incidences of this, we may have to support this.

[jekbradbury]

If the PyTorch API lets someone do something, they'll do it... but yeah recurrent batchnorm is the only situation I know of where you basically can't refactor such an auxiliary input out