Support of optim.lr_scheduler.ReduceLROnPlateau

[vikmary]

Hi,
I'm using pytorch-lightning as a base for a side project. My configure_optimizers looks this way:

def configure_optimizers(self):
  """
  return whatever optimizers we want here
  :return: list of optimizers
  """
  optimizer = optim.Adam(self.parameters(), lr=self.params.learning_rate)
  scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer,
                                                   mode='min',
                                                   factor=0.1,
                                                   patience=10,
                                                   min_lr=1e-6,
                                                   verbose=True)
  return [optimizer], [scheduler]

But that doesn't work out of the box, because the ReduceLROnPlateau scheduler needs scheduler.step(val_loss) call instead of scheduler.step(num_epochs).

The scheduler improves greatly nn convergence, it would nice to support it.

Thank you

[williamFalcon]

install from master and it should work. we pushed a fix for this a few weeks ago

[vikmary]

Are you talking about this fix?
e0c5406

It doesn't solve the issue.

It should be scheduler.step(val_loss, epochs=nb_epochs), not scheduler.step(nb_epochs). Right now scheduler is changing learning rate based on epoch number, not validation loss.

[williamFalcon]

got it. thanks for pointing that out. want to submit a PR?