This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Adam, AdaMax and FTML cannot be used with Trainer(update_on_kv=False) #13752

Closed

eric-haibin-lin opened this issue Jan 1, 2019 · 0 comments · Fixed by #14377

Labels

Bug Gluon Optimizer

Member

eric-haibin-lin commented Jan 1, 2019

These optimizers adjust the learning rate by the number of optimization steps, which is problematic if update_on_kv=False is set and multiple GPUs share the same optimizer object (leading to incorrect count of optimization steps). For example: https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/optimizer/optimizer.py#L1093

cc @szha @sandeep-krishnamurthy

The text was updated successfully, but these errors were encountered:

eric-haibin-lin added Bug Gluon Optimizer labels

ChaiBapchya mentioned this issue

Labelbot recommending already labeled issues MXNetEdge/mxnet-infrastructure#61

Open

ptrendx mentioned this issue

Correct update count with Gluon trainer and update_on_kvstore=False #14377

Merged

4 tasks

eric-haibin-lin closed this as completed in #14377

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.