Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

fix bug in nag optimizer #13683

Merged
merged 6 commits into from
Jan 16, 2019
Merged

fix bug in nag optimizer #13683

merged 6 commits into from
Jan 16, 2019

Commits on Dec 19, 2018

  1. fix bug in nag optimizer 

    ```
grad += wd * weight
mom[:] += grad
grad[:] += self.momentum * mom
weight[:] += -lr * grad
```
This will minus wd*weight twice, but in`state = momentum * state + grad + wd * weight   weight = weight - (lr * (grad + momentum * state)) ` only minus once.
    @solin319
    solin319 committed Dec 19, 2018
    Configuration menu
    Copy the full SHA
    1a065e8 View commit details
    Browse the repository at this point in the history

  2. fix bug in nag test 

    fix bug in nag test
    @solin319
    solin319 committed Dec 19, 2018
    Configuration menu
    Copy the full SHA
    641bf6c View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2018

  1. rewrite nag test

    @solin319
    solin319 committed Dec 20, 2018
    Configuration menu
    Copy the full SHA
    ea63cb1 View commit details
    Browse the repository at this point in the history

  2. rewrite nag

    @solin319
    solin319 committed Dec 20, 2018
    Configuration menu
    Copy the full SHA
    7633a1d View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2019

  1. fix nag with in-place operations

    @solin319
    solin319 committed Jan 16, 2019
    Configuration menu
    Copy the full SHA
    510dd33 View commit details
    Browse the repository at this point in the history

  2. fix nag with in-place operations

    @solin319
    solin319 committed Jan 16, 2019
    Configuration menu
    Copy the full SHA
    0fa1466 View commit details
    Browse the repository at this point in the history