Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RMSprop, AdaDelta and Adam solvers #564

Merged
merged 2 commits into from
Feb 3, 2016

Conversation

lukeyeager
Copy link
Member

Close #555

Disable solvers which aren't supported by the selected framework.

@gheinrich Torch can only do SGD for now, correct?

Disable solvers which aren't supported by the selected framework
@gheinrich
Copy link
Contributor

Torch-in-DIGITS currently supports SGD, NESTEROV and ADAGRAD through these switches though I don't recall testing them explicitly. RMSPROP, ADADELTA and ADAM could also be added with similar switches.

@lukeyeager
Copy link
Member Author

Torch-in-DIGITS currently supports SGD, NESTEROV and ADAGRAD through these switches

Oh, don't know how I missed that!

Looks like all of these solver types are supported for Torch (am I looking in the right place)? I've added the six solvers that Caffe now supports for Torch, but I don't know enough about the other solver types to feel comfortable adding them at the moment.

@gheinrich
Copy link
Contributor

Yes that's correct, Torch support a bunch of optimization methods through the optim package. Let's stick to those six solvers. I'll try your new patch though I'll be at a loss to verify the solver type is producing the desired effect.

@gheinrich
Copy link
Contributor

I just checked that the optimization method was passed to the Lua wrapper and that this produced slightly different learning curves.

@lukeyeager
Copy link
Member Author

Me too. Let's merge and address any solver-specific improvements in future PRs.

lukeyeager added a commit that referenced this pull request Feb 3, 2016
Add RMSprop, AdaDelta and Adam solvers
@lukeyeager lukeyeager merged commit ebfb188 into NVIDIA:master Feb 3, 2016
@lukeyeager lukeyeager deleted the add-new-solvers branch February 3, 2016 21:21
@gheinrich gheinrich added the torch label Feb 3, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants