Skip to content

Commit d9f3e01

Browse files
Lewuathemengxr
authored andcommitted
[SPARK-6336] LBFGS should document what convergenceTol means
LBFGS uses convergence tolerance. This value should be written in document as an argument. Author: lewuathe <[email protected]> Closes #5033 from Lewuathe/SPARK-6336 and squashes the following commits: e738b33 [lewuathe] Modify text to be more natural ac03c3a [lewuathe] Modify documentations 6ccb304 [lewuathe] [SPARK-6336] LBFGS should document what convergenceTol means
1 parent 4cca391 commit d9f3e01

File tree

2 files changed

+9
-1
lines changed

2 files changed

+9
-1
lines changed

docs/mllib-optimization.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,10 @@ regularization, as well as L2 regularizer.
203203
recommended.
204204
* `maxNumIterations` is the maximal number of iterations that L-BFGS can be run.
205205
* `regParam` is the regularization parameter when using regularization.
206+
* `convergenceTol` controls how much relative change is still allowed when L-BFGS
207+
is considered to converge. This must be nonnegative. Lower values are less tolerant and
208+
therefore generally cause more iterations to be run. This value looks at both average
209+
improvement and the norm of gradient inside [Breeze LBFGS](https://github.com/scalanlp/breeze/blob/master/math/src/main/scala/breeze/optimize/LBFGS.scala).
206210

207211
The `return` is a tuple containing two elements. The first element is a column matrix
208212
containing weights for every feature, and the second element is an array containing

mllib/src/main/scala/org/apache/spark/mllib/optimization/LBFGS.scala

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,8 @@ class LBFGS(private var gradient: Gradient, private var updater: Updater)
6060
/**
6161
* Set the convergence tolerance of iterations for L-BFGS. Default 1E-4.
6262
* Smaller value will lead to higher accuracy with the cost of more iterations.
63+
* This value must be nonnegative. Lower convergence values are less tolerant
64+
* and therefore generally cause more iterations to be run.
6365
*/
6466
def setConvergenceTol(tolerance: Double): this.type = {
6567
this.convergenceTol = tolerance
@@ -142,7 +144,9 @@ object LBFGS extends Logging {
142144
* one single data example)
143145
* @param updater - Updater function to actually perform a gradient step in a given direction.
144146
* @param numCorrections - The number of corrections used in the L-BFGS update.
145-
* @param convergenceTol - The convergence tolerance of iterations for L-BFGS
147+
* @param convergenceTol - The convergence tolerance of iterations for L-BFGS which is must be
148+
* nonnegative. Lower values are less tolerant and therefore generally
149+
* cause more iterations to be run.
146150
* @param maxNumIterations - Maximal number of iterations that L-BFGS can be run.
147151
* @param regParam - Regularization parameter
148152
*

0 commit comments

Comments
 (0)