Skip to content

[SPARK-26127][ML] Remove deprecated setters from tree regression and classification models #23093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mgaido91 wants to merge 4 commits into from
Closed

[SPARK-26127][ML] Remove deprecated setters from tree regression and classification models #23093

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
56d61f3
[SPARK-26127][ML] Remove deprecated setImpurity from tree regression …
mgaido91 Nov 20, 2018
8e78883
remove all deprecated setters in treeParams
mgaido91 Nov 20, 2018
ce4a02e
fix mima
mgaido91 Nov 20, 2018
762e4de
Merge branch 'master' into SPARK-26127
mgaido91 Nov 21, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,27 +55,27 @@ class DecisionTreeClassifier @Since("1.4.0") (

/** @group setParam */
@Since("1.4.0")
override def setMaxDepth(value: Int): this.type = set(maxDepth, value)
def setMaxDepth(value: Int): this.type = set(maxDepth, value)

/** @group setParam */
@Since("1.4.0")
override def setMaxBins(value: Int): this.type = set(maxBins, value)
def setMaxBins(value: Int): this.type = set(maxBins, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)
def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)
def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)
def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)
def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)

/**
* Specifies how often to checkpoint the cached node IDs.
Expand All @@ -87,15 +87,15 @@ class DecisionTreeClassifier @Since("1.4.0") (
* @group setParam
*/
@Since("1.4.0")
override def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)
def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)

/** @group setParam */
@Since("1.4.0")
override def setImpurity(value: String): this.type = set(impurity, value)
def setImpurity(value: String): this.type = set(impurity, value)
Copy link
Member

@srowen srowen Nov 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove these too. I assumed they were meant to be deprecated because the trait's implementation was deprecated, and there's no compelling reason to only remove it from the trait. The point was to use set().

Copy link
Contributor Author

@mgaido91 mgaido91 Nov 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think those can be removed. The ones which were deprecated are the setImpurity on the Models, not on the Classifiers/Regressors which build the models.

Copy link
Member

@srowen srowen Nov 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I think I'm wrong about the logic, given the first attempt to do this in #17867 Yes this is fine. In fact I think you're welcome to just revive that PR and remove a lot of similar deprecated methods again.

Copy link
Contributor Author

@mgaido91 mgaido91 Nov 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks. I'll update this PR removing all the other deprecated items for these models then. Thanks.


/** @group setParam */
@Since("1.6.0")
override def setSeed(value: Long): this.type = set(seed, value)
def setSeed(value: Long): this.type = set(seed, value)

override protected def train(
dataset: Dataset[_]): DecisionTreeClassificationModel = instrumented { instr =>
Expand Down
26 changes: 13 additions & 13 deletions mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,27 +69,27 @@ class GBTClassifier @Since("1.4.0") (

/** @group setParam */
@Since("1.4.0")
override def setMaxDepth(value: Int): this.type = set(maxDepth, value)
def setMaxDepth(value: Int): this.type = set(maxDepth, value)

/** @group setParam */
@Since("1.4.0")
override def setMaxBins(value: Int): this.type = set(maxBins, value)
def setMaxBins(value: Int): this.type = set(maxBins, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)
def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)
def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)
def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)
def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)

/**
* Specifies how often to checkpoint the cached node IDs.
Expand All @@ -101,7 +101,7 @@ class GBTClassifier @Since("1.4.0") (
* @group setParam
*/
@Since("1.4.0")
override def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)
def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)

/**
* The impurity setting is ignored for GBT models.
Expand All @@ -110,7 +110,7 @@ class GBTClassifier @Since("1.4.0") (
* @group setParam
*/
@Since("1.4.0")
override def setImpurity(value: String): this.type = {
def setImpurity(value: String): this.type = {
logWarning("GBTClassifier.setImpurity should NOT be used")
this
}
Expand All @@ -119,25 +119,25 @@ class GBTClassifier @Since("1.4.0") (

/** @group setParam */
@Since("1.4.0")
override def setSubsamplingRate(value: Double): this.type = set(subsamplingRate, value)
def setSubsamplingRate(value: Double): this.type = set(subsamplingRate, value)

/** @group setParam */
@Since("1.4.0")
override def setSeed(value: Long): this.type = set(seed, value)
def setSeed(value: Long): this.type = set(seed, value)

// Parameters from GBTParams:

/** @group setParam */
@Since("1.4.0")
override def setMaxIter(value: Int): this.type = set(maxIter, value)
def setMaxIter(value: Int): this.type = set(maxIter, value)

/** @group setParam */
@Since("1.4.0")
override def setStepSize(value: Double): this.type = set(stepSize, value)
def setStepSize(value: Double): this.type = set(stepSize, value)

/** @group setParam */
@Since("2.3.0")
override def setFeatureSubsetStrategy(value: String): this.type =
def setFeatureSubsetStrategy(value: String): this.type =
set(featureSubsetStrategy, value)

// Parameters from GBTClassifierParams:
Expand Down
24 changes: 12 additions & 12 deletions mllib/src/main/scala/org/apache/spark/ml/classification/RandomForestClassifier.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,27 +57,27 @@ class RandomForestClassifier @Since("1.4.0") (

/** @group setParam */
@Since("1.4.0")
override def setMaxDepth(value: Int): this.type = set(maxDepth, value)
def setMaxDepth(value: Int): this.type = set(maxDepth, value)

/** @group setParam */
@Since("1.4.0")
override def setMaxBins(value: Int): this.type = set(maxBins, value)
def setMaxBins(value: Int): this.type = set(maxBins, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)
def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)
def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)
def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)
def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)

/**
* Specifies how often to checkpoint the cached node IDs.
Expand All @@ -89,31 +89,31 @@ class RandomForestClassifier @Since("1.4.0") (
* @group setParam
*/
@Since("1.4.0")
override def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)
def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)

/** @group setParam */
@Since("1.4.0")
override def setImpurity(value: String): this.type = set(impurity, value)
def setImpurity(value: String): this.type = set(impurity, value)

// Parameters from TreeEnsembleParams:

/** @group setParam */
@Since("1.4.0")
override def setSubsamplingRate(value: Double): this.type = set(subsamplingRate, value)
def setSubsamplingRate(value: Double): this.type = set(subsamplingRate, value)

/** @group setParam */
@Since("1.4.0")
override def setSeed(value: Long): this.type = set(seed, value)
def setSeed(value: Long): this.type = set(seed, value)

// Parameters from RandomForestParams:

/** @group setParam */
@Since("1.4.0")
override def setNumTrees(value: Int): this.type = set(numTrees, value)
def setNumTrees(value: Int): this.type = set(numTrees, value)

/** @group setParam */
@Since("1.4.0")
override def setFeatureSubsetStrategy(value: String): this.type =
def setFeatureSubsetStrategy(value: String): this.type =
set(featureSubsetStrategy, value)

override protected def train(
Expand Down
18 changes: 9 additions & 9 deletions mllib/src/main/scala/org/apache/spark/ml/regression/DecisionTreeRegressor.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,27 +54,27 @@ class DecisionTreeRegressor @Since("1.4.0") (@Since("1.4.0") override val uid: S
// Override parameter setters from parent trait for Java API compatibility.
/** @group setParam */
@Since("1.4.0")
override def setMaxDepth(value: Int): this.type = set(maxDepth, value)
def setMaxDepth(value: Int): this.type = set(maxDepth, value)

/** @group setParam */
@Since("1.4.0")
override def setMaxBins(value: Int): this.type = set(maxBins, value)
def setMaxBins(value: Int): this.type = set(maxBins, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)
def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)
def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)
def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)
def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)

/**
* Specifies how often to checkpoint the cached node IDs.
Expand All @@ -86,15 +86,15 @@ class DecisionTreeRegressor @Since("1.4.0") (@Since("1.4.0") override val uid: S
* @group setParam
*/
@Since("1.4.0")
override def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)
def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)

/** @group setParam */
@Since("1.4.0")
override def setImpurity(value: String): this.type = set(impurity, value)
def setImpurity(value: String): this.type = set(impurity, value)

/** @group setParam */
@Since("1.6.0")
override def setSeed(value: Long): this.type = set(seed, value)
def setSeed(value: Long): this.type = set(seed, value)

/** @group setParam */
@Since("2.0.0")
Expand Down
27 changes: 13 additions & 14 deletions mllib/src/main/scala/org/apache/spark/ml/regression/GBTRegressor.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ import org.apache.spark.ml.util.DefaultParamsReader.Metadata
import org.apache.spark.ml.util.Instrumentation.instrumented
import org.apache.spark.mllib.tree.configuration.{Algo => OldAlgo}
import org.apache.spark.mllib.tree.model.{GradientBoostedTreesModel => OldGBTModel}
import org.apache.spark.rdd.RDD
import org.apache.spark.sql.{DataFrame, Dataset, Row}
import org.apache.spark.sql.functions._

Expand Down Expand Up @@ -69,27 +68,27 @@ class GBTRegressor @Since("1.4.0") (@Since("1.4.0") override val uid: String)

/** @group setParam */
@Since("1.4.0")
override def setMaxDepth(value: Int): this.type = set(maxDepth, value)
def setMaxDepth(value: Int): this.type = set(maxDepth, value)

/** @group setParam */
@Since("1.4.0")
override def setMaxBins(value: Int): this.type = set(maxBins, value)
def setMaxBins(value: Int): this.type = set(maxBins, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)
def setMinInstancesPerNode(value: Int): this.type = set(minInstancesPerNode, value)

/** @group setParam */
@Since("1.4.0")
override def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)
def setMinInfoGain(value: Double): this.type = set(minInfoGain, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)
def setMaxMemoryInMB(value: Int): this.type = set(maxMemoryInMB, value)

/** @group expertSetParam */
@Since("1.4.0")
override def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)
def setCacheNodeIds(value: Boolean): this.type = set(cacheNodeIds, value)

/**
* Specifies how often to checkpoint the cached node IDs.
Expand All @@ -101,7 +100,7 @@ class GBTRegressor @Since("1.4.0") (@Since("1.4.0") override val uid: String)
* @group setParam
*/
@Since("1.4.0")
override def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)
def setCheckpointInterval(value: Int): this.type = set(checkpointInterval, value)

/**
* The impurity setting is ignored for GBT models.
Expand All @@ -110,7 +109,7 @@ class GBTRegressor @Since("1.4.0") (@Since("1.4.0") override val uid: String)
* @group setParam
*/
@Since("1.4.0")
override def setImpurity(value: String): this.type = {
def setImpurity(value: String): this.type = {
logWarning("GBTRegressor.setImpurity should NOT be used")
this
}
Expand All @@ -119,21 +118,21 @@ class GBTRegressor @Since("1.4.0") (@Since("1.4.0") override val uid: String)

/** @group setParam */
@Since("1.4.0")
override def setSubsamplingRate(value: Double): this.type = set(subsamplingRate, value)
def setSubsamplingRate(value: Double): this.type = set(subsamplingRate, value)

/** @group setParam */
@Since("1.4.0")
override def setSeed(value: Long): this.type = set(seed, value)
def setSeed(value: Long): this.type = set(seed, value)

// Parameters from GBTParams:

/** @group setParam */
@Since("1.4.0")
override def setMaxIter(value: Int): this.type = set(maxIter, value)
def setMaxIter(value: Int): this.type = set(maxIter, value)

/** @group setParam */
@Since("1.4.0")
override def setStepSize(value: Double): this.type = set(stepSize, value)
def setStepSize(value: Double): this.type = set(stepSize, value)

// Parameters from GBTRegressorParams:

Expand All @@ -143,7 +142,7 @@ class GBTRegressor @Since("1.4.0") (@Since("1.4.0") override val uid: String)

/** @group setParam */
@Since("2.3.0")
override def setFeatureSubsetStrategy(value: String): this.type =
def setFeatureSubsetStrategy(value: String): this.type =
set(featureSubsetStrategy, value)

/** @group setParam */
Expand Down
Loading