[SPARK-28985][PYTHON][ML][FOLLOW-UP] Add _AFTSurvivalRegressionParams

[zero323]

What changes were proposed in this pull request?

Adds

_AFTSurvivalRegressionParams(HasFeaturesCol, HasLabelCol, HasPredictionCol,
                                   HasMaxIter, HasTol, HasFitIntercept,
                                   HasAggregationDepth): ...

with related Params and uses it to replace HasFitIntercept, HasMaxIter, HasTol and HasAggregationDepth in AFTSurvivalRegression base classes and JavaPredictionModel, in AFTSurvivalRegressionModel base classes.

Why are the changes needed?

Previous work (#25776) on SPARK-28985 replaced JavaEstimator, HasFeaturesCol, HasLabelCol, HasPredictionCol in AFTSurvivalRegression and JavaModel in AFTSurvivalRegressionModel with newly added JavaPredictor:

spark/python/pyspark/ml/wrapper.py

Line 377 in e97b55d

class JavaPredictor(JavaEstimator, JavaPredictorParams):

and JavaPredictionModel

spark/python/pyspark/ml/wrapper.py

Line 405 in e97b55d

class JavaPredictionModel(JavaModel, JavaPredictorParams):

respectively.

This however is inconsistent with Scala counterpart where both classes extend private AFTSurvivalRegressionBase

spark/mllib/src/main/scala/org/apache/spark/ml/regression/AFTSurvivalRegression.scala

Lines 48 to 50 in eb037a8

	private[regression] trait AFTSurvivalRegressionParams extends Params
	with HasFeaturesCol with HasLabelCol with HasPredictionCol with HasMaxIter
	with HasTol with HasFitIntercept with HasAggregationDepth with Logging {

This preserves some of the existing inconsistencies (variables as defined in the official example)

from pyspark.ml.regression import AFTSurvivalRegression, AFTSurvivalRegressionModel
from pyspark.ml.param.shared import HasMaxIter, HasTol, HasFitIntercept, HasAggregationDepth
from pyspark.ml.param import Param

issubclass(AFTSurvivalRegressionModel, HasMaxIter)
# False
hasattr(model, "maxIter")  and isinstance(model.maxIter, Param)
# True

issubclass(AFTSurvivalRegressionModel, HasTol)
# False
hasattr(model, "tol")  and isinstance(model.tol, Param)
# True

and can cause problems in the future, if Predictor / PredictionModel API changes (unlike IsotonicRegression, current implementation is technically speaking correct, though incomplete).

Does this PR introduce any user-facing change?

Yes, it adds a number of base classes to AFTSurvivalRegressionModel. These change purely additive and have negligible potential for breaking existing code (and none, compared to changes already made in #25776). Additionally affected API hasn't been released in the current form yet.

How was this patch tested?

• Existing unit tests.
• Manual testing.

CC @huaxingao, @zhengruifeng

[SparkQA]

Test build #111776 has finished for PR 26024 at commit 3279f9f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• class _AFTSurvivalRegressionParams(HasFeaturesCol, HasLabelCol, HasPredictionCol,
• class AFTSurvivalRegression(JavaEstimator, _AFTSurvivalRegressionParams,
• class AFTSurvivalRegressionModel(JavaModel, _AFTSurvivalRegressionParams,

[zhengruifeng]

Great Catch. LGTM

[huaxingao]

LGTM

[srowen]

Merged to master

[zero323]

Thanks @srowen, @huaxingao, @zhengruifeng!