-
Notifications
You must be signed in to change notification settings - Fork 29k
[MINOR][SQL] Remove unnecessary MiMa excludes #27729
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.regression.RandomForestRegressionModel.setMinInstancesPerNode"), | ||
| ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.regression.RandomForestRegressionModel.setNumTrees"), | ||
|
|
||
| // [SPARK-26124] Update plugins, including MiMa |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@huaxingao, can you point out why this is not needed anymore?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think these are not needed any more because of the following was added in #23086
// Data Source V2 API changes
(problem: Problem) => problem match {
case MissingClassProblem(cls) =>
!cls.fullName.startsWith("org.apache.spark.sql.sources.v2")
case MissingTypesProblem(newCls, _) =>
!newCls.fullName.startsWith("org.apache.spark.sql.sources.v2")
case InheritedNewAbstractMethodProblem(cls, _) =>
!cls.fullName.startsWith("org.apache.spark.sql.sources.v2")
case DirectMissingMethodProblem(meth) =>
!meth.owner.fullName.startsWith("org.apache.spark.sql.sources.v2")
case ReversedMissingMethodProblem(meth) =>
!meth.owner.fullName.startsWith("org.apache.spark.sql.sources.v2")
case _ => true
}
Then, because of #25700
org.apache.spark.sql.sources.v2.reader -> org.apache.spark.sql.connector.read
I think we can actually change the above code to the following
(problem: Problem) => problem match {
case MissingClassProblem(cls) =>
!cls.fullName.startsWith("org.apache.spark.sql.sources.v2")
case _ => true
|
Test build #119058 has finished for PR 27729 at commit
|
|
also cc @gatorsmile @cloud-fan |
HyukjinKwon
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, LGTM
|
Merged to master and branch-3.0. |
### What changes were proposed in this pull request? I found a few unnecessary MiMa excludes when auditing SQL binary incompatible changes. ### Why are the changes needed? These MiMa excludes are not required any more, so remove. ### Does this PR introduce any user-facing change? No ### How was this patch tested? Manually tested Closes #27729 from huaxingao/mima. Authored-by: Huaxin Gao <[email protected]> Signed-off-by: HyukjinKwon <[email protected]> (cherry picked from commit c0d4cc3) Signed-off-by: HyukjinKwon <[email protected]>
|
@HyukjinKwon to because of #25700 |
|
Sure, I think it might be best to make it as a followup of SPARK-28998. This PR alone can be better separate anyway because these Mima exclusions in this PR were added as of SPARK-26124. |
### What changes were proposed in this pull request? I found a few unnecessary MiMa excludes when auditing SQL binary incompatible changes. ### Why are the changes needed? These MiMa excludes are not required any more, so remove. ### Does this PR introduce any user-facing change? No ### How was this patch tested? Manually tested Closes apache#27729 from huaxingao/mima. Authored-by: Huaxin Gao <[email protected]> Signed-off-by: HyukjinKwon <[email protected]>
What changes were proposed in this pull request?
I found a few unnecessary MiMa excludes when auditing SQL binary incompatible changes.
Why are the changes needed?
These MiMa excludes are not required any more, so remove.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Manually tested