[SPARK-30412][SQL][TESTS] Eliminate warnings in Java tests regarding to deprecated Spark SQL API #27081
Conversation
| KeyValueGroupedDataset<String, Tuple2<String, Integer>> grouped = generateGroupedDataset(); | ||
| Dataset<Tuple2<String, Double>> agged = grouped.agg(typed.avg(v -> (double)(v._2() * 2))); | ||
| Dataset<Tuple2<String, Double>> agged = grouped.agg( | ||
| org.apache.spark.sql.expressions.javalang.typed.avg(v -> (double)(v._2() * 2))); |
There was a problem hiding this comment.
don't think it's a good idea to use full class name. you may use rename a class on import
import org.apache.spark.sql.expressions.javalang.{typed => javaTyped}
There was a problem hiding this comment.
The purpose of using full path to deprecated methods is to put any references to deprecated class under the annotation @SuppressWarnings("deprecation"). Even if we forget that your code doesn't compile in Java, it still refers to deprecated class.
|
|
||
| import org.apache.spark.sql.Dataset; | ||
| import org.apache.spark.sql.KeyValueGroupedDataset; | ||
| import org.apache.spark.sql.expressions.javalang.typed; |
There was a problem hiding this comment.
suggest to use rename a class on import, e.g. import org.apache.spark.sql.expressions.javalang.{typed => javaTyped}
instead of use full class name in rest of code
There was a problem hiding this comment.
It is Java not Scala. What do you propose doesn't work here.
|
Test build #116049 has finished for PR 27081 at commit
|
|
Merged to master. |
6 participants
What changes were proposed in this pull request?
In the PR, I propose to add the
@SuppressWarnings("deprecation")annotation to Java tests for deprecated Spark SQL APIs.Why are the changes needed?
This eliminates the following warnings:
and highlights warnings about real problems.
Does this PR introduce any user-facing change?
No
How was this patch tested?
By existing test suites
Java8DatasetAggregatorSuite.java,JavaDataFrameSuite.javaandJavaDatasetAggregatorSuite.java.