[GLUTEN-11341][CORE] Support Scala 2.13 with IntelliJ IDE#11342
Merged
baibaichen merged 7 commits intoapache:mainfrom Dec 31, 2025
Merged
[GLUTEN-11341][CORE] Support Scala 2.13 with IntelliJ IDE#11342baibaichen merged 7 commits intoapache:mainfrom
baibaichen merged 7 commits intoapache:mainfrom
Conversation
|
Run Gluten Clickhouse CI on x86 |
Contributor
There was a problem hiding this comment.
Can we use col("b")? The warning may be caused by b with single quote
Contributor
There was a problem hiding this comment.
| Syntax | Valid | Notes |
|---|---|---|
| df.repartition(4, 'b) | ✅ | Old-style, works |
| df.repartition(4, Symbol("b")) | ✅ | Same as above |
| df.repartition(4, col("b")) | ✅ | Recommended |
| df.repartition(4, 'b), | ❌ | Syntax error |
| } | ||
| } | ||
|
|
||
| @nowarn |
Contributor
There was a problem hiding this comment.
What's the warning message? Can we fix the warning?
Contributor
Author
There was a problem hiding this comment.
org.apache.iceberg.ContentFile::path() is marked as deprecated. Without introducing a shim, it's hard to fix properly, so we'll suppress the warning for now.
asFileScanTask(tasks.toList).foreach {
task =>
paths.add(
BackendsApiManager.getTransformerApiInstance
.encodeFilePathIfNeed(task.file().path().toString)) // => deprecatedAdditionally, @nowarn can only be added before method names; it cannot be applied before individual statements.
Contributor
There was a problem hiding this comment.
As the comment indicates, please use location
/**
* Returns fully qualified path to the file, suitable for constructing a Hadoop Path.
*
* @deprecated since 1.7.0, will be removed in 2.0.0; use {@link #location()} instead.
*/
@Deprecated
CharSequence path();
/** Return the fully qualified path to the file. */
default String location() {
return path().toString();
}
Contributor
Contributor
Author
There was a problem hiding this comment.
As the comment indicates, please use location
/** * Returns fully qualified path to the file, suitable for constructing a Hadoop Path. * * @deprecated since 1.7.0, will be removed in 2.0.0; use {@link #location()} instead. */ @Deprecated CharSequence path(); /** Return the fully qualified path to the file. */ default String location() { return path().toString(); }
No, location doesn't work in profiles in Spark 3.3 and earlier.
|
Run Gluten Clickhouse CI on x86 |
1 similar comment
Contributor
Author
|
Run Gluten Clickhouse CI on x86 |
…nings in pom.xml see apache/spark#43332
…o fix compilation error in Scala 2.13 with IntelliJ compiler: symbol literal is deprecated; use Symbol("i")
…reading in GlutenSQLQueryTestSuite see apache/spark#51911 which removes Spark's fileToString method from Spark code base.
…8 to `${java.version}` in the Scala 2.13 profiler to align it with `maven.compiler.target`
… to align with Spark API best practices
baibaichen
force-pushed
the
feature/intellij-java17
branch
from
85960a4 to
6d3e250
Compare
|
Run Gluten Clickhouse CI on x86 |
jinchengchenghh
approved these changes
Dec 31, 2025
Contributor
jinchengchenghh
left a comment
jinchengchenghh
left a comment
There was a problem hiding this comment.
Thanks for your fix!
QCLyu
pushed a commit
to QCLyu/incubator-gluten
that referenced
this pull request
Jan 8, 2026
* [Scala 2.13][IntelliJ] Remove suppression for lint-multiarg-infix warnings in pom.xml see apache/spark#43332 * [Scala 2.13][IntelliJ] Suppress warning for `ContentFile::path` * [Scala 2.13][IntelliJ] Suppress warning for ContextAwareIterator initialization * [Scala 2.13][IntelliJ] Refactor to use Symbol for column references to fix compilation error in Scala 2.13 with IntelliJ compiler: symbol literal is deprecated; use Symbol("i") * [Fix] Replace deprecated fileToString with Files.readString for file reading in GlutenSQLQueryTestSuite see apache/spark#51911 which removes Spark's fileToString method from Spark code base. * [Scala 2.13][IntelliJ] Update the Java compiler release version from 8 to `${java.version}` in the Scala 2.13 profiler to align it with `maven.compiler.target` * [Refactor] Replace usage of `Symbol` with `col` for column references to align with Spark API best practices --------- Co-authored-by: Chang chen <chenchang@apache.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #11341
What changes were proposed in this pull request?
This PR includes several improvements for Scala 2.13 compatibility and code modernization:
8to${java.version}in the Scala 2.13 profiler to align withmaven.compiler.targetpom.xmlfileToStringtoFiles.readStringinGlutenSQLQueryTestSuite(see apache/spark#51911)gluten-ut/spark40/src/test/scala/org/apache/spark/GlutenSQLQueryTestSuite.scalaSymbol()for column references to fix compilation error in Scala 2.13 with IntelliJ compilerGlutenDataSourceV2Suite.scala,ClickHouseAdaptiveQueryExecSuite.scala,VeloxAdaptiveQueryExecSuite.scalaContextAwareIteratorinitialization andContentFile::pathwarnings in Scala 2.13ColumnarArrowEvalPythonExec.scala,GlutenIcebergSourceUtil.scalapom.xmlWhy are the changes needed?
These changes are necessary to:
How was this patch tested?