Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HADOOP-19058. [JDK-17] Fix UT Failures in hadoop common, hdfs, yarn #6531

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from
+12 −1
Open

HADOOP-19058. [JDK-17] Fix UT Failures in hadoop common, hdfs, yarn #6531

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion hadoop-project/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,18 @@
<enforced.maven.version>[3.3.0,)</enforced.maven.version>

<!-- Plugin versions and config -->
<maven-surefire-plugin.argLine>-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError</maven-surefire-plugin.argLine>
<maven-surefire-plugin.argLine>-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for submitting this PR. We are looking forward to Hadoop supporting JDK11 and JDK17.

I have some questions:

Should we fully support JDK11 and JDK17, or should we try to avoid certain issues by adding some compilation parameters?

From my perspective, we should find ways to upgrade the code and make all modules fully compatible with the recommended syntax of JDK11 and JDK17.

@aajisaka @ayushtkn @steveloughran

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a big ongoing bit of work; aws have been busier than the rest of us...we do test against later java versions, but do the build/UT/IT against v8.

I'd like to move 3.5 and then 3.4.x to java17+ ..., where x is a release which comes out this year but is greater than 1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anyway, regarding this line: the exports thing is java11+ only, right? so it should only be set on java11+.

proposed: add a java17 profile and set some maven.surefire.java17.options string to it there, on the default java8 profile set it to ""

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spark added such JVM args unconditional with -XX:+IgnoreUnrecognizedVMOptions, this might be a simpler way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HBase uses maven profiles for these options, and it works well.

--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.math=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.net=ALL-UNNAMED
--add-opens=java.base/java.text=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
</maven-surefire-plugin.argLine>
<maven-surefire-plugin.version>3.0.0-M1</maven-surefire-plugin.version>
<maven-surefire-report-plugin.version>${maven-surefire-plugin.version}</maven-surefire-report-plugin.version>
<maven-failsafe-plugin.version>${maven-surefire-plugin.version}</maven-failsafe-plugin.version>
Expand Down
Loading