Skip to content

Conversation

@pan3793
Copy link
Member

@pan3793 pan3793 commented Jun 13, 2021

Why are the changes needed?

Close #550, relate to #559 #688 #689.

Add javax.xml.bind:jaxb-api and javax.activation:activation since those apis has been removed by JDK 11.

In GA, we saw a significant performance downgrade in the unit tests, and even with some changes like enlarging timeout, using in-memory catalog instead of hive catalog, the JDK11(~43min) tests cost 2 times than JDK8(~22min), this performance regression is tracked in #745.

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before make a pull request

@codecov-commenter
Copy link

codecov-commenter commented Jun 14, 2021

Codecov Report

Merging #693 (5706daf) into master (65996a4) will decrease coverage by 80.49%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #693       +/-   ##
==========================================
- Coverage   80.49%   0.00%   -80.50%     
==========================================
  Files         122     122               
  Lines        4748    4748               
  Branches      576     576               
==========================================
- Hits         3822       0     -3822     
- Misses        605    4748     +4143     
+ Partials      321       0      -321     
Impacted Files Coverage Δ
...mon/src/main/scala/org/apache/kyuubi/package.scala 0.00% <0.00%> (-100.00%) ⬇️
...main/scala/org/apache/kyuubi/KyuubiException.scala 0.00% <0.00%> (-100.00%) ⬇️
...ain/scala/org/apache/kyuubi/util/ThreadUtils.scala 0.00% <0.00%> (-100.00%) ⬇️
...in/scala/org/apache/kyuubi/engine/ShareLevel.scala 0.00% <0.00%> (-100.00%) ⬇️
...n/scala/org/apache/kyuubi/config/ConfigEntry.scala 0.00% <0.00%> (-100.00%) ⬇️
.../scala/org/apache/kyuubi/operation/Operation.scala 0.00% <0.00%> (-100.00%) ⬇️
...la/org/apache/kyuubi/ha/client/RetryPolicies.scala 0.00% <0.00%> (-100.00%) ⬇️
...scala/org/apache/kyuubi/cli/HandleIdentifier.scala 0.00% <0.00%> (-100.00%) ⬇️
...scala/org/apache/kyuubi/config/ConfigHelpers.scala 0.00% <0.00%> (-100.00%) ⬇️
...scala/org/apache/kyuubi/service/ServiceState.scala 0.00% <0.00%> (-100.00%) ⬇️
... and 106 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 65996a4...5706daf. Read the comment docs.

@pan3793 pan3793 force-pushed the jdk11 branch 3 times, most recently from acf755a to 7e20704 Compare July 3, 2021 07:39
@pan3793 pan3793 changed the title [WIP] [GA] Enable JDK 11 in GA [BUILD] [GA] Support JDK 11 and enable it in GitHub Action Jul 3, 2021
@pan3793 pan3793 self-assigned this Jul 3, 2021
@pan3793 pan3793 marked this pull request as ready for review July 3, 2021 12:52
@pan3793 pan3793 added this to the v1.3.0 milestone Jul 3, 2021
@yaooqinn
Copy link
Member

yaooqinn commented Jul 3, 2021

can you fix the conflicts? and open another issue to track the perf regress for jdk11?

@pan3793
Copy link
Member Author

pan3793 commented Jul 4, 2021

Rebased on master and open a new issue #745 to track the performance regression.

<artifactId>hive-service-rpc</artifactId>
</dependency>

<dependency>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

who/where use this in the common module?

Copy link
Member Author

@pan3793 pan3793 Jul 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use it. jaxb-api should be considered as a transitive dep of hadoop-client-runtime.
If we move jaxb-api to test scope, any accident invocation of Hadoop API which direct or indirect requires jaxb-api, the ut pass, but will cause runtime failures in production.

pom.xml Outdated
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
<filereports>TestSuite.txt</filereports>
<argLine>-Xmx4g -Xss4m -XX:ReservedCodeCacheSize=512M</argLine>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-Xmx4g -Xms4g -Xss32m -XX:ReservedCodeCacheSize=512M

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After changes, GA failed again, now shrink to -Xmx2g -Xms2g -Xss8m -XX:ReservedCodeCacheSize=512M and waiting GA results.


<profile>
<id>java-8</id>
<activation>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this for?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

auto activate profile by jdk version

@pan3793
Copy link
Member Author

pan3793 commented Jul 5, 2021

Thanks for reviewing, as all comments are addressed and GA passed, I'm going to merge it for v1.3.0.

@pan3793 pan3793 closed this in 5c5d43b Jul 5, 2021
@ulysses-you
Copy link
Contributor

late lgtm

@pan3793 pan3793 deleted the jdk11 branch July 5, 2021 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable JDK 11 in GA

4 participants