Skip to content

Comments

apache-kafka: replace ParNew to G1 YoungGeneration#100115

Closed
GTrunSec wants to merge 2 commits intoNixOS:masterfrom
GTrunSec:apache-kafka-gc
Closed

apache-kafka: replace ParNew to G1 YoungGeneration#100115
GTrunSec wants to merge 2 commits intoNixOS:masterfrom
GTrunSec:apache-kafka-gc

Conversation

@GTrunSec
Copy link
Contributor

@GTrunSec GTrunSec commented Oct 9, 2020

Motivation for this change
  1. the flag of UseParNewGC has been deprecated in JDK 9 and removed in JDK 10.

you can find info from here:

  1. remove CMS flags
    Dropping support for CMS and then removing the CMS code, or at least more thoroughly segregating it, will reduce the maintenance burden of the GC code base and accelerate new development. The G1 garbage collector is intended, in the long term, to be a replacement for most uses of CMS.

the issue output will be like this:

Unrecognized VM option 'UseParNewGC'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Things done
  • test flags
 more /nix/store/bq7qd8mmm63rwn1kzbw9rsfprnpa7q10-run-kafka/nix/store/y853yiia1ky7aipp73fv08xd2n6y3jyd-openjdk-14.0.2-ga/bin/java \
  -cp "/nix/store/kvq4jw9d9kzigdik4n2wrx1q9mzgapc2-apache-kafka-2.13-2.5.0/libs/*" \
  -Dlog4j.configuration=file:/nix/store/7qq35w8373pvbn9av0xk9c2rmwbas20m-log4j.properties \
  -server -Xmx1G -Xms1G -XX:+UseCompressedOops -XX:+UseG1GC -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSScavengeBeforeR
emark -XX:+DisableExplicitGC -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true \
  kafka.Kafka \
  /nix/store/ys10abnr79brsqh9ys1mxzx19jwmcd59-server.properties
  • output
    NOTICE: the ignoring warning will be sloved in second commit.
/nix/store/2jysm3dfsgby5sw5jgj43qjrb5v79ms9-bash-4.4-p23/bin/bash /nix/store/bq7qd8mmm63rwn1kzbw9rsfprnpa7q10-run-kafka
OpenJDK 64-Bit Server VM warning: Ignoring option UseConcMarkSweepGC; support was removed in 14.0
OpenJDK 64-Bit Server VM warning: Ignoring option CMSClassUnloadingEnabled; support was removed in 14.0
OpenJDK 64-Bit Server VM warning: Ignoring option CMSScavengeBeforeRemark; support was removed in 14.0
[2020-10-10 07:01:56,191] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2020-10-10 07:01:56,444] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2020-10-10 07:01:56,491] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2020-10-10 07:01:56,495] INFO starting (kafka.server.KafkaServer)
[2020-10-10 07:01:56,496] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2020-10-10 07:01:56,517] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2020-10-10 07:01:56,525] INFO Client environment:zookeeper.version=3.5.7-f0fdd52973d373ffd9c86b81d99842dc2c7f660e, built on 02/10/2020 11:30 GMT (org.apache.zookeeper.ZooKeeper)
[2020-10-10 07:01:56,525] INFO Client environment:host.name=gtcompile (org.apache.zookeeper.ZooKeeper)
[2020-10-10 07:01:56,525] INFO Client environment:java.version=14.0.2-internal (org.apache.zookeeper.ZooK
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. labels Oct 9, 2020
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there's a need to be explicit since G1 should be default GC since java 9.

Copy link
Contributor Author

@GTrunSec GTrunSec Oct 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/apache/kafka/blob/aba9036eb681597911343510e246de4fc547a479/bin/kafka-run-class.sh#L257. I think, the option of default should be matched with this line of the shell. otherwise, I can remove the default flags to list of example such as XX:+DisableExplicitGC was replaced by -XX:+ExplicitGCInvokesConcurrent in the default JVM settings. may this will be a good way to comment for users. it does not matter the flag is default setting or not. Or set the default list to empty(the example list as explain option).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants