Fix CVE-2022-34917 about kafka-client dependency #24097

adkharat · 2024-11-20T04:48:03Z

Description

CVE-2022-34917
Previous Kafka version to 2.3.1
Upgraded Kafka version to 3.9.0

A security vulnerability has been identified in Apache Kafka. It affects all releases since 2.8.0. The vulnerability allows malicious unauthenticated clients to allocate large amounts of memory on brokers. This can lead to brokers hitting OutOfMemoryException and causing denial of service. Example scenarios: - Kafka cluster without authentication: Any clients able to establish a network connection to a broker can trigger the issue. - Kafka cluster with SASL authentication: Any clients able to establish a network connection to a broker, without the need for valid SASL credentials, can trigger the issue. - Kafka cluster with TLS authentication: Only clients able to successfully authenticate via TLS can trigger the issue. We advise the users to upgrade the Kafka installations to one of the 3.2.3, 3.1.2, 3.0.2, 2.8.2 versions.

Motivation and Context

Impact

Test Plan

Contributor checklist

Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
Documented new properties (with its default value), SQL syntax, functions, or other functionality.
If release notes are required, they follow the release notes guidelines.
Adequate tests were added if applicable.
CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

Security Changes
* Upgrade kafka to 3.9.0 in response to `CVE-2022-34917 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-34917>`_. :pr:`24097`

adkharat · 2024-11-20T09:31:16Z

@imjalpreet

I see unit test failures after upgrading Kafka from version 2.3.1 to 2.8.2. (Kafka 2.8.2 fixes CVE-2022-34917)

On log analysis, I see,

.github->workflows-> product-tests-basic-environment.yml -> executes presto-product-tests/bin/run_on_docker.sh -> executes presto-product-tests/conf/docker/files/run-tempto.sh

run-tempto.sh shell script is used to execute test cases using the TemptoProductTestRunner.java

Integration tests for "Presto" is dependent on tempto-kafka/tempto-runner from repo prestodb/tempto. Tempto dependency is using older versions:

kafka = 0.11.0.2
zkclient = 0.10

Questions:

1. Upgrade Tempto dependencies?
Should I consider updating the Kafka and Zkclient versions within the [prestodb/tempto]. (https://github.com/prestodb/tempto/blob/master/build.gradle) repository? Are there any Tempto maintainers for [prestodb/tempto] as I don't see any activities on tempto repo in recent year.

2. Testing Tempto changes (if applicable)
If upgrading Kafka and depricated function in tempto is necessary, how can I effectively test the modified tempto functionality without breaking UT functionality ?

mvn dependency:tree -Dverbose

[INFO] +- io.prestodb.tempto:tempto-kafka:jar:1.53:compile
[INFO] |  +- org.apache.kafka:kafka_2.12:jar:2.8.2:compile
[INFO] |  |  +- org.apache.kafka:kafka-metadata:jar:2.8.2:compile
[INFO] |  |  +- org.apache.kafka:kafka-raft:jar:2.8.2:compile
[INFO] |  |  +- net.sourceforge.argparse4j:argparse4j:jar:0.7.0:compile
[INFO] |  |  +- com.fasterxml.jackson.module:jackson-module-scala_2.12:jar:2.10.5:compile
[INFO] |  |  |  \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.10.5:compile
[INFO] |  |  +- com.fasterxml.jackson.dataformat:jackson-dataformat-csv:jar:2.10.5:compile
[INFO] |  |  +- net.sf.jopt-simple:jopt-simple:jar:5.0.4:compile
[INFO] |  |  +- com.yammer.metrics:metrics-core:jar:2.2.0:compile
[INFO] |  |  +- org.scala-lang.modules:scala-collection-compat_2.12:jar:2.3.0:compile
[INFO] |  |  +- org.scala-lang.modules:scala-java8-compat_2.12:jar:0.9.1:compile
[INFO] |  |  +- org.scala-lang:scala-reflect:jar:2.12.13:compile
[INFO] |  |  \- com.typesafe.scala-logging:scala-logging_2.12:jar:3.9.2:compile
[INFO] |  \- com.101tec:zkclient:jar:0.10:compile
[INFO] |     \- org.apache.zookeeper:zookeeper:jar:3.4.14:compile
[INFO] |        +- com.github.spotbugs:spotbugs-annotations:jar:3.1.10:compile
[INFO] |        +- org.apache.yetus:audience-annotations:jar:0.5.0:compile
[INFO] |        \- io.netty:netty:jar:3.10.6.Final:compile

kafka.utils.ZkUtils was removed in Kafka - Version 2.4.0

2.0.0- > ZKUtils present
2.3.1 -> ZKUtils present
2.4.0 -> ZKUtils removed
2.8.2 -> ZKUtils not present (Version 2.8.2 has some vulnerability)
3.7.1 -> ZKUtils not present (No vulnerability)
3.9.0 -> ZKUtils not present (Latest version + No vulnerability)

Reference link:

steveburnett · 2024-11-20T15:43:53Z

Thanks for the great release note! Just one nit suggestion about kafka as related to #18721.

== RELEASE NOTES ==

Security Changes
* Upgrade Apache Kafka to 2.8.2 in response to `CVE-2022-34917 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-34917>`_. :pr:`24097`

imjalpreet · 2024-12-06T06:35:48Z

FYI this PR: #24211 uses the modified tempto with the kafka upgrade changes. All the tests passed.

pom.xml

ZacBlanco · 2024-12-06T21:36:11Z

It looks like there are some test failures:

2024-12-06T19:55:35.4935408Z 
2024-12-06T19:55:35.5122198Z [INFO] 
2024-12-06T19:55:35.5122509Z [INFO] Results:
2024-12-06T19:55:35.5122815Z [INFO] 
2024-12-06T19:55:35.5123088Z [ERROR] Failures: 
2024-12-06T19:55:35.5123585Z [ERROR]   TestKafkaIntegrationSmokeTest.destroy:144 NullPointer
2024-12-06T19:55:35.5126355Z [ERROR]   TestKafkaIntegrationSmokeTest>AbstractTestQueryFramework.init:85->createQueryRunner:45 » NoClassDefFound org/apache/zookeeper/client/ZKClientConfig
2024-12-06T19:55:35.5128199Z [ERROR]   TestMinimalFunctionality.startKafka:61 » NoClassDefFound org/apache/zookeeper/client/ZKClientConfig
2024-12-06T19:55:35.5129268Z [ERROR]   TestMinimalFunctionality.stopKafka:69 NullPointer
2024-12-06T19:55:35.5129982Z [ERROR] com.facebook.presto.kafka.TestMinimalFunctionality.tearDown
2024-12-06T19:55:35.5130711Z [ERROR]   Run 1: TestMinimalFunctionality.tearDown:93 NullPointer
2024-12-06T19:55:35.5131395Z [ERROR]   Run 2: TestMinimalFunctionality.tearDown:93 NullPointer
2024-12-06T19:55:35.5131909Z [INFO] 
2024-12-06T19:55:35.5132151Z [INFO] 
2024-12-06T19:55:35.5134318Z [ERROR] Tests run: 32, Failures: 5, Errors: 0, Skipped: 20
2024-12-06T19:55:35.5134842Z [INFO]

adkharat · 2024-12-11T10:43:49Z

It looks like there are some test failures:

2024-12-06T19:55:35.4935408Z 
2024-12-06T19:55:35.5122198Z [INFO] 
2024-12-06T19:55:35.5122509Z [INFO] Results:
2024-12-06T19:55:35.5122815Z [INFO] 
2024-12-06T19:55:35.5123088Z [ERROR] Failures: 
2024-12-06T19:55:35.5123585Z [ERROR]   TestKafkaIntegrationSmokeTest.destroy:144 NullPointer
2024-12-06T19:55:35.5126355Z [ERROR]   TestKafkaIntegrationSmokeTest>AbstractTestQueryFramework.init:85->createQueryRunner:45 » NoClassDefFound org/apache/zookeeper/client/ZKClientConfig
2024-12-06T19:55:35.5128199Z [ERROR]   TestMinimalFunctionality.startKafka:61 » NoClassDefFound org/apache/zookeeper/client/ZKClientConfig
2024-12-06T19:55:35.5129268Z [ERROR]   TestMinimalFunctionality.stopKafka:69 NullPointer
2024-12-06T19:55:35.5129982Z [ERROR] com.facebook.presto.kafka.TestMinimalFunctionality.tearDown
2024-12-06T19:55:35.5130711Z [ERROR]   Run 1: TestMinimalFunctionality.tearDown:93 NullPointer
2024-12-06T19:55:35.5131395Z [ERROR]   Run 2: TestMinimalFunctionality.tearDown:93 NullPointer
2024-12-06T19:55:35.5131909Z [INFO] 
2024-12-06T19:55:35.5132151Z [INFO] 
2024-12-06T19:55:35.5134318Z [ERROR] Tests run: 32, Failures: 5, Errors: 0, Skipped: 20
2024-12-06T19:55:35.5134842Z [INFO]

@ZacBlanco I have pushed changes fixing above test cases failure in presto-kafka module

pratyakshsharma · 2024-12-16T09:19:37Z

@adkharat can you please mark this as ready for review?

pratyakshsharma · 2024-12-17T08:36:39Z

presto-kafka/src/main/java/com/facebook/presto/kafka/KafkaConnectorConfig.java

+        return resourceConfigFiles;
+    }
+
+    @Config("kafka.config.resources")


why is this new config needed?

why is this new config needed?

@pratyakshsharma new config removed

Upgraded version to 3.7.1 upgraded to 3.9.0 fixed presto-kafka test cases excluded io.netty combined exclusion

prestodb-ci added the from:IBM PR from IBM label Nov 20, 2024

adkharat force-pushed the kafka_clients_cve_fix branch from 5c7e666 to 7d921b6 Compare November 20, 2024 08:41

adkharat mentioned this pull request Nov 28, 2024

upgraded_kafka_client_version_to_3.9.0 prestodb/tempto#279

Closed

imjalpreet requested a review from ZacBlanco December 5, 2024 23:03

imjalpreet mentioned this pull request Dec 6, 2024

[Test #24097] Fix CVE-2022-34917 #24211

Draft

imjalpreet reviewed Dec 6, 2024

View reviewed changes

pom.xml Outdated Show resolved Hide resolved

adkharat force-pushed the kafka_clients_cve_fix branch from 3d61ea4 to 0b0c472 Compare December 11, 2024 10:25

adkharat marked this pull request as ready for review December 16, 2024 11:09

adkharat requested a review from a team as a code owner December 16, 2024 11:09

adkharat requested a review from presto-oss December 16, 2024 11:09

pratyakshsharma reviewed Dec 17, 2024

View reviewed changes

This was referenced Dec 17, 2024

[Test #24097] Ut fix check kafka clients CVE fix #24268

Closed

[Test #24097] Ut fix check kafka clients CVE fix #24269

Draft

Fix CVE-2022-34917 about kafka-client dependency

74a1eb0

Upgraded version to 3.7.1 upgraded to 3.9.0 fixed presto-kafka test cases excluded io.netty combined exclusion

adkharat force-pushed the kafka_clients_cve_fix branch from 14977b7 to 8e64e1b Compare December 17, 2024 18:40

removed new config from kafka connector

52e6a76

adkharat force-pushed the kafka_clients_cve_fix branch from 8e64e1b to 52e6a76 Compare December 17, 2024 18:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix CVE-2022-34917 about kafka-client dependency #24097

Fix CVE-2022-34917 about kafka-client dependency #24097

Uh oh!

adkharat commented Nov 20, 2024 •

edited

Loading

Uh oh!

adkharat commented Nov 20, 2024 •

edited

Loading

Uh oh!

steveburnett commented Nov 20, 2024

Uh oh!

imjalpreet commented Dec 6, 2024

Uh oh!

Uh oh!

ZacBlanco commented Dec 6, 2024

Uh oh!

adkharat commented Dec 11, 2024 •

edited

Loading

Uh oh!

pratyakshsharma commented Dec 16, 2024

Uh oh!

pratyakshsharma Dec 17, 2024

Uh oh!

adkharat Dec 17, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Fix CVE-2022-34917 about kafka-client dependency #24097

Are you sure you want to change the base?

Fix CVE-2022-34917 about kafka-client dependency #24097

Uh oh!

Conversation

adkharat commented Nov 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Impact

Test Plan

Contributor checklist

Release Notes

Uh oh!

adkharat commented Nov 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steveburnett commented Nov 20, 2024

Uh oh!

imjalpreet commented Dec 6, 2024

Uh oh!

Uh oh!

ZacBlanco commented Dec 6, 2024

Uh oh!

adkharat commented Dec 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pratyakshsharma commented Dec 16, 2024

Uh oh!

pratyakshsharma Dec 17, 2024

Choose a reason for hiding this comment

Uh oh!

adkharat Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

adkharat commented Nov 20, 2024 •

edited

Loading

adkharat commented Nov 20, 2024 •

edited

Loading

adkharat commented Dec 11, 2024 •

edited

Loading

adkharat Dec 17, 2024 •

edited

Loading