Skip to content

Conversation

@yihua
Copy link
Contributor

@yihua yihua commented Apr 18, 2022

What is the purpose of the pull request

This PR fixes the NoClassDefFoundError due to org.apache.hudi.schema.FilebasedSchemaProvider. The class is not packaged to the hudi-kafka-connect-bundle because of changes in flink modules from #5072 (Right now, Kafka Connect Sink for Hudi temporarily relies on schema provider classes in Flink that do not use Spark context). The PR corrects the shade include rule for the flink module.

java.lang.NoClassDefFoundError: org/apache/hudi/schema/FilebasedSchemaProvider
    at org.apache.hudi.connect.writers.KafkaConnectConfigs.<clinit>(KafkaConnectConfigs.java:57)
    at org.apache.hudi.connect.HoodieSinkTask.start(HoodieSinkTask.java:80)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:308)
    at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:196)
    at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)
    at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:241)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Brief change log

  • Corrects the shade include rule of hudi-flink in packaging/hudi-kafka-connect-bundle/pom.xml.

Verify this pull request

The PR is verified by running the quick start guide of Kafka Connect Sink for Hudi and the NoClassDefFoundError is no longer thrown.

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@yihua yihua added the priority:blocker Production down; release blocker label Apr 18, 2022
@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@nsivabalan nsivabalan merged commit 52d878c into apache:master Apr 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority:blocker Production down; release blocker

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants