Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][io] Remove kafka-connect-avro-converter-shaded #19468

Merged
merged 3 commits into from
Feb 9, 2023

Conversation

tisonkun
Copy link
Member

@tisonkun tisonkun commented Feb 9, 2023

This can be a follow-up to #19010 and the original introduction PR is #6034.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)

kafka-connect-avro-converter-shaded as a temporary hack is no longer needed.

  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: tisonkun#23

@tisonkun tisonkun self-assigned this Feb 9, 2023
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Feb 9, 2023
@tisonkun tisonkun changed the title [build][io] Remove kafka-connect-avro-converter-shaded [improve][io] Remove kafka-connect-avro-converter-shaded Feb 9, 2023
@tisonkun
Copy link
Member Author

tisonkun commented Feb 9, 2023

Interesting. A steady unrelated test failure:

2023-02-09T05:21:31.9981562Z [ERROR] testDebeziumMySqlSourceAvro(org.apache.pulsar.tests.integration.io.sources.debezium.PulsarDebeziumSourcesTest)  Time elapsed: 59.421 s  <<< FAILURE!
2023-02-09T05:21:31.9982350Z java.lang.AssertionError: expected [1] but found [0]
2023-02-09T05:21:31.9983177Z 	at org.testng.Assert.fail(Assert.java:110)
2023-02-09T05:21:31.9983560Z 	at org.testng.Assert.failNotEquals(Assert.java:1413)
2023-02-09T05:21:31.9983926Z 	at org.testng.Assert.assertEqualsImpl(Assert.java:149)
2023-02-09T05:21:31.9984285Z 	at org.testng.Assert.assertEquals(Assert.java:131)
2023-02-09T05:21:31.9984627Z 	at org.testng.Assert.assertEquals(Assert.java:1240)
2023-02-09T05:21:31.9984973Z 	at org.testng.Assert.assertEquals(Assert.java:1274)
2023-02-09T05:21:31.9985580Z 	at org.apache.pulsar.tests.integration.io.sources.PulsarIOSourceRunner.getSourceStatus(PulsarIOSourceRunner.java:212)
2023-02-09T05:21:31.9987243Z 	at org.apache.pulsar.tests.integration.io.sources.debezium.PulsarIODebeziumSourceRunner.lambda$internalTestSource$0(PulsarIODebeziumSourceRunner.java:84)
2023-02-09T05:21:31.9988177Z 	at net.jodah.failsafe.Functions.lambda$toSupplier$10(Functions.java:262)
2023-02-09T05:21:31.9988574Z 	at net.jodah.failsafe.Functions.lambda$get$0(Functions.java:48)
2023-02-09T05:21:31.9989032Z 	at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66)
2023-02-09T05:21:31.9989494Z 	at net.jodah.failsafe.Execution.executeSync(Execution.java:128)
2023-02-09T05:21:31.9989922Z 	at net.jodah.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:379)
2023-02-09T05:21:31.9990391Z 	at net.jodah.failsafe.FailsafeExecutor.run(FailsafeExecutor.java:212)
2023-02-09T05:21:31.9991143Z 	at org.apache.pulsar.tests.integration.io.sources.debezium.PulsarIODebeziumSourceRunner.internalTestSource(PulsarIODebeziumSourceRunner.java:84)
2023-02-09T05:21:31.9992154Z 	at org.apache.pulsar.tests.integration.io.sources.debezium.PulsarIODebeziumSourceRunner.testSource(PulsarIODebeziumSourceRunner.java:66)
2023-02-09T05:21:31.9993162Z 	at org.apache.pulsar.tests.integration.io.sources.debezium.PulsarDebeziumSourcesTest.testDebeziumMySqlConnect(PulsarDebeziumSourcesTest.java:130)
2023-02-09T05:21:31.9994240Z 	at org.apache.pulsar.tests.integration.io.sources.debezium.PulsarDebeziumSourcesTest.testDebeziumMySqlSourceAvro(PulsarDebeziumSourcesTest.java:63)
2023-02-09T05:21:31.9995045Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-02-09T05:21:31.9995609Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2023-02-09T05:21:31.9996233Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2023-02-09T05:21:31.9996744Z 	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
2023-02-09T05:21:31.9997596Z 	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
2023-02-09T05:21:31.9998206Z 	at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
2023-02-09T05:21:31.9998769Z 	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
2023-02-09T05:21:31.9999327Z 	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
2023-02-09T05:21:31.9999810Z 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2023-02-09T05:21:32.0000294Z 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
2023-02-09T05:21:32.0000827Z 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
2023-02-09T05:21:32.0001240Z 	at java.base/java.lang.Thread.run(Thread.java:833)
2023-02-09T05:21:32.0001426Z 

May revisit or rerun later.

@tisonkun
Copy link
Member Author

tisonkun commented Feb 9, 2023

OK. Found another shade class reference.

@codecov-commenter
Copy link

Codecov Report

Merging #19468 (2a89a2f) into master (9ab9d01) will increase coverage by 0.43%.
The diff coverage is 65.03%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #19468      +/-   ##
============================================
+ Coverage     63.20%   63.64%   +0.43%     
+ Complexity     3475     3474       -1     
============================================
  Files          1832     1832              
  Lines        134159   134144      -15     
  Branches      14763    14756       -7     
============================================
+ Hits          84795    85370     +575     
+ Misses        41642    40986     -656     
- Partials       7722     7788      +66     
Flag Coverage Δ
inttests 24.87% <51.79%> (+0.23%) ⬆️
systests 25.46% <0.71%> (-0.12%) ⬇️
unittests 61.05% <65.03%> (+0.63%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...r/io/kafka/connect/AbstractKafkaConnectSource.java 58.67% <ø> (ø)
...he/pulsar/io/kafka/connect/KafkaConnectSource.java 60.71% <ø> (ø)
...nsaction/coordinator/TransactionMetadataStore.java 50.00% <ø> (ø)
...oordinator/impl/InMemTransactionMetadataStore.java 72.88% <25.00%> (-3.91%) ⬇️
...va/org/apache/pulsar/broker/service/ServerCnx.java 55.45% <54.79%> (+0.47%) ⬆️
...pulsar/broker/TransactionMetadataStoreService.java 60.75% <75.00%> (+2.06%) ⬆️
...n/coordinator/impl/MLTransactionMetadataStore.java 74.44% <75.00%> (-1.21%) ⬇️
...che/pulsar/broker/admin/impl/TransactionsBase.java 72.06% <100.00%> (+0.09%) ⬆️
...lsar/common/policies/data/TransactionMetadata.java 87.50% <100.00%> (+1.78%) ⬆️
...g/apache/pulsar/common/protocol/PulsarHandler.java 60.34% <100.00%> (+3.44%) ⬆️
... and 154 more

Copy link
Contributor

@nicoloboschi nicoloboschi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
cc @dlg99

@nicoloboschi nicoloboschi added this to the 3.0.0 milestone Feb 9, 2023
@tisonkun
Copy link
Member Author

tisonkun commented Feb 9, 2023

OWASP failed on stax2 which is unrelated to this patch I think.

@lhotari lhotari merged commit b969fe5 into apache:master Feb 9, 2023
@tisonkun tisonkun deleted the remove-kafka-avro-hack branch March 12, 2023 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants