Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][client] Create the retry producer async #23157

Merged
merged 3 commits into from
Aug 14, 2024

Conversation

omarkj
Copy link
Contributor

@omarkj omarkj commented Aug 12, 2024

Motivation

The consumer implementation is creating the retry producer synchronously, potentially unexpectedly blocking the calling thread.

Modifications

This changes modifies this behaviour to fit with how the dead letter queue producer is created—i.e. using the async APIs offloaded to the internal pinned executor.

Verifying this change

  • Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

The consumer implementation is creating the retry producer synchronously,
potentially unexpectedly blocking the calling thread.

This changes modifies this behaviour to fit with how the dead letter queue
producer is created—i.e. using the async APIs offloaded to the internal
pinned executor.
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Aug 12, 2024
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the contribution @omarkj

@omarkj
Copy link
Contributor Author

omarkj commented Aug 12, 2024

I'm unable to replicate the failed test locally.

@lhotari
Copy link
Member

lhotari commented Aug 13, 2024

I'm unable to replicate the failed test locally.

Unfortunately we have a lot of flaky tests. There's a solution called "pulsarbot" which can rerun the failed GitHub Action builds when you add a comment "/pulsarbot rerun-failure-checks" to the PR. I'll perform that now.

@lhotari
Copy link
Member

lhotari commented Aug 13, 2024

/pulsarbot rerun-failure-checks

Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check the comments about error handling the creation of the retry letter producer fails.

@omarkj omarkj requested a review from lhotari August 13, 2024 21:37
@codecov-commenter
Copy link

codecov-commenter commented Aug 14, 2024

Codecov Report

Attention: Patch coverage is 86.84211% with 5 lines in your changes missing coverage. Please review.

Project coverage is 74.56%. Comparing base (bbc6224) to head (4c5f92d).
Report is 520 commits behind head on master.

Files Patch % Lines
...va/org/apache/pulsar/client/impl/ConsumerImpl.java 86.84% 2 Missing and 3 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #23157      +/-   ##
============================================
+ Coverage     73.57%   74.56%   +0.98%     
- Complexity    32624    33637    +1013     
============================================
  Files          1877     1920      +43     
  Lines        139502   144434    +4932     
  Branches      15299    15803     +504     
============================================
+ Hits         102638   107693    +5055     
+ Misses        28908    28490     -418     
- Partials       7956     8251     +295     
Flag Coverage Δ
inttests 27.56% <71.05%> (+2.97%) ⬆️
systests 24.70% <0.00%> (+0.37%) ⬆️
unittests 73.92% <86.84%> (+1.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...va/org/apache/pulsar/client/impl/ConsumerImpl.java 79.91% <86.84%> (+2.34%) ⬆️

... and 496 files with indirect coverage changes

@lhotari lhotari merged commit a025938 into apache:master Aug 14, 2024
51 checks passed
Technoboy- pushed a commit that referenced this pull request Aug 14, 2024
lhotari pushed a commit that referenced this pull request Aug 14, 2024
Co-authored-by: Ómar Yasin <[email protected]>
(cherry picked from commit a025938)
@omarkj omarkj deleted the fix-create-retry-client-producer-async branch August 14, 2024 15:21
lhotari pushed a commit that referenced this pull request Aug 15, 2024
Co-authored-by: Ómar Yasin <[email protected]>
(cherry picked from commit a025938)
lhotari pushed a commit that referenced this pull request Aug 15, 2024
Co-authored-by: Ómar Yasin <[email protected]>
(cherry picked from commit a025938)
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Aug 16, 2024
Co-authored-by: Ómar Yasin <[email protected]>
(cherry picked from commit a025938)
(cherry picked from commit 507d402)
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Aug 16, 2024
Co-authored-by: Ómar Yasin <[email protected]>
(cherry picked from commit a025938)
(cherry picked from commit f2e1920)
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Aug 16, 2024
Co-authored-by: Ómar Yasin <[email protected]>
(cherry picked from commit a025938)
(cherry picked from commit 507d402)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Aug 20, 2024
Co-authored-by: Ómar Yasin <[email protected]>
(cherry picked from commit a025938)
(cherry picked from commit 507d402)
grssam pushed a commit to grssam/pulsar that referenced this pull request Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants