-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix][broker] Disable system topic message deduplication #22582
[fix][broker] Disable system topic message deduplication #22582
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
close reopen to trigger ci checks |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #22582 +/- ##
============================================
- Coverage 73.57% 71.62% -1.95%
+ Complexity 32624 4849 -27775
============================================
Files 1877 1889 +12
Lines 139502 149548 +10046
Branches 15299 17591 +2292
============================================
+ Hits 102638 107117 +4479
- Misses 28908 33869 +4961
- Partials 7956 8562 +606
Flags with carried forward coverage won't be shown. Click here to find out more.
|
(cherry picked from commit 5ff0fb9)
Motivation
Disable deduplication on system topic to avoid recovering deduplication WAL (especially from offloaded topic). Because the system topic usually is a precondition to other topics. Therefore, we should pay attention to topic loading time.
Note: If the system topic loading timeout may cause dependent topics to fail to run.
Dependency diagram: normal topic --rely on--> system topic --rely on--> deduplication recover --may rely on--> (tiered storage)
Modifications
Topic#isDeduplicationEnabled
to judge if we should enable deduplication.Topic#isDeduplicationEnabled
to returnfalse
.Verifying this change
Documentation
doc
doc-required
doc-not-needed
doc-complete