Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][broker] Release EntryBuffer after parse proto object #20170

Merged
merged 3 commits into from
Apr 24, 2023

Conversation

coderzc
Copy link
Member

@coderzc coderzc commented Apr 23, 2023

Motivation & Modifications

Release EntryBuffer after parsing the proto object, It was introduced by PR #20158

Reduce data copy on the addEntry

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Apr 23, 2023
@coderzc coderzc requested a review from mattisonchao April 23, 2023 15:52
@coderzc coderzc added the type/bug The PR fixed a bug or issue reported a bug label Apr 23, 2023
@@ -127,7 +127,9 @@ private CompletableFuture<Void> addSnapshotSegments(LedgerHandle ledgerHandle,
private SnapshotMetadata parseSnapshotMetadataEntry(LedgerEntry ledgerEntry) {
try {
ByteBuf entryBuffer = ledgerEntry.getEntryBuffer();
return SnapshotMetadata.parseFrom(entryBuffer.nioBuffer());
SnapshotMetadata snapshotMetadata = SnapshotMetadata.parseFrom(entryBuffer.nioBuffer());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should add try-finally here to aovid any exception.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

@coderzc coderzc added this to the 3.1.0 milestone Apr 23, 2023
@@ -140,6 +142,7 @@ private List<SnapshotSegment> parseSnapshotSegmentEntries(Enumeration<LedgerEntr
SnapshotSegment snapshotSegment = new SnapshotSegment();
ByteBuf entryBuffer = ledgerEntry.getEntryBuffer();
snapshotSegment.parseFrom(entryBuffer, entryBuffer.readableBytes());
entryBuffer.release();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

@coderzc coderzc force-pushed the fix_release_entry_buffer branch 6 times, most recently from de36c72 to 03f6154 Compare April 23, 2023 16:28
@coderzc coderzc force-pushed the fix_release_entry_buffer branch from 03f6154 to 950f73b Compare April 23, 2023 16:43
@codecov-commenter
Copy link

Codecov Report

Merging #20170 (950f73b) into master (795d60a) will increase coverage by 0.39%.
The diff coverage is 73.33%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #20170      +/-   ##
============================================
+ Coverage     72.55%   72.95%   +0.39%     
- Complexity     3661    31960   +28299     
============================================
  Files          1868     1868              
  Lines        138457   138468      +11     
  Branches      15235    15236       +1     
============================================
+ Hits         100458   101017     +559     
+ Misses        29953    29415     -538     
+ Partials       8046     8036      -10     
Flag Coverage Δ
inttests 24.18% <0.00%> (+0.07%) ⬆️
systests 24.77% <0.00%> (-0.04%) ⬇️
unittests 72.25% <73.33%> (+0.43%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...elayed/bucket/BookkeeperBucketSnapshotStorage.java 83.73% <73.33%> (-1.98%) ⬇️

... and 122 files with indirect coverage changes

@coderzc coderzc merged commit 35e9897 into apache:master Apr 24, 2023
@coderzc coderzc deleted the fix_release_entry_buffer branch April 24, 2023 02:34
poorbarcode pushed a commit that referenced this pull request May 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/broker cherry-picked/branch-3.0 doc-not-needed Your PR changes do not impact docs release/3.0.1 type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants