Skip to content

HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b… #2946

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
captainzmc merged 1 commit into from
Feb 10, 2022
Merged

HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b… #2946

captainzmc merged 1 commit into from
Feb 10, 2022

Conversation

@sadanand48
Copy link
Contributor

@sadanand48 sadanand48 commented Dec 24, 2021 •
edited
Loading

…ufferList.

What changes were proposed in this pull request?

This is a follow up to HDDS-5851. Define a limit/window on the bufferList size which is used for retry. Once this limit is hit , client will perform watchForCommit for the first flush size (first putBlock) worth of data.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-6138

How was this patch tested?

unit tests

@sadanand48 sadanand48 marked this pull request as ready for review January 12, 2022 07:51
@captainzmc
Copy link
Member

captainzmc commented Jan 21, 2022

Hi @sadanand48, can you update this PR?

@captainzmc
Copy link
Member

captainzmc commented Jan 27, 2022

Hi @sadanand48, I tried the cherry-pick PR in my local test, but this CI never passed. Can you confirm the problem of CI?

@sadanand48
Copy link
Contributor Author

sadanand48 commented Feb 9, 2022

@captainzmc I have fixed some tests that were failing. The current CI failure I think is related to HDDS-6239 . We should sync master with this branch to resolve this

@sadanand48 sadanand48 requested a review from captainzmc February 9, 2022 10:17
captainzmc
captainzmc reviewed Feb 9, 2022
View reviewed changes
@captainzmc
Copy link
Member

captainzmc commented Feb 9, 2022

@captainzmc I have fixed some tests that were failing. The current CI failure I think is related to HDDS-6239 . We should sync master with this branch to resolve this

Thanks @sadanand48 for the update. Just now I have synchronized HDDS-4454 with master. Could you rebase the code? The PR looks good overall, just left a minor comment.

@captainzmc
Copy link
Member

captainzmc commented Feb 10, 2022

Hi @sadanand48, In #3059 we had update ratis version and resolve dependency check issues. Currently #3059 has been merged, So you can remove the modification of these two parts in this PR.

@sadanand48
Copy link
Contributor Author

sadanand48 commented Feb 10, 2022

Thanks @captainzmc for taking care of the dependency check. I have rebased the PR now.

captainzmc
captainzmc approved these changes Feb 10, 2022
View reviewed changes
Copy link
Member

@captainzmc captainzmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 The change looks good.

@captainzmc captainzmc merged commit ad7b45e into apache:HDDS-4454 Feb 10, 2022
@captainzmc
Copy link
Member

captainzmc commented Feb 10, 2022

Merge this PR. Thanks for @sadanand48’s contribution and @bshashikant for the review.

SincereXIA pushed a commit to SincereXIA/ozone that referenced this pull request Feb 11, 2022
@sadanand48 @SincereXIA
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
3fe8965 
…ufferList. (apache#2946)

Co-authored-by: Sadanand Shenoy <[email protected]>
szetszwo pushed a commit that referenced this pull request Feb 16, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
bc87050 
…ufferList. (#2946)
szetszwo pushed a commit that referenced this pull request Mar 15, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
4f9265e 
…ufferList. (#2946)
szetszwo pushed a commit that referenced this pull request Mar 24, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
d290fba 
…ufferList. (#2946)
szetszwo pushed a commit to szetszwo/ozone that referenced this pull request May 6, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
1bbc76f 
…ufferList. (apache#2946)
szetszwo pushed a commit that referenced this pull request May 13, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
60fedb0 
…ufferList. (#2946)
szetszwo pushed a commit that referenced this pull request May 24, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
5dce655 
…ufferList. (#2946)
szetszwo pushed a commit that referenced this pull request Jun 9, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
32598ca 
…ufferList. (#2946)
captainzmc pushed a commit to captainzmc/hadoop-ozone that referenced this pull request Jul 4, 2022
@sadanand48 @captainzmc
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
30f8311 
…ufferList. (apache#2946)
szetszwo pushed a commit that referenced this pull request Oct 25, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
693eea7 
…ufferList. (#2946)

(cherry picked from commit 30f8311)
(cherry picked from commit 54361e5a4ded763d83c5b3116203deece0303960)
szetszwo pushed a commit that referenced this pull request Nov 7, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
340f959 
…ufferList. (#2946)

(cherry picked from commit 30f8311)
(cherry picked from commit 54361e5a4ded763d83c5b3116203deece0303960)
(cherry picked from commit 693eea7)
szetszwo pushed a commit that referenced this pull request Dec 1, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
3b1d2cf 
…ufferList. (#2946)

(cherry picked from commit 340f959)
szetszwo pushed a commit that referenced this pull request Dec 16, 2022
@sadanand48 @szetszwo
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
2657f5a 
…ufferList. (#2946)

(cherry picked from commit 340f959)
nishitpatira pushed a commit to nishitpatira/ozone that referenced this pull request Dec 16, 2022
@sadanand48
HDDS-6138.[Ozone-Streaming] Define a limit on the size of the retry b…
47145dc 
…ufferList. (apache#2946)

(cherry picked from commit 340f959)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bshashikant bshashikant bshashikant left review comments

@captainzmc captainzmc captainzmc approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sadanand48 @captainzmc @bshashikant