HDDS-3824: OM read requests should make SCM#refreshPipeline outside BUCKET_LOCK #1164
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xiaoyuyao
reviewed
Jul 7, 2020
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
Show resolved
Hide resolved
xiaoyuyao
reviewed
Jul 7, 2020
Contributor
xiaoyuyao
left a comment
xiaoyuyao
left a comment
There was a problem hiding this comment.
LGTM, just few minor comments inline...
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
Show resolved
Hide resolved
Contributor
bharatviswa504
left a comment
bharatviswa504
left a comment
There was a problem hiding this comment.
Overall LGTM.
While reading I found one more thing, generating of the secret token can also be moved outside lock.
…also be moved outside lock
Codecov Report
@@ Coverage Diff @@
## master #1164 +/- ##
============================================
- Coverage 73.50% 73.36% -0.14%
+ Complexity 10031 10025 -6
============================================
Files 973 974 +1
Lines 49663 49721 +58
Branches 4881 4896 +15
============================================
- Hits 36503 36477 -26
- Misses 10846 10914 +68
- Partials 2314 2330 +16 Continue to review full report at Codecov.
|
Contributor
Author
|
@xiaoyuyao @bharatviswa504, |
xiaoyuyao
reviewed
Jul 13, 2020
Contributor
xiaoyuyao
left a comment
xiaoyuyao
left a comment
There was a problem hiding this comment.
Thanks for the update, @rakeshadr . Latest update LGTM, just two more minor comments.
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
Outdated
Show resolved
Hide resolved
Contributor
Author
|
Thanks @xiaoyuyao for the comments. I have updated PR, kindly review it again. |
Contributor
Author
|
Note:- TestWatchForCommit#test2WayCommitForTimeoutException failure is unrelated to my changes. It is passing in my local env. |
xiaoyuyao
approved these changes
Jul 14, 2020
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
Show resolved
Hide resolved
Contributor
Author
|
Thanks @xiaoyuyao and @bharatviswa504 for the reviews and committing the PR. |
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Jul 17, 2020
* master: HDDS-3855. Add upgrade smoketest (apache#1142) HDDS-3964. Ratis config key mismatch (apache#1204) HDDS-3612. Allow mounting bucket under other volume (apache#1104) HDDS-3926. OM Token Identifier table should use in-house serialization. (apache#1182) HDDS-3824: OM read requests should make SCM#refreshPipeline outside BUCKET_LOCK (apache#1164)
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Jul 17, 2020
…erface * upstream/master: HDDS-3855. Add upgrade smoketest (apache#1142) HDDS-3964. Ratis config key mismatch (apache#1204) HDDS-3612. Allow mounting bucket under other volume (apache#1104) HDDS-3926. OM Token Identifier table should use in-house serialization. (apache#1182) HDDS-3824: OM read requests should make SCM#refreshPipeline outside BUCKET_LOCK (apache#1164) HDDS-3966. Disable flaky TestOMRatisSnapshots
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Jul 20, 2020
* master: HDDS-3984. Support filter and search the columns in recon UI (apache#1218) HDDS-3806. Support recognize aws v2 Authorization header. (apache#1098) HDDS-3955. Unable to list intermediate paths on keys created using S3G. (apache#1196) HDDS-3741. Reload old OM state if Install Snapshot from Leader fails (apache#1129) HDDS-3965. SCM failed to start up for duplicated pipeline detected. (apache#1210) HDDS-3855. Add upgrade smoketest (apache#1142) HDDS-3964. Ratis config key mismatch (apache#1204) HDDS-3612. Allow mounting bucket under other volume (apache#1104) HDDS-3926. OM Token Identifier table should use in-house serialization. (apache#1182) HDDS-3824: OM read requests should make SCM#refreshPipeline outside BUCKET_LOCK (apache#1164) HDDS-3966. Disable flaky TestOMRatisSnapshots
errose28
pushed a commit
to errose28/ozone
that referenced
this pull request
Jul 21, 2020
timmylicheng
pushed a commit
that referenced
this pull request
Aug 6, 2020
rakeshadr
added a commit
to rakeshadr/hadoop-ozone
that referenced
this pull request
Sep 3, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Refresh pipeline info does a call to SCM and it can be moved outside the BUCKET_LOCK, this would help to improve the performance of read/write mix workloads.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3824
How was this patch tested?
Using TestOzoneFileSystem unit test cases.