Skip to content

Conversation

@devmadhuu
Copy link
Contributor

What changes were proposed in this pull request?

This PR fixes the Ozone Client connection leak by closing all opened client connections in NSSummaryAdmin.java

NSSummaryAdmin commands leaves the Ozone client connections open which is not a good practice and may consume resources. So. this PR fixes that by closing all opened Ozone client connections.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-9944

How was this patch tested?

This patch was tested using existing integration test org.apache.hadoop.ozone.shell.TestNSSummaryAdmin

@devmadhuu
Copy link
Contributor Author

@adoroszlai Pls review.

Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @devmadhuu for working on this.

There is a second item in the task:

it should reuse client (or even bucket after lookup) for all checks

All namespace subcommands use these methods as the following:

if (parent.isObjectStoreBucket(path) ||
!parent.bucketIsPresentInThePath(path)) {
printBucketReminder();
}

I don't think we need to use two separate clients for checking properties of the same bucket. The two methods can be combined.

Also, isFileSystemOptimizedBucket seems to be unused.

@devmadhuu
Copy link
Contributor Author

Thanks @devmadhuu for working on this.

There is a second item in the task:

it should reuse client (or even bucket after lookup) for all checks

All namespace subcommands use these methods as the following:

if (parent.isObjectStoreBucket(path) ||
!parent.bucketIsPresentInThePath(path)) {
printBucketReminder();
}

I don't think we need to use two separate clients for checking properties of the same bucket. The two methods can be combined.

Also, isFileSystemOptimizedBucket seems to be unused.

Thanks @adoroszlai for your review. I have implemented 2nd part of this JIRA :
--- NSSummaryAdmin all sub-commands should reuse ozone client (or even bucket after lookup) for all checks

Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @devmadhuu for updating the patch with re-use of client. Some minor items suggested.

deveshsingh added 2 commits January 19, 2024 10:36
Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @devmadhuu for updating the patch.

@adoroszlai adoroszlai requested a review from xBis7 January 19, 2024 12:53
@adoroszlai
Copy link
Contributor

@ArafatKhan2198 please review

Copy link
Contributor

@xBis7 xBis7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@devmadhuu Thanks for the patch. I've tested it locally. LGTM!

* @throws IOException
*/
public boolean bucketIsPresentInThePath(String path) throws IOException {
public boolean isNotValidBucketOrOBSBucket(String path) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see that for every command, this check is preceded by a check to whether the path exists. Therefore, there is no issue of a NPException.

Copy link
Contributor

@ArafatKhan2198 ArafatKhan2198 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the modifications, @devmadhuu. I was just curious about why we are directly utilizing println() instead of employing logging. I believe even for a CLI command it usually does log the message on the console right?

System.out.println(
    "Bucket layout couldn't be verified for path: " + ofsPath +
        ". Exception: " + e);

@adoroszlai adoroszlai merged commit d813fdf into apache:master Jan 19, 2024
@adoroszlai
Copy link
Contributor

Thanks @devmadhuu for the fix, @ArafatKhan2198, @xBis7 for the review.

@devmadhuu
Copy link
Contributor Author

Thank you for the modifications, @devmadhuu. I was just curious about why we are directly utilizing println() instead of employing logging. I believe even for a CLI command it usually does log the message on the console right?

System.out.println(
    "Bucket layout couldn't be verified for path: " + ofsPath +
        ". Exception: " + e);

Thanks @ArafatKhan2198 for review. These are CLI based interface, so need to print on console.

Tejaskriya pushed a commit to Tejaskriya/ozone that referenced this pull request Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants