Skip to content

Use @PreDestroy to ensure KuduClientWrapper is closed#12149

Merged
kokosing merged 1 commit intotrinodb:masterfrom
leveyja:leveyja/netty-resource-leak-warning
Apr 27, 2022
Merged

Use @PreDestroy to ensure KuduClientWrapper is closed#12149
kokosing merged 1 commit intotrinodb:masterfrom
leveyja:leveyja/netty-resource-leak-warning

Conversation

@leveyja
Copy link
Member

@leveyja leveyja commented Apr 27, 2022

Description

Netty's ResourceLeakDetector logged a warning
that the resource was not being released via KuduClientWrapper.

Is this change a fix, improvement, new feature, refactoring, or other?

Fix/Improvement

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

Connector (Kudu)

How would you describe this change to a non-technical end user or system administrator?

Ensure closable resources are closed during graceful shutdown

Related issues, pull requests, and links

Documentation

(*) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

(*) No release notes entries required.
( ) Release notes entries required with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Apr 27, 2022
@leveyja leveyja force-pushed the leveyja/netty-resource-leak-warning branch from e96ffdd to ee793b2 Compare April 27, 2022 05:31
@leveyja leveyja force-pushed the leveyja/netty-resource-leak-warning branch from ee793b2 to af952dd Compare April 27, 2022 05:56
@leveyja leveyja changed the title Use LifeCycleManager to ensure KuduClientWrapper is closed Use @PreDestroy to ensure KuduClientWrapper is closed Apr 27, 2022
Copy link
Member

@hashhar hashhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

@hashhar
Copy link
Member

hashhar commented Apr 27, 2022

I don't suppose there's an easy way to test this? (Maybe we might just end up testing an airlift feature instead of our code which doesn't add value).

@leveyja
Copy link
Member Author

leveyja commented Apr 27, 2022

@kokosing - I'm wrong about the LifeCycleManager not being required :-(
Because KuduClientSession is the Guice-injected instance, and not the underlying KuduClientWrapper (which holds the resources which must be closed), we do need to explicitly call LifeCycleManager.addInstance(Obj obj) to have the lifecycle managed.

Update: Resolved by moving @PreDestroy to KuduClientSession and invoking client.close() from the new lifecycle managed method.

@leveyja leveyja force-pushed the leveyja/netty-resource-leak-warning branch from 358552a to 45a05d9 Compare April 27, 2022 07:01
@leveyja leveyja requested review from hashhar and kokosing April 27, 2022 07:02
@hashhar
Copy link
Member

hashhar commented Apr 27, 2022

Some context of "why" this works (Guice and Bootstrap magic) in commit message will likely help any future developer trying to copy what you did here. (The comments in #12149 (comment) have the explanation I think).

@leveyja leveyja force-pushed the leveyja/netty-resource-leak-warning branch from 45a05d9 to 0bb7637 Compare April 27, 2022 08:18
@leveyja
Copy link
Member Author

leveyja commented Apr 27, 2022

Some context of "why" this works (Guice and Bootstrap magic) in commit message will likely help any future developer trying to copy what you did here. (The comments in #12149 (comment) have the explanation I think).

I've updated the commit message:

    Use @PreDestroy to ensure KuduClientSession releases resources on closure

    Netty's ResourceLeakDetector<HashedWheelTimer> logged a warning
    that the timer was not being released via KuduClientWrapper.

    KuduClientSession now has a @PreDestroy annotated method to release
    the KuduClientWrapper. This is called by Airlift's LifeCycleManager
    due to the presence of the @PreDestroy/@PostContruct javax
    annotations.

Not 100% sure if it captures it or if it could be shorted?

…sure

Netty's ResourceLeakDetector<HashedWheelTimer> logged a warning
that the timer was not being released via KuduClientWrapper.

KuduClientSession now has a @PreDestroy annotated method to release
the KuduClientWrapper. This is called by Airlift's LifeCycleManager
due to the presence of the @PreDestroy/@PostContruct javax
annotations.
Copy link
Member

@hashhar hashhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracking this down. I did verify that the KuduAsyncClient and KuduClientWrapper are being closed via debugger.

Copy link
Member

@hashhar hashhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm % checkstyle

@leveyja leveyja force-pushed the leveyja/netty-resource-leak-warning branch from 0bb7637 to af3122f Compare April 27, 2022 12:18
@grantatspothero
Copy link
Contributor

Thanks for doing this @leveyja

@kokosing kokosing merged commit e3fb371 into trinodb:master Apr 27, 2022
@kokosing
Copy link
Member

Thanks!

@github-actions github-actions bot added this to the 379 milestone Apr 27, 2022
@leveyja leveyja deleted the leveyja/netty-resource-leak-warning branch April 27, 2022 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

4 participants