Skip to content

Fix Netty leak on event loop shutdown#97301

Merged
original-brownbear merged 1 commit intoelastic:mainfrom
original-brownbear:fix-netty-leak-clean
Jul 2, 2023
Merged

Fix Netty leak on event loop shutdown#97301
original-brownbear merged 1 commit intoelastic:mainfrom
original-brownbear:fix-netty-leak-clean

Conversation

@original-brownbear
Copy link
Copy Markdown
Contributor

Follow up to #96856, turns out this wasn't safe to remove after all. We can't go back to the previous solution though since that had double invocation issues => use notify-once for now until this is fixed in Netty.

Follow up to #96856, turns out this wasn't safe to remove after all.
We can't go back to the previous solution though since that had double invocation issues
=> use notify-once for now until this is fixed in Netty.
@original-brownbear original-brownbear added >test Issues or PRs that are addressing/adding tests :Distributed/Network Http and internode communication implementations labels Jul 1, 2023
@elasticsearchmachine elasticsearchmachine added Team:Distributed Meta label for distributed team. v8.10.0 labels Jul 1, 2023
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

Copy link
Copy Markdown
Member

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks safe to me 👍

@original-brownbear
Copy link
Copy Markdown
Contributor Author

Thanks David!

@original-brownbear original-brownbear merged commit fdcecd9 into elastic:main Jul 2, 2023
@original-brownbear original-brownbear deleted the fix-netty-leak-clean branch July 2, 2023 09:30
original-brownbear added a commit that referenced this pull request Jul 10, 2023
This is used on the rather hot path now due to #97301, lets apply the optimization
of saving one level of indirection here as well to make GC etc. a little cheaper on these.
@original-brownbear original-brownbear restored the fix-netty-leak-clean branch November 8, 2023 22:00
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this pull request Feb 8, 2024
Similar to elastic#97301, the fix in elastic#105293 was still not quite correct: we
could in principle shut down the transport after checking `isOpen()` but
before sending the message. Applying the same fix as for the transport
layer here.
DaveCTurner added a commit that referenced this pull request Feb 9, 2024
Similar to #97301, the fix in #105293 was still not quite correct: we
could in principle shut down the transport after checking `isOpen()` but
before sending the message. Applying the same fix as for the transport
layer here.
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this pull request Feb 9, 2024
Similar to elastic#97301, the fix in elastic#105293 was still not quite correct: we
could in principle shut down the transport after checking `isOpen()` but
before sending the message. Applying the same fix as for the transport
layer here.
elasticsearchmachine pushed a commit that referenced this pull request Feb 9, 2024
Similar to #97301, the fix in #105293 was still not quite correct: we
could in principle shut down the transport after checking `isOpen()` but
before sending the message. Applying the same fix as for the transport
layer here.
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this pull request Feb 14, 2024
We could still be manipulating a network message when the event loop
shuts down, causing us to close the message while it's still in use.
This is at best going to be a little surprising to the caller, and at
worst could be an outright use-after-free bug.

This commit moves the double-check for a leaked promise to happen
strictly after the event loop has fully terminated, so that we can be
sure we've finished using it by this point.

Relates elastic#105306, elastic#97301
elasticsearchmachine pushed a commit that referenced this pull request Feb 15, 2024
We could still be manipulating a network message when the event loop
shuts down, causing us to close the message while it's still in use.
This is at best going to be a little surprising to the caller, and at
worst could be an outright use-after-free bug.

This commit moves the double-check for a leaked promise to happen
strictly after the event loop has fully terminated, so that we can be
sure we've finished using it by this point.

Relates #105306, #97301
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this pull request Feb 15, 2024
We could still be manipulating a network message when the event loop
shuts down, causing us to close the message while it's still in use.
This is at best going to be a little surprising to the caller, and at
worst could be an outright use-after-free bug.

This commit moves the double-check for a leaked promise to happen
strictly after the event loop has fully terminated, so that we can be
sure we've finished using it by this point.

Relates elastic#105306, elastic#97301
elasticsearchmachine pushed a commit that referenced this pull request Feb 15, 2024
We could still be manipulating a network message when the event loop
shuts down, causing us to close the message while it's still in use.
This is at best going to be a little surprising to the caller, and at
worst could be an outright use-after-free bug.

This commit moves the double-check for a leaked promise to happen
strictly after the event loop has fully terminated, so that we can be
sure we've finished using it by this point.

Relates #105306, #97301
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed/Network Http and internode communication implementations Team:Distributed Meta label for distributed team. >test Issues or PRs that are addressing/adding tests v8.10.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants