Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transport: fixes established streams leak in loopy writer. #2610

Merged
merged 4 commits into from
Feb 12, 2019
Merged

transport: fixes established streams leak in loopy writer. #2610

merged 4 commits into from
Feb 12, 2019

Conversation

canguler
Copy link

@canguler canguler commented Feb 1, 2019

RSTStreamFrames used to be ignored by the server transport, if a trailer had already been put into the transport's control buffer. If loopy writer couldn't write anything into a stream because of an error on the client side, then this trailer would never be sent. At that point, server would receive an RSTStreamFrame from client. But this RSTStreamFrame would be ignored because a trailer was already put into the control buffer. This would keep the stream open and in memory on the server side.

With this change, a cleanupStream item is put into the transport's control buffer, whenever an RSTStreamFrame is received by the server, even after a trailer has been put into the buffer. When loopy writer handles this cleanupStream, the stream will be closed and cleaned up.

RSTStreamFrames used to be ignored by the server transport, if a trailer had already been put into the transport's control buffer. If loopy writer couldn't write anything into a stream because of an error on the client side, then this trailer would never be sent. At that point, server would receive an RSTStreamFrame from client. But this RSTStreamFrame would be ignored because a trailer was already put into the control buffer. This would keep the stream open and in memory on the server side.

With this change, a cleanupStream item is put into the transport's control buffer, whenever an RSTStreamFrame is received by the server, even after a trailer has been put into the buffer.
Copy link
Member

@dfawley dfawley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. Can you please add a test case that catches the fixed bug (making sure it fails without this change)?

… RST_STREAM, server gets these frames in the correct order.

When server receives the RST_STREAM, it marks the stream as done and defers the deletion of the stream to the loopy writer by putting a cleanupStream item into control buffer.
Then the server receives the header to initiate a stream. It acts on the header immediately and attempts to create the stream. But because the old stream is not deleted, it hits the number of streams limit and fails.
This commit solves this problem by letting server handle the deletion immediately after receiving the RST_STREAM.
@canguler canguler assigned dfawley and canguler and unassigned dfawley and canguler Feb 4, 2019
Copy link
Member

@dfawley dfawley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one minor nit about the test.

test/stream_cleanup_test.go Outdated Show resolved Hide resolved
@canguler canguler merged commit 32559e2 into grpc:master Feb 12, 2019
@menghanl menghanl changed the title Fixes established streams leak in the loopy writer. transport: fixes established streams leak in loopy writer. Feb 15, 2019
@menghanl menghanl added this to the 1.19 Release milestone Feb 15, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Aug 14, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants