Skip to content

quiche: add support of delayed close to QUIC session#9163

Merged
alyssawilk merged 17 commits intoenvoyproxy:masterfrom
danzh2010:delayclose
Dec 11, 2019
Merged

quiche: add support of delayed close to QUIC session#9163
alyssawilk merged 17 commits intoenvoyproxy:masterfrom
danzh2010:delayclose

Conversation

@danzh2010
Copy link
Copy Markdown
Contributor

Implement a TODO in QuicFilterManagerConnectionImpl::close() to delay close a quic connection.

The behavior mimics ConnectionImpl::close():
If there is buffered data at quic stream or connection layer, wait for QUIC to drain all the data with a timeout if delayed close timeout is configured. After data is drained, close the connection if FlushWrite.

If close() takes in FlushWriteAndWait and config has delayed close timeout, delay closing the connection even when there is no pending data or data has been drained.

Risk Level: low, no in production
Testing: added new unit tests
Part of #2557

Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
This reverts commit 5642cf8.

Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
@danzh2010
Copy link
Copy Markdown
Contributor Author

/assign @alyssawilk

delayed_close_timer_->enableTimer(delayed_close_timeout_);
} else {
closeConnectionImmediately();
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is data to write, don't we update the delay close timer to now + delay close?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timer is set when close() is called. I don't think we postpone it whenever we get a chance to write, do we?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs say it's timeout interval after the last write, e.g.

// The socket will be closed immediately after the buffer is flushed or if a period of
// inactivity after the last write event greater than or equal to delayed_close_timeout_ has
// elapsed.
CloseAfterFlush,

and I believe the TCP stack re-arms the timer after each write
https://github.com/envoyproxy/envoy/blob/master/source/common/network/connection_impl.cc#L588

cc @AndresGuedez

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woops, I totally missed this. I changed OnCanWrite() to re-arm timer for that case now.

} else {
delayed_close_state_ = DelayedCloseState::CloseAfterFlush;
}
} else {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could remove this else, and just have else if (hasDataToWrite()) {} else { // type == NoFlush

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point! Done

initializeDelayedCloseTimer();
}
// Update delay close state according to current call.
if (delayed_close_timeout_configured &&
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe have one if for delayed_close_timeout_configured and have the above code and this code in it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. But due to the intertwine of ConnectionCloseType and the presence of delayed_close_time in config, I feel it doesn't simplify the logic much...

quic_connection_->OnCanWrite();
closeConnectionImmediately();
} else {
// Quic connection doesn't have unsent data. It's upto caller and
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upto -> up to the

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
@danzh2010
Copy link
Copy Markdown
Contributor Author

I sync'ed with #8496 and added client side implementation and integration test. PTAL

Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Dan Zhang <danzh@google.com>
Copy link
Copy Markdown
Contributor

@alyssawilk alyssawilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks much better - just a few more nits!

"with the configured filter chain.");
connection_stats_ = std::make_unique<ConnectionStats>(stats);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are format fixes for previous change


void EnvoyQuicServerSession::OnCanWrite() {
quic::QuicServerSessionBase::OnCanWrite();
// Do no update delay close state according to connection level packet egress because that is
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do no -> do not
Also yay for detailed commenting :-)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

write_buffer_watermark_simulation_.checkLowWatermark(bytes_to_send_);
}

void QuicFilterManagerConnectionImpl::maybeHandleDelayedClose() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could use better naming. By name I assumed this would be handling the close(... delayClose) call not applying the delay close policy after a write.
The comments in the .h file are great but if we can make the name more clear that'd be even better.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to maybeApplyDelayClosePolicy()

Signed-off-by: Dan Zhang <danzh@google.com>
@danzh2010
Copy link
Copy Markdown
Contributor Author

Ping? @alyssawilk

@alyssawilk alyssawilk merged commit ec2d06c into envoyproxy:master Dec 11, 2019
prakhag1 pushed a commit to prakhag1/envoy that referenced this pull request Jan 3, 2020
Implement a TODO in QuicFilterManagerConnectionImpl::close() to delay close a quic connection.

The behavior mimics ConnectionImpl::close():
If there is buffered data at quic stream or connection layer, wait for QUIC to drain all the data with a timeout if delayed close timeout is configured. After data is drained, close the connection if FlushWrite.

If close() takes in FlushWriteAndWait and config has delayed close timeout, delay closing the connection even when there is no pending data or data has been drained.

Risk Level: low, no in production
Testing: added new unit tests
Part of envoyproxy#2557

Signed-off-by: Dan Zhang <danzh@google.com>
Signed-off-by: Prakhar <prakhar_au@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants