Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix an infinite loop when downgrading HTTP/2 errors #1318

Merged
merged 1 commit into from
Oct 19, 2021

Conversation

olix0r
Copy link
Member

@olix0r olix0r commented Oct 19, 2021

578d979 introduced a bug: when the proxy handles errors on
HTTP/2-upgraded connections, it can get stuck in an infinite loop when
searching the error chain for an HTTP/2 error.

This change fixes this inifite loop and adds a test that exercises this
behavior to ensure that downgrade_h2_error behaves as expected.

Fixes linkerd/linkerd2#7103

578d979 introduced a bug: when the proxy handles errors on
HTTP/2-upgraded connections, it can get stuck in an infinite loop when
searching the error chain for an HTTP/2 error.

This change fixes this inifite loop and adds a test that exercises this
behavior to ensure that `downgrade_h2_error` behaves as expected.

Fixes linkerd/linkerd2#7103
@olix0r olix0r requested a review from a team October 19, 2021 22:15
Copy link
Contributor

@kleimkuhler kleimkuhler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should, uh, probably double check that no other error downcasting loops do this 🙃

Comment on lines +158 to +190
fn test_downgrade_h2_error() {
assert!(
downgrade_h2_error(h2::H2Error::from(h2::Reason::PROTOCOL_ERROR)).is::<DowngradedH2Error>(),
"h2 errors must be downgraded"
);

#[derive(Debug, Error)]
#[error("wrapped h2 error: {0}")]
struct WrapError(#[source] Error);
assert!(
downgrade_h2_error(WrapError(
h2::H2Error::from(h2::Reason::PROTOCOL_ERROR).into()
))
.is::<DowngradedH2Error>(),
"wrapped h2 errors must be downgraded"
);

assert!(
downgrade_h2_error(WrapError(
WrapError(h2::H2Error::from(h2::Reason::PROTOCOL_ERROR).into()).into()
))
.is::<DowngradedH2Error>(),
"double-wrapped h2 errors must be downgraded"
);

assert!(
!downgrade_h2_error(std::io::Error::new(
std::io::ErrorKind::Other,
"non h2 error"
))
.is::<DowngradedH2Error>(),
"other h2 errors must not be downgraded"
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit/TIOLI: it might be nice to have these cases as separate tests, so that we can see all the failures if there are multiples, rather than just the first one? not a big deal

@olix0r
Copy link
Member Author

olix0r commented Oct 19, 2021

@hawkw quick spot check shows that the other error handlers look okay. most of them are recursive, which is probably better in this case ;)

@olix0r olix0r merged commit ec441e5 into main Oct 19, 2021
@olix0r olix0r deleted the ver/h2-downgrade-fix branch October 19, 2021 22:29
olix0r added a commit to linkerd/linkerd2 that referenced this pull request Oct 19, 2021
This release fixes a bug where the outbound proxy could loop infinitely
while handling errors on meshed HTTP/1 connections. This would typically
cause proxies to be fail health checks and be restarted.

Furthermore, the proxy now requires identity. Proxies will log an error
and fail to start if identity is disabled.

---

* dns: Avoid allocating in `Name::is_localhost` (linkerd/linkerd2-proxy#1303)
* metrics: Implement FmtMetrics for Option (linkerd/linkerd2-proxy#1302)
* tracing: simplify subscriber construction with `Box`ed layers (linkerd/linkerd2-proxy#1304)
* Require identity configuration (linkerd/linkerd2-proxy#1305)
* build(deps): bump thiserror from 1.0.29 to 1.0.30 (linkerd/linkerd2-proxy#1306)
* build(deps): bump tower from 0.4.8 to 0.4.9 (linkerd/linkerd2-proxy#1308)
* build(deps): bump trust-dns-resolver (linkerd/linkerd2-proxy#1311)
* build(deps): bump actions/checkout from 2.3.4 to 2.3.5 (linkerd/linkerd2-proxy#1313)
* dns-name: Remove `webpki` dependency (linkerd/linkerd2-proxy#1316)
* build(deps): bump libc from 0.2.103 to 0.2.104 (linkerd/linkerd2-proxy#1315)
* inbound: Add a box layer to reduce compile times (linkerd/linkerd2-proxy#1317)
* Split cryptographic dependencies into a dedicated crate (linkerd/linkerd2-proxy#1307)
* Fix an infinite loop when downgrading HTTP/2 errors (linkerd/linkerd2-proxy#1318)
olix0r added a commit to linkerd/linkerd2 that referenced this pull request Oct 20, 2021
This release fixes a bug where the outbound proxy could loop infinitely
while handling errors on meshed HTTP/1 connections. This would typically
cause proxies to be fail health checks and be restarted.

Furthermore, the proxy now requires identity. Proxies will log an error
and fail to start if identity is disabled.

---

* dns: Avoid allocating in `Name::is_localhost` (linkerd/linkerd2-proxy#1303)
* metrics: Implement FmtMetrics for Option (linkerd/linkerd2-proxy#1302)
* tracing: simplify subscriber construction with `Box`ed layers (linkerd/linkerd2-proxy#1304)
* Require identity configuration (linkerd/linkerd2-proxy#1305)
* build(deps): bump thiserror from 1.0.29 to 1.0.30 (linkerd/linkerd2-proxy#1306)
* build(deps): bump tower from 0.4.8 to 0.4.9 (linkerd/linkerd2-proxy#1308)
* build(deps): bump trust-dns-resolver (linkerd/linkerd2-proxy#1311)
* build(deps): bump actions/checkout from 2.3.4 to 2.3.5 (linkerd/linkerd2-proxy#1313)
* dns-name: Remove `webpki` dependency (linkerd/linkerd2-proxy#1316)
* build(deps): bump libc from 0.2.103 to 0.2.104 (linkerd/linkerd2-proxy#1315)
* inbound: Add a box layer to reduce compile times (linkerd/linkerd2-proxy#1317)
* Split cryptographic dependencies into a dedicated crate (linkerd/linkerd2-proxy#1307)
* Fix an infinite loop when downgrading HTTP/2 errors (linkerd/linkerd2-proxy#1318)
olix0r added a commit that referenced this pull request Oct 22, 2021
578d979 introduced a bug: when the proxy handles errors on
HTTP/2-upgraded connections, it can get stuck in an infinite loop when
searching the error chain for an HTTP/2 error.

This change fixes this inifite loop and adds a test that exercises this
behavior to ensure that `downgrade_h2_error` behaves as expected.

Fixes linkerd/linkerd2#7103
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

linkerd-proxy 2.11 crashes when the application issues requests resulting in Connection refused
3 participants