Skip to content

[http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process#10886

Merged
mattklein123 merged 3 commits intoenvoyproxy:masterfrom
antoniovicente:inner_lws
Apr 29, 2020
Merged

[http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process#10886
mattklein123 merged 3 commits intoenvoyproxy:masterfrom
antoniovicente:inner_lws

Conversation

@antoniovicente
Copy link
Contributor

@antoniovicente antoniovicente commented Apr 21, 2020

Description: Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.
Risk Level: low
Testing: unit and integration
Docs Changes: n/a
Release Notes: n/a
Fixes #10270

@antoniovicente
Avoid stripping LWS from the middle of HTTP1 header values which are …
24ec37f 
…read into different buffer slices.

Signed-off-by: Antonio Vicente <avd@google.com>
asraa
asraa reviewed Apr 22, 2020
View reviewed changes
@antoniovicente
fix nit
c796667 
Signed-off-by: Antonio Vicente <avd@google.com>
asraa
asraa previously approved these changes Apr 28, 2020
View reviewed changes
Copy link
Contributor

@asraa asraa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the positive/negative test coverage!

mattklein123
mattklein123 requested changes Apr 28, 2020
View reviewed changes
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks. Just small comment typo from a quick skim.

/wait

@mattklein123 mattklein123 self-assigned this Apr 28, 2020
@antoniovicente
fix typo
31e0501 
Signed-off-by: Antonio Vicente <avd@google.com>
@mattklein123 mattklein123 merged commit b962a8e into envoyproxy:master Apr 29, 2020
@antoniovicente antoniovicente mentioned this pull request Jul 21, 2020
Closed
@antoniovicente
Copy link
Contributor Author

antoniovicente commented Jul 22, 2020

/backport

@repokitteh-read-only repokitteh-read-only bot added the backport/review Request to backport to stable releases label Jul 22, 2020
@lambdai lambdai added backport/approved Approved backports to stable releases and removed backport/review Request to backport to stable releases labels Jul 22, 2020
@PiotrSikora
Copy link
Contributor

PiotrSikora commented Jul 27, 2020

@lambdai Someone has to backport those changes to older branches.

cc @antoniovicente for visibility and/or volunteering to backport.
cc @lizan to see if he can volunteer someone to help.

lambdai pushed a commit to lambdai/envoy-dai that referenced this pull request Jul 28, 2020
@antoniovicente @lambdai
[http1] Preserve LWS from the middle of HTTP1 header values that requ…
f945472 
…ire multiple dispatch calls to process (envoyproxy#10886)

Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
@lambdai lambdai mentioned this pull request Jul 28, 2020
Merged
lambdai pushed a commit to lambdai/envoy-dai that referenced this pull request Jul 28, 2020
@antoniovicente @lambdai
[http1] Preserve LWS from the middle of HTTP1 header values that requ…
d5f6c65 
…ire multiple dispatch calls to process (envoyproxy#10886)

Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Signed-off-by: Antonio Vicente <avd@google.com>

Signed-off-by: antonio <avd@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
@lambdai lambdai mentioned this pull request Jul 28, 2020
Merged
lambdai pushed a commit to lambdai/envoy-dai that referenced this pull request Jul 28, 2020
@antoniovicente @lambdai
[http1] Preserve LWS from the middle of HTTP1 header values that requ…
14e6cc0 
…ire multiple dispatch calls to process (envoyproxy#10886)

Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>

Signed-off-by: antonio <avd@google.com>
@lambdai lambdai mentioned this pull request Jul 28, 2020
Merged
lambdai added a commit that referenced this pull request Jul 29, 2020
@lambdai
[1.13] [http1] Preserve LWS from the middle of HTTP1 header values th…
334b47a 
…at requ… (#12320)

[http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process (#10886)

Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
lambdai added a commit that referenced this pull request Jul 29, 2020
@lambdai
[http1] Preserve LWS from the middle of HTTP1 header values that requ…
886377e 
…ire multiple dispatch calls to process (#12323)

Cherry-pick [http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process (#10886) 
Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
lambdai added a commit that referenced this pull request Jul 31, 2020
@lambdai
[1.12] [http1] Preserve LWS from the middle of HTTP1 header values th…
a624dee 
…at requ… (#12319)

* [http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process (#10886)

Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Also various fix to allow build on clang-10

Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>
brian-avery pushed a commit to istio/envoy that referenced this pull request Oct 7, 2020
@lambdai
[1.4] Fix CVE-2020-25017 (#46)
b64ea00 
* doc: version 1.12.7

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* Preserve LWS from the middle of HTTP1 header values that requ… (envoyproxy#12319)

* [http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process (envoyproxy#10886)

Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Also various fix to allow build on clang-10

Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* http: header map security fixes for duplicate headers (#197) (#204)

Previously header matching did not match on all headers for
non-inline headers. This patch changes the default behavior to
always logically match on all headers. Multiple individual
headers will be logically concatenated with ',' similar to what
is done with inline headers. This makes the behavior effectively
consistent. This behavior can be temporary reverted by setting
the runtime value "envoy.reloadable_features.header_match_on_all_headers"
to "false".

Targeted fixes have been additionally performed on the following
extensions which make them consider all duplicate headers by default as
a comma concatenated list:
1) Any extension using CEL matching on headers.
2) The header to metadata filter.
3) The JWT filter.
4) The Lua filter.
Like primary header matching used in routing, RBAC, etc. this behavior
can be disabled by setting the runtime value
"envoy.reloadable_features.header_match_on_all_headers" to false.

Finally, the setCopy() header map API previously only set the first
header in the case of duplicate non-inline headers. setCopy() now
behaves similiarly to the other set*() APIs and replaces all found
headers with a single value. This may have had security implications
in the extauth filter which uses this API. This behavior can be disabled
by setting the runtime value
"envoy.reloadable_features.http_set_copy_replace_all_headers" to false.

Fixes https://github.com/envoyproxy/envoy-setec/issues/188

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* fix wasm compile

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* fix typo

Signed-off-by: Yuchen Dai <silentdai@gmail.com>
istio-testing pushed a commit to istio/envoy that referenced this pull request Oct 8, 2020
@brian-avery @lambdai @PiotrSikora
[1.5] Fix CVE-2020-25017 (#45) (#271)
936fdef 
* docs: kick-off 1.13.5 release. (envoyproxy#12164)

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* Preserve LWS from the middle of HTTP1 header values that requ… (envoyproxy#12320)

[http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process (envoyproxy#10886)

Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* http: header map security fixes for duplicate headers (#197) (#203)

Previously header matching did not match on all headers for
non-inline headers. This patch changes the default behavior to
always logically match on all headers. Multiple individual
headers will be logically concatenated with ',' similar to what
is done with inline headers. This makes the behavior effectively
consistent. This behavior can be temporary reverted by setting
the runtime value "envoy.reloadable_features.header_match_on_all_headers"
to "false".

Targeted fixes have been additionally performed on the following
extensions which make them consider all duplicate headers by default as
a comma concatenated list:
1) Any extension using CEL matching on headers.
2) The header to metadata filter.
3) The JWT filter.
4) The Lua filter.
Like primary header matching used in routing, RBAC, etc. this behavior
can be disabled by setting the runtime value
"envoy.reloadable_features.header_match_on_all_headers" to false.

Finally, the setCopy() header map API previously only set the first
header in the case of duplicate non-inline headers. setCopy() now
behaves similiarly to the other set*() APIs and replaces all found
headers with a single value. This may have had security implications
in the extauth filter which uses this API. This behavior can be disabled
by setting the runtime value
"envoy.reloadable_features.http_set_copy_replace_all_headers" to false.

Fixes https://github.com/envoyproxy/envoy-setec/issues/188

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* fixed compile

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* fix test

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

Co-authored-by: Piotr Sikora <piotrsikora@google.com>

Co-authored-by: Yuchen Dai <silentdai@gmail.com>
Co-authored-by: Piotr Sikora <piotrsikora@google.com>
istio-testing pushed a commit to istio/envoy that referenced this pull request Oct 8, 2020
@brian-avery @lambdai
[1.4] Fix CVE-2020-25017 (#46) (#272)
e43f60f 
* doc: version 1.12.7

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* Preserve LWS from the middle of HTTP1 header values that requ… (envoyproxy#12319)

* [http1] Preserve LWS from the middle of HTTP1 header values that require multiple dispatch calls to process (envoyproxy#10886)

Correctly preserve linear whitespace in the middle of HTTP1 header values. The fix in 6a95a21 trimmed away both leading and trailing whitespace when accepting header value fragments which can result in inner LWS in header values being stripped away if the LWS lands at the beginning or end of a buffer slice.

Also various fix to allow build on clang-10

Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* http: header map security fixes for duplicate headers (#197) (#204)

Previously header matching did not match on all headers for
non-inline headers. This patch changes the default behavior to
always logically match on all headers. Multiple individual
headers will be logically concatenated with ',' similar to what
is done with inline headers. This makes the behavior effectively
consistent. This behavior can be temporary reverted by setting
the runtime value "envoy.reloadable_features.header_match_on_all_headers"
to "false".

Targeted fixes have been additionally performed on the following
extensions which make them consider all duplicate headers by default as
a comma concatenated list:
1) Any extension using CEL matching on headers.
2) The header to metadata filter.
3) The JWT filter.
4) The Lua filter.
Like primary header matching used in routing, RBAC, etc. this behavior
can be disabled by setting the runtime value
"envoy.reloadable_features.header_match_on_all_headers" to false.

Finally, the setCopy() header map API previously only set the first
header in the case of duplicate non-inline headers. setCopy() now
behaves similiarly to the other set*() APIs and replaces all found
headers with a single value. This may have had security implications
in the extauth filter which uses this API. This behavior can be disabled
by setting the runtime value
"envoy.reloadable_features.http_set_copy_replace_all_headers" to false.

Fixes https://github.com/envoyproxy/envoy-setec/issues/188

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* fix wasm compile

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

* fix typo

Signed-off-by: Yuchen Dai <silentdai@gmail.com>

Co-authored-by: Yuchen Dai <silentdai@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mattklein123 mattklein123 mattklein123 approved these changes

+1 more reviewer

@asraa asraa asraa left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@asraa asraa

@mattklein123 mattklein123

Labels

backport/approved Approved backports to stable releases

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

LWS in H1 header values may be stripped out when proxying.

5 participants

@antoniovicente @PiotrSikora @asraa @mattklein123 @lambdai