Skip to content

sync from master.#133

Merged
wangfakang merged 57 commits intowangfakang:masterfrom
envoyproxy:main
Jul 28, 2021
Merged

sync from master.#133
wangfakang merged 57 commits intowangfakang:masterfrom
envoyproxy:main

Conversation

@wangfakang
Copy link
Copy Markdown
Owner

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]

phlax and others added 30 commits July 19, 2021 06:33
Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Manish Kumar <manish.kumar1@india.nec.com>
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Risk Level: low
Testing: unit test improvements
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Pradeep Rao <pcrao@google.com>
This is PR 2/4 towards implementing shadow requests for thrift.

This moves all the stats into the common RequestOwner interface so that
they can be reused. The next, similar sized, PR will decouple
UpstreamRequest from the Router so that it can be reused by the upcoming
ShadowRequest class.

The final PR should introduce the ShadowWriter and ShadowRequest classes
and make use of RequestOwner to tie them together.

Risk Level: low, refactor
Testing: existing tests pass
Docs Changes: n/a
Release Notes: n/a
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
…PROVIDED policy instead when configuring an original destination cluster. (#17230)

Risk Level: low
Testing: bazel test //test/...

Signed-off-by: Tianyu Xia <tyxia@google.com>
1) Deprecate existing reuse_port field
2) Add new enable_reuse_port field which uses a WKT
3) Make the new default hot restart aware so the default is
   not changed during hot restart.
4) Allow the default to be reverted using the
   "envoy.reloadable_features.listener_reuse_port_default_enabled"
   feature flag.
5) Change listener init so that almost all error handling occurs on
   the main thread. This a) vastly simplifies error handling and
   b) makes it so that we pre-create all sockets on the main thread
   and can use them all during hot restart.
6) Change hot restart to pass reuse port sockets by socket/worker
   index. This works around a race condition in which a draining
   listener has a new connection on its accept queue, but it's
   never accepted by the old process worker. It will be dropped.
   By passing all sockets (even reuse port sockets) we make sure
   the accept queue is fully processed.

Fixes #15794

Risk Level: High, scary stuff involving hot restart and listener init
Testing: New and existing tests. It was very hard to get the tests to pass which gives me more confidence.
Docs Changes: N/A
Release Notes: Added
Platform Specific Features: N/A

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
Signed-off-by: Long Dai <long0dai@foxmail.com>
Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Ryan Northey <ryan@synca.io>
…d08 (#17328)

Commit Message: Modify QUIC_BUG implementation to log with rate limit like ENVOY_BUG in release mode. Fix a QuicMemSliceSpanImpl life time issue which was exposed by new QUICHE change

Risk Level: low
Testing: added unit tests for QUIC_BUG and QuicMemSliceSpanImpl
Signed-off-by: Dan Zhang <danzh@google.com>
Fixes #15562

Signed-off-by: Matt Klein <mklein@lyft.com>
Removing support of hidden_deprecated_use_alpha

Risk Level: LOW
Testing: unit (i.e. bazel test under bazel test test/extensions/filters/http/ext_authz/... )

Signed-off-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Risk Level: Low
Testing: n/a
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Ryan Northey <ryan@synca.io>
Risk Level: low, new telemetry
Testing: added tests

Signed-off-by: Jose Nino <jnino@lyft.com>
The version of clang that ships with Xcode 12 has false positives with
this warning that might be fixed by https://reviews.llvm.org/D73007

In the meantime we can disable it entirely as discussed on
#17393

Signed-off-by: Keith Smiley <keithbsmiley@gmail.com>
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Update xray segment data to contain  error, fault & throttle tag in the
"Errors and exceptions" section as per the xray documentation at
https://docs.aws.amazon.com/xray/latest/devguide/xray-api-segmentdocuments.html

Signed-off-by: Sunil Narasimhamurthy <13044744+suniltheta@users.noreply.github.com>
Corrects all Envoy uses of ForwardedProto which actually want request URI over to :scheme

As a reminder, XFP indicates the encryption of the (original) downstream connection where :scheme is part of the URI and the resource requested. It's legal (though unusual) to request http:// urls over a TLS connection for HTTP/2. It's possible (if ill advised) to have an internal mesh forwarding https schemed requests in the clear.

Current uses of X-Forwarded-Proto are

in the HCM, clearing XFP from untrusted users (unchanged)
in the HCM, setting absent XFP based on downstream transport security (unchanged)
in the HCM setting absent :scheme to XFP (unchanged)
in buildOriginalUri, changing from using XFP to scheme (changed. new URIs should be based on original URIs not on transport security.
in the router, clearing default port based on XFP (unchanged)
in the router serving redirect URLs based on scheme (changed - used to be XFP but is now based on the scheme of the original URI)
in the router, applying SSL route redirect based on XFP (unchanged)
in the router, using :scheme for internal redirect url checks (changed - used to use XFP. new URIs should be based on original URI)
in the cache filter, using :scheme to serve content (changed we used to serve based on XFP but if http://foo.com/ differs from https://foo.com and the http version is requested over a TLS connection the http response should be served)
in oath2 serving redirect URLs based on scheme (changed this used to be based on SFP but URLs should be based on original URL scheme)
Risk Level: High
Testing: updated tests
Docs Changes: inline
Release Notes: inline
Runtime guard: envoy.reloadable_features.correct_scheme_and_xfp
Fixes #14587

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: Ryan Northey <ryan@synca.io>
#17110 the EnvoyMobileHttpConnectionManager but it turns out it's instantiated either as a standard network filter (needs a name) or as an API listener (which only accepted HttpConnectionManager)

Fixing both issues.

Risk Level: low
Testing: unit tests
Docs Changes: n/a
Release Notes: n/a
Part of envoyproxy/envoy-mobile#1540

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Implement this mode, which allows an external processor to receive the body one chunk at a time and examine or modify the body while it is being processed.

Risk Level: Medium. The mode is only engaged if used. However an external processor must be coded carefully, particularly if it expects to stream both request and response bodies simultaneously, since each will proceed in its own ordering.

Testing: New unit and integration tests.

Docs Changes: Updated proto docs to reflect that the mode is now supported.

Release Notes: External processing servers may now use the STREAMED processing mode. In this mode, chunks of the body are forwarded to the external processing server when they arrive. Depending on how the upstream system is implemented, request body chunks may be delivered before or after the response headers, and request and response body chunks may be interleaved if the upstream system delivers them that way. An external processor should be carefully coded so that it does not assume that a particular ordering will be implemented.

Signed-off-by: Gregory Brail <gregbrail@google.com>
This fixes:

```
WARNING: Option 'experimental_strict_action_env' is deprecated: Use --incompatible_strict_action_env instead
```

Signed-off-by: Keith Smiley <keithbsmiley@gmail.com>
* add QUIC stats to upstream connections

Signed-off-by: Renjie Tang <renjietang@google.com>

* format

Signed-off-by: Renjie Tang <renjietang@google.com>

* minor ci fixes

Signed-off-by: Renjie Tang <renjietang@google.com>

* format

Signed-off-by: Renjie Tang <renjietang@google.com>

* fix compiler error

Signed-off-by: Renjie Tang <renjietang@google.com>

* address comments.

Signed-off-by: Renjie Tang <renjietang@google.com>

* clean up unused variables.

Signed-off-by: Renjie Tang <renjietang@google.com>

* remove unused includes.

Signed-off-by: Renjie Tang <renjietang@google.com>

* fix ci compile time option

Signed-off-by: Renjie Tang <renjietang@google.com>

* fix an unrelated Windows ci failure.

Signed-off-by: Renjie Tang <renjietang@google.com>

* add docs and fix a compile option issue.

Signed-off-by: Renjie Tang <renjietang@google.com>

* fix include

Signed-off-by: Renjie Tang <renjietang@google.com>

* fix windows CI failure in config_impl_test

Signed-off-by: Renjie Tang <renjietang@google.com>

* fix unsuccessful merge to upstream.

Signed-off-by: Renjie Tang <renjietang@google.com>
Signed-off-by: Harvey Tuch <htuch@google.com>
Signed-off-by: Ilya Lobkov <lobkovilya@yandex.ru>
Manish Kumar and others added 27 commits July 22, 2021 14:05
* Add Jwt cache.
* Added release docs.
* Added unit test.

Signed-off-by: Manish Kumar <manish.kumar1@india.nec.com>
Co-authored-by: Wayne Zhang <qiwzhang@google.com>
Signed-off-by: Adam Kotwasinski <adam.kotwasinski@gmail.com>
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
This is the CPU you target when building on an Apple Silicon mac. All
the apple settings should apply.

Signed-off-by: Keith Smiley <keithbsmiley@gmail.com>
Signed-off-by: Taylor Barrella <tabarr@google.com>
…e experience (#17413)

Signed-off-by: vamsee.lakamsani <lakamsani@gmail.com>
Risk Level: LOW
Testing: CI

Signed-off-by: Tianyu Xia <tyxia@google.com>
This has been broken for several versions, possibly by commit 980c84d.

Risk Level: Low
Testing: Added new test


Signed-off-by: Greg Greenway <ggreenway@apple.com>
Signed-off-by: Kateryna Nezdolii <nezdolik@spotify.com>
This is a common source of user confusion.

Signed-off-by: Matt Klein <mklein@lyft.com>
Previously there was a docs recommendation to set the path manually when
building on macOS. Since we can use the `build:macos` config we can do
this automatically. This also allows us to transparently solve PATH
differences with Apple Silicon homebrew and intel homebrew, for the host
configuration as well.

This fixes this issue
bazel-contrib/rules_foreign_cc#672 that intel
homebrew didn't have because `--incompatible_strict_action_env` contains
`/usr/local/bin` but not `/opt/homebrew/bin`

This also reorders these so that we prefer pre-installed tools for
stability.

Signed-off-by: Keith Smiley <keithbsmiley@gmail.com>
This clarifies the language on the documentation page for the ext_proc
filter to clarify the implementation status and to link to the
reference page where it is kept up to date.

Signed-off-by: Gregory Brail <gregbrail@google.com>
Signed-off-by: Le Yao <le.yao@intel.com>
Signed-off-by: Ryan Northey <ryan@synca.io>
Risk Level: low
Testing: n/a
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Long Dai <long0dai@foxmail.com>
Delete connection pools when they have no connections anymore. This
fixes unbounded memory use for cases where a new connection pool is
needed for each downstream connection, such as when using upstream
PROXY protocol.

This reverts commit b7bc539.
This reverts PR #17319, by re-adding #17302 and #16948.

Signed-off-by: Greg Greenway <ggreenway@apple.com>
Co-authored-by: Craig Radcliffe <craig.radcliffe@broadcom.com>
Signed-off-by: Sotiris Nanopoulos <sonanopo@microsoft.com>
According to https://luajit.org/install.html

> Important: this relates to LuaJIT 2.0 only — use LuaJIT 2.1 to avoid these complications.

Since we have updated past 2.1 we shouldn't need these anymore which is
great since it breaks on Apple Silicon

#16482 (comment)

Signed-off-by: Keith Smiley <keithbsmiley@gmail.com>
…17430)

Signed-off-by: Gregory Brail <gregbrail@google.com>
While testing the feature I noticed that we can not retrieve the
redirect records for the following curl request:

`curl.exe -s -o NUL -D - -I -w StatusCode:%{http_code} -L http://edition.cnn.com`

Increase the buffer size to 2kb to resolve the issue.

Signed-off-by: Sotiris Nanopoulos <sonanopo@microsoft.com>
Adding unit tests and bumping tracer coverage

Risk Level: n/a (test only)
Testing: unit tests
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: Snow Pettersen <snowp@lyft.com>
This moves links used by example Docker configuration to a 1st party
repository.

Fixes #16866

Signed-off-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: Manish Kumar <manish.kumar1@india.nec.com>
This is needed so that it can be reused by the upcoming shadowing
implementation, which will introduce a dependency from Router to
ShadowRouter to UpstreamRequest, so this breaks the circular
dependency.

Risk Level: low (refactor)
Unit Tests: existing
Docs Changes: n/a
Release Notes: n/a
Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
@wangfakang wangfakang merged commit 5318196 into wangfakang:master Jul 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.