Skip to content

sync from master#229

Merged
wangfakang merged 110 commits intowangfakang:masterfrom
envoyproxy:main
Nov 19, 2021
Merged

sync from master#229
wangfakang merged 110 commits intowangfakang:masterfrom
envoyproxy:main

Conversation

@wangfakang
Copy link
Copy Markdown
Owner

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

dependabot bot and others added 30 commits November 2, 2021 04:32
…18851)

Bumps [wrapt](https://github.com/GrahamDumpleton/wrapt) from 1.13.2 to 1.13.3.
- [Release notes](https://github.com/GrahamDumpleton/wrapt/releases)
- [Changelog](https://github.com/GrahamDumpleton/wrapt/blob/develop/docs/changes.rst)
- [Commits](GrahamDumpleton/wrapt@1.13.2...1.13.3)

---
updated-dependencies:
- dependency-name: wrapt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Docs Changes: Removes an extra word

Signed-off-by: Peter Jausovec <peter.jausovec@gmail.com>
- Add response trailers for the bandwidth limit filter delays.
- Add new metric request_enforced and response_enforced.
- Change the following metrics type from Gauge to Counter (request_incoming_size, response_incoming_size, request_allowed_size, response_allowed_size) to better calculate the network bytes per second.

This is useful when downstream want to understand how much delays that is caused by bandwidth limit filter.

Risk Level: Low
Testing: UT added.
Docs Changes: Updated.
Release Notes: Added.

Signed-off-by: gayang <gayang@microsoft.com>
In preparation for #18305, I split the implementation of io_socket_handle on different platforms to make the code more readable and separated. The new Windows class leverages the common implementation and now is responsible for registering the events.

This will allow us to cleanly implement MSG_PEEK by read in a consequent PR in a self-contained way.

Signed-off-by: Sotiris Nanopoulos <sonanopo@microsoft.com>
Fixing extension name in multiple places.

Signed-off-by: Jojy George Varghese <jojy_varghese@apple.com>
The copy assignment operator is being invoked for SocketOptionName implicitly. Explicitly request it.

This fixes compiling with clang-14.

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Commit Message: Allow cert validation by only leaf trusted CAs CRL
Additional Description: Close #18268. In the previous implementation, we don't have availability to validate certs when all trusted CAs don't have their own CRLs if any trusted CAs have that. This feature allows validating even if all trusted CAs don't have CRLs.
Risk Level: Low
Testing: Unit
Docs Changes: Required
Release Notes: Required

Signed-off-by: Shikugawa <rei@tetrate.io>
…18858)

This allows to generate tcp proxy's upstream CONNECT header values using downstream info.
Examples include modifying the host/:authority headers
Or emulate the proxy protocol via CONNECT as in the test cases.

Use with caution: a RFC non-compliant upstream header can be introduced because stream info may carry arbitrary bytes.

Signed-off-by: Yuchen Dai <silentdai@gmail.com>
* build(deps): bump frozendict from 2.0.6 to 2.0.7 in /tools/base

Bumps [frozendict](https://github.com/Marco-Sulla/python-frozendict) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/Marco-Sulla/python-frozendict/releases)
- [Commits](https://github.com/Marco-Sulla/python-frozendict/commits)

---
updated-dependencies:
- dependency-name: frozendict
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Ryan Northey <ryan@synca.io>

* build(deps): bump setuptools from 58.2.0 to 58.3.0 in /tools/base

Bumps [setuptools](https://github.com/pypa/setuptools) from 58.2.0 to 58.3.0.
- [Release notes](https://github.com/pypa/setuptools/releases)
- [Changelog](https://github.com/pypa/setuptools/blob/main/CHANGES.rst)
- [Commits](pypa/setuptools@v58.2.0...v58.3.0)

---
updated-dependencies:
- dependency-name: setuptools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Ryan Northey <ryan@synca.io>

* build(deps): bump pyparsing from 2.4.7 to 3.0.1 in /tools/dependency

Bumps [pyparsing](https://github.com/pyparsing/pyparsing) from 2.4.7 to 3.0.1.
- [Release notes](https://github.com/pyparsing/pyparsing/releases)
- [Changelog](https://github.com/pyparsing/pyparsing/blob/master/CHANGES)
- [Commits](pyparsing/pyparsing@pyparsing_2.4.7...pyparsing_3.0.1)

---
updated-dependencies:
- dependency-name: pyparsing
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Ryan Northey <ryan@synca.io>

* build(deps): bump protobuf in /examples/grpc-bridge/client

Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 3.18.0 to 3.19.0.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/master/generate_changelog.py)
- [Commits](protocolbuffers/protobuf@v3.18.0...v3.19.0)

---
updated-dependencies:
- dependency-name: protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Ryan Northey <ryan@synca.io>

* build(deps): bump pyjwt from 2.1.0 to 2.3.0 in /tools/dependency

Bumps [pyjwt](https://github.com/jpadilla/pyjwt) from 2.1.0 to 2.3.0.
- [Release notes](https://github.com/jpadilla/pyjwt/releases)
- [Changelog](https://github.com/jpadilla/pyjwt/blob/master/CHANGELOG.rst)
- [Commits](jpadilla/pyjwt@2.1.0...2.3.0)

---
updated-dependencies:
- dependency-name: pyjwt
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Ryan Northey <ryan@synca.io>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Fix #18764

Signed-off-by: Ryan Northey <ryan@synca.io>
https://github.com/google/quiche/compare/1e04d1e30..0b75841d5

2021-11-01 dschinazi Internal change
2021-11-01 dschinazi Platformize AsciiUrlDecode
2021-11-01 wub Pass ParsedClientHello to QuicDispatcher::CreateQuicSession. This makes it easier to pass CHLO information into CreateQuicSession, which is responsible for creating a QUIC session.
2021-11-01 wub Deprecate --gfe2_reloadable_flag_quic_tls_disable_resumption_refactor.
2021-11-01 haoyuewang Internal change

Risk Level: low
Testing: n/a
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
… folks (#18872)

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Risk Level: Low
Testing: unit test
Docs Changes: N/A
Release Notes: added
Platform Specific Features: n.a
Signed-off-by: Zhangdong Ma <zhdma_xd@163.com>
…18890)

upstream: Inline multiplexed_upstream_integration_test.h into multiplexed_upstream_integration_test.cc

Also rename Http2UpstreamIntegrationTest to MultiplexedUpstreamIntegrationTest since the
tests run on HTTP/3 as well as HTTP/2.

Risk Level: N/A
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

Signed-off-by: Ryan Hamilton <rch@google.com>
…y are provided (#18353)

Signed-off-by: Shmakov Aleksey <alexcei88@yandex.ru>
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: theidexisted <invalid_ms_user@live.com>
Fixes #1975
Adds a CLI parameter that can be repeated that acts as universal tags for all stats

Additional Description:
Risk Level: Low, additional parameter
Testing: Unit + integration + manual
Docs Changes: Provided
Release Notes: Pending
Platform Specific Features: N/A

Signed-off-by: Sotiris Nanopoulos <sonanopo@microsoft.com>
This allows sinks to inspect the log line details without parsing the formatted message.

Signed-off-by: Snow Pettersen <snowp@lyft.com>
…8705)

Signed-off-by: YaoZengzeng <yaozengzeng@huawei.com>
moving to an accessor now that it exists.

Risk Level: low
Testing: n/a (existing tests apply)
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
…8901)

Bumps [setuptools](https://github.com/pypa/setuptools) from 58.3.0 to 58.5.2.
- [Release notes](https://github.com/pypa/setuptools/releases)
- [Changelog](https://github.com/pypa/setuptools/blob/main/CHANGES.rst)
- [Commits](pypa/setuptools@v58.3.0...v58.5.2)

---
updated-dependencies:
- dependency-name: setuptools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ection (#17469)

Add a retry mechanism to the grpc access logger. This retry mechanism currently only supports a simple retry count. 

Also, at the moment, retries are only fired when the gRPC stream fails to be established, and nothing happens if the stream is successfully established once and a reset is issued.

Risk Level: Low
Testing: Unit

Signed-off-by: Shikugawa <rei@tetrate.io>
Signed-off-by: Adi Suissa-Peleg <adip@google.com>
Signed-off-by: Wayne Zhang <qiwzhang@google.com>
Signed-off-by: Peter Jausovec <peter.jausovec@gmail.com>
Signed-off-by: Shikugawa <rei@tetrate.io>
Part of #14696 (comment). The refactoring that enables this is mainly within `FilterConfigSubscription::onConfigUpdate`. The rest is adding and removing template parameters.
Risk Level: Low
Testing: Existing (refactoring)
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A
#14696

Signed-off-by: Taylor Barrella <tabarr@google.com>
RyanTheOptimist and others added 29 commits November 15, 2021 14:43
quic: Disable flaky test QuicHttpIntegrationTest.AdminDrainDrainsListeners while we work on a fix to the underlying issue.

Signed-off-by: Ryan Hamilton rch@google.com

Risk Level: Low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

Signed-off-by: Ryan Hamilton <rch@google.com>
Risk Level: n/a
Testing: n/a
part of #18844

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
* wasm: update V8 to v9.6.180.12.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* review: use_lld.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
…9003)

Bumps [setuptools](https://github.com/pypa/setuptools) from 58.5.3 to 59.0.1.
- [Release notes](https://github.com/pypa/setuptools/releases)
- [Changelog](https://github.com/pypa/setuptools/blob/main/CHANGES.rst)
- [Commits](pypa/setuptools@v58.5.3...v59.0.1)

---
updated-dependencies:
- dependency-name: setuptools
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Florin Coras <fcoras@cisco.com>
Signed-off-by: Peter Jausovec <peter.jausovec@gmail.com>
BaseIntegrationTest::waitForAccessLog expects one newline per access log so if the 2nd entry was waited for with an access log string ending in "\n" it'd return earlier than expected.

Risk Level: low
Testing: Ran tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
…ectly_coalesce_cookies (#19014)

Signed-off-by: wbpcode <comems@msn.com>
JA3 fingerprint information is available at https://github.com/salesforce/ja3 and at https://ja3er.com. 

Fixes #16622

Signed-off-by: Jason Miller <jason.miller@stackpath.com>
Co-authored-by: Ryland Degnan <rdegnan@twitter.com>
upstream: Implement Happy Eyeballs address sorting with address families interleaved, as per Section 4 of RFC 8305, Happy Eyeballs v2. Sorting as per Section 6 of RFC6724 already happens in ares_getaddrinfo() and is not part of this.

Risk Level: Low - Happy Eyeballs is not yet used
Testing: Unit tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

Signed-off-by: Ryan Hamilton rch@google.com
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: Tarun Sharma <starun.1998@gmail.com>
Known large scale production use.

Signed-off-by: Matt Klein <mklein@lyft.com>
Currently FilterStateFormatter::formatValue is converting ProtobufTypes::Message to ProtobufWkt::Value through intermediate json string format, which is not efficient and may throw exception in data plane. Add serializeAsProtoValue virtual method to do the conversion directly.
Additional Description: NA
Risk Level: Low
Testing: NA
Docs Changes: NA
Release Notes: NA

Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Every update to the stream into interface requires updates to the impl, the mocks, and the test class. Making the test class inherit from the impl to reduce churn.

Risk Level: n/a (test only)
Testing: n/a
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: Jose Nino <jnino@lyft.com>
…ng (#18998)

Signed-off-by: Rex Chang <chiyc@amazon.com>
This PR allows creating server connection but the abilitity to connect to such a listener is located in #18105.

Signed-off-by: Yuchen Dai <silentdai@gmail.com>
…ity (#18999)

Signed-off-by: YaoZengzeng <yaozengzeng@huawei.com>
Risk Level: low
Testing: new unit, integration tests
Docs Changes:
Release Notes:
Part of envoyproxy/envoy-mobile#1520

Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Proxying 102s from upstream

Risk Level: Low (minor refactor)
Testing: new integration test
Docs Changes: n/a
Release Notes: inline
Runtime guard: envoy.reloadable_features.proxy_102
Fixes #18844
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
…9013)

* Update QUICHE from 0b75841d5 to 4f552f349
https://github.com/google/quiche/compare/0b75841d5..4f552f349

$ git log 0b75841d5..4f552f349 --date=short --no-merges --format="%ad %al %s"

2021-11-15 wub Deprecate --gfe2_reloadable_flag_quic_bbr_start_new_aggregation_epoch_after_a_full_round
2021-11-15 wub Deprecate --gfe2_reloadable_flag_quic_bbr2_check_cwnd_limited_before_aggregation_epoch.
2021-11-15 fayang Make QUIC_BUG of "Trying to start blackhole detection without no bytes in flight" server side only.
2021-11-15 bnc Deprecate --gfe2_reloadable_flag_quic_reject_invalid_chars_in_field_value.
2021-11-15 quiche-dev Consolidates half_closed_local tracking. This is preparation for a refactoring of stream close events.
2021-11-12 quiche-dev Fixes stream unregistration in OgHttp2Session::CloseStream().
2021-11-12 quiche-dev Adds unit tests demonstrating bugs in existing oghttp2 handling of stream close events.
2021-11-12 wub Change QuicCryptoServerConfig::ParseSourceAddressToken from taking a 'SourceAddressTokens*' to a 'SourceAddressTokens&'.  The function dereferences this pointer without checking for nullptr, changing it to reference prevents caller from passing in a nullptr.
2021-11-12 wub Add a regression test for b/206077990.
2021-11-12 quiche-dev Factors out `MaybeSendBufferedData()`, and moves the functionality into `SendQueuedFrames()`.
2021-11-12 fayang Set chrome_value false for gfe2_reloadable_flag_quic_add_cached_network_parameters_to_address_token.
2021-11-12 quiche-dev Add missing nullptr check.
2021-11-11 wub Add retry_token, resumption_attempted and early_data_attempted to quic::ParsedClientHello.
2021-11-11 quiche-dev Another small fix from debugging: only mark a request stream ready to write if it includes a body.
2021-11-11 quiche-dev Fixes frame length calculations to use the actual serialized length.
2021-11-10 dschinazi Add support for draft-ietf-quic-version-negotiation-05
2021-11-10 vasilvv Make sure WebTransport over HTTP/3 is enabled on the client even when SETTINGS_ENABLE_CONNECT_PROTOCOL is not present.
2021-11-09 dschinazi Refactor QUIC version downgrade prevention, part 2
2021-11-09 dschinazi Refactor QUIC version downgrade prevention
2021-11-09 dschinazi Fix a QuicConnection connection close log
2021-11-08 danzh Internal change
2021-11-08 fayang Internal change
2021-11-08 fayang In QUIC, do not check amplification limit if there is pending timer credit. This would guarantee CRYPTO frame be retransmitted because of 1) PTO fires 2) bundled with outgoing ACKs.
2021-11-05 fayang Set gfe2_reloadable_flag_quic_verify_request_headers_2 chrome_value to true.
2021-11-05 wub Allow QuicToyClient to provide a client certificate to the server, if requested.
2021-11-04 quiche-dev Automated g4 rollback of changelist 386316152.
2021-11-04 danzh Validate QUIC request/response headers against invalid token and disallowed headers. Add empty string to disallow-list. Split --gfe2_reloadable_flag_quic_verify_request_headers into 2 flags: --gfe2_reloadable_flag_quic_verify_request_headers_2 to validate QUIC request/response headers against invalid request with ratio monitoring; mark H2 request with empty string header as invalid earlier in H2 stack. --gfe2_reloadable_flag_quic_act_upon_invalid_header return error response upon any invalid QUIC request header.
2021-11-04 quiche-dev Call visitor_.OnInvalidFrame() for oghttp2 header errors.
2021-11-04 wub Add CachedNetworkParameters to address token for IETF QUIC, and - min_rtt received from a validated token will be used to set the initial rtt, if connection option 'TRTT' is set. - Enable bandwidth resumption for IETF QUIC when connection options BWRE or BWMX exists.
2021-11-03 renjietang Add connection option to trigger path degrading on 1 PTO.
2021-11-03 quiche-dev Add default return statements to switch statements to appease GCC.
2021-11-03 quiche-dev Wrap OgHttp2Session callbacks with a latched_error_ check.
2021-11-03 quiche-dev Add mock methods to MockSpdyFramerVisitor.
2021-11-02 quiche-dev Change Http2VisitorInterface::OnInvalidFrame() to accept an InvalidFrameError enum.
2021-11-02 vasilvv Perform header-based draft version negotiation in WebTransport over HTTP/3.
2021-11-02 haoyuewang Use absl::optional<StatelessResetToken> in place of a separate boolean and token on QuicConnection::PathState.
2021-11-02 dschinazi Internal change
2021-11-02 wub Add mTLS support for IETF QUIC.
2021-11-02 wub Internal change
2021-11-02 vasilvv Internal change
2021-11-02 haoyuewang Update quic::IsAppleMobile to quic::IsAppleClient for better coverage of Apple related QUIC reverse path validation crash: 1) Added coverage for iOS Youtube (from cr/406134264) & iMM (from Sherlog) user agents. 2) Added coverage for Mac traffic.

Signed-off-by: Dan Zhang <danzh@google.com>
…8978)

This PR attempts to address #18589 and istio/istio#35575 by adding escaping logic to the prometheus stats code. An attempt was made to mirror the logic in the prometheus common golang code for text serialization.

Fixes #18589

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>
…edPtr (#19008)

If QuicFilterManagerConnectionImpl::ConnectionInfoProviderSharedPtr is
called after the connection is closed, return nullptr.

Risk Level: Low
Testing: New unit tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

Signed-off-by: Ryan Hamilton <rch@google.com>
)

Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@intel.com>
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
…override and legacy code paths (#19016)

Remove envoy.reloadable_features.upstream_http2_flood_checks runtime override and legacy code paths.

Risk Level: Low
Testing: Unit Tests
Docs Changes: proto comments updated
Release Notes: Yes
Platform Specific Features: N/A
Fixes #18449

Signed-off-by: Yan Avlasov <yavlasov@google.com>
…ustom header (Fixes #19024) (#19038)

This PR adds a new command operator in the header formatter called `%REQUESTED_SERVER_NAME%` which can be used to extract the SNI information from the stream and set it as a custom header. 

**Commit Message:** Added support for `%REQUESTED_SERVER_NAME%` to extract the SNI as a custom header.
**Additional Description:** Adds a new command operator called `%REQUESTED_SERVER_NAME%` in the http header formatter which can be used to extract the SNI information from the stream and set it as a custom header.
**Risk Level:** Low
**Testing:** Unit Tests
**Docs Changes:** Added description on `%REQUESTED_SERVER_NAME%` in the docs.
**Release Notes:** Added
**Platform Specific Features:** N/A

Signed-off-by: Rohit Agrawal <rohit.agrawal@databricks.com>
Bumps [grpcio](https://github.com/grpc/grpc) from 1.41.1 to 1.42.0.
- [Release notes](https://github.com/grpc/grpc/releases)
- [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md)
- [Commits](grpc/grpc@v1.41.1...v1.42.0)

---
updated-dependencies:
- dependency-name: grpcio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@wangfakang wangfakang merged commit 0886f78 into wangfakang:master Nov 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.