Merged
Conversation
Commit Message: To prevent long event loop when too many UDP packets are in the queue, limit how many packets to read in each event loop. If haven't finished reading, artifacts a READ event to continue in the next event loop. Additional Description: Add numPacketsExpectedPerEventLoop() callback to UdpListenerCallback, so that QUIC listener can tell how many packets it wants to read in each loop. The actually number of packets read are still bound by MAX_NUM_PACKETS_PER_EVENT_LOOP (6000). Quic listener returns numPacketsExpectedPerEventLoop() based on number of connections it has at the moment and the configured envoy::config::listener::QuicProtocolOptions.packets_to_read_to_connection_count_ratio. Made InjectableSingleton really thread safe. Risk Level: medium, other than quic listener, other UdpListenerCallbacks return max size_t for numPacketsExpectedPerEventLoop(). This will cause those callbacks to read 6000 packets per READ event. Testing: added udp listener unit tests. Fixes #16335 #16278 Part of #16198 #16493 Signed-off-by: Dan Zhang <danzh@google.com>
These new histograms should be useful for tracking large requests and responses in deployments using the Thrift Proxy. This is a follow-up to #15884. Risk Level: low Testing: tests added & updated. Docs Changes: yes, the new histograms and the details of their behavior has been documented. Release Notes: added. Platform Specific Features: n/a Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
Risk Level: low (http/3 only) Testing: new unit tests Docs Changes: n/a Release Notes: n/a Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Risk Level: n/a (test only) Testing: manual Docs Changes: n/a Release Notes: n/a Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
* event: update fd registration mask even if it hasn't changed. Updates to the fd mask can result in new events when operating in EDGE trigger mode. Doing this update unconditionally is specially important in cases where there was a synthetic event scheduled and setEnabled ends up clearing it before the call to updateEvents since by skipping the update we prevent the generation of a new real event to replace the lost synthetic event. Without this change, calling close(Flush) a socket that is readDisabled and had a pending synthetic write event can result in th write event never being delivered so the final flush will fail due to timeout. * Remove call to dispatcher exit that is not needed Signed-off-by: Antonio Vicente <avd@google.com>
…allbacks for a stream that has been terminated due to timeout. (#16590) This change doesn't reduce the failure rate or time to detect failures caused by undetected connection disconnects that eventually result in failures due to stream timeout, it just makes the failure condition more clear by avoiding undefined behaviors and crashes in the fake upstream. Signed-off-by: Antonio Vicente <avd@google.com>
This extension simply allows to add user-defined dynamic metadata. This is intended to be used with matchers, so that the "result" of a matching process can imply specific metadata. See issue #16266 Risk Level: Medium Testing: Unit tests for the extension are added Docs Changes: A small documentation for the extension is added in docs/root/configuration/http/http_filters Signed-off-by: Adrien Guinet <adrien@reblaze.com>
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Risk Level: Low Testing: Added integration test; possibly need more unit tests Docs Changes: Documented in protos Release Notes: Added Signed-off-by: Greg Greenway <ggreenway@apple.com>
This is a PR for scoping token buckets in the local rate limiting flow on a per connection basis as opposed to scoping it on the entire envoy instance. More details in #15637 Currently, the HTTP local rate limiter's token bucket is shared across all workers, thus causing the rate limits to be applied per Envoy instance/process. This could potentially result in bad actors quickly exhausting limits on a given envoy instance before legitimate users have had a fair chance. We achieve this by adding an instance of the LocalRateLimit::LocalRateLimiterImpl to each connection object, if there isn't one already, via FilterState data Risk Level: Low Testing: Added unit tests to local_ratelimit Manually tested via curl'ing against a locally patched envoy instance. One can send multiple requests on the same connection via curl using the following: curl -vI example.com example.com Docs Changes: Added new toggle to local rate limit configuration to enable per connection local rate limiting // Specifies the scope of the rate limiter's token bucket. // If set to false, the token bucket is shared across all worker threads // thus the rate limits are applied per Envoy process. // If set to true, a token bucket is allocated for each connection. // Thus the rate limits are applied per connection thereby allowing // one to rate limit requests on a per connection basis. // If unspecified, the default value is false. bool local_rate_limit_per_downstream_connection Sample configuration typed_config: "@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit stat_prefix: http_local_rate_limiter token_bucket: max_tokens: 10000 tokens_per_fill: 1000 fill_interval: 1s filter_enabled: runtime_key: local_rate_limit_enabled default_value: numerator: 100 denominator: HUNDRED filter_enforced: runtime_key: local_rate_limit_enforced default_value: numerator: 100 denominator: HUNDRED response_headers_to_add: - append: false header: key: x-local-rate-limit value: 'true' local_rate_limit_per_downstream_connection: true Fixes #15637 Signed-off-by: Gokul Nair <gnair@twitter.com>
Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Ryan Northey <ryan@synca.io>
Adding HTTP/3 stream reset stats Removing a bunch of unused HTTP/3 stats. Doccing up the stats which exist and adding tests. While I'm in there, adding scope strings to the test client and test server, so it's easier to differentiate which stats are Envoy's vs test code. Also changing wait for counter calls to not do infinite waits. Risk Level: Low (http/3 stats addition) Testing: new integration tests Docs Changes: documenting HTT/3 stats Release Notes: n/a Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
… = (#16636) [fuzz]: fix oss fuzz 30088, envoy:stat_merger_fuzz_test: ASSERT: name = Commit Message: Update for oss-fuzz Issue 30088 Additional Description: Note that #4709 covers a number of oss-fuzz error This is part of the ongoing effort started for #4709. Risk Level: Low Testing: test/common/stats Docs Changes: None Release Notes: Platform Specific Features: Signed-off-by: Tim Walsh <temporal.differential@gmail.com>
* reference issues in getting started doc for installing envoy using getenvoy with centos and rhel Signed-off-by: Abhay Narayan Katare <abhay.katare@india.nec.com>
Risk Level: Low (http/3 only) Testing: existing flow control tests pass Docs Changes: n/a Release Notes: n/a Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Add an option in connection manager which allow to strip trailing host dot from Host/authority header (e.g transforms "example.com." to "example.com"). Signed-off-by: maheshkurund <mahesh.kurund@oneconvergence.com>
the filter smoothens the flow of data in both direction up to the specified bandwidth limit. Risk Level: Low, new filter Testing: UTs added. Adding more UTs and integration tests. Docs Changes: Added Release Notes: Added Fixes #13604 Signed-off-by: Nitin Goyal <nigoyal@microsoft.com>
…s. (#16623) Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
…on response (#16645) Signed-off-by: Jonathan Stewmon <jstewmon@gmail.com>
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
As discussed in #16214 there is a case when flow control blocks and as a result the reponse gets truncated. To deflake the test we reduce the size of the headers and the limit. Validated with no flakes in 10k test runs. Signed-off-by: Sotiris Nanopoulos <sonanopo@microsoft.com>
Signed-off-by: Piotr Sikora <piotrsikora@google.com>
…otocol, not per origin, to match the alt-svc spec. (#16624) grid: Modify the AlternateProtocolsCache to specify expiration per-protocol, not per origin, to match the alt-svc spec. Risk Level: Low Testing: Unit tests Docs Changes: N/A Release Notes: N/A Platform Specific Features: N/A Signed-off-by: Ryan Hamilton <rch@google.com>
Signed-off-by: Adi Suissa-Peleg <adip@google.com>
…16679) The stats were only incorrect until the parent process terminated. Signed-off-by: Greg Greenway <ggreenway@apple.com>
Signed-off-by: Tony Allen <tony@allen.gg>
There's no good reason for the email addresses that security announcements are sent to to be public. They are already maintained on the relevant mailing lists. Listing them publicly just increases spam potential. Signed-off-by: Greg Greenway <ggreenway@apple.com>
…ove test coverage (#16656) Signed-off-by: Pradeep Rao <pcrao@google.com>
Signed-off-by: Nitin <nigoyal@microsoft.com> Signed-off-by: Gokul Nair <gnair@twitter.com> Co-authored-by: Nitin Goyal <nigoyal@microsoft.com> Co-authored-by: Teju Nareddy <nareddyt@google.com> Co-authored-by: phlax <phlax@users.noreply.github.com> Co-authored-by: Ivan Zemlyanskiy <QtIvan@gmail.com> Co-authored-by: DavidKorczynski <david@adalogics.com> Co-authored-by: Sotiris Nanopoulos <sonanopo@microsoft.com> Co-authored-by: Sunil Narasimhamurthy <sunnrs@amazon.com> Co-authored-by: Alex Konradi <akonradi@google.com> Co-authored-by: David Schinazi <DavidSchinazi@users.noreply.github.com> Co-authored-by: Peter Jausovec <peterj@users.noreply.github.com> Co-authored-by: asraa <asraa@google.com> Co-authored-by: Wayne Zhang <qiwzhang@google.com> Co-authored-by: Snow Pettersen <snowp@lyft.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: danzh <danzh2010@users.noreply.github.com> Co-authored-by: Kevin Baichoo <kbaichoo@google.com> Co-authored-by: alyssawilk <alyssar@chromium.org> Co-authored-by: Jose Ulises Nino Rivera <junr03@users.noreply.github.com> Co-authored-by: yanavlasov <yavlasov@google.com> Co-authored-by: Anirudha Pratap Singh <ps.anirudha@gmail.com> Co-authored-by: Ryan Hamilton <rch@google.com> Co-authored-by: Auni Ahsan <auni@google.com> Co-authored-by: Jared Tan <jian.tan@daocloud.io> Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io> Co-authored-by: Manish Kumar <manish.kumar1@india.nec.com> Co-authored-by: Mark D. Roth <roth@google.com> Co-authored-by: Rei Shimizu <rei@tetrate.io> Co-authored-by: htuch <htuch@users.noreply.github.com> Co-authored-by: Adi (Suissa) Peleg <adip@google.com> Co-authored-by: Greg Brail <gbrail@users.noreply.github.com> Co-authored-by: williamsfu99 <32112201+williamsfu99@users.noreply.github.com> Co-authored-by: Yuchen Dai <silentdai@gmail.com> Co-authored-by: Ravindra Akella <raakella@gmail.com> Co-authored-by: Dan Zhang <danzh@google.com> Co-authored-by: RenjieTang <renjietang27@gmail.com> Co-authored-by: Renjie Tang <renjietang@google.com> Co-authored-by: OutOfControl <whereisgww@outlook.com> Co-authored-by: Piotr Sikora <piotrsikora@google.com> Co-authored-by: tavishvaidya <tavishvaidya@users.noreply.github.com> Co-authored-by: YaoLe <le.yao@intel.com> Co-authored-by: Adrien Guinet <adrien@guinet.me> Co-authored-by: Kornél Dávid <47784437+davidkornel@users.noreply.github.com> Co-authored-by: Oleksiy Pylypenko <oleksiy.pylypenko@gmail.com> Co-authored-by: James Mulcahy <james@manque.net> Co-authored-by: Christoph Pakulski <christoph@tetrate.io> Co-authored-by: chaoqin-li1123 <55518381+chaoqin-li1123@users.noreply.github.com> Co-authored-by: Mike Schore <mike.schore@gmail.com> Co-authored-by: Charissa Plattner <charissa.plattner@gmail.com> Co-authored-by: yihuaz <yihuaz@google.com> Co-authored-by: Gokul Nair <gnair@twitter.com> Co-authored-by: Eleonora Kiziv <37271985+ekiziv@users.noreply.github.com> Co-authored-by: Taylor Barrella <tabarr@google.com> Co-authored-by: Bryce Anderson <banderson@twitter.com>
Signed-off-by: Greg Greenway <ggreenway@apple.com>
Signed-off-by: Long Dai <long0dai@foxmail.com>
Signed-off-by: Ryan Northey <ryan@synca.io>
Signed-off-by: Ashley Hedberg <ahedberg@google.com>
…set_config (#16647) Commit Message: This pull request try to resolve fuzz bug(https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34650), key of subset selector can't be empty when use_single_host_per_subset is set to true. Additional Description: Risk Level: Testing: Docs Changes: Release Notes: Platform Specific Features: Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Unifying test upstream and client codec types, but leaving legacy typedefs for folks who don't want to update all their filters/tests. Risk Level: Low (enum change, theoretical legacy support) Testing: n/a Docs Changes: n/a Release Notes: n/a Signed-off-by: Alyssa Wilk <alyssar@chromium.org>
Signed-off-by: Mike Schore <mike.schore@gmail.com>
Signed-off-by: Keerthan Ekbote <saiskee@gmail.com>
As with any other dependency, BoringSSL is not a fixed thing. #14600 added an enumeration over all BoringSSL errors. This incorrectly assumes we'd never add more errors, and unnecessarily adds an dependency on errors (e.g. SSL_ERROR_WANT_CHANNEL_ID_LOOKUP) that Envoy will never encounter and may be removed in the future. Instead, the correct function is SSL_error_description. The original code enumerated errors because Envoy tries to support an old version of BoringSSL, but in that case the future-proof scheme would be to use a BORINGSSL_API_VERSION ifdef. Next, this rewrites the test. The tests assume SSL_ERROR_* constants are stable, which is invalid, and they assume that 19 will never be allocated when it has been and, in fact, we allocate them consecutively. Instead, use the constants, test a few error codes that Envoy already depends on, and use -1 as the sample unknown error. This ensures Envoy's logging reflect future values BoringSSL may add and avoids this code breaking Envoy in a future version of BoringSSL. Signed-off-by: David Benjamin <davidben@google.com>
…proxy filter (#16294) Signed-off-by: Sixiang Gu <sgu@twitter.com>
Per-request buffer accounting is still a work in progress so behavior changes are protected by 'envoy.test_only.per_stream_buffer_accounting' which is disabled by default. Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Signed-off-by: Long Dai <long0dai@foxmail.com>
Signed-off-by: Long Dai <long0dai@foxmail.com>
Signed-off-by: Long Dai <long0dai@foxmail.com>
Signed-off-by: Long Dai <long0dai@foxmail.com>
Currently, envoy ignores the value of present match. This commit adjust envoy's behavior. When the value is True, envoy will check the header present. When the value is False, envoy will check the header absent. Signed-off-by: He Jie Xu <hejie.xu@intel.com>
#16371) Signed-off-by: Peter Woodman <peter@shortbus.org>
Signed-off-by: Piotr Sikora <piotrsikora@google.com>
Signed-off-by: Jose Nino <jnino@lyft.com>
Signed-off-by: Ryan Northey <ryan@synca.io>
It is to fix: #14556 (comment) Currently, remote Jwks is fetched on-demand, in the worker thread after the requests come. The first few requests need to pause to wait for the Jwks. Add a new feature to fetch remote Jwks in the main thread, before the listener is activated. Detail changes: * Change the filter config to add async_fetch field inside RemoteJwks message * Add a new class: JwksAsyncFetcher class to handle this new config. * Add two new statistics counters `jwks_fetch_success` and `jwks_fetch_fail`. Risk Level: Low since new feature is guarded by the new config. Testing: unit-tested and integration tested Docs Changes: None Release Notes: Yes Signed-off-by: Wayne Zhang <qiwzhang@google.com>
Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Ryan Northey <ryan@synca.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Ryan Northey <ryan@synca.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]