Merged
Conversation
Update overview doc to include user-specified logging. Signed-off-by: jiajunye <jiajunye@google.com>
Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
Adding config factories for RequestSourcePlugins. Specifically starting with FileBasedRequestSourcePlugin. These will be provided as options to be used inside RequestSourceFactory. This is part of a series of PRs for providing the ability to use statically linked requestSources. Signed-off-by: William Juan <66322422+wjuan-AFK@users.noreply.github.com>
…ies (#512) - Make the test server use the shared configuration handling code. - Convert its tests to use the new facilities that are shared accross extensions. Last in a series of PRs to fix #498 This is on par with what we did for the dynamic delay and timing extensions, but this extension was slightly more complex to begin with, so saving the best for last ;-) Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
- Update Envoy to 2097fe908f2abb718757dbd4087d793c861d7c5a - Update HdrHistogram_c to 0.11.2 Adds `--define tcmalloc=gperftools` to the opt build we use to push images, to (continue to) allow cpu profiling. We need this because of envoyproxy/envoy@4c0d2d2 Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
Notable changes: - Handle the new `Envoy::Http::StreamResetReason:: ConnectError` & amend related test - Handle that `Envoy::Http::HeaderMap::get()` now returns a `Envoy::Http::HeaderMap::GetResult`. - Squelch Envoy's format check error that scans for urls in `bazel/repositories.bzl` Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
This is a more usable version of the Request Source Config Factory similar to the FIle Based Config Factory which is easier to pass through the CLI. Signed-off-by: William Juan <66322422+wjuan-AFK@users.noreply.github.com>
Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
Commit to add RequestSourcePlugin option to the CLI. base PR: #560. This is necessary to make use of the previous PRs adding the requestsourcepluginfactory via the CLI. Signed-off-by: William Juan <66322422+wjuan-AFK@users.noreply.github.com>
Signed-off-by: Kush Trivedi <kushthedude@gmail.com>
As discussed yestereday, make clang-tidy ignore failures to stop the excessive flaking from being a blocker to merge. Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
- Handle return value of encodeHeaders() now that there is one. - MetricSnapshotImpl now needs a TimeSource in its constructor. - Avoid assert, use valid request header in StreamDecoderTest Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
Adding implementation in the factory_impl class for loading request source plugins. This also introduces python tests for the previous PR and undoes the temporary hack to reduce test_integration_coverage threshold in issue #564. Signed-off-by: William Juan <66322422+wjuan-AFK@users.noreply.github.com>
Signed-off-by: Yan Avlasov <yavlasov@google.com>
Verified that all integration tests pass after updating Envoy to commit `588d9344b31e6544869547c4bcd359b3b0f1d4cf`, so this PR unblocks #575. Our next steps will be adding a good warning and a compatibility flag for users of Nighthawk. If they do send configs with Envoy API v2, we will break by default, but allow them to continue with the compatibility flag. Summary of performed changes: - changing `config` to `typed_config` and listing the correct type. - migrating from deprecated field `tls_context` to `transport_socket`. - changing filter names to ones that match extension names in [extensions_build_config.bzl](https://github.com/envoyproxy/nighthawk/blob/master/extensions_build_config.bzl). - cosmetic changes of enum value from `auto` to `AUTO`. Also: - updating README and help displayed by the CLI in regards to passing in the `--tls-context` flag since this behavior is mirrored by one of the edited integration tests. - Adding the `test_request_source_plugin.py` integration test as a dependency of the `integration_test` py_binary which was forgotten before. Works on #580 Signed-off-by: Jakub Sobon <mumak@google.com>
We don't run integration tests in IPv6 mode on CI (See #578). As a result some of the test expectations became invalid. Also: - Pytest now displays more logging on test failures including the stderr and stdout of the started nighthawk test server. - fixing the default IPv6 address, as per `man 3 inet_pton`, the address `::/0` isn't valid, while `::` is. Signed-off-by: Jakub Sobon <mumak@google.com>
Signed-off-by: Jakub Sobon <mumak@google.com>
* chore: add issue and pull request templates Signed-off-by: Kush Trivedi <kushthedude@gmail.com> * add link to contributing.md Signed-off-by: Kush Trivedi <kushthedude@gmail.com>
Adds a new schedule option to the proto api., which allows specifying a date/time at which execution should start. Should also fix #569 by reducing reliance on `Envoy::SystemTime` to a minimum: Based on the value schedule option, a monotonic time will be computed and propagated. This makes us use `Envoy::MonotonicTime` again in places where sub ms resolution is a must have. (The fix will need confirmation, as I couldn't get `--config=libc++` to work for other reasons.) Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
- Adds a new proto/cli option, which allows requesting continued use of the about to be deprecated v2 api's. - Adds a new fixture / integration test for the test server, as a sanity check for making sure that when we update Envoy to the revision that deprecated v2 things will keep working as expected, while documenting how to set up the runtime configuration. Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
- Fixes to accommodate upstream connection pool changes. - Fixes to accommodate upstream cluster related changes. Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
Verified that an Envoy built after `588d9344b31e6544869547c4bcd359b3b0f1d4cf` can load the configuration from the documents. Also: - fixing one typo in the documentation. - cosmetic changes of enum values to uppercase form. Works on #580 Signed-off-by: Jakub Sobon <mumak@google.com>
…ers. (#588) Adds a new field `v3_response_headers` alongside the old v2 `response_headers` in `api/server/response_options.proto` and marks the old field as deprecated. The old field retains its functionality for backward compatibility. Since these are repeated fields, we cannot use a oneof. Instead validation is added to ensure the filters report an error on configuration that has both the v2 and the v3 fields set. `Envoy::EnvoyException` is thrown on validation errors to comply with the `Envoy::Server::Configuration::NamedHttpFilterConfigFactory` interface. Also: - sorting imports in `response_options.proto ` alphabetically. - adding missing anonymous namespace in `http_dynamic_delay_filter_integration_test.cc`. Works on #580. Signed-off-by: Jakub Sobon <mumak@google.com>
…urce. (#589) Converting the current field into a `oneof` and marking the `v2` primitive as deprecated. The old field retains its functionality for backward compatibility. Also: - sorting imports in `api/request_source/service.proto`. - migrating our dummy request source to using the `v3` primitive. - improving error message emitted on test failures in `test/request_stream_grpc_client_test.cc`, the current message just dumps byte representation of the two compared header objects. Fixes #580. Signed-off-by: Jakub Sobon mumak@google.com
Fails the integration tests if any unknown warnings or errors are found in the logs of the Nighthawk test server. Includes an ignore list that (for now) accepts known warnings found in the logs, the ignore list can have different content per each test case. Also: - moving the `Attributes:` docstring section on `IntegrationTestBase` into its class docstring. `__init__` is just a method and should only have the `Args:` section. Fixes #577 Signed-off-by: Jakub Sobon <mumak@google.com>
Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
List of changes: - Update Envoy to 8188e232a9e0b15111d30f4724cbc7bf77d3964a (Dec 8th) - Sync .bazelrc - Modify extensions_build_config.bzl, add required DISABLED_BY_DEFAULT_EXTENSIONS - Rework our h1 connection pool derivation to keep our own prefetching feature working. Note: We enable the allow_prefetch runtime feature for this. - Changes to reflect changed Envoy constructors and method signatures: OnPoolReady(), ProdClusterManagerFactory(), allocateConnPool() - Modified log filtering in the integration test: `SIGTERM` -> `ENVOY_SIGTERM`. - Dropped include of `ares.h` as small cleanup, we no longer need that. - In `options_impl.cc` there's a small change to how we define `elapsed_since_epoch` to unbreak building with `libc++`: a regression of #569. Filed #594 to avoid regression. Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
- The way we can obtain a http connection pool changed. Amend. - Cleanup by fix_format: strip superfluous .Times(1) in tests. Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
…598) The character literal overload is more efficient. Signed-off-by: Yan Avlasov <yavlasov@google.com>
- Unbreak changed allocateConnPool() method usage: changes with respect to ALPN allow negotiation of a protocol, which is backed in Envoy by a pool which supports >1 protocols. This update leaves a code-level comment plus a RELEASE_ASSERT when a multi-protocol pool is allocated. - Avoid MOCK_METHODn as per new check_format objections: Change our mocks to use MOCK_METHOD instead. Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
- Transitively updates the grpc dependency, which eliminated the grpc_impl namespace. s/grpc_impl::/grpc::/ - Envoy::LocalInfo::LocalInfoImpl constructor now requires passing a symbol table. Pass that in. Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
Fixes a bug observed while working on horizontal scaling support: SignalHandler would fire on destruction. Fix & add tests. Note: pre-emptive, I don't think this is an issue in the current state of master. But some of the CI errors in #600 seemed familiar and hopefully merging this in there helps. (Split out from https://github.com/oschaaf/nighthawk/tree/horizontal-scaling) Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
- `Envoy::StreamInfo::StreamInfoImpl` no longer contains methods that set local and remote socket addresses. It instead requires a `Envoy::Network::SocketAddressProviderSharedPtr` with the addresses already set. Pass in a `Envoy::Network::SocketAddressSetterImpl` so that we can continue setting the remote address for tests. - Copied `.bazelversion` and `.bazelrc` from Envoy. Signed-off-by: Jakub Sobon <mumak@google.com>
Updates the Envoy dependency to revision: 17e815122ff53d0ac6cb2d64cdbf1bfc547bb7e8 Non mechanical change: - LocalinfoImpl constructor changed Signed-off-by: Otto van der Schaaf <oschaaf@we-amp.com>
This fixes the broken docker ci workflow. Signed-off-by: Jakub Sobon <mumak@google.com>
- tryCreateNewConnection now returns a ConnPoolImplBase::ConnectionResult instead of a bool. Signed-off-by: Jakub Sobon <mumak@google.com>
* Updating references to branch name master->main. The branch was renamed recently. Signed-off-by: Jakub Sobon <mumak@google.com>
Changing github repository paths from oschaaf to envoyproxy. Signed-off-by: Jakub Sobon <mumak@google.com>
Signed-off-by: Otto van der Schaaf <ovanders@redhat.com>
A mock for AdaptiveLoadController for use in unit tests of the upcoming adaptive load CLI executable. Part of #416 Signed-off-by: eric846 <56563761+eric846@users.noreply.github.com>
I think the root cause is that the alpine-glibc image updates, and through that dependency requirements can no longer be met. This PR - pins the alpine image at the latest version - changes the dependency versions so they work again Note: the `>=` for python was changed to `=` as otherwise we'd write a file to disk mentioning the version, this seemed contra intent. There's a complaint [1] emitted during the image build, so we need to see if everything is OK when CI finishes. [1] /usr/glibc-compat/sbin/ldconfig: /usr/glibc-compat/lib/ld-linux-x86-64.so.2 is not a symbolic link Signed-off-by: Otto van der Schaaf <ovanders@redhat.com>
* Update Envoy to cfe0391 (Feb 5th 2021). - synchronizing `.bazelrc` from Envoy repository. - the API of `getJsonStringFromMessage` changed, it now returns `ProtobufUtil::StatusOr` instead of `std::string`. Switching our code over to `getJsonStringFromMessageOrDie` which continues to return `std::string`. - Increased the `no_output_timeout` parameter to 30m for test_gcc which was timing out. Signed-off-by: Jakub Sobon <mumak@google.com>
Step by step guide on how to perform the update. Marking our local changes in .bazelrc with #unique. To make it easier to spot them. Reformatting MAINTAINERS.md, adding line breaks at 80 char where appropriate. Signed-off-by: Jakub Sobon <mumak@google.com>
Being able to serialize and deserialize statistics serves several goals: - Allow over-the-wire transportation in horizontally scaled setups. - Facilitate storage of high res data. - Could be used as a means to decouple our workers from the main thread, by avoiding reliance on a shared process namespace to merge statistics. (not a goal right now) This adds an abstraction to Statistic, implements it for HdrHistogram, and adds some tests. Part of horizontal scaling effort, split out from: Split out from https://github.com/oschaaf/nighthawk/tree/horizontal-scaling Signed-off-by: Otto van der Schaaf <ovanders@redhat.com>
An adaptive load command line executable that drives a Nighthawk Service with a series of benchmarks, varying the RPS or another input variable dynamically. `bazel build -c opt :nighthawk` `bazel-bin/nighthawk_adaptive_load_client` - `--spec-file` _pathname_ - Pathname of an `AdaptiveLoadSessionSpec` textproto defining the session - Required - Example:`test/adaptive_load/test_data/valid_session_spec.textproto` - `--output-file` _pathname_ - Pathname to write the `AdaptiveLoadSessionOutput` textproto session output - Required - `--nighthawk-service-address` _host:port_ - Address of the Nighthawk Service to be driven by the adaptive load controller over gRPC - Default `localhost:8443` - `--use-tls` - Use TLS for the gRPC connection when sending commands to the Nighthawk Service - Default false (insecure connection) Note that to run an adaptive load session, you must also have a Nighthawk Service running, e.g.: `bazel-bin/nighthawk_service &` (to run at localhost:8443) Part of #416. Signed-off-by: eric846 <56563761+eric846@users.noreply.github.com>
- don't set up a drain callback when there are no active connections - set up a timer that will cap the amount of time we wait for the pool to drain. - disable latency measurement when commencing the drain procedure, as by that time we are no longer interested in it, and in particular don't want to hear about any warnings issued by the Statistic implementation about recorded values being too large. Fixes #627 Signed-off-by: Otto van der Schaaf <ovanders@redhat.com>
Fixes #629 Signed-off-by: Otto van der Schaaf <ovanders@redhat.com>
Signed-off-by: Nathan Perry <nbperry@google.com>
At some point clang-tidy runs broke in CI, and this couldn't be reproduced locally. The after a while local runs would exit with a failure seemingly mid- run. This PR addresses both these things. - Update the CircleCI SHA of the envoy build docker image. - Sync the script we use to run Envoy's build docker image. - Make clang-tidy runs use libstdc++ to avoid standard library headers from producing errors. - Sync run_clang_tidy.sh - Update MAINTAINERS.md to mention we should keep more things in sync. Fixes #546 Signed-off-by: Otto van der Schaaf <ovanders@redhat.com>
* Fix python dependency version for benchmark image Signed-off-by: abaptiste <abaptiste@users.noreply.github.com>
Recommend not to check for strict equality on error messages. This is aligned with [Google Cloud APIs error model](https://cloud.google.com/apis/design/errors#error_messages). We are considering adding extra debug information to the underlying JSON parser library. The extra debug information breaks this test if it checks for strict equality. Signed-off-by: Teju Nareddy <nareddyt@google.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.