-
Notifications
You must be signed in to change notification settings - Fork 5.3k
http fault: implement header controlled faults #6318
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
b34e7ff
2bc39f5
97f2905
7def5ef
eda7fa8
92cfdd6
606a364
c6ea7fc
70506d1
574385b
a79a8b0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,15 +16,6 @@ The scope of failures is restricted to those that are observable by an | |
| application communicating over the network. CPU and disk failures on the | ||
| local host cannot be emulated. | ||
|
|
||
| Currently, the fault injection filter has the following limitations: | ||
|
|
||
| * Abort codes are restricted to HTTP status codes only | ||
| * Delays are restricted to fixed duration. | ||
|
|
||
| Future versions will include support for restricting faults to specific | ||
| routes, injecting *gRPC* and *HTTP/2* specific error codes and delay | ||
| durations based on distributions. | ||
|
|
||
| Configuration | ||
| ------------- | ||
|
|
||
|
|
@@ -36,33 +27,83 @@ Configuration | |
| * :ref:`v2 API reference <envoy_api_msg_config.filter.http.fault.v2.HTTPFault>` | ||
| * This filter should be configured with the name *envoy.fault*. | ||
|
|
||
| .. _config_http_filters_fault_injection_http_header: | ||
|
|
||
| Controlling fault injection via HTTP headers | ||
| -------------------------------------------- | ||
|
|
||
| The fault filter has the capability to allow fault configuration to be specified by the caller. | ||
| This is useful in certain scenarios in which it is desired to allow the client to specify its own | ||
| fault configuration. The currently supported header controls are: | ||
|
|
||
| * Request delay configuration via the *x-envoy-fault-delay-request* header. The header value | ||
| should be an integer that specifies the number of milliseconds to throttle the latency for. | ||
mattklein123 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| * Response rate limit configuration via the *x-envoy-fault-throughput-response* header. The | ||
| header value should be an integer that specified the limit in KiB/s and must be > 0. | ||
mattklein123 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| .. attention:: | ||
|
|
||
| Allowing header control is inherently dangerous if exposed to untrusted clients. In this case, | ||
| it is suggested to use the :ref:`max_active_faults | ||
| <envoy_api_field_config.filter.http.fault.v2.HTTPFault.max_active_faults>` setting to limit the | ||
| maximum concurrent faults that can be active at any given time. | ||
|
|
||
| The following is an example configuration that enables header control for both of the above | ||
| options: | ||
|
|
||
| .. code-block:: yaml | ||
|
|
||
| name: envoy.fault | ||
| config: | ||
| max_active_faults: 100 | ||
| delay: | ||
| header_delay: {} | ||
| percentage: | ||
| numerator: 100 | ||
| response_rate_limit: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. per previous discussion, i suggest cutting the percentage here and opting for a semantic like, "if you specify delay, it will be injected with 100% certainty". That would make it easier for people to test, imo
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See above comment. |
||
| header_limit: {} | ||
| percentage: | ||
| numerator: 100 | ||
|
|
||
| .. _config_http_filters_fault_injection_runtime: | ||
|
|
||
| Runtime | ||
mattklein123 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ------- | ||
|
|
||
| The HTTP fault injection filter supports the following global runtime settings: | ||
|
|
||
| .. attention:: | ||
|
|
||
| Some of the following runtime keys require the filter to be configured for the specific fault | ||
| type and some do not. Please consult the documentation for each key for more information. | ||
|
|
||
| fault.http.abort.abort_percent | ||
| % of requests that will be aborted if the headers match. Defaults to the | ||
| *abort_percent* specified in config. If the config does not contain an | ||
| *abort* block, then *abort_percent* defaults to 0. | ||
| *abort* block, then *abort_percent* defaults to 0. For historic reasons, this runtime key is | ||
| available regardless of whether the filter is :ref:`configured for abort | ||
| <envoy_api_field_config.filter.http.fault.v2.HTTPFault.abort>`. | ||
|
|
||
| fault.http.abort.http_status | ||
| HTTP status code that will be used as the of requests that will be | ||
| aborted if the headers match. Defaults to the HTTP status code specified | ||
| in the config. If the config does not contain an *abort* block, then | ||
| *http_status* defaults to 0. | ||
| *http_status* defaults to 0. For historic reasons, this runtime key is | ||
| available regardless of whether the filter is :ref:`configured for abort | ||
| <envoy_api_field_config.filter.http.fault.v2.HTTPFault.abort>`. | ||
|
|
||
| fault.http.delay.fixed_delay_percent | ||
| % of requests that will be delayed if the headers match. Defaults to the | ||
| *delay_percent* specified in the config or 0 otherwise. | ||
| *delay_percent* specified in the config or 0 otherwise. This runtime key is only available when | ||
| the filter is :ref:`configured for delay | ||
| <envoy_api_field_config.filter.http.fault.v2.HTTPFault.delay>`. | ||
|
|
||
| fault.http.delay.fixed_duration_ms | ||
| The delay duration in milliseconds. If not specified, the | ||
| *fixed_duration_ms* specified in the config will be used. If this field | ||
| is missing from both the runtime and the config, no delays will be | ||
| injected. | ||
| injected. This runtime key is only available when the filter is :ref:`configured for delay | ||
| <envoy_api_field_config.filter.http.fault.v2.HTTPFault.delay>`. | ||
|
|
||
| fault.http.max_active_faults | ||
| The maximum number of active faults (of all types) that Envoy will will inject via the fault | ||
|
|
@@ -72,10 +113,10 @@ fault.http.max_active_faults | |
| <envoy_api_field_config.filter.http.fault.v2.HTTPFault.max_active_faults>` setting will be used. | ||
|
|
||
| fault.http.rate_limit.response_percent | ||
| % of requests which will have a response rate limit fault injected, if the filter is | ||
| :ref:`configured <envoy_api_field_config.filter.http.fault.v2.HTTPFault.response_rate_limit>` to | ||
| do so. Defaults to the value set in the :ref:`percentage | ||
| <envoy_api_field_config.filter.fault.v2.FaultRateLimit.percentage>` field. | ||
| % of requests which will have a response rate limit fault injected. Defaults to the value set in | ||
| the :ref:`percentage <envoy_api_field_config.filter.fault.v2.FaultRateLimit.percentage>` field. | ||
| This runtime key is only available when the filter is :ref:`configured for response rate limiting | ||
| <envoy_api_field_config.filter.http.fault.v2.HTTPFault.response_rate_limit>`. | ||
|
|
||
| *Note*, fault filter runtime settings for the specific downstream cluster | ||
| override the default ones if present. The following are downstream specific | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| licenses(["notice"]) # Apache 2 | ||
|
|
||
| load( | ||
| "//bazel:envoy_build_system.bzl", | ||
| "envoy_cc_library", | ||
| "envoy_package", | ||
| ) | ||
|
|
||
| envoy_package() | ||
|
|
||
| envoy_cc_library( | ||
| name = "fault_config_lib", | ||
| srcs = ["fault_config.cc"], | ||
| hdrs = ["fault_config.h"], | ||
| deps = [ | ||
| "//include/envoy/http:header_map_interface", | ||
| "//source/common/protobuf:utility_lib", | ||
| "@envoy_api//envoy/config/filter/fault/v2:fault_cc", | ||
| ], | ||
| ) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| #include "extensions/filters/common/fault/fault_config.h" | ||
|
|
||
| #include "common/protobuf/utility.h" | ||
|
|
||
| namespace Envoy { | ||
| namespace Extensions { | ||
| namespace Filters { | ||
| namespace Common { | ||
| namespace Fault { | ||
|
|
||
| FaultDelayConfig::FaultDelayConfig(const envoy::config::filter::fault::v2::FaultDelay& delay_config) | ||
| : percentage_(delay_config.percentage()) { | ||
| switch (delay_config.fault_delay_secifier_case()) { | ||
| case envoy::config::filter::fault::v2::FaultDelay::kFixedDelay: | ||
| provider_ = std::make_unique<FixedDelayProvider>( | ||
| std::chrono::milliseconds(PROTOBUF_GET_MS_REQUIRED(delay_config, fixed_delay))); | ||
| break; | ||
| case envoy::config::filter::fault::v2::FaultDelay::kHeaderDelay: | ||
| provider_ = std::make_unique<HeaderDelayProvider>(); | ||
| break; | ||
| case envoy::config::filter::fault::v2::FaultDelay::FAULT_DELAY_SECIFIER_NOT_SET: | ||
| NOT_REACHED_GCOVR_EXCL_LINE; | ||
| } | ||
| } | ||
|
|
||
| absl::optional<std::chrono::milliseconds> | ||
| FaultDelayConfig::HeaderDelayProvider::duration(const Http::HeaderEntry* header) const { | ||
| if (header == nullptr) { | ||
| return absl::nullopt; | ||
| } | ||
|
|
||
| uint64_t value; | ||
| if (!StringUtil::atoull(header->value().getStringView().data(), value)) { | ||
| return absl::nullopt; | ||
| } | ||
|
|
||
| return std::chrono::milliseconds(value); | ||
| } | ||
|
|
||
| FaultRateLimitConfig::FaultRateLimitConfig( | ||
| const envoy::config::filter::fault::v2::FaultRateLimit& rate_limit_config) | ||
| : percentage_(rate_limit_config.percentage()) { | ||
| switch (rate_limit_config.limit_type_case()) { | ||
| case envoy::config::filter::fault::v2::FaultRateLimit::kFixedLimit: | ||
| provider_ = | ||
| std::make_unique<FixedRateLimitProvider>(rate_limit_config.fixed_limit().limit_kbps()); | ||
| break; | ||
| case envoy::config::filter::fault::v2::FaultRateLimit::kHeaderLimit: | ||
| provider_ = std::make_unique<HeaderRateLimitProvider>(); | ||
| break; | ||
| case envoy::config::filter::fault::v2::FaultRateLimit::LIMIT_TYPE_NOT_SET: | ||
| NOT_REACHED_GCOVR_EXCL_LINE; | ||
| } | ||
| } | ||
|
|
||
| absl::optional<uint64_t> | ||
| FaultRateLimitConfig::HeaderRateLimitProvider::rateKbps(const Http::HeaderEntry* header) const { | ||
| if (header == nullptr) { | ||
| return absl::nullopt; | ||
| } | ||
|
|
||
| uint64_t value; | ||
| if (!StringUtil::atoull(header->value().getStringView().data(), value)) { | ||
| return absl::nullopt; | ||
| } | ||
|
|
||
| if (value == 0) { | ||
| return absl::nullopt; | ||
| } | ||
|
|
||
| return value; | ||
| } | ||
|
|
||
| } // namespace Fault | ||
| } // namespace Common | ||
| } // namespace Filters | ||
| } // namespace Extensions | ||
| } // namespace Envoy |
Uh oh!
There was an error while loading. Please reload this page.