diff --git a/docs/configuration/http_conn_man/route_config/route.rst b/docs/configuration/http_conn_man/route_config/route.rst index 9872b8cfb1fd6..8a5f42830dffe 100644 --- a/docs/configuration/http_conn_man/route_config/route.rst +++ b/docs/configuration/http_conn_man/route_config/route.rst @@ -49,7 +49,7 @@ cluster *path_redirect* is not specified), one of *cluster* or *weighted_clusters* must be specified. With the *weighted_clusters* option, multiple upstream clusters can be specified for a given route. The request is forwarded to one of the upstream clusters based on weights assigned - to each cluster. See :ref:`traffic splitting ` + to each cluster. See :ref:`traffic splitting ` for additional documentation. .. _config_http_conn_man_route_table_route_host_redirect: @@ -122,7 +122,9 @@ Runtime ------- A :ref:`runtime ` route configuration can be used to roll out route changes -in a gradual manner without full code/config deploys. +in a gradual manner without full code/config deploys. Refer to +:ref:`traffic shifting ` docs +for additional documentation. .. code-block:: json diff --git a/docs/configuration/http_conn_man/route_config/route_config.rst b/docs/configuration/http_conn_man/route_config/route_config.rst index a37ef752f3fb6..d8317c8069a89 100644 --- a/docs/configuration/http_conn_man/route_config/route_config.rst +++ b/docs/configuration/http_conn_man/route_config/route_config.rst @@ -55,5 +55,4 @@ response_headers_to_remove vcluster rate_limits route_matching - traffic_shifting - weighted_routing + traffic_splitting diff --git a/docs/configuration/http_conn_man/route_config/traffic_shifting.rst b/docs/configuration/http_conn_man/route_config/traffic_shifting.rst deleted file mode 100644 index 69c4e1911fd24..0000000000000 --- a/docs/configuration/http_conn_man/route_config/traffic_shifting.rst +++ /dev/null @@ -1,49 +0,0 @@ -.. _config_http_conn_man_route_table_traffic_shifting: - -Traffic Shifting -================ - -Envoy's router can accomplish traffic shifting between routes of a virtual host -based on the :ref:`runtime ` object in the route configuration. -A common use case is shifting traffic between clusters with different versions of a service deployed on them. - -.. code-block:: json - - { - "route_config": { - "virtual_hosts": [ - { - "name": "service", - "domains": ["*"], - "routes": [ - { - "prefix": "/", - "cluster": "service_v1", - "runtime": { - "key": "routing.traffic_shift.service", - "default": 50 - } - }, - { - "prefix": "/", - "cluster": "service_v2", - } - ] - } - ] - } - } - -Envoy matches routes with a :ref:`first match ` policy. -If the route has a runtime object, the request will be additionally matched based on the runtime -:ref:`value ` -(or the default, if no value is specified). Thus, by placing routes back-to-back in the above example and specifying -a runtime object in the first route, traffic shifting can be accomplished by changing the runtime value. The flow would -look something like this: - -1. Set ``routing.traffic_shift.service`` to ``100``. This would mean that all requests to the ``service`` virtual host - would match with the v1 route. -2. Set ``routing.traffic_shift.service`` to values ``0 < x < 100``. For instance at ``50``, half of the requests to the - ``service`` virtual host will not match the v1 route and fall through to the v2 route. -3. Set ``routing.traffic_shift.service`` to ``0``. This means no requests to the ``service`` virtual host will match - to the v1 route and they will all fall through to the v2 route. diff --git a/docs/configuration/http_conn_man/route_config/traffic_splitting.rst b/docs/configuration/http_conn_man/route_config/traffic_splitting.rst new file mode 100644 index 0000000000000..c8d90b58b14b2 --- /dev/null +++ b/docs/configuration/http_conn_man/route_config/traffic_splitting.rst @@ -0,0 +1,194 @@ +.. _config_http_conn_man_route_table_traffic_splitting: + +Traffic Shifting/Splitting +=========================================== + +Envoy's router can split traffic to a route in a virtual host across +two or more upstream clusters. There are two common use cases. + +1. Version upgrades: traffic to a route is shifted gradually +from one cluster to another. The +:ref:`traffic shifting ` +section describes this scenario in more detail. + +2. A/B testing or multivariate testing: ``two or more versions`` of +the same service are tested simultaneously. The traffic to the route has to +be *split* between clusters running different versions of the same +service. The +:ref:`traffic splitting ` +section describes this scenario in more detail. + +.. _config_http_conn_man_route_table_traffic_splitting_shift: + +Traffic shifting between two upstreams +====================================== + +The :ref:`runtime ` object +in the route configuration determines the probability of selecting a +particular route (and hence its cluster). By using the runtime +configuration, traffic to a particular route in a virtual host can be +gradually shifted from one cluster to another. Consider the following +example configuration, where two versions ``helloworld_v1`` and +``helloworld_v2`` of a service named ``helloworld`` are declared in the +envoy configuration file. + +.. code-block:: json + + { + "route_config": { + "virtual_hosts": [ + { + "name": "helloworld", + "domains": ["*"], + "routes": [ + { + "prefix": "/", + "cluster": "helloworld_v1", + "runtime": { + "key": "routing.traffic_shift.helloworld", + "default": 50 + } + }, + { + "prefix": "/", + "cluster": "helloworld_v2", + } + ] + } + ] + } + } + +Envoy matches routes with a :ref:`first match ` policy. +If the route has a runtime object, the request will be additionally matched based on the runtime +:ref:`value ` +(or the default, if no value is specified). Thus, by placing routes +back-to-back in the above example and specifying a runtime object in the +first route, traffic shifting can be accomplished by changing the runtime +value. The following are the approximate sequence of actions required to +accomplish the task. + +1. In the beginning, set ``routing.traffic_shift.helloworld`` to ``100``, + so that all requests to the ``helloworld`` virtual host would match with + the v1 route and be served by the ``helloworld_v1`` cluster. +2. To start shifting traffic to ``helloworld_v2`` cluster, set + ``routing.traffic_shift.helloworld`` to values ``0 < x < 100``. For + instance at ``90``, 1 out of every 10 requests to the ``helloworld`` + virtual host will not match the v1 route and will fall through to the v2 + route. +3. Gradually decrease the value set in ``routing.traffic_shift.helloworld`` + so that a larger percentage of requests match the v2 route. +4. When ``routing.traffic_shift.helloworld`` is set to ``0``, no requests + to the ``helloworld`` virtual host will match to the v1 route. All + traffic would now fall through to the v2 route and be served by the + ``helloworld_v2`` cluster. + + +.. _config_http_conn_man_route_table_traffic_splitting_split: + +Traffic splitting across multiple upstreams +=========================================== + +Consider the ``helloworld`` example again with three versions (v1, v2 and +v3). Envoy provides two ways to split traffic evenly across the three +versions (i.e., ``33%, 33%, 34%``): + +.. _config_http_conn_man_route_table_traffic_splitting_split_percentages: + +(a) Weight-based cluster selection +---------------------------------- + +The first option is to use a **single** :ref:`route ` with +:ref:`weighted_clusters `, +where multiple upstream cluster targets are specified for a single route, +along with weights that indicate the **percentage** of traffic to be sent +to each upstream cluster. + +.. code-block:: json + + { + "route_config": { + "virtual_hosts": [ + { + "name": "helloworld", + "domains": ["*"], + "routes": [ + { + "prefix": "/", + "weighted_clusters": { + "runtime_key_prefix" : "routing.traffic_split.helloworld", + "clusters" : [ + { "name" : "helloworld_v1", "weight" : 33 }, + { "name" : "helloworld_v2", "weight" : 33 }, + { "name" : "helloworld_v3", "weight" : 34 } + ] + } + } + ] + } + ] + } + } + +The weights assigned to each cluster can be dynamically adjusted using the +following runtime variables: ``routing.traffic_split.helloworld.helloworld_v1``, +``routing.traffic_split.helloworld.helloworld_v2`` and +``routing.traffic_split.helloworld.helloworld_v3``. + +.. _config_http_conn_man_route_table_traffic_splitting_split_probabilities: + +(b) Probabilistic route selection +--------------------------------- + +The second option is to use **multiple** :ref:`routes ` +as shown in the traffic shifting example, with :ref:`runtimes ` +that specify the **probability** of selecting a route. +Since Envoy matches routes with a :ref:`first match ` +policy, the related routes (one for each upstream cluster) must be placed back-to-back, +along with a runtime in all but the last route. + +.. code-block:: json + + { + "route_config": { + "virtual_hosts": [ + { + "name": "helloworld", + "domains": ["*"], + "routes": [ + { + "prefix": "/", + "cluster": "helloworld_v1", + "runtime": { + "key": "routing.traffic_split.helloworld.helloworld_v1", + "default": 33 + } + }, + { + "prefix": "/", + "cluster": "helloworld_v2", + "runtime": { + "key": "routing.traffic_split.helloworld.helloworld_v2", + "default": 50 + } + }, + { + "prefix": "/", + "cluster": "helloworld_v3", + } + ] + } + ] + } + } + +In the configuration above, + +1. ``routing.traffic_split.helloworld.helloworld_v1`` is set to ``33``, so that there is a + *33\% probability* that the v1 route will be selected by Envoy. +2. ``routing.traffic_split.helloworld.helloworld_v2`` is set to ``50``, so that if the v1 route + is not selected, between v2 and v3, there is a *50\% probability* that the v2 route will + be selected by Envoy. If v2 is not selected the traffic falls through to the v3 route. + +This distribution of probabilities ensures that the traffic will be split evenly across +all three routes (i.e. ``33%, 33%, 34%``). diff --git a/docs/configuration/http_conn_man/route_config/weighted_routing.rst b/docs/configuration/http_conn_man/route_config/weighted_routing.rst deleted file mode 100644 index c6f252b0ae60c..0000000000000 --- a/docs/configuration/http_conn_man/route_config/weighted_routing.rst +++ /dev/null @@ -1,120 +0,0 @@ -.. _config_http_conn_man_route_table_weighted_routing: - -Traffic splitting across multiple upstreams -=========================================== - -Envoy's router can split traffic to a route in a virtual host across -multiple upstream clusters. There are two common use cases. The first use -case is version upgrades, where traffic to a route is shifted gradually -from one cluster to another. The -:ref:`traffic shifting ` -describes this scenario in more detail. - -The second use case, discussed in this section, is A/B testing or multivariate -testing, where ``two or more versions`` of the same service are tested simultaneously. -The traffic to the route has to be *split* between clusters running different versions -of the same service. - -Consider a simple example ``service`` with three versions (v1, v2 and -v3). Envoy provides two ways to split traffic evenly across the three -versions (i.e., ``33%, 33%, 34%``): - -.. _config_http_conn_man_route_table_weighted_routing_percentages: - -(a) Weight-based cluster selection ----------------------------------- - -The first option is to use a **single** :ref:`route ` with -:ref:`weighted_clusters `, -where multiple upstream cluster targets are specified for a single route, -along with weights that indicate the **percentage** of traffic to be sent -to each upstream cluster. - -.. code-block:: json - - { - "route_config": { - "virtual_hosts": [ - { - "name": "service", - "domains": ["*"], - "routes": [ - { - "prefix": "/", - "weighted_clusters": { - "runtime_key_prefix" : "routing.traffic_split.service", - "clusters" : [ - { "name" : "service_v1", "weight" : 33 }, - { "name" : "service_v2", "weight" : 33 }, - { "name" : "service_v3", "weight" : 34 } - ] - } - } - ] - } - ] - } - } - -The weights assigned to each cluster can be dynamically adjusted using the -following runtime variables: ``routing.traffic_split.service.service_v1``, -``routing.traffic_split.service.service_v2`` and -``routing.traffic_split.service.service_v3``. - -.. _config_http_conn_man_route_table_weighted_routing_probabilities: - -(b) Probabilistic route selection ---------------------------------- - -The second option is to use **multiple** :ref:`routes ` -with :ref:`runtimes ` that specify the -**probability** of selecting a route. -Since Envoy matches routes with a :ref:`first match ` -policy, the related routes (one for each upstream cluster) must be placed back-to-back, -along with a runtime in all but the last route. - -.. code-block:: json - - { - "route_config": { - "virtual_hosts": [ - { - "name": "service", - "domains": ["*"], - "routes": [ - { - "prefix": "/", - "cluster": "service_v1", - "runtime": { - "key": "routing.traffic_split.service.service_v1", - "default": 33 - } - }, - { - "prefix": "/", - "cluster": "service_v2", - "runtime": { - "key": "routing.traffic_split.service.service_v2", - "default": 50 - } - }, - { - "prefix": "/", - "cluster": "service_v3", - } - ] - } - ] - } - } - -In the configuration above, - -1. ``routing.traffic_split.service.service_v1`` is set to ``33``, so that there is a - *33\% probability* that the v1 route will be selected by Envoy. -2. ``routing.traffic_split.service.service_v2`` is set to ``50``, so that if the v1 route - is not selected, between v2 and v3, there is a *50\% probability* that the v2 route will - be selected by Envoy. If v2 is not selected the traffic falls through to the v3 route. - -This distribution of probabilities ensures that the traffic will be split evenly across -all three routes (i.e. ``33%, 33%, 34%``). diff --git a/docs/intro/arch_overview/http_routing.rst b/docs/intro/arch_overview/http_routing.rst index 5a93ce91c2c7e..52795e2807e69 100644 --- a/docs/intro/arch_overview/http_routing.rst +++ b/docs/intro/arch_overview/http_routing.rst @@ -27,8 +27,10 @@ request. The router filter supports the following features: * Request timeout specified either via :ref:`HTTP header ` or via :ref:`route configuration `. -* Runtime configuration routing rules. Runtime configuration can be used to create :ref:`traffic - shifting `. +* Probability-based route selection via :ref:`runtime values ` + (see :ref:`traffic shifting/shifting `). +* :ref:`Weight/percentage-based ` upstream cluster selection + for a given route (see :ref:`traffic shifting/splitting `). * Arbitrary headers :ref:`routing rules `. * Virtual cluster specifications. A virtual cluster is specified at the virtual host level and is used by Envoy to generate additional statistics on top of the standard cluster level ones. Virtual