implement rds api

docs/configuration/cluster_manager/cds.rst

@@ -11,7 +11,7 @@ clusters depending on what is required.
 
   {
     "cluster": "{...}",
-    "refresh_interval_ms": "{...}"
+    "refresh_interval_ms": "..."
   }
 
 :ref:`cluster <config_cluster_manager_cluster>`
@@ -20,10 +20,10 @@ clusters depending on what is required.
   <config_cluster_manager_cds_api>`.
 
 refresh_interval_ms
-  *(optional, integer)* The delay, in milliseconds, between fetches to the CDS API for each
-  configured CDS cluster. Envoy will add an additional random jitter to the delay that is between
-  zero and *refresh_interval_ms* milliseconds. Thus the longest possible refresh delay is
-  2 \* *refresh_interval_ms*. Default value is 30000ms (30 seconds).
+  *(optional, integer)* The delay, in milliseconds, between fetches to the CDS API. Envoy will add
+  an additional random jitter to the delay that is between zero and *refresh_interval_ms*
+  milliseconds. Thus the longest possible refresh delay is 2 \* *refresh_interval_ms*. Default value
+  is 30000ms (30 seconds).
 
 .. _config_cluster_manager_cds_api:
 

docs/configuration/http_conn_man/http_conn_man.rst

@@ -14,6 +14,7 @@ HTTP connection manager
     "config": {
       "codec_type": "...",
       "stat_prefix": "...",
+      "rds": "{...}",
       "route_config": "{...}",
       "filters": [],
       "add_user_agent": "...",
@@ -54,9 +55,17 @@ stat_prefix
   connection manager. See the :ref:`statistics <config_http_conn_man_stats>` documentation
   for more information.
 
+:ref:`rds <config_http_conn_man_rds>`
+  *(sometimes required, object)* The connection manager configuration must specify one of *rds* or
+  *route_config*. If *rds* is specified, the connection manager's route table will be dynamically
+  loaded via the RDS API. See the :ref:`documentation <config_http_conn_man_rds>` for more
+  information.
+
 :ref:`route_config <config_http_conn_man_route_table>`
-  *(required, object)* The :ref:`route table <arch_overview_http_routing>` for the connection
-  manager. All connection managers must have a route table, even if it is empty.
+  *(sometimes required, object)* The connection manager configuration must specify one of *rds* or
+  *route_config*. If *route_config* is specified, the :ref:`route table <arch_overview_http_routing>`
+  for the connection manager is static and is specified in this property. All connection managers must have
+  a route table, even if it is empty.
 
 :ref:`filters <config_http_conn_man_filters>`
   *(required, array)* A list of individual :ref:`HTTP filters <arch_overview_http_filters>` that
@@ -74,7 +83,7 @@ add_user_agent
 :ref:`tracing <config_http_conn_man_tracing>`
   *(optional, object)* Presence of the object defines whether the connection manager
   emits :ref:`tracing <arch_overview_tracing>` data to the :ref:`configured tracing provider <config_tracing>`.
-  
+
 .. _config_http_conn_man_http_codec_options:
 
 http_codec_options
@@ -145,3 +154,4 @@ generate_request_id
   headers
   stats
   runtime
+  rds

docs/configuration/http_conn_man/rds.rst

@@ -0,0 +1,70 @@
+.. _config_http_conn_man_rds:
+
+Route discovery service
+=======================
+
+The route discovery service (RDS) API is an optional API that Envoy will call to dynamically fetch
+route tables. Each :ref:`HTTP connection manager filter <config_http_conn_man>` can independently
+fetch its own route table via the API.
+
+.. code-block:: json
+
+  {
+    "cluster": "{...}",
+    "route_config_name": "...",
+    "refresh_interval_ms": "..."
+  }
+
+cluster
+  *(required, string)* The name of an upstream :ref:`cluster <config_cluster_manager_cluster>` that
+  hosts the route discovery service. The cluster must run a REST service that implements the
+  :ref:`RDS HTTP API <config_http_conn_man_rds_api>`.
+
+route_config_name
+  *(required, string)* The name of the route table. This name will be passed to the
+  :ref:`RDS HTTP API <config_http_conn_man_rds_api>` so that different route tables can be used by
+  independent filters.
+
+refresh_interval_ms
+  *(optional, integer)* The delay, in milliseconds, between fetches to the RDS API. Envoy will add
+  an additional random jitter to the delay that is between zero and *refresh_interval_ms*
+  milliseconds. Thus the longest possible refresh delay is 2 \* *refresh_interval_ms*. Default
+  value is 30000ms (30 seconds).
+
+.. _config_http_conn_man_rds_api:
+
+REST API
+--------
+
+.. http:get:: /v1/routes/(string: route_config_name)/(string: service_cluster)/(string: service_node)
+
+Asks the discovery service to return the route table for a particular `route_config_name`,
+`service_cluster`, and `service_node`. `route_config_name` corresponds to the RDS configuration
+parameter above. `service_cluster` corresponds to the :option:`--service-cluster` CLI option.
+`service_node` corresponds to the :option:`--service-node` CLI option. Responses are a single JSON
+object that contains a route table as defined in the :ref:`route table documentation
+<config_http_conn_man_route_table>`.
+
+A new route table will be gracefully swapped in such that existing requests are not effected.
+
+.. attention::
+
+  Route tables that are loaded via RDS are *not* checked to see if referenced clusters are known
+  to the :ref:`cluster manager <config_cluster_manager>`. The RDS API has been designed to work
+  alongside the :ref:`CDS API <config_cluster_manager_cds>` such that Envoy assumes eventually
+  consistent updates. If a route references an unknown cluster a 404 response will be returned by
+  the router filter.
+
+Statistics
+----------
+
+RDS has a statistics tree rooted at *http.<stat_prefix>.rds.* with the following statistics:
+
+.. csv-table::
+  :header: Name, Type, Description
+  :widths: 1, 1, 2
+
+  config_reload, Counter, Total API fetches that resulted in a config reload due to a different config
+  update_attempt, Counter, Total API fetches attempted
+  update_success, Counter, Total API fetches completed successfully
+  update_failure, Counter, Total API fetches that failed (either network or schema errors)

include/envoy/router/rds.h

@@ -0,0 +1,24 @@
+#pragma once
+
+#include "envoy/router/router.h"
+
+namespace Router {
+
+/**
+ * A provider for constant route configurations.
+ */
+class RouteConfigProvider {
+public:
+  virtual ~RouteConfigProvider() {}
+
+  /**
+   * @return Router::ConfigPtr a route configuration for use during a single request. The returned
+   *         config may be different on a subsequent call so a new config should be acquired for
+   *         each request flow.
+   */
+  virtual Router::ConfigPtr config() PURE;
+};
+
+typedef std::unique_ptr<RouteConfigProvider> RouteConfigProviderPtr;
+
+} // Router

include/envoy/router/router.h

@@ -264,6 +264,6 @@ class Config {
   virtual bool usesRuntime() const PURE;
 };
 
-typedef std::unique_ptr<Config> ConfigPtr;
+typedef std::shared_ptr<const Config> ConfigPtr;
 
 } // Router

source/common/CMakeLists.txt

@@ -90,6 +90,7 @@ add_library(
   redis/conn_pool_impl.cc
   redis/proxy_filter.cc
   router/config_impl.cc
+  router/rds_impl.cc
   router/retry_state_impl.cc
   router/router.cc
   router/router_ratelimit.cc

source/common/http/conn_manager_impl.cc

@@ -287,8 +287,9 @@ void ConnectionManagerImpl::chargeTracingStats(const Tracing::Reason& tracing_re
 
 ConnectionManagerImpl::ActiveStream::ActiveStream(ConnectionManagerImpl& connection_manager)
     : connection_manager_(connection_manager),
-      stream_id_(ConnectionManagerUtility::generateStreamId(
-          connection_manager.config_.routeConfig(), connection_manager.random_generator_)),
+      snapped_route_config_(connection_manager.config_.routeConfigProvider().config()),
+      stream_id_(ConnectionManagerUtility::generateStreamId(*snapped_route_config_,
+                                                            connection_manager.random_generator_)),
       request_timer_(connection_manager_.stats_.named_.downstream_rq_time_.allocateSpan()),
       request_info_(connection_manager_.codec_->protocol()) {
   connection_manager_.stats_.named_.downstream_rq_total_.inc();
@@ -434,7 +435,7 @@ void ConnectionManagerImpl::ActiveStream::decodeHeaders(HeaderMapPtr&& headers,
 
   ConnectionManagerUtility::mutateRequestHeaders(
       *request_headers_, connection_manager_.read_callbacks_->connection(),
-      connection_manager_.config_, connection_manager_.random_generator_,
+      connection_manager_.config_, *snapped_route_config_, connection_manager_.random_generator_,
       connection_manager_.runtime_);
 
   // Check if tracing is enabled at all.
@@ -582,7 +583,7 @@ void ConnectionManagerImpl::ActiveStream::encodeHeaders(ActiveStreamEncoderFilte
   connection_manager_.config_.dateProvider().setDateHeader(headers);
   headers.insertServer().value(connection_manager_.config_.serverName());
   ConnectionManagerUtility::mutateResponseHeaders(headers, *request_headers_,
-                                                  connection_manager_.config_);
+                                                  *snapped_route_config_);
 
   // See if we want to drain/close the connection. Send the go away frame prior to encoding the
   // header block.
@@ -820,8 +821,8 @@ AccessLog::RequestInfo& ConnectionManagerImpl::ActiveStreamFilterBase::requestIn
 
 Router::RoutePtr ConnectionManagerImpl::ActiveStreamFilterBase::route() {
   if (!parent_.cached_route_.valid()) {
-    parent_.cached_route_.value(parent_.connection_manager_.config_.routeConfig().route(
-        *parent_.request_headers_, parent_.stream_id_));
+    parent_.cached_route_.value(
+        parent_.snapped_route_config_->route(*parent_.request_headers_, parent_.stream_id_));
   }
 
   return parent_.cached_route_.value();

source/common/http/conn_manager_impl.h

@@ -10,6 +10,7 @@
 #include "envoy/network/connection.h"
 #include "envoy/network/drain_decision.h"
 #include "envoy/network/filter.h"
+#include "envoy/router/rds.h"
 #include "envoy/runtime/runtime.h"
 #include "envoy/stats/stats_macros.h"
 #include "envoy/tracing/http_tracer.h"
@@ -159,9 +160,10 @@ class ConnectionManagerConfig {
   virtual const Optional<std::chrono::milliseconds>& idleTimeout() PURE;
 
   /**
-   * @return const Router::Config& the route configuration for all connection manager requests.
+   * @return Router::RouteConfigProvider& the configuration provider used to acquire a route
+   *         config for each request flow.
    */
-  virtual const Router::Config& routeConfig() PURE;
+  virtual Router::RouteConfigProvider& routeConfigProvider() PURE;
 
   /**
    * @return const std::string& the server name to write into responses.
@@ -396,6 +398,7 @@ class ConnectionManagerImpl : Logger::Loggable<Logger::Id::http>,
     };
 
     ConnectionManagerImpl& connection_manager_;
+    Router::ConfigPtr snapped_route_config_;
     Tracing::SpanPtr active_span_;
     const uint64_t stream_id_;
     StreamEncoder* response_encoder_{};

source/common/http/conn_manager_utility.cc

@@ -25,6 +25,7 @@ uint64_t ConnectionManagerUtility::generateStreamId(const Router::Config& route_
 void ConnectionManagerUtility::mutateRequestHeaders(Http::HeaderMap& request_headers,
                                                     Network::Connection& connection,
                                                     ConnectionManagerConfig& config,
+                                                    const Router::Config& route_config,
                                                     Runtime::RandomGenerator& random,
                                                     Runtime::Loader& runtime) {
   // Clean proxy headers.
@@ -79,7 +80,7 @@ void ConnectionManagerUtility::mutateRequestHeaders(Http::HeaderMap& request_hea
     request_headers.removeEnvoyExpectedRequestTimeoutMs();
     request_headers.removeEnvoyForceTrace();
 
-    for (const Http::LowerCaseString& header : config.routeConfig().internalOnlyHeaders()) {
+    for (const Http::LowerCaseString& header : route_config.internalOnlyHeaders()) {
       request_headers.remove(header);
     }
   }
@@ -122,16 +123,16 @@ void ConnectionManagerUtility::mutateRequestHeaders(Http::HeaderMap& request_hea
 
 void ConnectionManagerUtility::mutateResponseHeaders(Http::HeaderMap& response_headers,
                                                      const Http::HeaderMap& request_headers,
-                                                     ConnectionManagerConfig& config) {
+                                                     const Router::Config& route_config) {
   response_headers.removeConnection();
   response_headers.removeTransferEncoding();
 
-  for (const Http::LowerCaseString& to_remove : config.routeConfig().responseHeadersToRemove()) {
+  for (const Http::LowerCaseString& to_remove : route_config.responseHeadersToRemove()) {
     response_headers.remove(to_remove);
   }
 
   for (const std::pair<Http::LowerCaseString, std::string>& to_add :
-       config.routeConfig().responseHeadersToAdd()) {
+       route_config.responseHeadersToAdd()) {
     response_headers.addStatic(to_add.first, to_add.second);
   }
 

source/common/http/conn_manager_utility.h

@@ -17,11 +17,12 @@ class ConnectionManagerUtility {
 
   static void mutateRequestHeaders(Http::HeaderMap& request_headers,
                                    Network::Connection& connection, ConnectionManagerConfig& config,
+                                   const Router::Config& route_config,
                                    Runtime::RandomGenerator& random, Runtime::Loader& runtime);
 
   static void mutateResponseHeaders(Http::HeaderMap& response_headers,
                                     const Http::HeaderMap& request_headers,
-                                    ConnectionManagerConfig& config);
+                                    const Router::Config& route_config);
 
 private:
   // NOTE: This is used for stable randomness in the case where the route table does not use any

source/common/http/filter/fault_filter.h

@@ -4,7 +4,6 @@
 #include "envoy/runtime/runtime.h"
 #include "envoy/stats/stats_macros.h"
 
-#include "common/json/json_loader.h"
 #include "common/router/config_impl.h"
 
 namespace Http {

source/common/json/config_schemas.cc

@@ -69,6 +69,23 @@ const std::string Json::Schema::CLIENT_SSL_NETWORK_FILTER_SCHEMA(R"EOF(
   }
   )EOF");
 
+const std::string Json::Schema::RDS_CONFIGURATION_SCHEMA(R"EOF(
+  {
+    "$schema": "http://json-schema.org/schema#",
+    "properties" : {
+      "cluster" : {"type": "string"},
+      "route_config_name" : {"type": "string"},
+      "refresh_interval_ms" : {
+        "type" : "integer",
+        "minimum" : 0,
+        "exclusiveMinimum" : true
+      }
+    },
+    "required" : ["cluster", "route_config_name"],
+    "additionalProperties" : false
+  }
+  )EOF");
+
 const std::string Json::Schema::HTTP_CONN_NETWORK_FILTER_SCHEMA(R"EOF(
   {
     "$schema": "http://json-schema.org/schema#",
@@ -201,6 +218,7 @@ const std::string Json::Schema::HTTP_CONN_NETWORK_FILTER_SCHEMA(R"EOF(
         "enum" : ["http1", "http2", "auto"]
       },
       "stat_prefix" : {"type" : "string"},
+      "rds" : {"type": "object"},
       "route_config" : {"type": "object"},
       "filters" : {
         "type" : "array",
@@ -242,7 +260,7 @@ const std::string Json::Schema::HTTP_CONN_NETWORK_FILTER_SCHEMA(R"EOF(
       "use_remote_address" : {"type" : "boolean"},
       "generate_request_id" : {"type" : "boolean"}
     },
-    "required" : ["codec_type", "stat_prefix", "route_config", "filters"],
+    "required" : ["codec_type", "stat_prefix", "filters"],
     "additionalProperties" : false
   }
   )EOF");

source/common/json/config_schemas.h

@@ -23,6 +23,7 @@ class Schema {
   static const std::string VIRTUAL_HOST_CONFIGURATION_SCHEMA;
   static const std::string ROUTE_ENTRY_CONFIGURATION_SCHEMA;
   static const std::string HTTP_RATE_LIMITS_CONFIGURATION_SCHEMA;
+  static const std::string RDS_CONFIGURATION_SCHEMA;
 
   // HTTP Filter Schemas
   static const std::string BUFFER_HTTP_FILTER_SCHEMA;