redis: auth support

api/envoy/config/filter/network/redis_proxy/v2/BUILD

@@ -5,4 +5,7 @@ licenses(["notice"])  # Apache 2
 api_proto_library_internal(
     name = "redis_proxy",
     srcs = ["redis_proxy.proto"],
+    deps = [
+        "//envoy/api/v2/core:base",
+    ],
 )

api/envoy/config/filter/network/redis_proxy/v2/redis_proxy.proto

@@ -7,6 +7,8 @@ option java_multiple_files = true;
 option java_package = "io.envoyproxy.envoy.config.filter.network.redis_proxy.v2";
 option go_package = "v2";
 
+import "envoy/api/v2/core/base.proto";
+
 import "google/protobuf/duration.proto";
 
 import "validate/validate.proto";
@@ -141,4 +143,23 @@ message RedisProxy {
   // <arch_overview_redis_configuration>` of the architecture overview for recommendations on
   // configuring the backing clusters.
   PrefixRoutes prefix_routes = 5 [(gogoproto.nullable) = false];
+
+  // Authenticate Redis client connections locally by forcing downstream clients to issue a 'Redis
+  // AUTH command <https://redis.io/commands/auth>`_ with this password before enabling any other
+  // command. If an AUTH command's password matches this password, an "OK" response will be returned
+  // to the client. If the AUTH command password does not match this password, then an "ERR invalid
+  // password" error will be returned. If any other command is received before AUTH when this
+  // password is set, then a "NOAUTH Authentication required." error response will be sent to the
+  // client. If an AUTH command is received when the password is not set, then an "ERR Client sent
+  // AUTH, but no password is set" error will be returned.
+  envoy.api.v2.core.DataSource downstream_auth_password = 6;
 }
+
+// RedisProtocolOptions specifies Redis upstream protocol options. This object is used in
+// :ref:`extension_protocol_options<envoy_api_field_Cluster.extension_protocol_options>`, keyed
+// by the name `envoy.redis_proxy`.
+message RedisProtocolOptions {
+  // Upstream server password as defined by the `requirepass directive
+  // <https://redis.io/topics/config>`_ in the server's configuration file.
+  envoy.api.v2.core.DataSource auth_password = 1;
+}

docs/root/intro/arch_overview/redis.rst

@@ -25,6 +25,7 @@ The Redis project offers a thorough reference on partitioning as it relates to R
 * Active and passive healthchecking.
 * Hash tagging.
 * Prefix routing.
+* Separate downstream client and upstream server authentication.
 
 **Planned future enhancements**:
 
@@ -85,10 +86,13 @@ Supported commands
 At the protocol level, pipelines are supported. MULTI (transaction block) is not.
 Use pipelining wherever possible for the best performance.
 
-At the command level, Envoy only supports commands that can be reliably hashed to a server. PING
-is the only exception, which Envoy responds to immediately with PONG. Arguments to PING are not
-allowed. All other supported commands must contain a key. Supported commands are functionally
-identical to the original Redis command except possibly in failure scenarios.
+At the command level, Envoy only supports commands that can be reliably hashed to a server. AUTH and PING
+are the only exceptions. AUTH is processed locally by Envoy if a downstream password has been configured, 
+and no other commands will be processed until authentication is successful when a password has been 
+configured. Envoy will transparently issue AUTH commands upon connecting to upstream servers, if upstream 
+authentication passwords are configured for the cluster. Envoy responds to PING immediately with PONG. 
+Arguments to PING are not allowed. All other supported commands must contain a key. Supported commands are 
+functionally identical to the original Redis command except possibly in failure scenarios.
 
 For details on each command's usage see the official
 `Redis command reference <https://redis.io/commands>`_.
@@ -97,6 +101,7 @@ For details on each command's usage see the official
   :header: Command, Group
   :widths: 1, 1
 
+  AUTH, Authentication
   PING, Connection
   DEL, Generic
   DUMP, Generic
@@ -227,6 +232,12 @@ Envoy can also generate its own errors in response to the client.
   responded with a response that not conform to the Redis protocol."
   wrong number of arguments for command, "Certain commands check in Envoy that the number of
   arguments is correct."
+  "NOAUTH Authentication required.", "The command was rejected because a downstream authentication
+  password has been set and the client has not successfully authenticated."
+  ERR invalid password, "The authentication command failed due to an invalid password."
+  "ERR Client sent AUTH, but no password is set", "An authentication command was received, but no 
+  downstream authentication password has been configured."
+
 
 In the case of MGET, each individual key that cannot be fetched will generate an error response.
 For example, if we fetch five keys and two of the keys' backends time out, we would get an error

docs/root/intro/version_history.rst

@@ -31,6 +31,7 @@ Version history
 * redis: added
   :ref:`max_buffer_size_before_flush <envoy_api_field_config.filter.network.redis_proxy.v2.RedisProxy.ConnPoolSettings.max_buffer_size_before_flush>` to batch commands together until the encoder buffer hits a certain size, and
   :ref:`buffer_flush_timeout <envoy_api_field_config.filter.network.redis_proxy.v2.RedisProxy.ConnPoolSettings.buffer_flush_timeout>` to control how quickly the buffer is flushed if it is not full.
+* redis: added auth support :ref:`downstream_auth_password <envoy_api_field_config.filter.network.redis_proxy.v2.RedisProxy.downstream_auth_password>` for downstream client authentication, and :ref:`auth_password <envoy_api_field_config.filter.network.redis_proxy.v2.RedisProtocolOptions.auth_password>` to configure authentication passwords for upstream server clusters.
 * router: add support for configuring a :ref:`grpc timeout offset <envoy_api_field_route.RouteAction.grpc_timeout_offset>` on incoming requests.
 * router: added ability to control retry back-off intervals via :ref:`retry policy <envoy_api_msg_route.RetryPolicy.RetryBackOff>`.
 * router: added ability to issue a hedged retry in response to a per try timeout via a :ref:`hedge policy <envoy_api_msg_route.HedgePolicy>`.

source/extensions/clusters/redis/BUILD

@@ -20,6 +20,7 @@ envoy_cc_library(
         "//include/envoy/upstream:cluster_factory_interface",
         "//include/envoy/upstream:cluster_manager_interface",
         "//include/envoy/upstream:upstream_interface",
+        "//source/common/config:datasource_lib",
         "//source/common/config:metadata_lib",
         "//source/common/event:dispatcher_lib",
         "//source/common/json:config_schemas_lib",
@@ -33,6 +34,8 @@ envoy_cc_library(
         "//source/extensions/filters/network/common/redis:client_interface",
         "//source/extensions/filters/network/common/redis:client_lib",
         "//source/extensions/filters/network/common/redis:codec_interface",
+        "//source/extensions/filters/network/common/redis:utility_lib",
+        "//source/extensions/filters/network/redis_proxy:config_interface",
         "//source/extensions/transport_sockets/raw_buffer:config",
         "//source/server:transport_socket_config_lib",
         "@envoy_api//envoy/config/cluster/redis:redis_cluster_cc",

source/extensions/clusters/redis/redis_cluster.cc

@@ -7,11 +7,15 @@ namespace Extensions {
 namespace Clusters {
 namespace Redis {
 
+namespace {
+Extensions::NetworkFilters::Common::Redis::Client::DoNothingPoolCallbacks null_pool_callbacks;
+} // namespace
+
 RedisCluster::RedisCluster(
     const envoy::api::v2::Cluster& cluster,
     const envoy::config::cluster::redis::RedisClusterConfig& redisCluster,
     NetworkFilters::Common::Redis::Client::ClientFactory& redis_client_factory,
-    Upstream::ClusterManager& clusterManager, Runtime::Loader& runtime,
+    Upstream::ClusterManager& clusterManager, Runtime::Loader& runtime, Api::Api& api,
     Network::DnsResolverSharedPtr dns_resolver,
     Server::Configuration::TransportSocketFactoryContext& factory_context,
     Stats::ScopePtr&& stats_scope, bool added_via_api)
@@ -28,7 +32,7 @@ RedisCluster::RedisCluster(
                            ? cluster.load_assignment()
                            : Config::Utility::translateClusterHosts(cluster.hosts())),
       local_info_(factory_context.localInfo()), random_(factory_context.random()),
-      redis_discovery_session_(*this, redis_client_factory) {
+      redis_discovery_session_(*this, redis_client_factory), api_(api) {
   const auto& locality_lb_endpoints = load_assignment_.endpoints();
   for (const auto& locality_lb_endpoint : locality_lb_endpoints) {
     for (const auto& lb_endpoint : locality_lb_endpoint.lb_endpoints()) {
@@ -38,7 +42,14 @@ RedisCluster::RedisCluster(
           locality_lb_endpoint, lb_endpoint));
     }
   }
-};
+
+  auto options =
+      info()->extensionProtocolOptionsTyped<NetworkFilters::RedisProxy::ProtocolOptionsConfigImpl>(
+          NetworkFilters::NetworkFilterNames::get().RedisProxy);
+  if (options) {
+    auth_password_datasource_ = options->auth_password_datasource();
+  }
+}
 
 void RedisCluster::startPreInit() {
   for (const DnsDiscoveryResolveTargetPtr& target : dns_discovery_resolve_targets_) {
@@ -221,6 +232,14 @@ void RedisCluster::RedisDiscoverySession::startResolve() {
     client->host_ = current_host_address_;
     client->client_ = client_factory_.create(host, dispatcher_, *this);
     client->client_->addConnectionCallbacks(*client);
+    std::string auth_password =
+        Envoy::Config::DataSource::read(parent_.auth_password_datasource_, true, parent_.api_);
+    if (!auth_password.empty()) {
+      // Send an AUTH command to the upstream server.
+      client->client_->makeRequest(
+          Extensions::NetworkFilters::Common::Redis::Utility::makeAuthCommand(auth_password),
+          null_pool_callbacks);
+    }
   }
 
   current_request_ = client->client_->makeRequest(ClusterSlotsRequest::instance_, *this);
@@ -301,8 +320,9 @@ Upstream::ClusterImplBaseSharedPtr RedisClusterFactory::createClusterWithConfig(
   }
   return std::make_shared<RedisCluster>(
       cluster, proto_config, NetworkFilters::Common::Redis::Client::ClientFactoryImpl::instance_,
-      context.clusterManager(), context.runtime(), selectDnsResolver(cluster, context),
-      socket_factory_context, std::move(stats_scope), context.addedViaApi());
+      context.clusterManager(), context.runtime(), context.api(),
+      selectDnsResolver(cluster, context), socket_factory_context, std::move(stats_scope),
+      context.addedViaApi());
 }
 
 REGISTER_FACTORY(RedisClusterFactory, Upstream::ClusterFactory);

source/extensions/clusters/redis/redis_cluster.h

@@ -39,6 +39,7 @@
 #include "common/common/callback_impl.h"
 #include "common/common/enum_to_int.h"
 #include "common/common/logger.h"
+#include "common/config/datasource.h"
 #include "common/config/metadata.h"
 #include "common/config/well_known_names.h"
 #include "common/network/address_impl.h"
@@ -56,6 +57,8 @@
 #include "extensions/filters/network/common/redis/client.h"
 #include "extensions/filters/network/common/redis/client_impl.h"
 #include "extensions/filters/network/common/redis/codec.h"
+#include "extensions/filters/network/common/redis/utility.h"
+#include "extensions/filters/network/redis_proxy/config.h"
 
 namespace Envoy {
 namespace Extensions {
@@ -92,7 +95,7 @@ class RedisCluster : public Upstream::BaseDynamicClusterImpl {
   RedisCluster(const envoy::api::v2::Cluster& cluster,
                const envoy::config::cluster::redis::RedisClusterConfig& redisCluster,
                NetworkFilters::Common::Redis::Client::ClientFactory& client_factory,
-               Upstream::ClusterManager& clusterManager, Runtime::Loader& runtime,
+               Upstream::ClusterManager& clusterManager, Runtime::Loader& runtime, Api::Api& api,
                Network::DnsResolverSharedPtr dns_resolver,
                Server::Configuration::TransportSocketFactoryContext& factory_context,
                Stats::ScopePtr&& stats_scope, bool added_via_api);
@@ -261,6 +264,9 @@ class RedisCluster : public Upstream::BaseDynamicClusterImpl {
 
   Upstream::HostVector hosts_;
   Upstream::HostMap all_hosts_;
+
+  envoy::api::v2::core::DataSource auth_password_datasource_;
+  Api::Api& api_;
 };
 
 class RedisClusterFactory : public Upstream::ConfigurableClusterFactoryBase<

source/extensions/filters/network/common/redis/BUILD

@@ -14,6 +14,12 @@ envoy_cc_library(
     deps = ["//include/envoy/buffer:buffer_interface"],
 )
 
+envoy_cc_library(
+    name = "utility_interface",
+    hdrs = ["utility.h"],
+    deps = [":codec_interface"],
+)
+
 envoy_cc_library(
     name = "codec_lib",
     srcs = ["codec_impl.cc"],
@@ -62,3 +68,12 @@ envoy_cc_library(
         "@envoy_api//envoy/config/filter/network/redis_proxy/v2:redis_proxy_cc",
     ],
 )
+
+envoy_cc_library(
+    name = "utility_lib",
+    srcs = ["utility.cc"],
+    hdrs = ["utility.h"],
+    deps = [
+        ":codec_lib",
+    ],
+)

source/extensions/filters/network/common/redis/client.h

@@ -52,6 +52,18 @@ class PoolCallbacks {
   virtual bool onRedirection(const Common::Redis::RespValue& value) PURE;
 };
 
+/**
+ * DoNothingPoolCallbacks is used for internally generated commands whose response is
+ * transparently filtered, and redirection never occurs (e.g., "asking", "auth", etc.).
+ */
+class DoNothingPoolCallbacks : public PoolCallbacks {
+public:
+  // PoolCallbacks
+  void onResponse(Common::Redis::RespValuePtr&&) override {}
+  void onFailure() override {}
+  bool onRedirection(const Common::Redis::RespValue&) override { return false; }
+};
+
 /**
  * A single redis client connection.
  */

source/extensions/filters/network/common/redis/client_impl.h

@@ -71,6 +71,8 @@ class ClientImpl : public Client, public DecoderCallbacks, public Network::Conne
   void flushBufferAndResetTimer();
 
 private:
+  friend class RedisClientImplTest;
+
   struct UpstreamReadFilter : public Network::ReadFilterBaseImpl {
     UpstreamReadFilter(ClientImpl& parent) : parent_(parent) {}
 

source/extensions/filters/network/common/redis/supported_commands.h

@@ -45,6 +45,11 @@ struct SupportedCommands {
     CONSTRUCT_ON_FIRST_USE(std::vector<std::string>, "del", "exists", "touch", "unlink");
   }
 
+  /**
+   * @return auth command
+   */
+  static const std::string& auth() { CONSTRUCT_ON_FIRST_USE(std::string, "auth"); }
+
   /**
    * @return mget command
    */

source/extensions/filters/network/common/redis/utility.cc

@@ -0,0 +1,26 @@
+#include "extensions/filters/network/common/redis/utility.h"
+
+namespace Envoy {
+namespace Extensions {
+namespace NetworkFilters {
+namespace Common {
+namespace Redis {
+namespace Utility {
+
+Redis::RespValue makeAuthCommand(const std::string& password) {
+  Redis::RespValue auth_command, value;
+  auth_command.type(Redis::RespType::Array);
+  value.type(Redis::RespType::BulkString);
+  value.asString() = "auth";
+  auth_command.asArray().push_back(value);
+  value.asString() = password;
+  auth_command.asArray().push_back(value);
+  return auth_command;
+}
+
+} // namespace Utility
+} // namespace Redis
+} // namespace Common
+} // namespace NetworkFilters
+} // namespace Extensions
+} // namespace Envoy

source/extensions/filters/network/common/redis/utility.h

@@ -0,0 +1,21 @@
+#pragma once
+
+#include <string>
+
+#include "extensions/filters/network/common/redis/codec.h"
+
+namespace Envoy {
+namespace Extensions {
+namespace NetworkFilters {
+namespace Common {
+namespace Redis {
+namespace Utility {
+
+Redis::RespValue makeAuthCommand(const std::string& password);
+
+} // namespace Utility
+} // namespace Redis
+} // namespace Common
+} // namespace NetworkFilters
+} // namespace Extensions
+} // namespace Envoy

source/extensions/filters/network/redis_proxy/BUILD

@@ -20,6 +20,17 @@ envoy_cc_library(
     ],
 )
 
+envoy_cc_library(
+    name = "config_interface",
+    hdrs = ["config.h"],
+    deps = [
+        "//source/common/config:datasource_lib",
+        "//source/extensions/filters/network:well_known_names",
+        "//source/extensions/filters/network/common:factory_base_lib",
+        "@envoy_api//envoy/config/filter/network/redis_proxy/v2:redis_proxy_cc",
+    ],
+)
+
 envoy_cc_library(
     name = "conn_pool_interface",
     hdrs = ["conn_pool.h"],
@@ -62,6 +73,7 @@ envoy_cc_library(
     srcs = ["conn_pool_impl.cc"],
     hdrs = ["conn_pool_impl.h"],
     deps = [
+        ":config_interface",
         ":conn_pool_interface",
         "//include/envoy/thread_local:thread_local_interface",
         "//include/envoy/upstream:cluster_manager_interface",
@@ -73,6 +85,7 @@ envoy_cc_library(
         "//source/common/upstream:load_balancer_lib",
         "//source/common/upstream:upstream_lib",
         "//source/extensions/filters/network/common/redis:client_lib",
+        "//source/extensions/filters/network/common/redis:utility_lib",
         "@envoy_api//envoy/config/filter/network/redis_proxy/v2:redis_proxy_cc",
     ],
 )
@@ -88,6 +101,7 @@ envoy_cc_library(
         "//include/envoy/upstream:cluster_manager_interface",
         "//source/common/buffer:buffer_lib",
         "//source/common/common:assert_lib",
+        "//source/common/config:datasource_lib",
         "//source/common/config:utility_lib",
         "//source/extensions/filters/network/common/redis:codec_interface",
         "@envoy_api//envoy/config/filter/network/redis_proxy/v2:redis_proxy_cc",
@@ -105,6 +119,7 @@ envoy_cc_library(
         "//source/extensions/filters/network/common:factory_base_lib",
         "//source/extensions/filters/network/common/redis:codec_lib",
         "//source/extensions/filters/network/redis_proxy:command_splitter_lib",
+        "//source/extensions/filters/network/redis_proxy:conn_pool_lib",
         "//source/extensions/filters/network/redis_proxy:proxy_filter_lib",
         "//source/extensions/filters/network/redis_proxy:router_lib",
     ],