Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/envoy/mixer/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ cc_proto_library(
cc_library(
name = "filter_lib",
srcs = [
"config.cc",
"config.h",
"http_control.cc",
"http_control.h",
"http_filter.cc",
Expand Down
37 changes: 28 additions & 9 deletions src/envoy/mixer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,7 @@ This Proxy will use Envoy and talk to Mixer server.
curl http://localhost:7070/echo -d "hello world"
```

## How to configurate HTTP filters

### *mixer* filter:
## How to configurate HTTP Mixer filters

This filter will intercept all HTTP requests and call Mixer. Here is its config:

Expand All @@ -72,17 +70,25 @@ This filter will intercept all HTTP requests and call Mixer. Here is its config:
"forward_attributes" : {
"attribute_name1": "attribute_value1",
"attribute_name2": "attribute_value2"
}
},
"quota_name": "RequestCount",
"quota_amount": "1",
"check_cache_keys": [
"request.host",
"request.path",
"origin.user"
]
}
```

Notes:
* mixer_server is required
* mixer_attributes: these attributes will be send to the mixer
* forward_attributes: these attributes will be forwarded to the upstream istio/proxy.
* "quota.name" and "quota.amount" are used for quota call. "quota.amount" is default to 1 if missing.
* mixer_attributes: these attributes will be sent to the mixer in both Check and Report calls.
* forward_attributes: these attributes will be forwarded to the upstream istio/proxy. It will send them to mixer in Check and Report calls.
* quota_name, quota_amount are used for making quota call. quota_amount is default to 1 if missing.
* check_cache_keys is to cache check calls. If missing or empty, check calls are not cached.

By default, mixer filter forwards attributes and does not invoke mixer server. You can customize this behavior per HTTP route by supplying an opaque config:
By default, mixer filter forwards attributes and does not invoke mixer server. You can customize this behavior per HTTP route by supplying an opaque config in the route config:

```
"opaque_config": {
Expand All @@ -91,4 +97,17 @@ By default, mixer filter forwards attributes and does not invoke mixer server. Y
}
```

This config reverts the behavior by sending requests to mixer server but not forwarding any attributes.
This route opaque config reverts the behavior by sending requests to mixer server but not forwarding any attributes.


## How to enable quota (rate limiting)

Quota (rate limiting) is enforced by the mixer. Mixer needs to be configured with Quota in its global config and service config. Its quota config will have
"quota name", its limit within a window. If "Quota" is added but param is missing, the default config is: quota name is "RequestCount", the limit is 10 with 1 second window. Essentially, it is imposing 10 qps rate limiting.

Mixer client can be configured to make Quota call for all requests. If "quota_name" is specified in the mixer filter config, mixer client will call Quota with the specified quota name. If "quota_amount" is specified, it will call with that amount, otherwise the used amount is 1.


## How to pass some attributes from client proxy to mixer.

Usually client proxy is not configured to call mixer (it can be enabled in the route opaque_config). Client proxy can pass some attributes to mixer by using "forward_attributes" field. Its attributes will be sent to the upstream proxy (the server proxy). If the server proxy is calling mixer, these attributes will be sent to the mixer.
96 changes: 96 additions & 0 deletions src/envoy/mixer/config.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
/* Copyright 2017 Istio Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#include "src/envoy/mixer/config.h"

using ::istio::mixer_client::Attributes;

namespace Http {
namespace Mixer {
namespace {

// The Json object name for mixer-server.
const std::string kJsonNameMixerServer("mixer_server");

// The Json object name for static attributes.
const std::string kJsonNameMixerAttributes("mixer_attributes");

// The Json object name to specify attributes which will be forwarded
// to the upstream istio proxy.
const std::string kJsonNameForwardAttributes("forward_attributes");

// The Json object name for quota name and amount.
const std::string kJsonNameQuotaName("quota_name");
const std::string kJsonNameQuotaAmount("quota_amount");

// The Json object name for check cache keys.
const std::string kJsonNameCheckCacheKeys("check_cache_keys");

void ReadString(const Json::Object& json, const std::string& name,
std::string* value) {
if (json.hasObject(name)) {
*value = json.getString(name);
}
}

void ReadStringMap(const Json::Object& json, const std::string& name,
std::map<std::string, std::string>* map) {
if (json.hasObject(name)) {
json.getObject(name)->iterate(
[map](const std::string& key, const Json::Object& obj) -> bool {
(*map)[key] = obj.asString();
return true;
});
}
}

void ReadStringVector(const Json::Object& json, const std::string& name,
std::vector<std::string>* value) {
if (json.hasObject(name)) {
auto v = json.getStringArray(name);
value->swap(v);
}
}

} // namespace

void MixerConfig::Load(const Json::Object& json) {
ReadString(json, kJsonNameMixerServer, &mixer_server);

ReadStringMap(json, kJsonNameMixerAttributes, &mixer_attributes);
ReadStringMap(json, kJsonNameForwardAttributes, &forward_attributes);

ReadString(json, kJsonNameQuotaName, &quota_name);
ReadString(json, kJsonNameQuotaAmount, &quota_amount);

ReadStringVector(json, kJsonNameCheckCacheKeys, &check_cache_keys);
}

void MixerConfig::ExtractQuotaAttributes(Attributes* attr) const {
if (!quota_name.empty()) {
attr->attributes[ ::istio::mixer_client::kQuotaName] =
Attributes::StringValue(quota_name);

int64_t amount = 1; // default amount to 1.
if (!quota_amount.empty()) {
amount = std::stoi(quota_amount);
}
attr->attributes[ ::istio::mixer_client::kQuotaAmount] =
Attributes::Int64Value(amount);
}
}

} // namespace Mixer
} // namespace Http
53 changes: 53 additions & 0 deletions src/envoy/mixer/config.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/* Copyright 2017 Istio Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#pragma once

#include "precompiled/precompiled.h"

#include "envoy/json/json_object.h"
#include "include/attribute.h"

namespace Http {
namespace Mixer {

// A config for mixer filter
struct MixerConfig {
// the mixer server address
std::string mixer_server;

// These static attributes will be send to mixer in both
// Check and Report.
std::map<std::string, std::string> mixer_attributes;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may lose some context here. Just wondering what is the relation between mixer_attributes and forward_attributes? And why we need a standalone quota attributes?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mixer_attributes will be send to mixer server
forward_attributes will be forwarded to upstream so that next hop proxy can send them to mixer server.
Above attributes are for Check and Report.
Quota is a separate call. Here we use some standalone attributes to trigger the calls.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will mixer_attributes and forward_attributes have overlap/different entries?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they have overlapped entries, mixer_attributes win.


// These attributes will be forwarded to upstream.
std::map<std::string, std::string> forward_attributes;

// Quota attributes.
std::string quota_name;
std::string quota_amount;

// The attribute names for check cache.
std::vector<std::string> check_cache_keys;

// Load the config from envoy config.
void Load(const Json::Object& json);

// Extract quota attributes.
void ExtractQuotaAttributes(::istio::mixer_client::Attributes* attr) const;
};

} // namespace Mixer
} // namespace Http
12 changes: 9 additions & 3 deletions src/envoy/mixer/envoy.conf.template
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,15 @@
"mixer_server": "${MIXER_SERVER}",
"mixer_attributes": {
"target.uid": "POD222",
"target.namespace": "XYZ222",
"quota.name": "RequestCount"
}
"target.namespace": "XYZ222"
},
"quota_name": "RequestCount",
"quota_amount": "1",
"check_cache_keys": [
"request.host",
"request.path",
"origin.user"
]
}
},
{
Expand Down
29 changes: 8 additions & 21 deletions src/envoy/mixer/http_control.cc
Original file line number Diff line number Diff line change
Expand Up @@ -108,29 +108,16 @@ void FillRequestInfoAttributes(const AccessLog::RequestInfo& info,

} // namespace

HttpControl::HttpControl(const std::string& mixer_server,
std::map<std::string, std::string>&& attributes)
: config_attributes_(std::move(attributes)) {
HttpControl::HttpControl(const MixerConfig& mixer_config)
: mixer_config_(mixer_config) {
::istio::mixer_client::MixerClientOptions options;
options.mixer_server = mixer_server;
options.mixer_server = mixer_config_.mixer_server;
options.check_options.cache_keys.insert(
mixer_config_.check_cache_keys.begin(),
mixer_config_.check_cache_keys.end());
mixer_client_ = ::istio::mixer_client::CreateMixerClient(options);

// Extract quota attributes
auto it = config_attributes_.find(::istio::mixer_client::kQuotaName);
if (it != config_attributes_.end()) {
quota_attributes_.attributes[ ::istio::mixer_client::kQuotaName] =
Attributes::StringValue(it->second);
config_attributes_.erase(it);

int64_t amount = 1; // default amount to 1.
it = config_attributes_.find(::istio::mixer_client::kQuotaAmount);
if (it != config_attributes_.end()) {
amount = std::stoi(it->second);
config_attributes_.erase(it);
}
quota_attributes_.attributes[ ::istio::mixer_client::kQuotaAmount] =
Attributes::Int64Value(amount);
}
mixer_config_.ExtractQuotaAttributes(&quota_attributes_);
}

void HttpControl::FillCheckAttributes(HeaderMap& header_map, Attributes* attr) {
Expand All @@ -148,7 +135,7 @@ void HttpControl::FillCheckAttributes(HeaderMap& header_map, Attributes* attr) {

FillRequestHeaderAttributes(header_map, attr);

for (const auto& attribute : config_attributes_) {
for (const auto& attribute : mixer_config_.mixer_attributes) {
SetStringAttribute(attribute.first, attribute.second, attr);
}
}
Expand Down
8 changes: 4 additions & 4 deletions src/envoy/mixer/http_control.h
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
#include "common/http/headers.h"
#include "envoy/http/access_log.h"
#include "include/client.h"
#include "src/envoy/mixer/config.h"

namespace Http {
namespace Mixer {
Expand All @@ -37,8 +38,7 @@ typedef std::shared_ptr<HttpRequestData> HttpRequestDataPtr;
class HttpControl final : public Logger::Loggable<Logger::Id::http> {
public:
// The constructor.
HttpControl(const std::string& mixer_server,
std::map<std::string, std::string>&& attributes);
HttpControl(const MixerConfig& mixer_config);

// Make mixer check call.
void Check(HttpRequestDataPtr request_data, HeaderMap& headers,
Expand All @@ -56,8 +56,8 @@ class HttpControl final : public Logger::Loggable<Logger::Id::http> {

// The mixer client
std::unique_ptr<::istio::mixer_client::MixerClient> mixer_client_;
// The attributes read from the config file.
std::map<std::string, std::string> config_attributes_;
// The mixer config
const MixerConfig& mixer_config_;
// Quota attributes; extracted from envoy filter config.
::istio::mixer_client::Attributes quota_attributes_;
};
Expand Down
36 changes: 11 additions & 25 deletions src/envoy/mixer/http_filter.cc
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
#include "envoy/server/instance.h"
#include "envoy/ssl/connection.h"
#include "server/config/network/http_connection_manager.h"
#include "src/envoy/mixer/config.h"
#include "src/envoy/mixer/http_control.h"
#include "src/envoy/mixer/utils.h"

Expand All @@ -33,16 +34,6 @@ namespace Http {
namespace Mixer {
namespace {

// The Json object name for mixer-server.
const std::string kJsonNameMixerServer("mixer_server");

// The Json object name for static attributes.
const std::string kJsonNameMixerAttributes("mixer_attributes");

// The Json object name to specify attributes which will be forwarded
// to the upstream istio proxy.
const std::string kJsonNameForwardAttributes("forward_attributes");

// Switch to turn off attribute forwarding
const std::string kJsonNameForwardSwitch("mixer_forward");

Expand Down Expand Up @@ -100,35 +91,30 @@ class Config : public Logger::Loggable<Logger::Id::http> {
std::shared_ptr<HttpControl> http_control_;
Upstream::ClusterManager& cm_;
std::string forward_attributes_;
MixerConfig mixer_config_;

public:
Config(const Json::Object& config, Server::Instance& server)
: cm_(server.clusterManager()) {
std::string mixer_server;
if (config.hasObject(kJsonNameMixerServer)) {
mixer_server = config.getString(kJsonNameMixerServer);
} else {
mixer_config_.Load(config);
if (mixer_config_.mixer_server.empty()) {
log().error(
"mixer_server is required but not specified in the config: {}",
__func__);
} else {
log().debug("Called Mixer::Config constructor with mixer_server: ",
mixer_config_.mixer_server);
}

Utils::StringMap attributes =
Utils::ExtractStringMap(config, kJsonNameForwardAttributes);
if (!attributes.empty()) {
std::string serialized_str = Utils::SerializeStringMap(attributes);
if (!mixer_config_.forward_attributes.empty()) {
std::string serialized_str =
Utils::SerializeStringMap(mixer_config_.forward_attributes);
forward_attributes_ =
Base64::encode(serialized_str.c_str(), serialized_str.size());
log().debug("Mixer forward attributes set: ", serialized_str);
}

std::map<std::string, std::string> mixer_attributes =
Utils::ExtractStringMap(config, kJsonNameMixerAttributes);

http_control_ = std::make_shared<HttpControl>(mixer_server,
std::move(mixer_attributes));
log().debug("Called Mixer::Config constructor with mixer_server: ",
mixer_server);
http_control_ = std::make_shared<HttpControl>(mixer_config_);
}

std::shared_ptr<HttpControl>& http_control() { return http_control_; }
Expand Down
Loading