Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion api/envoy/config/cluster/v3/cluster.proto
Original file line number Diff line number Diff line change
Expand Up @@ -486,7 +486,7 @@ message Cluster {
}

// Common configuration for all load balancer implementations.
// [#next-free-field: 8]
// [#next-free-field: 9]
message CommonLbConfig {
option (udpa.annotations.versioning).previous_message_type =
"envoy.api.v2.Cluster.CommonLbConfig";
Expand Down Expand Up @@ -595,6 +595,16 @@ message Cluster {

// Common Configuration for all consistent hashing load balancers (MaglevLb, RingHashLb, etc.)
ConsistentHashingLbConfig consistent_hashing_lb_config = 7;

// Expected override host statuses. The result of load balancing is allowed to be overwritten
// only when the target host status is as expected when stateful session stickiness directly
// specifies a target host.
//
// If this is set and and has no any item then any host from stateful session stickiness will be
// ignored.
// If this is not set then healthy statuses (``UNKNOWN``, ``HEALTHY`` and ``DEGRADED`` for now)
// will be applied by default.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this field only relevant when the stateful_session extension is used?
If so, can you add this to the comment, and also add a reference to it?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be referring to the specific extension here at all? Or should we instead have this talk about the host override feature in general? I can imagine other extensions wanting to use host overrides as well

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the moment, it does. I will add some more comments. Thanks.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that I do not fully grok is what is expected if a config has this field set, but doesn't have the sticky_session extension configured.

@wbpcode wbpcode Jan 26, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be referring to the specific extension here at all? Or should we instead have this talk about the host override feature in general? I can imagine other extensions wanting to use host overrides as well

@htuch @snowp This configuration is designed for general host override. But LoadBalancerContext::overrideHostToSelect() is internaly C++ API which is annomymous for public users and the stateful session is the only related implemenatation for now. Although we will have more extensions in the future, now we still need a specific example to help users to understand this config.

The result of load balancing is allowed to be overwritten by the L4/L7 filters specified host.
For example the `stateful session filter <........>` will specfic upstream host directly accoriding to the 
downstream request.
But the overridden will only worked when the status of specficed host is as expected in the configuration.

how about some comments like this? cc @htuch cc @adisuissa

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit lost reading this. I think there is context assumed of the reader that isn't likely to be there in most cases. Can you explain this section as if the reader had never heard of this override feature, pointing them to the relevant docs with RST links and/or some examples?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The override host feature is currently only available through the L4/L7 extensions. Now stateful session is the only extension that is ready.

So my intention is to use the stateful session as an example to help the reader understand the functionality and the configuration.

I will think about how to make it clearer and easier to understand. 🤔

core.v3.HealthStatusSet override_host_status = 8;
}

message RefreshRate {
Expand Down
5 changes: 5 additions & 0 deletions api/envoy/config/core/v3/health_check.proto
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ enum HealthStatus {
DEGRADED = 5;
}

message HealthStatusSet {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you expect this message to have more fields?
If not, what's the advantage over just specifying the repeated field directly (in your specific case renaming the field to override_host_status_set)?

@wbpcode wbpcode Jan 26, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because we need to distinguish between the case where no override_host_status is configured at all and the case where 0 items are configured. The former will provide a default value.

// An order-independent set of health status.
repeated HealthStatus statuses = 1;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be an empty set? If not, then add a PGV constraint.

@wbpcode wbpcode Jan 26, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, it can be an empty set.

Then override_host_status will be zero and all the override host from LoadBalancerContext::overrideHostToSelect() will be ignored and stateful session stickiness will be disabled for current cluster.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like the case when the override_host_status field is set which is technically different from being set with a zero list. Is there a distinction between override_host_status not being set and it being set with an empty list of statuses?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@snowp Yep. If override_host_status is set with an empty list of statuses then the override host will be ignored. And if override_host_status is not set completely, we will give some default statuses.

}

// [#next-free-field: 25]
message HealthCheck {
option (udpa.annotations.versioning).previous_message_type = "envoy.api.v2.core.HealthCheck";
Expand Down
17 changes: 3 additions & 14 deletions envoy/upstream/load_balancer.h
Original file line number Diff line number Diff line change
Expand Up @@ -89,22 +89,11 @@ class LoadBalancerContext {
*/
virtual Network::TransportSocketOptionsConstSharedPtr upstreamTransportSocketOptions() const PURE;

// Using uint32_t to express expected status of override host. Every bit in the OverrideHostStatus
// represent an enum value of Host::Health. The specific correspondence is shown below:
//
// * 0b001: Host::Health::Unhealthy
// * 0b010: Host::Health::Degraded
// * 0b100: Host::Health::Healthy
//
// If multiple bit fields are set, it is acceptable as long as the status of override host is in
// any of these statuses.
using OverrideHostStatus = uint32_t;
using OverrideHost = std::pair<std::string, OverrideHostStatus>;

using OverrideHost = absl::string_view;
/**
* Returns the host the load balancer should select directly. If the expected host exists and
* the health status of the host matches the expectation, the load balancer can bypass the load
* balancing algorithm and return the corresponding host directly.
* the host can be selected directly, the load balancer can bypass the load balancing algorithm
* and return the corresponding host directly.
*/
virtual absl::optional<OverrideHost> overrideHostToSelect() const PURE;
};
Expand Down
8 changes: 1 addition & 7 deletions source/common/router/router.h
Original file line number Diff line number Diff line change
Expand Up @@ -431,13 +431,7 @@ class Filter : Logger::Loggable<Logger::Id::router>,
return {};
}

auto override_host = callbacks_->upstreamOverrideHost();
if (override_host.has_value()) {
// TODO(wbpcode): Currently we need to provide additional expected host status to the load
// balancer. This should be resolved after the `overrideHostToSelect()` refactoring.
return std::make_pair(std::string(override_host.value()), ~static_cast<uint32_t>(0));
}
return {};
return callbacks_->upstreamOverrideHost();
}

/**
Expand Down
72 changes: 46 additions & 26 deletions source/common/upstream/load_balancer_impl.cc
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,13 @@ static const std::string RuntimeZoneEnabled = "upstream.zone_routing.enabled";
static const std::string RuntimeMinClusterSize = "upstream.zone_routing.min_cluster_size";
static const std::string RuntimePanicThreshold = "upstream.healthy_panic_threshold";

static constexpr uint32_t UnhealthyStatus = 1u << static_cast<size_t>(Host::Health::Unhealthy);
static constexpr uint32_t DegradedStatus = 1u << static_cast<size_t>(Host::Health::Degraded);
static constexpr uint32_t HealthyStatus = 1u << static_cast<size_t>(Host::Health::Healthy);
constexpr uint32_t singleHealthStatusToUint(Host::Health status) {
return 1u << static_cast<size_t>(status);
Comment thread
htuch marked this conversation as resolved.
Outdated
}

static constexpr uint32_t DefaultOverrideHostStatus =
singleHealthStatusToUint(Host::Health::Degraded) |
singleHealthStatusToUint(Host::Health::Healthy);

bool tooManyPreconnects(size_t num_preconnect_picks, uint32_t healthy_hosts) {
// Currently we only allow the number of preconnected connections to equal the
Expand Down Expand Up @@ -118,7 +122,8 @@ LoadBalancerBase::LoadBalancerBase(
: stats_(stats), runtime_(runtime), random_(random),
default_healthy_panic_percent_(PROTOBUF_PERCENT_TO_ROUNDED_INTEGER_OR_DEFAULT(
common_config, healthy_panic_threshold, 100, 50)),
priority_set_(priority_set) {
priority_set_(priority_set),
override_host_status_(LoadBalancerContextBase::createOverrideHostStatus(common_config)) {
for (auto& host_set : priority_set_.hostSetsPerPriority()) {
recalculatePerPriorityState(host_set->priority(), priority_set_, per_priority_load_,
per_priority_health_, per_priority_degraded_, total_healthy_hosts_);
Expand Down Expand Up @@ -517,20 +522,37 @@ bool ZoneAwareLoadBalancerBase::earlyExitNonLocalityRouting() {
return false;
}

bool LoadBalancerContextBase::validateOverrideHostStatus(Host::Health health,
OverrideHostStatus status) {
switch (health) {
case Host::Health::Unhealthy:
return status & UnhealthyStatus;
case Host::Health::Degraded:
return status & DegradedStatus;
case Host::Health::Healthy:
return status & HealthyStatus;
uint32_t LoadBalancerContextBase::createOverrideHostStatus(
const envoy::config::cluster::v3::Cluster::CommonLbConfig& common_config) {
if (!common_config.has_override_host_status()) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if there is but it is empty?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then override_host_status will be zero and all the override host from LoadBalancerContext::overrideHostToSelect() will be ignored and stateful session stickiness will be disabled for current cluster.

return DefaultOverrideHostStatus;
}

uint32_t override_host_status = 0;

for (auto single_status : common_config.override_host_status().statuses()) {
switch (static_cast<envoy::config::core::v3::HealthStatus>(single_status)) {
case envoy::config::core::v3::HealthStatus::UNKNOWN:
case envoy::config::core::v3::HealthStatus::HEALTHY:
override_host_status |= singleHealthStatusToUint(Host::Health::Healthy);
break;
case envoy::config::core::v3::HealthStatus::UNHEALTHY:
case envoy::config::core::v3::HealthStatus::DRAINING:
case envoy::config::core::v3::HealthStatus::TIMEOUT:
override_host_status |= singleHealthStatusToUint(Host::Health::Unhealthy);
break;
case envoy::config::core::v3::HealthStatus::DEGRADED:
override_host_status |= singleHealthStatusToUint(Host::Health::Degraded);
break;
default:
break;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this may hide a case where the server sends a config with some other health status.
Consider adding [(validate.rules).enum.defined_only = true] to the statuses, and maybe a emit a log here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get it.

}
}
return false;
return override_host_status;
}

HostConstSharedPtr LoadBalancerContextBase::selectOverrideHost(const HostMap* host_map,
uint32_t status,
LoadBalancerContext* context) {
if (context == nullptr) {
return nullptr;
Expand All @@ -545,7 +567,7 @@ HostConstSharedPtr LoadBalancerContextBase::selectOverrideHost(const HostMap* ho
return nullptr;
}

auto host_iter = host_map->find(override_host.value().first);
auto host_iter = host_map->find(override_host.value());

// The override host cannot be found in the host map.
if (host_iter == host_map->end()) {
Expand All @@ -555,17 +577,15 @@ HostConstSharedPtr LoadBalancerContextBase::selectOverrideHost(const HostMap* ho
HostConstSharedPtr host = host_iter->second;
ASSERT(host != nullptr);

// Verify the host status.
if (LoadBalancerContextBase::validateOverrideHostStatus(host->health(),
override_host.value().second)) {
if (singleHealthStatusToUint(host->health()) & status) {
return host;
}
return nullptr;
}

HostConstSharedPtr ZoneAwareLoadBalancerBase::chooseHost(LoadBalancerContext* context) {
HostConstSharedPtr host =
LoadBalancerContextBase::selectOverrideHost(cross_priority_host_map_.get(), context);
HostConstSharedPtr host = LoadBalancerContextBase::selectOverrideHost(
cross_priority_host_map_.get(), override_host_status_, context);
if (host != nullptr) {
return host;
}
Expand Down Expand Up @@ -652,8 +672,8 @@ uint32_t ZoneAwareLoadBalancerBase::tryChooseLocalLocalityHosts(const HostSet& h
return random_.random() % number_of_localities;
}

// Random sampling to select specific locality for cross locality traffic based on the additional
// capacity in localities.
// Random sampling to select specific locality for cross locality traffic based on the
// additional capacity in localities.
uint64_t threshold = random_.random() % state.residual_capacity_[number_of_localities - 1];

// This potentially can be optimized to be O(log(N)) where N is the number of localities.
Expand Down Expand Up @@ -903,8 +923,8 @@ HostConstSharedPtr EdfLoadBalancerBase::peekAnotherHost(LoadBalancerContext* con

// As has been commented in both EdfLoadBalancerBase::refresh and
// BaseDynamicClusterImpl::updateDynamicHostList, we must do a runtime pivot here to determine
// whether to use EDF or do unweighted (fast) selection. EDF is non-null iff the original weights
// of 2 or more hosts differ.
// whether to use EDF or do unweighted (fast) selection. EDF is non-null iff the original
// weights of 2 or more hosts differ.
if (scheduler.edf_ != nullptr) {
return scheduler.edf_->peekAgain([this](const Host& host) { return hostWeight(host); });
} else {
Expand All @@ -929,8 +949,8 @@ HostConstSharedPtr EdfLoadBalancerBase::chooseHostOnce(LoadBalancerContext* cont

// As has been commented in both EdfLoadBalancerBase::refresh and
// BaseDynamicClusterImpl::updateDynamicHostList, we must do a runtime pivot here to determine
// whether to use EDF or do unweighted (fast) selection. EDF is non-null iff the original weights
// of 2 or more hosts differ.
// whether to use EDF or do unweighted (fast) selection. EDF is non-null iff the original
// weights of 2 or more hosts differ.
if (scheduler.edf_ != nullptr) {
auto host = scheduler.edf_->pickAndAdd([this](const Host& host) { return hostWeight(host); });
return host;
Expand Down
21 changes: 18 additions & 3 deletions source/common/upstream/load_balancer_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -155,17 +155,32 @@ class LoadBalancerBase : public LoadBalancer {
// The total count of healthy hosts across all priority levels.
uint32_t total_healthy_hosts_;

// Expected override host statues. Type uint32_t is used to speed up host status validate.
// Every bit in the OverrideHostStatus represent an enum value of Host::Health. The specific
// correspondence is shown below:
//
// * 0b001: Host::Health::Unhealthy
// * 0b010: Host::Health::Degraded
// * 0b100: Host::Health::Healthy
//
// If multiple bit fields are set, it is acceptable as long as the status of override host is in
// any of these statuses.
const uint32_t override_host_status_{};

private:
Common::CallbackHandlePtr priority_update_cb_;
};

class LoadBalancerContextBase : public LoadBalancerContext {
public:
static bool validateOverrideHostStatus(Host::Health health, OverrideHostStatus status);

static HostConstSharedPtr selectOverrideHost(const HostMap* host_map,
// A utility function to select override host from host map according to load balancer context.
static HostConstSharedPtr selectOverrideHost(const HostMap* host_map, uint32_t status,
LoadBalancerContext* context);

// A utility function to create override host status from lb config.
static uint32_t createOverrideHostStatus(
const envoy::config::cluster::v3::Cluster::CommonLbConfig& common_config);

absl::optional<uint64_t> computeHashKey() override { return {}; }

const Network::Connection* downstreamConnection() const override { return nullptr; }
Expand Down
7 changes: 4 additions & 3 deletions source/common/upstream/subset_lb.cc
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ SubsetLoadBalancer::SubsetLoadBalancer(
original_local_priority_set_(local_priority_set),
locality_weight_aware_(subsets.localityWeightAware()),
scale_locality_weight_(subsets.scaleLocalityWeight()), list_as_any_(subsets.listAsAny()),
time_source_(time_source) {
time_source_(time_source),
override_host_status_(LoadBalancerContextBase::createOverrideHostStatus(common_config)) {
ASSERT(subsets.isEnabled());

if (fallback_policy_ != envoy::config::cluster::v3::Cluster::LbSubsetConfig::NO_FALLBACK) {
Expand Down Expand Up @@ -283,8 +284,8 @@ void SubsetLoadBalancer::initSelectorFallbackSubset(
}

HostConstSharedPtr SubsetLoadBalancer::chooseHost(LoadBalancerContext* context) {
HostConstSharedPtr override_host =
LoadBalancerContextBase::selectOverrideHost(cross_priority_host_map_.get(), context);
HostConstSharedPtr override_host = LoadBalancerContextBase::selectOverrideHost(
cross_priority_host_map_.get(), override_host_status_, context);
if (override_host != nullptr) {
return override_host;
}
Expand Down
2 changes: 2 additions & 0 deletions source/common/upstream/subset_lb.h
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,8 @@ class SubsetLoadBalancer : public LoadBalancer, Logger::Loggable<Logger::Id::ups

TimeSource& time_source_;

const uint32_t override_host_status_{};

friend class SubsetLoadBalancerDescribeMetadataTester;
};

Expand Down
6 changes: 3 additions & 3 deletions source/common/upstream/thread_aware_lb_impl.cc
Original file line number Diff line number Diff line change
Expand Up @@ -142,8 +142,8 @@ ThreadAwareLoadBalancerBase::LoadBalancerImpl::chooseHost(LoadBalancerContext* c
return nullptr;
}

HostConstSharedPtr host =
LoadBalancerContextBase::selectOverrideHost(cross_priority_host_map_.get(), context);
HostConstSharedPtr host = LoadBalancerContextBase::selectOverrideHost(
cross_priority_host_map_.get(), override_host_status_, context);
if (host != nullptr) {
return host;
}
Expand Down Expand Up @@ -188,7 +188,7 @@ LoadBalancerPtr ThreadAwareLoadBalancerBase::LoadBalancerFactoryImpl::create() {
lb->degraded_per_priority_load_ = degraded_per_priority_load_;
lb->per_priority_state_ = per_priority_state_;
lb->cross_priority_host_map_ = cross_priority_host_map_;

lb->override_host_status_ = override_host_status_;
return lb;
}

Expand Down
8 changes: 5 additions & 3 deletions source/common/upstream/thread_aware_lb_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ class ThreadAwareLoadBalancerBase : public LoadBalancerBase, public ThreadAwareL
Random::RandomGenerator& random,
const envoy::config::cluster::v3::Cluster::CommonLbConfig& common_config)
: LoadBalancerBase(priority_set, stats, runtime, random, common_config),
factory_(new LoadBalancerFactoryImpl(stats, random)) {}
factory_(new LoadBalancerFactoryImpl(stats, random, override_host_status_)) {}

private:
struct PerPriorityState {
Expand Down Expand Up @@ -138,6 +138,7 @@ class ThreadAwareLoadBalancerBase : public LoadBalancerBase, public ThreadAwareL

ClusterStats& stats_;
Random::RandomGenerator& random_;
uint32_t override_host_status_{};
std::shared_ptr<std::vector<PerPriorityStatePtr>> per_priority_state_;
std::shared_ptr<HealthyLoad> healthy_per_priority_load_;
std::shared_ptr<DegradedLoad> degraded_per_priority_load_;
Expand All @@ -147,14 +148,15 @@ class ThreadAwareLoadBalancerBase : public LoadBalancerBase, public ThreadAwareL
};

struct LoadBalancerFactoryImpl : public LoadBalancerFactory {
LoadBalancerFactoryImpl(ClusterStats& stats, Random::RandomGenerator& random)
: stats_(stats), random_(random) {}
LoadBalancerFactoryImpl(ClusterStats& stats, Random::RandomGenerator& random, uint32_t status)
: stats_(stats), random_(random), override_host_status_(status) {}

// Upstream::LoadBalancerFactory
LoadBalancerPtr create() override;

ClusterStats& stats_;
Random::RandomGenerator& random_;
uint32_t override_host_status_{};
absl::Mutex mutex_;
std::shared_ptr<std::vector<PerPriorityStatePtr>> per_priority_state_ ABSL_GUARDED_BY(mutex_);
// This is split out of PerPriorityState so LoadBalancerBase::ChoosePriority can be reused.
Expand Down
3 changes: 1 addition & 2 deletions test/common/router/router_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -6601,8 +6601,7 @@ TEST_F(RouterTest, RequestWithUpstreamOverrideHost) {
.WillOnce(Return(absl::make_optional<absl::string_view>("1.2.3.4")));

auto override_host = router_.overrideHostToSelect();
EXPECT_EQ("1.2.3.4", override_host->first);
EXPECT_EQ(~static_cast<uint32_t>(0), override_host->second);
EXPECT_EQ("1.2.3.4", override_host.value());

Http::TestRequestHeaderMapImpl headers{{"x-envoy-retry-on", "5xx"}, {"x-envoy-internal", "true"}};
HttpTestUtility::addDefaultHeaders(headers);
Expand Down
Loading