Proxy supports multiple algorithms for load balancing across destinations

[samsp-msft]

Complete the set of common load balancing schemes

• Round Robin Implement Round-Robin Load Balancing #72
• Random Implement Random Load-Balancing #73
• Least loaded of 2 random "The Power of Two Choices" Load-Balancing Algorithm #75

• First healthy endpoint (already implemented)

[Tratcher]

Current implementation:

reverse-proxy/src/ReverseProxy.Core/Service/Proxy/LoadBalancer.cs

Lines 21 to 36 in be96d64

	switch (loadBalancingOptions.Mode)
	{
	case BackendConfig.BackendLoadBalancingOptions.LoadBalancingMode.First:
	// TODO: Remove, this is a silly load balancing mode
	if (healthyEndpoints.Count == 0)
	{
	return null;
	}
	return healthyEndpoints[0];
	case BackendConfig.BackendLoadBalancingOptions.LoadBalancingMode.Random:
	throw new NotImplementedException();
	case BackendConfig.BackendLoadBalancingOptions.LoadBalancingMode.PowerOfTwoChoices:
	throw new NotImplementedException();
	default:
	throw new ReverseProxyException($"Load balancing mode '{loadBalancingOptions.Mode}' is not supported.");

Note that load balancing is configurable per backend set. Do we need that level of granularity, or could this be specified server wide?

[analogrelay]

Oh it should definitely be per backend. It's very common for different services to have different load profiles and benefit from different kinds of load balancing. It's also something Envoy configures per-cluster which serves as some prior art.

[Tratcher]

Another one: Least busy
We have the metadata we need for this, it just requires one pass through the list.

[Kahbazi]

Is it possible to add a feature for blue green deployment to load balancer? or is it out of the scope of load balancer?

[Tratcher]

Is it possible to add a feature for blue green deployment to load balancer? or is it out of the scope of load balancer?

I think that's a different feature from load balancing that would be handled earlier at the routing phase. Load balancing assumes all nodes are equivalent, plus or minus health/load.

[analogrelay]

I'm going to break this up into smaller issues for individual algorithms. @Tratcher and I were talking about some of them being fairly easy and high-value for preview 1 (while others are more complicated).

[analogrelay]

Created #72 #73 #74 #75

Closing this out now.