-

Over the years many webscale companies have desinged massivley scalable and highly available services using loadbalancer solutions based on commodity Linux servers. Traditional middleboxes are completley replaced with software loadbalancers. In this blog we will see common building blocks across Microsoft’s Ananta, +

Over the years many webscale companies have designed massivley scalable and highly available services using loadbalancer solutions based on commodity Linux servers. Traditional middleboxes are completely replaced with software loadbalancers. In this blog we will see common building blocks across Microsoft’s Ananta, Google’s Maglev, Facebook’s Shiv, Github GLB and Yahoo L3 DSR. We will see how Kube-router has implemented some of these building blocks for Kuberentes, and how you can leverage them to build a highly-available and scalable ingress in bare-metal deployments.

@@ -135,11 +135,11 @@

Network Desgin

Network requirements

-

Below are some of the standard mechanisams used.

+

Below are some of the standard mechanisms used.

Use of BGP + ECMP

-

You have second tier fleet of L4 directors, each of which is a BGP speaker and advertising service VIP to the BGP router. Routers has equal cost mutliple paths to the VIP through the L4 directors. +

You have second tier fleet of L4 directors, each of which is a BGP speaker and advertising service VIP to the BGP router. Routers has equal cost multiple paths to the VIP through the L4 directors. Running the BGP protocol on the L4 director provides automatic failure detection and recovery. If a L4 director fails or shuts down unexpectedly, the router detects this failure via the BGP protocol and automatically stops sending traffic to that L4 director. Similarly, when the L4 director comes up, it can start announcing the routes and the router will start forwarding traffic to it.

@@ -147,7 +147,7 @@

L3/L4 network load balancing

Since router has multiple paths to advertised vip, it can perform ECMP load balancing. In case router does L3 does balancing, router distributes the traffic across the tier-2 L4 directors. Router can also do hash (on packets source, destination ip and port etc) based load balancing. Where traffic corresponding to a same flow always gets forwarded to same L4 director. Even if there are -more than one router (for redundency) even then traffic can get forwarded to same L4 director by both the routers if consistent hashing is used.

+more than one router (for redundancy) even then traffic can get forwarded to same L4 director by both the routers if consistent hashing is used.

L4 director

@@ -155,12 +155,12 @@

L4 director

Direct server return

-

In typical load balancer acting as proxy, packets are DNAT’ed to real server IP. Return traffic must go through the same loadbalancer so that packets gets SNAT’ed (to VIP as source IP). This hinders scale-out approach particulalry when routers are sharding traffic across the L4 directors. To overcome the limitation, as mentioned above L4 director simply forward the packet. It also does tunnel the packets so that original packet is delivered to the service point as is. Various solution are -available (IPVS/LVS DR mode, use of GRE/IPIP tunnels etc) to send the traffic to endpoint. Since endpoint when it recives the packets, it sees the traffic destined to the VIP (ofcourse endoint needs to be setup to accept traffic to VIP) from the original client. Return traffic is directly sent to the client.

+

In typical load balancer acting as proxy, packets are DNAT’ed to real server IP. Return traffic must go through the same loadbalancer so that packets gets SNAT’ed (to VIP as source IP). This hinders scale-out approach particularly when routers are sharding traffic across the L4 directors. To overcome the limitation, as mentioned above L4 director simply forward the packet. It also does tunnel the packets so that original packet is delivered to the service point as is. Various solution are +available (IPVS/LVS DR mode, use of GRE/IPIP tunnels etc) to send the traffic to endpoint. Since endpoint when it receives the packets, it sees the traffic destined to the VIP (ofcourse endoint needs to be setup to accept traffic to VIP) from the original client. Return traffic is directly sent to the client.

L4/L7 split design

-

Above basic mechanisams can be extended to implement application load balancing. Whats is called L4/L7 split design as shown below.

+

Above basic mechanisms can be extended to implement application load balancing. Whats is called L4/L7 split design as shown below.

Network requirements

diff --git a/post/index.html b/post/index.html index 0c6bd67..abe42f0 100644 --- a/post/index.html +++ b/post/index.html @@ -113,7 +113,7 @@

Kube-router: Highly-available and scalable ingress for ba

- Over the years many webscale companies have desinged massivley scalable and highly available services using loadbalancer solutions based on commodity Linux servers. Traditional middleboxes are completley replaced with software loadbalancers. In this blog we will see common building blocks across Microsoft’s Ananta, Google’s Maglev, Facebook’s Shiv, Github GLB and Yahoo L3 DSR. We will see how Kube-router has implemented some of these building blocks for Kuberentes, and how you can leverage them to build a highly-available and scalable ingress in bare-metal deployments. + Over the years many webscale companies have designed massivley scalable and highly available services using loadbalancer solutions based on commodity Linux servers. Traditional middleboxes are completely replaced with software loadbalancers. In this blog we will see common building blocks across Microsoft’s Ananta, Google’s Maglev, Facebook’s Shiv, Github GLB and Yahoo L3 DSR. We will see how Kube-router has implemented some of these building blocks for Kuberentes, and how you can leverage them to build a highly-available and scalable ingress in bare-metal deployments. [Read More]
diff --git a/post/index.xml b/post/index.xml index 0c62806..a7fc43b 100644 --- a/post/index.xml +++ b/post/index.xml @@ -18,7 +18,7 @@ Wed, 01 Nov 2017 00:00:00 +0000 info@cloudnativelabs.net (Cloudnative Labs) https://cloudnativelabs.github.io/post/2017-11-01-kube-high-available-ingress/ - Over the years many webscale companies have desinged massivley scalable and highly available services using loadbalancer solutions based on commodity Linux servers. Traditional middleboxes are completley replaced with software loadbalancers. In this blog we will see common building blocks across Microsoft’s Ananta, Google’s Maglev, Facebook’s Shiv, Github GLB and Yahoo L3 DSR. We will see how Kube-router has implemented some of these building blocks for Kuberentes, and how you can leverage them to build a highly-available and scalable ingress in bare-metal deployments. + Over the years many webscale companies have designed massivley scalable and highly available services using loadbalancer solutions based on commodity Linux servers. Traditional middleboxes are completely replaced with software loadbalancers. In this blog we will see common building blocks across Microsoft’s Ananta, Google’s Maglev, Facebook’s Shiv, Github GLB and Yahoo L3 DSR. We will see how Kube-router has implemented some of these building blocks for Kuberentes, and how you can leverage them to build a highly-available and scalable ingress in bare-metal deployments.