Adding IPVS blog post #9431
Adding IPVS blog post #9431
Conversation
k8s-ci-robot
added
cncf-cla: yes
|
Deploy preview for kubernetes-io-master-staging ready! Built with commit e6586d7 https://deploy-preview-9431--kubernetes-io-master-staging.netlify.com |
|
Deploy preview for kubernetes-io-master-staging ready! Built with commit 19470a1 https://deploy-preview-9431--kubernetes-io-master-staging.netlify.com |
|
/assign @zacharysarah |
There was a problem hiding this comment.
thanks you, for this blog post @kbarnard10 and the other authors! 👍 💯
|
|
||
| **Parameter: --ipvs-min-sync-period** Minimum interval of how often the IPVS rules are refreshed (e.g. '5s', '1m'). Must be greater than 0. | ||
|
|
||
| **Parameter: --ipvs-exclude-cidrs** A comma-separated list of CIDR's which the IPVS proxier should not touch when cleaning up IPVS rules because IPVS proxier can't distinguish kube-proxy created ipvs rules from user original ipvs rules. If you are using IPVS proxier with your own IPVS rules in the environment, this parameter should be specified, otherwise your original rule will be cleaned. |
There was a problem hiding this comment.
uppercase ipvs -> IPVS.
distinguish kube-proxy created IPVS rules from user original IPVS rules.
There was a problem hiding this comment.
possibly style --ipvs-exclude-cidrs, --ipvs-min-sync-period, --ipvs-sync-period with " ` "
even if they are in bold text?
|
|
||
| **IPVS** (**IP Virtual Server**) is built on top of the Netfilter and implements transport-layer load balancing as part of the Linux kernel. | ||
|
|
||
| IPVS is incorporated into the LVS(Linux Virtual Server), where it runs on a host and acts as a load balancer in front of a cluster of real servers. IPVS can direct requests for TCP- and UDP-based services to the real servers, and make services of the real servers appear as virtual services on a single IP address. Therefore, IPVS naturally supports Kubernetes Service. |
There was a problem hiding this comment.
add space before bracket:
LVS(Linux Virtual Server) -> LVS (Linux Virtual Server)
|
|
||
| Even though Kubernetes already support 5000 nodes in release v1.6, the kube-proxy with iptables is actually a bottleneck to scale the cluster to 5000 nodes. One example is that with NodePort Service in a 5000-node cluster, if we have 2000 services and each services have 10 pods, this will cause at least 20000 iptable records on each worker node, and this can make the kernel pretty busy. | ||
|
|
||
| On the other hand, using IPVS-based in-cluster service load balancing can help a lot for such cases. IPVS is specifically designed for load balancing and uses more efficient data structures(hash tables) allowing for almost unlimited scale under the hood. |
|
|
||
| ### Parameter Changes | ||
|
|
||
| **Parameter: --proxy-mode** In addition to existing userspace and iptables modes, IPVS mode is configured via `--proxy-mode=ipvs`. It implicitly uses IPVS NAT mode for Service port mapping. |
|
|
||
| **Parameter: --ipvs-scheduler** | ||
|
|
||
| A new kube-proxy parameter will be added to specify the IPVS load balancing algorithm, with the parameter being `--ipvs-scheduler`. If it’s not configured, then round-robin (rr) is the default value. |
There was a problem hiding this comment.
re: will be added
--ipvs-scheduler was fist added in v1.8.0-beta.0,
while the scheduler methods were introduced around v1.9.0-alpha.2, if i'm not mistaken.
perhaps reword the sentence to use paste tense?
|
|
||
| When creating a ClusterIP type Service, IPVS proxier will do the following three things: | ||
|
|
||
| * Make sure a dummy interface exists in the node, default to kube-ipvs0 |
There was a problem hiding this comment.
, default to ->, defaults to ?
| -> 10.244.1.237:8080 Masq 1 0 0 | ||
| ``` | ||
|
|
||
| Please note that the relationship between a Kubernetes Service and IPVS virtual servers is `1:N`. Consider a Kubernetes Service that has more than one IP addresses, for example, an External IP type Service has two IP addresses(ClusterIP and External IP). Then the IPVS proxier will create 2 IPVS virtual servers - one for Cluster IP and another one for External IP. The relationship between a Kubernetes Endpoint(each IP+Port pair) and an IPVS virtual server is `1:1`. |
There was a problem hiding this comment.
spaces before (each IP+Port pair) and (ClusterIP and External IP).
i would reword:
Consider a Kubernetes Service that has more than one IP addresses, for example, an External IP type Service has two IP addresses(ClusterIP and External IP).
to:
For example, consider a Kubernetes Service that has more than one IP address. An External IP type Service has two IP addresses - ClusterIP and External IP.
|
|
||
| #### Port Mapping | ||
|
|
||
| There are three proxy modes in IPVS: NAT (masq), IPIP and DR. Only NAT mode supports port mapping. Kube-proxy leverages NAT mode for port mapping. The following example shows IPVS mapping Service port 3080 to Pod port 8080. |
There was a problem hiding this comment.
there is an extra space before mapping and 3080.
|
|
||
| #### Iptables & Ipset in IPVS Proxier | ||
|
|
||
| IPVS is for load balancing, it can't handle other workarounds in kube-proxy, e.g. packet filtering, hairpin-masquerade tricks, SNAT, etc. |
|
|
||
| ### Run kube-proxy in IPVS Mode | ||
|
|
||
| Currently, local-up scripts, GCE scripts, and kubeadm support switching IPVS proxy mode via exporting environment variables(`KUBE_PROXY_MODE=ipvs`) or specifying flag(`--proxy-mode=ipvs`). Before running IPVS proxier, please ensure IPVS required kernel modules are already installed. |
|
@neolit123 Thanks! Edits made. |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: zacharysarah The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
For 5 Days of K8s series.
/assign @natekartchner