Skip to content

Commit

Permalink
doc: update and format all documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
aauren committed Oct 19, 2023
1 parent 6442689 commit dad4991
Show file tree
Hide file tree
Showing 19 changed files with 290 additions and 352 deletions.
75 changes: 39 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
<!-- markdownlint-disable-next-line first-line-heading -->
![logo](https://cdn.rawgit.com/cloudnativelabs/kube-router/64f7700e/Documentation/img/logo-full.svg)

[![Build Status](https://github.com/cloudnativelabs/kube-router/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/cloudnativelabs/kube-router/actions/workflows/ci.yml?query=branch%3Amaster)
[![Go Report Card](https://goreportcard.com/badge/github.com/cloudnativelabs/kube-router)](https://goreportcard.com/report/github.com/cloudnativelabs/kube-router)
[![Slack](https://img.shields.io/badge/slack-join%20chat%20%E2%86%92-e01563.svg)](https://kubernetes.slack.com/messages/C8DCQGTSB/)
[![Docker Pulls kube-router](https://img.shields.io/docker/pulls/cloudnativelabs/kube-router.svg?label=docker+pulls)](https://hub.docker.com/r/cloudnativelabs/kube-router/)
[![](https://images.microbadger.com/badges/image/cloudnativelabs/kube-router.svg)](https://microbadger.com/images/cloudnativelabs/kube-router "Get your own image badge on microbadger.com")
[![](https://images.microbadger.com/badges/image/cloudnativelabs/kube-router.svg)](https://microbadger.com/images/cloudnativelabs/kube-router)
[![](https://img.shields.io/github/release/cloudnativelabs/kube-router/all.svg?style=flat-square)](https://github.com/cloudnativelabs/kube-router/releases)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)

Expand All @@ -20,19 +21,20 @@ single DaemonSet/Binary. It doesn't get any easier.

### IPVS/LVS based service proxy | `--run-service-proxy`

kube-router uses the Linux kernel's LVS/IPVS features to implement its K8s Services
Proxy. Kube-router fully leverages power off LVS/IPVS to provide rich set of [scheduling options](/docs#load-balancing-scheduling-algorithms) and unique features like DSR (Direct Server Return), L3 load balancing with ECMP for deployments where high throughput, minimal latency and high-availability are crucial.
kube-router uses the Linux kernel's LVS/IPVS features to implement its K8s Services Proxy. Kube-router fully leverages
power of LVS/IPVS to provide a rich set of scheduling options and unique features like DSR (Direct Server Return), L3
load balancing with ECMP for deployments where high throughput, minimal latency and high-availability are crucial.

Read more about the advantages of IPVS for container load balancing:

- [Kubernetes network services proxy with IPVS/LVS](https://cloudnativelabs.github.io/post/2017-05-10-kube-network-service-proxy/)
- [Highly-available and scalable ingress for baremetal Kubernetes clusters](https://cloudnativelabs.github.io/post/2017-11-01-kube-high-available-ingress/)

### Pod Networking | `--run-router`

kube-router handles Pod networking efficiently with direct routing thanks to the
BGP protocol and the GoBGP Go library. It uses the native Kubernetes API to
maintain distributed pod networking state. That means no dependency on a
separate datastore to maintain in your cluster.
kube-router handles Pod networking efficiently with direct routing thanks to the BGP protocol and the GoBGP Go library.
It uses the native Kubernetes API to maintain distributed pod networking state. That means no dependency on a separate
datastore to maintain in your cluster.

kube-router's elegant design also means there is no dependency on another CNI
plugin. The
Expand All @@ -41,6 +43,7 @@ provided by the CNI project is all you need -- and chances are you already have
it in your CNI binary directory!

Read more about the advantages and potential of BGP with Kubernetes:

- [Kubernetes pod networking and beyond with BGP](https://cloudnativelabs.github.io/post/2017-05-22-kube-pod-networking)

### Network Policy Controller | `--run-firewall`
Expand All @@ -54,44 +57,42 @@ Kube-router supports the networking.k8s.io/NetworkPolicy API or network policy V
[semantics](https://github.com/kubernetes/kubernetes/pull/39164#issue-197243974) and also network policy beta semantics.

Read more about kube-router's approach to Kubernetes Network Policies:

- [Enforcing Kubernetes network policies with iptables](https://cloudnativelabs.github.io/post/2017-05-1-kube-network-policies/)

### Advanced BGP Capabilities

If you have other networking devices or SDN systems that talk BGP, kube-router
will fit in perfectly. From a simple full node-to-node mesh to per-node peering
configurations, most routing needs can be attained. The configuration is
Kubernetes native (annotations) just like the rest of kube-router, so use the
tools you already know! Since kube-router uses GoBGP, you have access to a
modern BGP API platform as well right out of the box. Kube-router also provides
a way to expose services outside the cluster by advertising ClusterIP and externalIPs to
configured BGP peers. Kube-routes also support MD5 password based authentication and
uses strict export policies so you can be assured routes are advertised to underlay
only as you intended.
If you have other networking devices or SDN systems that talk BGP, kube-router will fit in perfectly. From a simple full
node-to-node mesh to per-node peering configurations, most routing needs can be attained. The configuration is
Kubernetes native (annotations) just like the rest of kube-router, so use the tools you already know! Since kube-router
uses GoBGP, you have access to a modern BGP API platform as well right out of the box. Kube-router also provides a way
to expose services outside the cluster by advertising ClusterIP and externalIPs to configured BGP peers. Kube-routesalso
support MD5 password based authentication and uses strict export policies so you can be assured routes are advertised to
the underlay only as you intended.

For more details please refer to the [BGP documentation](docs/bgp.md).

### Standard Linux Networking

A key design tenet of Kube-router is to use standard Linux networking stack and toolset. There is no overlays or
SDN pixie dust, but just plain good old networking. You can use standard Linux networking tools like iptables, ipvsadm, ipset,
iproute, traceroute, tcpdump etc. to troubleshoot or observe data path. When kube-router is ran as a daemonset, image also ships with these [tools](./docs/pod-toolbox.md#pod-toolbox) automatically configured for your cluster.
A key design tenet of Kube-router is to use standard Linux networking stack and toolset. There is no overlays or SDN
pixie dust, but just plain good old networking. You can use standard Linux networking tools like iptables, ipvsadm,
ipset, iproute, traceroute, tcpdump etc. to troubleshoot or observe data path. When kube-router is ran as a daemonset,
the official kube-router image also ships with these [tools](./docs/pod-toolbox.md#pod-toolbox) automatically configured
for your cluster.

### Small Footprint

Although it does the work of several of its peers in one binary, kube-router
does it all with a relatively [tiny codebase](https://github.com/cloudnativelabs/kube-router/tree/master/pkg/controllers), partly because IPVS is already
there on your Kuberneres nodes waiting to help you do amazing things.
kube-router brings that and GoBGP's modern BGP interface to you in an elegant
package designed from the ground up for Kubernetes.
Although it does the work of several of its peers in one binary, kube-router does it all with a relatively
[tiny codebase](https://github.com/cloudnativelabs/kube-router/tree/master/pkg/controllers), partly because IPVS is
already there on your Kuberneres nodes waiting to help you do amazing things. kube-router brings that and GoBGP's modern
BGP interface to you in an elegant package designed from the ground up for Kubernetes.

### High Performance

A primary motivation for kube-router is performance. The combination of BGP for
inter-node Pod networking and IPVS for load balanced proxy Services is a perfect
recipe for high-performance cluster networking at scale. BGP ensures that the
data path is dynamic and efficient, and IPVS provides in-kernel load balancing
that has been thoroughly tested and optimized.
A primary motivation for kube-router is performance. The combination of BGP for inter-node Pod networking and IPVS for
load balanced proxy Services is a perfect recipe for high-performance cluster networking at scale. BGP ensures that the
data path is dynamic and efficient, and IPVS provides in-kernel load balancing that has been thoroughly tested and
optimized.

## Getting Started

Expand All @@ -103,7 +104,9 @@ that has been thoroughly tested and optimized.

## Project status

Kube-router is being used in several production clusters by diverse set of users ranging from financial firms, gaming companies to universities. For almost a year we have listened to users and incorporated feedback. The core functionality is stable. We are working toward GA release.
Kube-router is being used in several production clusters by diverse set of users ranging from financial firms, gaming
companies to universities. For years we have listened to users and incorporated feedback. The core functionality is now
very stable.

## Contributing

Expand All @@ -119,8 +122,8 @@ for quick help. Feel free to leave feedback or raise questions by opening an iss

Kube-router build upon following libraries:

- Iptables: https://github.com/coreos/go-iptables
- GoBGP: https://github.com/osrg/gobgp
- Netlink: https://github.com/vishvananda/netlink
- Ipset: https://github.com/janeczku/go-ipset
- IPVS: https://github.com/docker/libnetwork/
- [iptables](https://github.com/coreos/go-iptables)
- [GoBGP](https://github.com/osrg/gobgp)
- [Netlink](https://github.com/vishvananda/netlink)
- [ipset](https://github.com/janeczku/go-ipset)
- [IPVS](https://github.com/docker/libnetwork/)
5 changes: 0 additions & 5 deletions docs/Observability.md

This file was deleted.

1 change: 0 additions & 1 deletion docs/README.md

This file was deleted.

10 changes: 7 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
# Architecture

Kube-router is built around concept of watchers and controllers. Watchers use Kubernetes watch API to get notification on events related to create, update, delete of Kubernetes objects. Each watcher gets notification related to a particular API object. On receiving an event from API server, watcher broadcasts events. Controller registers to get event updates from the watchers and act up on the events.
Kube-router is built around concept of watchers and controllers. Watchers use Kubernetes watch API to get notification
on events related to create, update, delete of Kubernetes objects. Each watcher gets notification related to a
particular API object. On receiving an event from API server, watcher broadcasts events. Controller registers to get
event updates from the watchers and act up on the events.

Kube-router consists of 3 core controllers and multiple watchers as depicted in below diagram.

![Arch](./img/kube-router-arch.png)

Each of the [controller](https://github.com/cloudnativelabs/kube-router/tree/master/app/controllers) follows below structure
Each of the [controller](https://github.com/cloudnativelabs/kube-router/tree/master/app/controllers) follows below
structure:

```
```go
func Run() {
for {
Sync() // control loop that runs for ever and perfom sync at periodic interval
Expand Down
23 changes: 11 additions & 12 deletions docs/bgp.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ annotation value as the ASN number for the node.

Users can annotate node objects with the following command:

```
```sh
kubectl annotate node <kube-node> "kube-router.io/node.asn=64512"
```

Expand All @@ -41,13 +41,13 @@ route reflector servers and with route reflector clients enabling reflection.

Users can annotate node objects with the following command for Route Reflector server mode:

```
```sh
kubectl annotate node <kube-node> "kube-router.io/rr.server=42"
```

and for Route Reflector client mode:

```
```sh
kubectl annotate node <kube-node> "kube-router.io/rr.client=42"
```

Expand All @@ -68,7 +68,7 @@ peer router by specifying a slice of BGP peers.

For example:

```
```sh
--peer-router-ips="192.168.1.99,192.168.1.100"
--peer-router-asns=65000,65000
```
Expand All @@ -77,10 +77,10 @@ For example:

Alternatively, each node can be configured with one or more node specific BGP peers. Information regarding node specific
BGP peer is read from node API object annotations:

- `kube-router.io/peer.ips`
- `kube-router.io/peer.asns`


For example, users can annotate node object with below commands:

```shell
Expand Down Expand Up @@ -144,7 +144,7 @@ U2VjdXJlUGFzc3dvcmQ=

In this CLI flag example the first router (192.168.1.99) uses a password, while the second (192.168.1.100) does not.

```
```sh
--peer-router-ips="192.168.1.99,192.168.1.100"
--peer-router-asns="65000,65000"
--peer-router-passwords="U2VjdXJlUGFzc3dvcmQK,"
Expand All @@ -171,7 +171,7 @@ Finally, to include peer passwords as a file you would run kube-router with the
The password file, closely follows the syntax of the command-line and node annotation options.
Here, the first peer IP (192.168.1.99) would be configured with a password, while the second would not.

```
```sh
U2VjdXJlUGFzc3dvcmQK,
```

Expand Down Expand Up @@ -247,9 +247,8 @@ network.

This configuration would have the following effects:

- Peering Outside the Cluster
(https://github.com/cloudnativelabs/kube-router/blob/master/docs/bgp.md#peering-outside-the-cluster) via one of the
many means that kube-router makes available
- [Peering Outside the Cluster](https://github.com/cloudnativelabs/kube-router/blob/master/docs/bgp.md#peering-outside-the-cluster)
via one of themany means that kube-router makes available
- Overriding Next Hop
- Enabling overlays in either full mode or with nodes in different subnets

Expand All @@ -259,5 +258,5 @@ given. If this happens it may cause some network flows to become un-routable.

Specifically, people need to take care when combining `--override-nexthop` and `--enable-overlay` and make sure that
they understand their network, the flows they desire, how the kube-router logic works, and the possible side effects
that are created from their configuration. Please refer to this PR for the risk and impact discussion
https://github.com/cloudnativelabs/kube-router/pull/1025.
that are created from their configuration. Please refer to [this PR](https://github.com/cloudnativelabs/kube-router/pull/1025)
for the risk and impact discussion.
Loading

0 comments on commit dad4991

Please sign in to comment.