Skip to content

Commit

Permalink
Merge pull request #5 from didil/4-config-crd
Browse files Browse the repository at this point in the history
Replace config annotations with config CRD
  • Loading branch information
didil authored Feb 24, 2023
2 parents 8c98cd9 + fe0e8c1 commit b470f51
Show file tree
Hide file tree
Showing 20 changed files with 696 additions and 68 deletions.
8 changes: 8 additions & 0 deletions PROJECT
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,12 @@ resources:
kind: LoadBalancer
path: github.com/didil/paperlb/api/v1alpha1
version: v1alpha1
- api:
crdVersion: v1
namespaced: true
domain: paperlb.com
group: lb
kind: LoadBalancerConfig
path: github.com/didil/paperlb/api/v1alpha1
version: v1alpha1
version: "3"
66 changes: 52 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ PaperLB is implemented as a kubernetes "Operator":

The idea is:

- You create a Kubernetes LoadBalancer type service and add some PaperLB annotations
- The controller notices the service and annotations and creates a "LoadBalancer" object
- The controller notices the "LoadBalancer" object and updates your network load balancer using the config data from the annotations + the service/nodes info
- You create a Kubernetes LoadBalancer type service and a LoadBalancerConfig configuration object
- The controller notices the service and LoadBalancerConfig and creates a "LoadBalancer" object
- The controller notices the "LoadBalancer" object and updates your network load balancer using the config data + the service/nodes info

## Features
- Works with TCP or UDP L4 load balancers
Expand All @@ -49,6 +49,7 @@ You’ll need a kubernetes cluster to run against. You can use a local cluster f
**Note:** Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster `kubectl cluster-info` shows).

### Usage
You can find the full example in the demo/ directory
Service example:
````yaml
apiVersion: v1
Expand All @@ -57,11 +58,9 @@ metadata:
labels:
app: k8s-pod-info-api
name: k8s-pod-info-api-service
annotations:
lb.paperlb.com/http-updater-url: "http://192.168.64.1:3000/api/v1/lb"
lb.paperlb.com/load-balancer-host: "192.168.64.1"
lb.paperlb.com/load-balancer-port: "8100"
lb.paperlb.com/load-balancer-protocol: "TCP"
#optional annotation to use a config different than the default config
#annotations:
# lb.paperlb.com/config-name: "my-special-config"
spec:
ports:
- port: 5000
Expand All @@ -70,14 +69,53 @@ spec:
selector:
app: k8s-pod-info-api
type: LoadBalancer
````

LoadBalancerConfig example:
````yaml
apiVersion: lb.paperlb.com/v1alpha1
kind: LoadBalancerConfig
metadata:
name: default-lb-config
namespace: paperlb-system
spec:
default: true
httpUpdaterURL: "http://192.168.64.1:3000/api/v1/lb"
host: "192.168.64.1"
portRange:
low: 8100
high: 8200
````

LoadBalancerConfig fields:
- `.spec.default`: "true" if this should be the default config, false otherwise
- `.spec.httpUpdaterURL`: URL where the http lb updater instance can be called. The API is explained here: https://github.com/didil/nginx-lb-updater#api
- `.spec.host`: Load Balancer Host
- `.spec.portRange`: The controller will select a load balancer port from this range
- `.spec.portRange.low`: Lowest of the available ports on the load balancer
- `.spec.portRange.high`: Highest of the available ports on the load balancer

When you apply these manifests, a load balancer resource should be created. To get the load balancer connection info you can run:
````bash
$ k get loadbalancer k8s-pod-info-api-service
NAME HOST PORT PROTOCOL TARGETCOUNT STATUS
k8s-pod-info-api-service 192.168.64.1 8100 TCP 2 READY
````

Annotations:
- `lb.paperlb.com/http-updater-url`: URL where the http lb updater instance can be called. The API is explained here: https://github.com/didil/nginx-lb-updater#api
- `lb.paperlb.com/load-balancer-host`: Load Balancer Host
- `lb.paperlb.com/load-balancer-port`: Load Balancer Port
- `lb.paperlb.com/load-balancer-protocol`: Load Balancer Protocol (`TCP` or `UDP`)

Testing with Curl
````bash
$ curl -s 192.168.64.1:8100/api/v1/info|jq
"pod": {
"name": "k8s-pod-info-api-84dc7c9bdd-mz74t",
"ip": "10.42.0.27",
"namespace": "default",
"serviceAccountName": "default"
},
"node": {
"name": "k3s-local-server"
}
}
````


### Run tests
Expand Down
3 changes: 3 additions & 0 deletions api/v1alpha1/loadbalancer_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ import (

// LoadBalancerSpec defines the desired state of LoadBalancer
type LoadBalancerSpec struct {
// ConfigName is the loadbalancer config name
// +kubebuilder:validation:Required
ConfigName string `json:"configName,omitempty"`
// HTTPUpdater is the http updater
// +kubebuilder:validation:Required
HTTPUpdater HTTPUpdater `json:"httpUpdater,omitempty"`
Expand Down
80 changes: 80 additions & 0 deletions api/v1alpha1/loadbalancerconfig_types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
/*
Copyright 2023.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha1

import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.

// LoadBalancerConfigSpec defines the desired state of LoadBalancerConfig
type LoadBalancerConfigSpec struct {
// Default defines if this config is the default config
// +kubebuilder:validation:Required
Default bool `json:"default,omitempty"`
// HTTPUpdaterURL is the http updater url
// +kubebuilder:validation:Required
HTTPUpdaterURL string `json:"httpUpdaterURL,omitempty"`
// Host is the load balancer host
// +kubebuilder:validation:Required
Host string `json:"host,omitempty"`
// PortRange is the load balancer port range
// +kubebuilder:validation:Required
PortRange PortRange `json:"portRange,omitempty"`
}

// PortRange defines the load balancer port range
type PortRange struct {
// Low is the lower limit of the port range
// +kubebuilder:validation:Required
// +kubebuilder:validation:Minimum=1
Low int `json:"low,omitempty"`
// High is the higher limit of the port range
// +kubebuilder:validation:Required
// +kubebuilder:validation:Maximum=65535
High int `json:"high,omitempty"`
}

// LoadBalancerConfigStatus defines the observed state of LoadBalancerConfig
type LoadBalancerConfigStatus struct {
}

//+kubebuilder:object:root=true
//+kubebuilder:subresource:status

// LoadBalancerConfig is the Schema for the loadbalancerconfigs API
type LoadBalancerConfig struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

Spec LoadBalancerConfigSpec `json:"spec,omitempty"`
Status LoadBalancerConfigStatus `json:"status,omitempty"`
}

//+kubebuilder:object:root=true

// LoadBalancerConfigList contains a list of LoadBalancerConfig
type LoadBalancerConfigList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []LoadBalancerConfig `json:"items"`
}

func init() {
SchemeBuilder.Register(&LoadBalancerConfig{}, &LoadBalancerConfigList{})
}
105 changes: 105 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

68 changes: 68 additions & 0 deletions config/crd/bases/lb.paperlb.com_loadbalancerconfigs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.11.1
creationTimestamp: null
name: loadbalancerconfigs.lb.paperlb.com
spec:
group: lb.paperlb.com
names:
kind: LoadBalancerConfig
listKind: LoadBalancerConfigList
plural: loadbalancerconfigs
singular: loadbalancerconfig
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: LoadBalancerConfig is the Schema for the loadbalancerconfigs
API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: LoadBalancerConfigSpec defines the desired state of LoadBalancerConfig
properties:
default:
description: Default defines if this config is the default config
type: boolean
host:
description: Host is the load balancer host
type: string
httpUpdaterURL:
description: HTTPUpdaterURL is the http updater url
type: string
portRange:
description: PortRange is the load balancer port range
properties:
high:
description: High is the higher limit of the port range
maximum: 65535
type: integer
low:
description: Low is the lower limit of the port range
minimum: 1
type: integer
type: object
type: object
status:
description: LoadBalancerConfigStatus defines the observed state of LoadBalancerConfig
type: object
type: object
served: true
storage: true
subresources:
status: {}
3 changes: 3 additions & 0 deletions config/crd/bases/lb.paperlb.com_loadbalancers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,9 @@ spec:
spec:
description: LoadBalancerSpec defines the desired state of LoadBalancer
properties:
configName:
description: ConfigName is the loadbalancer config name
type: string
host:
description: Host is the lb host
type: string
Expand Down
Loading

0 comments on commit b470f51

Please sign in to comment.