Skip to content

OpenSDS Integration with Kubernetes Flexvolume

Jump to bottom
Leon Wang edited this page Jun 25, 2018 · 1 revision

Prerequisite

ubuntu

  • Version information

     root@proxy:~# cat /etc/issue
 Ubuntu 16.04.2 LTS \n \l

docker

  • Version information

     root@proxy:~# docker version
 Client:
  Version:      1.12.6
  API version:  1.24
  Go version:   go1.6.2
  Git commit:   78d1802
  Built:        Tue Jan 31 23:35:14 2017
  OS/Arch:      linux/amd64
 
 Server:
  Version:      1.12.6
  API version:  1.24
  Go version:   go1.6.2
  Git commit:   78d1802
  Built:        Tue Jan 31 23:35:14 2017
  OS/Arch:      linux/amd64

golang

  • Version information

     root@proxy:~# go version
 go version go1.9.2 linux/amd64

  • You can install golang by executing commands blow:

     wget https://storage.googleapis.com/golang/go1.9.2.linux-amd64.tar.gz
 tar -C /usr/local -xzf go1.9.2.linux-amd64.tar.gz
 export PATH=$PATH:/usr/local/go/bin
 export GOPATH=$HOME/gopath

kubernetes local cluster

  • Version information

     root@proxy:~# kubectl version
 Client Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.0-beta.0-dirty", GitCommit:"a0fb3baa71f1559fd42d1acd9cbdd8a55ab4dfff", GitTreeState:"dirty", BuildDate:"2017-12-13T09:22:09Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
 Server Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.0-beta.0-dirty", GitCommit:"a0fb3baa71f1559fd42d1acd9cbdd8a55ab4dfff", GitTreeState:"dirty", BuildDate:"2017-12-13T09:22:09Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}

  • You can startup the k8s local cluster by executing commands blow:

     cd $HOME
 git clone https://github.com/kubernetes/kubernetes.git
 cd $HOME/kubernetes
 git checkout v1.9.0
 make
 echo alias kubectl='$HOME/kubernetes/cluster/kubectl.sh' >> /etc/profile
 RUNTIME_CONFIG=settings.k8s.io/v1alpha1=true AUTHORIZATION_MODE=Node,RBAC hack/local-up-cluster.sh -O

NOTE:

Due to opensds using etcd as the database which is same with kubernetes so you should startup kubernetes firstly.

opensds local cluster

Testing steps

  • Load some ENVs which is setted before.

    source /etc/profile

  • Download nbp source code.

    using git clone

     git clone https://github.com/opensds/nbp.git  $GOPATH/src/github.com/opensds/nbp

    or using go get

     go get -v  github.com/opensds/nbp/...

  • Build the FlexVolume.

     cd $GOPATH/src/github.com/opensds/nbp/flexvolume
 go build -o opensds ./cmd/flex-plugin/

    FlexVolume plugin binary is on the current directory.

  • Copy the OpenSDS FlexVolume binary file to k8s kubelet volume-plugin-dir.
    if you don't specify the volume-plugin-dir, you can execute commands blow:

     mkdir -p /usr/libexec/kubernetes/kubelet-plugins/volume/exec/opensds.io~opensds/
 cp $GOPATH/src/github.com/opensds/nbp/flexvolume/opensds /usr/libexec/kubernetes/kubelet-plugins/volume/exec/opensds.io~opensds/

    NOTE:

    OpenSDS FlexVolume will get the opensds api endpoint from the environment variable `OPENSDS_ENDPOINT`, if you don't specify it, the FlexVolume will use the default vaule: `http://127.0.0.1:50040`. if you want to specify the `OPENSDS_ENDPOINT` executing command `export OPENSDS_ENDPOINT=http://ip:50040` and restart the k8s local cluster.

    • Build the provisioner docker image.

       cd $GOPATH/src/github.com/opensds/nbp/opensds-provisioner
 make container

    • Create service account, role and bind them.

       cd $GOPATH/src/github.com/opensds/nbp/opensds-provisioner/examples
 kubectl create -f serviceaccount.yaml
 kubectl create -f clusterrole.yaml
 kubectl create -f clusterrolebinding.yaml

    • Change the opensds endpoint IP in pod-provisioner.yaml
      The IP (192.168.56.106) should be replaced with the OpenSDS osdslet actual endpoint IP.

      kind: Pod
apiVersion: v1
metadata:
  name: opensds-provisioner
spec:
  serviceAccount: opensds-provisioner
  containers:
    - name: opensds-provisioner
      image: opensdsio/opensds-provisioner
      securityContext:
      args:
        - "-endpoint=http://192.168.56.106:50040" # should be replaced
      imagePullPolicy: "IfNotPresent"

    • Create provisioner pod.

       kubectl create -f pod-provisioner.yaml

      Execute kubectl get pod to check if the opensds-provisioner is ok.

      root@nbp:~/go/src/github.com/opensds/nbp/opensds-provisioner/examples# kubectl get pod
NAME                  READY     STATUS    RESTARTS   AGE
opensds-provisioner   1/1       Running   0          42m

    • You can use the following cammands to test the OpenSDS FlexVolume and Proversioner functions.

      Create storage class.

       kubectl create -f sc.yaml              # Create StorageClass

      Execute kubectl get sc to check if the storage class is ok.

       root@nbp:~/go/src/github.com/opensds/nbp/opensds-provisioner/examples# kubectl get sc
 NAME                 PROVISIONER               AGE
 opensds              opensds/nbp-provisioner   46m
 standard (default)   kubernetes.io/host-path   49m

      Create PVC.

       kubectl create -f pvc.yaml             # Create PVC

      Execute kubectl get pvc to check if the pvc is ok.

       root@nbp:~/go/src/github.com/opensds/nbp/opensds-provisioner/examples# kubectl get pvc
 NAME          STATUS    VOLUME                                 CAPACITY   ACCESS MODES   STORAGECLASS   AGE
 opensds-pvc   Bound     731da41e-c9ee-4180-8fb3-d1f6c7f65378   1Gi        RWO            opensds        48m

      Create busybox pod.

       kubectl create -f pod-application.yaml # Create busybox pod and mount the block storage.

      Execute kubectl get pod to check if the busybox pod is ok.

      root@nbp:~/go/src/github.com/opensds/nbp/opensds-provisioner/examples# kubectl get pod
NAME                  READY     STATUS    RESTARTS   AGE
busy-pod              1/1       Running   0          49m
opensds-provisioner   1/1       Running   0          50m

      Execute the findmnt|grep opensds to confirm whether the volume has been provided. If there is some thing that goes wrong, you can check the log files in directory /var/log/opensds.

    Clean up steps

    kubectl delete -f pod-application.yaml
kubectl delete -f pvc.yaml
kubectl delete -f sc.yaml

kubectl delete -f pod-provisioner.yaml
kubectl delete -f clusterrolebinding.yaml
kubectl delete -f clusterrole.yaml
kubectl delete -f serviceaccount.yaml
Clone this wiki locally