PVC mount support #462

bacongobbler · 2020-11-30T20:00:08Z

Adds support for mounting PersistentVolumeClaims to a Pod via the Container Storage Interface.

Closes #182

bacongobbler · 2021-01-20T23:40:04Z

I got everything working here according to the CSI examples. I DO NOT expect anyone else to be able to reproduce the same environment, but if you want to try giving it a whirl... Here goes:

compile the following projects from HEAD, and move them into your PATH:
a. https://github.com/kubernetes-csi/csi-driver-host-path
b. https://github.com/kubernetes-csi/node-driver-registrar
c. https://github.com/kubernetes-csi/external-provisioner
In separate terminals spawn the following:

# hostpathplugin will need root permissions to execute 'mount'
# change your node ID to the name of your Krustlet node. In my case it's 'ultralisk'
./hostpathplugin --nodeid ultralisk --endpoint unix://csi/csi.sock -v 5
./csi-node-driver-registrar --csi-address /csi/csi.sock --kubelet-registration-path /csi/csi.sock --plugin-registration-path ~/.krustlet/plugins
./csi-provisioner --csi-address /csi/csi.sock --kubeconfig ~/.kube/config

Start Krustlet. Approve the bootstrap TLS certificate.
Create the storage class: kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-host-path/master/examples/csi-storageclass.yaml
Create the PVC: kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-host-path/master/examples/csi-pvc.yaml
Create the modified k8s.yaml example in demos/wasi

bacongobbler · 2021-01-20T23:42:46Z

Note: this is blocked until kflansburg/k8s-csi#1 is merged and a new version of k8s-csi is released, as it will not compile until it's been re-compiled against tonic 0.3. I've pointed Krustlet at a temporary fork for the time being.

thomastaylor312

This is looking pretty good! We definitely should document all the steps required (basically at least the components needed) and a bug "THIS IS VERY ALPHA" label somewhere

crates/kubelet/src/provider/mod.rs

crates/kubelet/src/volume/mod.rs

crates/kubelet/src/volume/persistentvolumeclaim.rs

crates/kubelet/src/volume/secret.rs

demos/wasi/hello-world-rust/k8s.yaml

kflansburg

I don't understand this enough to approve but had some comments.

crates/kubelet/Cargo.toml

crates/kubelet/src/provider/mod.rs

crates/kubelet/src/volume/persistentvolumeclaim.rs

crates/kubelet/src/volume/mod.rs

crates/kubelet/src/volume/persistentvolumeclaim.rs

itowlson

I have basically no idea whether this is right or wrong, but I sure do have a lot of opinions about how to organise code I don't understand!

crates/kubelet/Cargo.toml

crates/kubelet/src/kubelet.rs

crates/kubelet/src/provider/mod.rs

crates/kubelet/src/volume/mod.rs

crates/kubelet/src/volume/configmap.rs

crates/kubelet/src/volume/persistentvolumeclaim.rs

thomastaylor312

Docs look really good. I realized we were missing some things this second go around

crates/kubelet/src/volume/configmap.rs

crates/kubelet/src/volume/mod.rs

crates/kubelet/src/volume/persistentvolumeclaim.rs

docs/howto/csi.md

This is a simple change that allows the registrar to be built for MacOS. All of the `unix` package being used works for Mac as well. The reason for enabling this is completely selfish I'll admit. We are adding [CSI Support](krustlet/krustlet#462) to the Krustlet project, which supports Mac as well. As most CSI implementations don't support registering themselves with the Kubelet directly, we wanted to unblock any users to need to register their plugins with a Krustlet Kubelet. This change shouldn't affect normal container-based usage of the registrar

thomastaylor312 · 2021-01-28T23:26:13Z

Looks like the doc tests are failing @bacongobbler. I also am seeing a lot of clippy errors, but I can fix those later

bacongobbler · 2021-01-29T01:26:44Z

Oh yeah I see that. Never run the doc tests locally... My laptop is having a hard enough time as it is with just cargo build lol. Will fix those right up!

Thanks for catching the clippy errors as well.

thomastaylor312 · 2021-01-29T18:03:36Z

One last doc add I just thought of. In some clusters, the system:nodes group doesn't have access to read StorageClass objects. So if the Pod is erroring saying that the kubelet doesn't have access, you'll have to create the following cluster role and role binding to allow the nodes group to access them:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: storageclass-reader
rules:
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: node-storageclass-reader
subjects:
  - kind: Group
    name: system:nodes
    apiGroup: rbac.authorization.k8s.io
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: storageclass-reader

Signed-off-by: Matthew Fisher <matt.fisher@microsoft.com>

bacongobbler · 2021-01-29T18:07:15Z

Good call. I just added it to the HOWTO doc as an addendum. Let me know if you'd like me to make further changes/tweaks there.

bacongobbler · 2021-02-01T16:34:22Z

Thanks @thomastaylor312!

@itowlson are there any further changes you'd like me to address here?

itowlson

LGTM

This is a simple change that allows the registrar to be built for MacOS. All of the `unix` package being used works for Mac as well. The reason for enabling this is completely selfish I'll admit. We are adding [CSI Support](krustlet/krustlet#462) to the Krustlet project, which supports Mac as well. As most CSI implementations don't support registering themselves with the Kubelet directly, we wanted to unblock any users to need to register their plugins with a Krustlet Kubelet. This change shouldn't affect normal container-based usage of the registrar

bacongobbler force-pushed the pvc-mount-support branch 7 times, most recently from 42c0fb5 to ee2dcdf Compare January 12, 2021 22:57

bacongobbler marked this pull request as ready for review January 12, 2021 23:17

bacongobbler force-pushed the pvc-mount-support branch from ee2dcdf to 0ba4c5f Compare January 12, 2021 23:22

bacongobbler force-pushed the pvc-mount-support branch from 0ba4c5f to ecf4658 Compare January 20, 2021 23:33

bacongobbler force-pushed the pvc-mount-support branch from ecf4658 to 92c3d07 Compare January 20, 2021 23:41

thomastaylor312 requested changes Jan 25, 2021

View reviewed changes

kflansburg reviewed Jan 25, 2021

View reviewed changes

itowlson suggested changes Jan 26, 2021

View reviewed changes

bacongobbler force-pushed the pvc-mount-support branch 14 times, most recently from 894c7ca to 8559b4b Compare January 28, 2021 17:10

bacongobbler force-pushed the pvc-mount-support branch 3 times, most recently from bb4a0ca to 638031b Compare January 28, 2021 17:15

thomastaylor312 requested changes Jan 28, 2021

View reviewed changes

crates/kubelet/src/volume/configmap.rs Outdated Show resolved Hide resolved

crates/kubelet/src/volume/mod.rs Outdated Show resolved Hide resolved

crates/kubelet/src/volume/persistentvolumeclaim.rs Show resolved Hide resolved

docs/howto/csi.md Outdated Show resolved Hide resolved

thomastaylor312 mentioned this pull request Jan 28, 2021

feat(util): Allows building for darwin targets kubernetes-csi/node-driver-registrar#133

Merged

bacongobbler force-pushed the pvc-mount-support branch 9 times, most recently from 480dd32 to 9acdac1 Compare January 28, 2021 22:44

bacongobbler force-pushed the pvc-mount-support branch from 9acdac1 to a27b60d Compare January 29, 2021 17:58

PVC mount support

e42772c

Signed-off-by: Matthew Fisher <matt.fisher@microsoft.com>

bacongobbler force-pushed the pvc-mount-support branch from a27b60d to e42772c Compare January 29, 2021 18:06

thomastaylor312 approved these changes Jan 29, 2021

View reviewed changes

itowlson approved these changes Feb 1, 2021

View reviewed changes

bacongobbler merged commit b720389 into krustlet:master Feb 1, 2021

bacongobbler deleted the pvc-mount-support branch February 1, 2021 19:46

PVC mount support #462

PVC mount support #462

Uh oh!

Conversation

bacongobbler commented Nov 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bacongobbler commented Jan 20, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bacongobbler commented Jan 20, 2021

Uh oh!

thomastaylor312 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kflansburg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

itowlson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thomastaylor312 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thomastaylor312 commented Jan 28, 2021

Uh oh!

bacongobbler commented Jan 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thomastaylor312 commented Jan 29, 2021

Uh oh!

bacongobbler commented Jan 29, 2021

Uh oh!

bacongobbler commented Feb 1, 2021

Uh oh!

itowlson left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bacongobbler commented Nov 30, 2020 •

edited

Loading

bacongobbler commented Jan 20, 2021 •

edited

Loading

bacongobbler commented Jan 29, 2021 •

edited

Loading