Skip to content

Commit b805c1f

Browse files
humblechumblec
committed
Kubernetes 1.27: CSI node expand secret support moves to Beta
1 parent ef2f738 commit b805c1f

File tree

1 file changed

+169
-0
lines changed

1 file changed

+169
-0
lines changed

‎content/en/blog/_posts/2023-04-27-node-expand-secret-moves-beta‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
+---
2+
+layout: blog
3+
+title: "Kubernetes 1.27: CSI node expand secret support moves to Beta"
4+
+date: 2023-04-27
5+
+slug: csi-node-expand-secret-support-beta
6+
+ ---
7+
+
8+
+**Author:** Humble Chirammal, Louis Koo
9+
10+
In Kubernetes v1.27, support for authenticating during CSI storage resize operations has moved
11+
from alpha to beta. That feature was originally introduced as alpha in Kubernetes v1.25. This post
12+
summarizes the changes that accompany the graduation to beta.
13+
14+
## Authentication secrets for CSI storage resizing
15+
16+
Kubernetes uses [CSI](https://kubernetes-csi.github.io) to integrate with third party storage.
17+
That integration lets your cluster cluster grow storage volumes created or managed by the CSI driver.
18+
The authentication secret support for resizes, now beta, lets you expand volumes
19+
even in scenarios where the expansion operation of the underlying storage has to
20+
make use of credentials to perform the actual operation in the backend cluster.
21+
For example, you might need to provide a credential for accessing a SAN/NAS fabric.
22+
Without this feature, it was a shortcoming for the CSI drivers to expand a volume at _node_ level
23+
in cases where the storage component requires authentication for resize operations;
24+
there was no straightforward way for the CSI driver to receive the credentials as part of the
25+
node-driven resize.
26+
27+
It is worth mentioning that, it is not only the node level expansion operation which was problematic.
28+
Within SIG Storage, we have seen use cases like below:
29+
30+
At times, the CSI driver needs to check the actual size of the backend block storage (or image) before proceeding with a node-level filesystem expand operation. This avoids false positive returns from the backend storage cluster during file system expansion.
31+
When a PersistentVolume represents encrypted block storage (for example using LUKS) you need to provide a passphrase in order to expand the device, and also to make it possible to grow the filesystem on that device.
32+
33+
## What's new in the beta?
34+
35+
With the promotion to beta, the feature is now enabled by default. That means you usually do not need
36+
to enable the feature gate in the control plane components - which was required for the alpha.
37+
As part of the graduation to beta, the support has also added in the CSI `external-provisioner` sidecar
38+
controller.
39+
You need to be running the external provisioner sidecar controller at v3.3.0 or above to take advantage
40+
of this feature.
41+
42+
## How do I use authenticated CSI storage resizing?
43+
44+
Assuming all the required components (including CSI driver) are already deployed and running on your cluster,
45+
and you have a CSI Driver that supports resizing, you can try a `NodeExpansion` operation on a CSI volume.
46+
47+
The credentials for that CSI `NodeExpand` operation can be provided as Kubernetes
48+
[Secret](/docs/concepts/configuration/secret/) object; you specify which Secret via the
49+
StorageClass.
50+
51+
Here is an example manifest for a Secret that holds credentials:
52+
53+
manifest for a Secret that holds credentials:
54+
55+
```yaml
56+
57+
---
58+
apiVersion: v1
59+
kind: Secret
60+
metadata:
61+
name: test-secret
62+
namespace: default
63+
data:
64+
stringData:
65+
username: admin
66+
password: t0p-Secret
67+
68+
```
69+
70+
Here's an example manifest for a StorageClass that refers to those credentials:
71+
72+
```yaml
73+
74+
---
75+
76+
apiVersion: storage.k8s.io/v1
77+
kind: StorageClass
78+
metadata:
79+
name: csi-blockstorage-sc
80+
parameters:
81+
csi.storage.k8s.io/node-expand-secret-name: test-secret # the name of the Secret
82+
csi.storage.k8s.io/node-expand-secret-namespace: default # the namespace that the Secret is in
83+
provisioner: blockstorage.cloudprovider.example
84+
reclaimPolicy: Delete
85+
volumeBindingMode: Immediate
86+
allowVolumeExpansion: true
87+
```
88+
89+
If the PersistentVolumeClaim (PVC) was created successfully, you can see that configuration
90+
within the `.spec.csi` field of the PersistentVolume (look for `.spec.csi.nodeExpandSecretRef`).
91+
Check that it worked by running `kubectl get persistentvolume <pv_name> -o yaml`. You should see something like:
92+
93+
```yaml
94+
95+
---
96+
apiVersion: v1
97+
kind: PersistentVolume
98+
metadata:
99+
annotations:
100+
pv.kubernetes.io/provisioned-by: blockstorage.cloudprovider.example
101+
creationTimestamp: "2023-02-26T13:14:07Z"
102+
finalizers:
103+
- kubernetes.io/pv-protection
104+
name: pvc-95eb531a-d675-49f6-940b-9bc3fde83eb0
105+
resourceVersion: “178817"
106+
uid: 6fa824d7-8a06-4e0c-b722-d3f897dcbd65
107+
spec:
108+
accessModes:
109+
- ReadWriteOnce
110+
capacity:
111+
storage: 6Gi
112+
claimRef:
113+
apiVersion: v1
114+
kind: PersistentVolumeClaim
115+
name: csi-pvc
116+
namespace: default
117+
resourceVersion: "178817"
118+
uid: 95eb531a-d675-49f6-940b-9bc3fde83eb0
119+
csi:
120+
driver: blockstorage.cloudprovider.example
121+
nodeExpandSecretRef:
122+
name: test-secret
123+
namespace: default
124+
volumeAttributes:
125+
storage.kubernetes.io/csiProvisionerIdentity: 1648042783218-8081-blockstorage.cloudprovider.example
126+
volumeHandle: e21c7809-aabb-11ec-917a-2e2e254eb4cf
127+
nodeAffinity:
128+
required:
129+
nodeSelectorTerms:
130+
- matchExpressions:
131+
- key: topology.hostpath.csi/node
132+
operator: In
133+
values:
134+
- racknode01
135+
persistentVolumeReclaimPolicy: Delete
136+
storageClassName: csi-blockstorage-sc
137+
volumeMode: Filesystem
138+
status:
139+
phase: Bound
140+
141+
```
142+
If you then trigger online storage expansion, the kubelet passes the appropriate credentials to the CSI driver, by loading that Secret and passing the data to the storage driver.
143+
Here's an example debug log:
144+
145+
```
146+
147+
I0330 03:29:51.966241 1 server.go:101] GRPC call: /csi.v1.Node/NodeExpandVolume
148+
I0330 03:29:51.966261 1 server.go:105] GRPC request: {"capacity_range":{"required_bytes":7516192768},"secrets":"***stripped***","staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/blockstorage.cloudprovider.example/f7c62e6e08ce21e9b2a95c841df315ed4c25a15e91d8fcaf20e1c2305e5300ab/globalmount","volume_capability":{"AccessType":{"Mount":{}},"access_mode":{"mode":7}},"volume_id":"e21c7809-aabb-11ec-917a-2e2e254eb4cf","volume_path":"/var/lib/kubelet/pods/bcb1b2c4-5793-425c-acf1-47163a81b4d7/volumes/kubernetes.io~csi/pvc-95eb531a-d675-49f6-940b-9bc3fde83eb0/mount"}
149+
150+
I0330 03:29:51.966360 1 nodeserver.go:459] req:volume_id:"e21c7809-aabb-11ec-917a-2e2e254eb4cf" volume_path:"/var/lib/kubelet/pods/bcb1b2c4-5793-425c-acf1-47163a81b4d7/volumes/kubernetes.io~csi/pvc-95eb531a-d675-49f6-940b-9bc3fde83eb0/mount" capacity_range:<required_bytes:7516192768 > staging_target_path:"/var/lib/kubelet/plugins/kubernetes.io/csi/blockstorage.cloudprovider.example/f7c62e6e08ce21e9b2a95c841df315ed4c25a15e91d8fcaf20e1c2305e5300ab/globalmount" volume_capability:<mount:<> access_mode:<mode:SINGLE_NODE_MULTI_WRITER > > secrets:<key:"XXXXXX" value:"XXXXX" > secrets:<key:"XXXXX" value:"XXXXXX" >
151+
152+
```
153+
154+
## Future plan
155+
156+
Depending on feedback and adoption, the Kubernetes team plans to push the CSI NodeExpandSecret implementation to GA in either 1.28 or 1.29.
157+
158+
159+
## Want to get involved or learn more?
160+
161+
The enhancement proposal includes lots of detail about the history and technical implementation of this feature.
162+
163+
To learn more about StorageClass based dynamic provisioning in Kubernetes, please refer
164+
to [storage class](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#class)
165+
and to the overall [PersistentVolumes](/docs/concepts/storage/persistent-volumes/)
166+
documentation.
167+
168+
Please get involved by joining the Kubernetes Storage SIG (Special Interest Group) to help us enhance this feature. There are a lot of good ideas already and we'd be thrilled to have more!
169+

0 commit comments

Comments
 (0)