Liveness probe memory leak

[juliohm1978]

What happened:

After upgrading to v0.6.0, we are seeing regular pod restarts, some of them being OOMKilled.

Looking at the pod metrics, we can see that the liveness probe in the node pod seems to be leaking memory over time.

Environment:

• CSI Driver version: v0.6.0
• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.4", GitCommit:"d360454c9bcd1634cf4cc52d1867af5491dc9c5f", GitTreeState:"archive", BuildDate:"2020-11-25T13:19:56Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.5", GitCommit:"e338cf2c6d297aa603b50ad3a301f761b4173aa6", GitTreeState:"clean", BuildDate:"2020-12-09T11:10:32Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}

• OS (e.g. from /etc/os-release): ubuntu 18.04 LTS
• Kernel (e.g. uname -a): Linux jhempc 5.10.13-arch1-2 #1 SMP PREEMPT Sat, 06 Feb 2021 11:07:29 +0000 x86_64 GNU/Linux
• Install tools: Kubespray

[juliohm1978]

Could this be related?

kubernetes-csi/livenessprobe#102

[juliohm1978]

A memory leak appears to have been fixed in the liveness probe image 2.2.0.

kubernetes-csi/livenessprobe#94

https://github.com/kubernetes-csi/livenessprobe/blob/master/CHANGELOG/CHANGELOG-2.2.md

[andyzhangx]

the master branch is already using v2.2.0:

csi-driver-smb/deploy/csi-smb-controller.yaml

Lines 46 to 47 in 2d3ef8a

	- name: liveness-probe
	image: k8s.gcr.io/sig-storage/livenessprobe:v2.2.0

Would you kubectl edit ds csi-driver-smb-node -n kube-system to upgrade to v2.2.0 directly in your cluster?

[juliohm1978]

Just changed my values.yaml and redeployed. Will be back to confirm if this fixes it.