After the operator was upgraded from v0.47.3 to v0.48.0, all pods in the original cr vmcluster restarted

[pity77]

Hi bro!

I installed victoria-metrics-operator-0.34.8.tgz (operator:v0.47.3) using Helm and created a vmcluster. Everything was working fine at that point. However, this afternoon I noticed that a new version of the operator (v0.48.0) was available.

After upgrading, I found that all the pods in mzc/vmcluster-01 were restarted, and some containers were in ImagePullBackOff status because my images are stored in a private registry, and some of the worker nodes didn't have the images cached.

Upon running kubectl describe, I discovered that the vmcluster.spec.imagePullSecrets configuration was missing, and the StatefulSet also lacked this configuration.

I suspect that after upgrading the operator, it performed a validation check on the vmcluster, which resulted in the loss of the vmcluster.spec.imagePullSecrets configuration. This may have triggered the restart mechanism for both the StatefulSet and Deployment.

After my test, adding or subtracting imagePullSecrets in kubernetes v1.27.7 does cause StatefulSet and Deployment to restart its pods.

Looking forward to your reply.
[email protected]

Below is my vmcluster configuration : )
---
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMCluster
metadata:
  name: vmcluster-01
  namespace: mzc
spec:
  imagePullSecrets: 
    - name: "2000102274.private.registrykey"
  clusterVersion: "v1.103.0-cluster"
  retentionPeriod: "1"
  replicationFactor: 2
  vmstorage:
    replicaCount: 2
    storageDataPath: "/vm-data"
    image:
      repository: "hub-cn-beijing-6.kce.ksyun.com/victoriametrics/vmstorage"
    storage:
      volumeClaimTemplate:
        spec:
          resources:
            requests:
              storage: "10Gi"
    resources:
      limits:
        cpu: "0.5"
        memory: "500Mi"
  vmselect:
    replicaCount: 2
    cacheMountPath: "/select-cache"
    image:
      repository: "hub-cn-beijing-6.kce.ksyun.com/victoriametrics/vmselect"
    storage:
      volumeClaimTemplate:
        spec:
          resources:
            requests:
              storage: "10Gi"
    resources:
      limits:
        cpu: "0.5"
        memory: "500Mi"
      requests:
        cpu: "0.5"
        memory: "500Mi"
  vminsert:
    replicaCount: 2
    image:
      repository: "hub-cn-beijing-6.kce.ksyun.com/victoriametrics/vminsert"
    resources:
      limits:
        cpu: "0.5"
        memory: "500Mi"
      requests:
        cpu: "0.5"
        memory: "500Mi"

[f41gh7]

Thanks for reporting! The issue will be fixed soon.

[f41gh7]

Issue was fixed at v0.48.3 release.