Skip to content

Comments

STOR-1804: use new snapshot options from ClusterCSIDriver#226

Merged
openshift-merge-bot[bot] merged 2 commits intoopenshift:masterfrom
RomanBednar:driver-config
Apr 19, 2024
Merged

STOR-1804: use new snapshot options from ClusterCSIDriver#226
openshift-merge-bot[bot] merged 2 commits intoopenshift:masterfrom
RomanBednar:driver-config

Conversation

@RomanBednar
Copy link
Contributor

@RomanBednar RomanBednar commented Mar 7, 2024

Related PRs:

Basic smoke test:

Check current vSphere config map:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
  cloud.conf: |+
    # Labels with topology values are added dynamically via operator
    [Global]
    cluster-id = rbednar-vsphere-01-cwv8p

    [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
    insecure-flag           = true
    datacenters             = DEVQEdatacenter
    migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

kind: ConfigMap
metadata:
  creationTimestamp: "2024-03-06T09:46:40Z"
  name: vsphere-csi-config
  namespace: openshift-cluster-csi-drivers
  resourceVersion: "126687"
  uid: f6968303-81d8-4048-99c1-d8211363d0fa

Add global snapshot limit:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 10}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Verify operator reflected changes in config map:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
  cloud.conf: |+
    # Labels with topology values are added dynamically via operator
    [Global]
    cluster-id = rbednar-vsphere-01-cwv8p

    [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
    insecure-flag           = true
    datacenters             = DEVQEdatacenter
    migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

    [Snapshot]
    global-max-snapshots-per-block-volume = 10

kind: ConfigMap
metadata:
  creationTimestamp: "2024-03-06T09:46:40Z"
  name: vsphere-csi-config
  namespace: openshift-cluster-csi-drivers
  resourceVersion: "127118"
  uid: f6968303-81d8-4048-99c1-d8211363d0fa

Verify cloud.conf mounted to pod is updated correctly:

$ oc -n openshift-cluster-csi-drivers exec pod/vmware-vsphere-csi-driver-controller-6f88544d99-flnsw cat /etc/kubernetes/vsphere-csi-config/cloud.conf
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Defaulted container "csi-driver" out of: csi-driver, driver-kube-rbac-proxy, csi-provisioner, provisioner-kube-rbac-proxy, csi-attacher, attacher-kube-rbac-proxy, csi-resizer, resizer-kube-rbac-proxy, csi-snapshotter, snapshotter-kube-rbac-proxy, csi-liveness-probe, vsphere-syncer, syncer-kube-rbac-proxy
# Labels with topology values are added dynamically via operator
[Global]
cluster-id = rbednar-vsphere-01-cwv8p

[VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
insecure-flag           = true
datacenters             = DEVQEdatacenter
migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

[Snapshot]
global-max-snapshots-per-block-volume = 10

Remove the newly added option:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=json -p='[{"op": "remove", "path": "/spec/driverConfig/vSphere"}]'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Verify the config map is restored:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
  cloud.conf: |+
    # Labels with topology values are added dynamically via operator
    [Global]
    cluster-id = rbednar-vsphere-01-cwv8p

    [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
    insecure-flag           = true
    datacenters             = DEVQEdatacenter
    migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

kind: ConfigMap
metadata:
  creationTimestamp: "2024-03-06T09:46:40Z"
  name: vsphere-csi-config
  namespace: openshift-cluster-csi-drivers
  resourceVersion: "127692"
  uid: f6968303-81d8-4048-99c1-d8211363d0fa

Try to set all available config options:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 10, "granularMaxSnapshotsPerBlockVolumeInVSAN": 5, "granularMaxSnapshotsPerBlockVolumeInVVOL": 3}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Verify all options are set:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
  cloud.conf: |+
    # Labels with topology values are added dynamically via operator
    [Global]
    cluster-id = rbednar-vsphere-01-cwv8p

    [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
    insecure-flag           = true
    datacenters             = DEVQEdatacenter
    migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

    [Snapshot]
    global-max-snapshots-per-block-volume        = 10
    granular-max-snapshots-per-block-volume-vsan = 5
    granular-max-snapshots-per-block-volume-vvol = 3

kind: ConfigMap
metadata:
  creationTimestamp: "2024-03-06T09:46:40Z"
  name: vsphere-csi-config
  namespace: openshift-cluster-csi-drivers
  resourceVersion: "137738"

Clean up all config options:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=json -p='[{"op": "remove", "path": "/spec/driverConfig/vSphere"}]'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Set global snapshot limit to 5:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 5}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Create 6 snapshots and verify the 6th failed to create:

$ oc get volumesnapshot
NAME             READYTOUSE   SOURCEPVC   SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS     SNAPSHOTCONTENT                                    CREATIONTIME   AGE
pvc-1-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-24979f14-ff20-43c6-8869-c06ac1cee714   102s           107s
pvc-2-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-c14c034b-e281-4292-9893-18010a28d95f   82s            90s
pvc-3-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-4884d8aa-4656-4e5c-8b98-75a87f6b8c3a   73s            80s
pvc-4-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-800e4957-2487-4645-8b29-97f61fe84c84   62s            68s
pvc-5-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-9e753c61-bac3-4d7d-9ae0-e855089bd445   52s            60s
pvc-6-snapshot   false        pvc-1                                             csi-vsphere-vsc   snapcontent-0b1f840f-1a10-4dd8-93b9-e6e7864d0097                  40s

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 7, 2024
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Mar 7, 2024

@RomanBednar: This pull request references STOR-1804 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 7, 2024
@openshift-ci openshift-ci bot requested review from jsafrane and tsmetana March 7, 2024 13:28
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 7, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: RomanBednar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 7, 2024
@RomanBednar RomanBednar force-pushed the driver-config branch 4 times, most recently from 5ccf8cd to 69ee81c Compare March 13, 2024 15:54
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Mar 15, 2024

@RomanBednar: This pull request references STOR-1804 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

Details

In response to this:

Related PRs:

Basic smoke test:

Check current vSphere config map:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
 cloud.conf: |+
   # Labels with topology values are added dynamically via operator
   [Global]
   cluster-id = rbednar-vsphere-01-cwv8p

   [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
   insecure-flag           = true
   datacenters             = DEVQEdatacenter
   migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

kind: ConfigMap
metadata:
 creationTimestamp: "2024-03-06T09:46:40Z"
 name: vsphere-csi-config
 namespace: openshift-cluster-csi-drivers
 resourceVersion: "126687"
 uid: f6968303-81d8-4048-99c1-d8211363d0fa

Add global snapshot limit:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 10}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Verify operator reflected changes in config map:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
 cloud.conf: |+
   # Labels with topology values are added dynamically via operator
   [Global]
   cluster-id = rbednar-vsphere-01-cwv8p

   [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
   insecure-flag           = true
   datacenters             = DEVQEdatacenter
   migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

   [Snapshot]
   global-max-snapshots-per-block-volume = 10

kind: ConfigMap
metadata:
 creationTimestamp: "2024-03-06T09:46:40Z"
 name: vsphere-csi-config
 namespace: openshift-cluster-csi-drivers
 resourceVersion: "127118"
 uid: f6968303-81d8-4048-99c1-d8211363d0fa

Verify cloud.conf mounted to pod is updated correctly:

$ oc -n openshift-cluster-csi-drivers exec pod/vmware-vsphere-csi-driver-controller-6f88544d99-flnsw cat /etc/kubernetes/vsphere-csi-config/cloud.conf
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Defaulted container "csi-driver" out of: csi-driver, driver-kube-rbac-proxy, csi-provisioner, provisioner-kube-rbac-proxy, csi-attacher, attacher-kube-rbac-proxy, csi-resizer, resizer-kube-rbac-proxy, csi-snapshotter, snapshotter-kube-rbac-proxy, csi-liveness-probe, vsphere-syncer, syncer-kube-rbac-proxy
# Labels with topology values are added dynamically via operator
[Global]
cluster-id = rbednar-vsphere-01-cwv8p

[VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
insecure-flag           = true
datacenters             = DEVQEdatacenter
migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

[Snapshot]
global-max-snapshots-per-block-volume = 10

Remove the newly added option:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=json -p='[{"op": "remove", "path": "/spec/driverConfig/vSphere"}]'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Verify the config map is restored:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
 cloud.conf: |+
   # Labels with topology values are added dynamically via operator
   [Global]
   cluster-id = rbednar-vsphere-01-cwv8p

   [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
   insecure-flag           = true
   datacenters             = DEVQEdatacenter
   migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

kind: ConfigMap
metadata:
 creationTimestamp: "2024-03-06T09:46:40Z"
 name: vsphere-csi-config
 namespace: openshift-cluster-csi-drivers
 resourceVersion: "127692"
 uid: f6968303-81d8-4048-99c1-d8211363d0fa

Try to set all available config options:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 10, "granularMaxSnapshotsPerBlockVolumeInVSAN": 5, "granularMaxSnapshotsPerBlockVolumeInVVOL": 3}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Verify all options are set:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
 cloud.conf: |+
   # Labels with topology values are added dynamically via operator
   [Global]
   cluster-id = rbednar-vsphere-01-cwv8p

   [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
   insecure-flag           = true
   datacenters             = DEVQEdatacenter
   migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

   [Snapshot]
   global-max-snapshots-per-block-volume        = 10
   granular-max-snapshots-per-block-volume-vsan = 5
   granular-max-snapshots-per-block-volume-vvol = 3

kind: ConfigMap
metadata:
 creationTimestamp: "2024-03-06T09:46:40Z"
 name: vsphere-csi-config
 namespace: openshift-cluster-csi-drivers
 resourceVersion: "137738"

Clean up all config options:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=json -p='[{"op": "remove", "path": "/spec/driverConfig/vSphere"}]'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Set global snapshot limit to 5:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 5}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Create 6 snapshots and verify the 6th failed to create:

$ oc get volumesnapshot
NAME             READYTOUSE   SOURCEPVC   SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS     SNAPSHOTCONTENT                                    CREATIONTIME   AGE
pvc-1-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-24979f14-ff20-43c6-8869-c06ac1cee714   102s           107s
pvc-2-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-c14c034b-e281-4292-9893-18010a28d95f   82s            90s
pvc-3-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-4884d8aa-4656-4e5c-8b98-75a87f6b8c3a   73s            80s
pvc-4-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-800e4957-2487-4645-8b29-97f61fe84c84   62s            68s
pvc-5-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-9e753c61-bac3-4d7d-9ae0-e855089bd445   52s            60s
pvc-6-snapshot   false        pvc-1                                             csi-vsphere-vsc   snapcontent-0b1f840f-1a10-4dd8-93b9-e6e7864d0097                  40s

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@RomanBednar
Copy link
Contributor Author

Re-tested with latest openshift/api PR, looks ok:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 10}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
  cloud.conf: |
    # Labels with topology values are added dynamically via operator
    [Global]
    cluster-id = ci-ln-ddsrk9t-c1627-7ztq2

    [VirtualCenter "v8c-2-vcenter.ocp2.dev.cluster.com"]
    insecure-flag           = true
    datacenters             = IBMCloud
    migration-datastore-url = ds:///vmfs/volumes/vsan:52bbf3f0bbdd6a6a-81552b2d8f65444e/

    [Snapshot]
    global-max-snapshots-per-block-volume = 10
kind: ConfigMap
metadata:
  creationTimestamp: "2024-04-08T09:01:44Z"
  name: vsphere-csi-config
  namespace: openshift-cluster-csi-drivers
  resourceVersion: "44188"
  uid: 8b411f39-1205-439e-b324-195c8ce41071

@Phaow
Copy link
Contributor

Phaow commented Apr 8, 2024

/label qe-approved
@RomanBednar Thanks for helping use the temp api version to pre merge testing. After the API PR merged we could bump the api version to the latest offical one and merge this PR.
Pre verify passed with 4.16.0-0.test-2024-04-08-003725-ci-ln-sv9g1z2-latest (build 4.16.0-0.ci,openshift/api#1783,openshift/vmware-vsphere-csi-driver-operator#226,openshift/cluster-storage-operator#464)

1. Check the default setting should be `3`.

# Create 4 snapshots and verify the 4th failed to create:
$ oc -n default get volumesnapshots/app-pvc-snapshot-4 -oyaml
...
 error:
    message: 'Failed to check and update snapshot content: failed to take snapshot of the volume cf20c294-9af7-4722-89fd-b95921b30df0: "rpc error: code = FailedPrecondition desc = the number of snapshots on the source volume cf20c294-9af7-4722-89fd-b95921b30df0 reaches the configured maximum (3)"'
...

2. Change the `global-max-snapshots-per-block-volume = 10` and `granular-max-snapshots-per-block-volume-vsan = 5 `, check the granular-max-snapshots-per-block-volume-vsan  should overwrite the max value for vSAN volumes as expected.
$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 10}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched 
$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"granularMaxSnapshotsPerBlockVolumeInVSAN": 5}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched 

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
  cloud.conf: |
    # Labels with topology values are added dynamically via operator
    [Global]
    cluster-id = pewang-0408vv-pf8v5

    [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
    insecure-flag           = true
    datacenters             = DEVQEdatacenter
    migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

    [Snapshot]
    global-max-snapshots-per-block-volume        = 10
    granular-max-snapshots-per-block-volume-vsan = 5
kind: ConfigMap
metadata:
  creationTimestamp: "2024-04-08T01:27:45Z"
  name: vsphere-csi-config
  namespace: openshift-cluster-csi-drivers
  resourceVersion: "330478"
  uid: c6d0dac1-6ece-48d3-a0db-1932c18ff06e

# Create 6 snapshots and verify the 6th failed to create:
$ oc -n mystorage get volumesnapshots/vsan-pvc-snapshot-6 -oyaml
...
status:
  boundVolumeSnapshotContentName: snapcontent-ada27b34-f71a-42d6-8e0c-c4a4368f42c4
  error:
    message: 'Failed to check and update snapshot content: failed to take snapshot of the volume 314d491b-d972-4177-8024-04b41c9306d5: "rpc error: code = FailedPrecondition desc = the number of snapshots on the source volume 314d491b-d972-4177-8024-04b41c9306d5 reaches the configured maximum (5)"'
    time: '2024-04-08T03:27:57Z'
  readyToUse: false
...

3. Remove the configurations it should be recoverd to be the default value.

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=json -p='[{"op": "remove", "path": "/spec/driverConfig/vSphere"}]'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
  cloud.conf: |
    # Labels with topology values are added dynamically via operator
    [Global]
    cluster-id = pewang-0408vv-pf8v5

    [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
    insecure-flag           = true
    datacenters             = DEVQEdatacenter
    migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/
kind: ConfigMap
metadata:
  creationTimestamp: "2024-04-08T01:27:45Z"
  name: vsphere-csi-config
  namespace: openshift-cluster-csi-drivers
  resourceVersion: "333640"
  uid: c6d0dac1-6ece-48d3-a0db-1932c18ff06e

# Create 4 snapshots and verify the 4th failed to create:
$ oc -n default get volumesnapshots/app-pvc-snapshot-4 -oyaml
...
  error:
    message: 'Failed to check and update snapshot content: failed to take snapshot of the volume 314d491b-d972-4177-8024-04b41c9306d5: "rpc error: code = FailedPrecondition desc = the number of snapshots on the source volume 314d491b-d972-4177-8024-04b41c9306d5 reaches the configured maximum (3)"'
    time: '2024-04-08T14:20:02Z'
  readyToUse: false
...

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Apr 8, 2024
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Apr 8, 2024

@RomanBednar: This pull request references STOR-1804 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

Details

In response to this:

Related PRs:

Basic smoke test:

Check current vSphere config map:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
 cloud.conf: |+
   # Labels with topology values are added dynamically via operator
   [Global]
   cluster-id = rbednar-vsphere-01-cwv8p

   [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
   insecure-flag           = true
   datacenters             = DEVQEdatacenter
   migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

kind: ConfigMap
metadata:
 creationTimestamp: "2024-03-06T09:46:40Z"
 name: vsphere-csi-config
 namespace: openshift-cluster-csi-drivers
 resourceVersion: "126687"
 uid: f6968303-81d8-4048-99c1-d8211363d0fa

Add global snapshot limit:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 10}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Verify operator reflected changes in config map:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
 cloud.conf: |+
   # Labels with topology values are added dynamically via operator
   [Global]
   cluster-id = rbednar-vsphere-01-cwv8p

   [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
   insecure-flag           = true
   datacenters             = DEVQEdatacenter
   migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

   [Snapshot]
   global-max-snapshots-per-block-volume = 10

kind: ConfigMap
metadata:
 creationTimestamp: "2024-03-06T09:46:40Z"
 name: vsphere-csi-config
 namespace: openshift-cluster-csi-drivers
 resourceVersion: "127118"
 uid: f6968303-81d8-4048-99c1-d8211363d0fa

Verify cloud.conf mounted to pod is updated correctly:

$ oc -n openshift-cluster-csi-drivers exec pod/vmware-vsphere-csi-driver-controller-6f88544d99-flnsw cat /etc/kubernetes/vsphere-csi-config/cloud.conf
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Defaulted container "csi-driver" out of: csi-driver, driver-kube-rbac-proxy, csi-provisioner, provisioner-kube-rbac-proxy, csi-attacher, attacher-kube-rbac-proxy, csi-resizer, resizer-kube-rbac-proxy, csi-snapshotter, snapshotter-kube-rbac-proxy, csi-liveness-probe, vsphere-syncer, syncer-kube-rbac-proxy
# Labels with topology values are added dynamically via operator
[Global]
cluster-id = rbednar-vsphere-01-cwv8p

[VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
insecure-flag           = true
datacenters             = DEVQEdatacenter
migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

[Snapshot]
global-max-snapshots-per-block-volume = 10

Remove the newly added option:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=json -p='[{"op": "remove", "path": "/spec/driverConfig/vSphere"}]'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Verify the config map is restored:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
 cloud.conf: |+
   # Labels with topology values are added dynamically via operator
   [Global]
   cluster-id = rbednar-vsphere-01-cwv8p

   [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
   insecure-flag           = true
   datacenters             = DEVQEdatacenter
   migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

kind: ConfigMap
metadata:
 creationTimestamp: "2024-03-06T09:46:40Z"
 name: vsphere-csi-config
 namespace: openshift-cluster-csi-drivers
 resourceVersion: "127692"
 uid: f6968303-81d8-4048-99c1-d8211363d0fa

Try to set all available config options:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 10, "granularMaxSnapshotsPerBlockVolumeInVSAN": 5, "granularMaxSnapshotsPerBlockVolumeInVVOL": 3}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Verify all options are set:

$ oc -n openshift-cluster-csi-drivers get cm/vsphere-csi-config -o yaml
apiVersion: v1
data:
 cloud.conf: |+
   # Labels with topology values are added dynamically via operator
   [Global]
   cluster-id = rbednar-vsphere-01-cwv8p

   [VirtualCenter "vcenter.devqe.ibmc.devcluster.openshift.com"]
   insecure-flag           = true
   datacenters             = DEVQEdatacenter
   migration-datastore-url = ds:///vmfs/volumes/vsan:527320283a8c3163-2faa6dc5949a3a28/

   [Snapshot]
   global-max-snapshots-per-block-volume        = 10
   granular-max-snapshots-per-block-volume-vsan = 5
   granular-max-snapshots-per-block-volume-vvol = 3

kind: ConfigMap
metadata:
 creationTimestamp: "2024-03-06T09:46:40Z"
 name: vsphere-csi-config
 namespace: openshift-cluster-csi-drivers
 resourceVersion: "137738"

Clean up all config options:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=json -p='[{"op": "remove", "path": "/spec/driverConfig/vSphere"}]'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Set global snapshot limit to 5:

$ oc patch clustercsidriver/csi.vsphere.vmware.com --type=merge -p '{"spec":{"driverConfig":{"vSphere":{"globalMaxSnapshotsPerBlockVolume": 5}}}}'
clustercsidriver.operator.openshift.io/csi.vsphere.vmware.com patched

Create 6 snapshots and verify the 6th failed to create:

$ oc get volumesnapshot
NAME             READYTOUSE   SOURCEPVC   SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS     SNAPSHOTCONTENT                                    CREATIONTIME   AGE
pvc-1-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-24979f14-ff20-43c6-8869-c06ac1cee714   102s           107s
pvc-2-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-c14c034b-e281-4292-9893-18010a28d95f   82s            90s
pvc-3-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-4884d8aa-4656-4e5c-8b98-75a87f6b8c3a   73s            80s
pvc-4-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-800e4957-2487-4645-8b29-97f61fe84c84   62s            68s
pvc-5-snapshot   true         pvc-1                               1Gi           csi-vsphere-vsc   snapcontent-9e753c61-bac3-4d7d-9ae0-e855089bd445   52s            60s
pvc-6-snapshot   false        pvc-1                                             csi-vsphere-vsc   snapcontent-0b1f840f-1a10-4dd8-93b9-e6e7864d0097                  40s

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@RomanBednar RomanBednar changed the title WIP: STOR-1804: use new snapshot options from ClusterCSIDriver STOR-1804: use new snapshot options from ClusterCSIDriver Apr 16, 2024
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 16, 2024
@RomanBednar
Copy link
Contributor Author

/retest-required

1 similar comment
@RomanBednar
Copy link
Contributor Author

/retest-required

@jsafrane
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 17, 2024
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Apr 18, 2024
@RomanBednar
Copy link
Contributor Author

/label px-approved
/label docs-approved

@openshift-ci openshift-ci bot added px-approved Signifies that Product Support has signed off on this PR docs-approved Signifies that Docs has signed off on this PR labels Apr 18, 2024
@RomanBednar
Copy link
Contributor Author

/retest-required

1 similar comment
@RomanBednar
Copy link
Contributor Author

/retest-required

@Phaow
Copy link
Contributor

Phaow commented Apr 19, 2024

/test e2e-vsphere-ovn-upgrade

2 similar comments
@Phaow
Copy link
Contributor

Phaow commented Apr 19, 2024

/test e2e-vsphere-ovn-upgrade

@Phaow
Copy link
Contributor

Phaow commented Apr 19, 2024

/test e2e-vsphere-ovn-upgrade

@jsafrane
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 19, 2024
@jsafrane
Copy link
Contributor

/override ci/prow/e2e-vsphere-ovn-upgrade
Tracked in OCPBUGS-29126

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 19, 2024

@jsafrane: Overrode contexts on behalf of jsafrane: ci/prow/e2e-vsphere-ovn-upgrade

Details

In response to this:

/override ci/prow/e2e-vsphere-ovn-upgrade
Tracked in OCPBUGS-29126

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 19, 2024

@RomanBednar: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit b518a76 into openshift:master Apr 19, 2024
@openshift-bot
Copy link

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-vmware-vsphere-csi-driver-operator-container-v4.16.0-202404191609.p0.gb518a76.assembly.stream.el9 for distgit ose-vmware-vsphere-csi-driver-operator.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. docs-approved Signifies that Docs has signed off on this PR jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. px-approved Signifies that Product Support has signed off on this PR qe-approved Signifies that QE has signed off on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants