[CORS-2666] Add ControlPlaneMachineSet for Nutanix

[thunderboltsid]

Set ControlPlaneMachineSet to Active. This generates a ControlPlaneMachineSet manifest in the openshift directory when running create manifests command.

How was this tested?

1. Create a CPMSO image with the changes in cluster-control-plane-machine-set-operator#200
2. Create a new release image based on a recent 4.14 release image using oc adm release new --from-release registry.ci.openshift.org/ocp/release:4.14.0-0.nightly-2023-04-19-125337 cluster-control-plane-machine-set-operator=image-registry:cluster-control-plane-machine-set-operator --to-image image-registry:release and set the release image override environment variable to the newly created release image using export OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=image-registry:release
3. Create a install-config using openshift-installer create install-config
4. Run openshift-installer create manifests and verify ControlPlaneMachineSet manifest generated is as expected

apiVersion: machine.openshift.io/v1
kind: ControlPlaneMachineSet
metadata:
  creationTimestamp: null
  labels:
    machine.openshift.io/cluster-api-cluster: openshiftsid-wvs7z
  name: cluster
  namespace: openshift-machine-api
spec:
  replicas: 3
  selector:
    matchLabels:
      machine.openshift.io/cluster-api-cluster: openshiftsid-wvs7z
      machine.openshift.io/cluster-api-machine-role: master
      machine.openshift.io/cluster-api-machine-type: master
  strategy: {}
  template:
    machineType: machines_v1beta1_machine_openshift_io
    machines_v1beta1_machine_openshift_io:
      failureDomains:
        platform: ""
      metadata:
        labels:
          machine.openshift.io/cluster-api-cluster: openshiftsid-wvs7z
          machine.openshift.io/cluster-api-machine-role: master
          machine.openshift.io/cluster-api-machine-type: master
      spec:
        lifecycleHooks: {}
        metadata: {}
        providerSpec:
          value:
            apiVersion: machine.openshift.io/v1
            bootType: ""
            categories: null
            cluster:
              type: uuid
              uuid: 0005b0f1-8f43-a0f2-02b7-3cecef193712
            credentialsSecret:
              name: nutanix-credentials
            image:
              name: openshiftsid-wvs7z-rhcos
              type: name
            kind: NutanixMachineProviderConfig
            memorySize: 16Gi
            metadata:
              creationTimestamp: null
            project:
              type: ""
            subnets:
            - type: uuid
              uuid: c7938dc6-7659-453e-a688-e26020c68e43
            systemDiskSize: 120Gi
            userDataSecret:
              name: master-user-data
            vcpuSockets: 8
            vcpusPerSocket: 1
status: {}

5. Create a cluster successfully using openshift-install create cluster

$ oc get machines -A
NAMESPACE               NAME                              PHASE         TYPE   REGION    ZONE    AGE
openshift-machine-api   openshiftsid-v2tv8-master-0       Running       AHV    Unnamed   ganon   22m
openshift-machine-api   openshiftsid-v2tv8-master-1       Running       AHV    Unnamed   ganon   22m
openshift-machine-api   openshiftsid-v2tv8-master-2       Running       AHV    Unnamed   ganon   22m
openshift-machine-api   openshiftsid-v2tv8-worker-6xc8s   Running       AHV    Unnamed   ganon   12m
openshift-machine-api   openshiftsid-v2tv8-worker-c57mc   Running       AHV    Unnamed   ganon   12m
openshift-machine-api   openshiftsid-v2tv8-worker-pnc5c   Running       AHV    Unnamed   ganon   12m

6. Verify that the ControlPlaneMachineSet is Active and deleting ControlPlane and Worker machines leads to creation of new ones

$ oc -n openshift-machine-api get controlplanemachineset
NAME      DESIRED   CURRENT   READY   UPDATED   UNAVAILABLE   STATE    AGE
cluster   3         3         3       3                       Active   75m

$ oc -n openshift-machine-api delete machine openshiftsid-v2tv8-master-0
machine.machine.openshift.io "openshiftsid-v2tv8-master-0" deleted

$ oc -n openshift-machine-api delete machine openshiftsid-v2tv8-worker-pnc5c
machine.machine.openshift.io "openshiftsid-v2tv8-worker-pnc5c" deleted

$ oc -n openshift-machine-api get machines
NAME                                PHASE     TYPE   REGION    ZONE    AGE
openshiftsid-v2tv8-master-1         Running   AHV    Unnamed   ganon   75m
openshiftsid-v2tv8-master-2         Running   AHV    Unnamed   ganon   75m
openshiftsid-v2tv8-master-4rpfd-0   Running   AHV    Unnamed   ganon   24m
openshiftsid-v2tv8-worker-6xc8s     Running   AHV    Unnamed   ganon   65m
openshiftsid-v2tv8-worker-c57mc     Running   AHV    Unnamed   ganon   65m
openshiftsid-v2tv8-worker-ps5tn     Running   AHV    Unnamed   ganon   21m

[thunderboltsid]

/test e2e-nutanix-ovn

[JoelSpeed]

Code initially looks correct, can we observe what the generated CPMS output from this is? Does it look correct? Is the cluster bootstrapping correctly with the CPMS installed?

[openshift-ci]

@thunderboltsid: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/okd-scos-e2e-aws-ovn	93b3197	link	false	/test okd-scos-e2e-aws-ovn
ci/prow/okd-e2e-aws-ovn-upgrade	93b3197	link	false	/test okd-e2e-aws-ovn-upgrade
ci/prow/e2e-agent-ha-dualstack	93b3197	link	false	/test e2e-agent-ha-dualstack

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[thunderboltsid]

Code initially looks correct, can we observe what the generated CPMS output from this is? Does it look correct? Is the cluster bootstrapping correctly with the CPMS installed?

Yeah, good point! I updated the PR description with some of the verifications I've done.

[thunderboltsid]

/assign @damdo
/assign @JoelSpeed

[rna-afk]

/lgtm

[damdo]

Thanks for putting this together @thunderboltsid
LGTM

[JoelSpeed]

/lgtm

[thunderboltsid]

/test e2e-nutanix-ovn
/test e2e-nutanix-sdn

[rna-afk]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rna-afk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [rna-afk]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[deepsm007]

/label jira/valid-bug