Skip to content

[vsphere] support for multi-zone/region installation #5911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-robot merged 9 commits into from
Aug 26, 2022
Merged

[vsphere] support for multi-zone/region installation #5911

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
b897299
add multi-zone/region support to the vSphere platform specification
rvanderp3 May 13, 2022
ff5ccf1
add zones to machinepool specification
rvanderp3 May 17, 2022
86e2ece
add zone/region awareness to machineset/machines
rvanderp3 May 18, 2022
43a8874
include cloud-provider-vsphere
rvanderp3 May 23, 2022
a2e363f
add go-yaml dependency
rvanderp3 Jun 3, 2022
8b61465
generate out of tree cloud provider config
rvanderp3 May 23, 2022
cbd3d9c
add support for credentials across multiple vCenters
rvanderp3 May 24, 2022
8163969
add multi-zone/region platform provisioning validation
rvanderp3 May 26, 2022
e94d5aa
add region/zone labels to the machine spec
rvanderp3 Jun 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
265 changes: 265 additions & 0 deletions data/data/install.openshift.io_installconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -706,6 +706,11 @@ spec:
format: int32
type: integer
type: object
zones:
description: Zones defines available zones
items:
type: string
type: array
type: object
type: object
replicas:
Expand Down Expand Up @@ -1344,6 +1349,11 @@ spec:
format: int32
type: integer
type: object
zones:
description: Zones defines available zones
items:
type: string
type: array
type: object
type: object
replicas:
Expand Down Expand Up @@ -2972,7 +2982,67 @@ spec:
format: int32
type: integer
type: object
zones:
description: Zones defines available zones
items:
type: string
type: array
type: object
deploymentZones:
description: vSphere location where openshift rhcos virtual machines
will be deployed based on vCenter and failureDomain If this
is omitted failure domains (regions and zones) will not be used.
items:
description: DeploymentZone holds the association between a
vCenter, failure domain and the virtual machine placementConstraints
properties:
controlPlane:
description: ControlPlane determines if this failure domain
is suitable for use by control plane machines.
enum:
- Allowed
- NotAllowed
type: string
failureDomain:
description: failureDomain is the name of FailureDomain
used for this DeploymentZone
maxLength: 256
minLength: 1
type: string
name:
description: name is the abritary name of the DeploymentZone
This name will be used in the install MachinePool
maxLength: 256
minLength: 1
type: string
placementConstraint:
description: PlacementConstraint encapsulates the placement
constraints used within this deployment zone.
properties:
folder:
description: folder is the name or inventory path of
the folder in which the virtual machine is created/located.
maxLength: 2048
minLength: 1
type: string
resourcePool:
description: resourcePool is the absolute path of the
resource pool where virtual machines will be created.
The absolute path is of the form /<datacenter>/host/<cluster>/Resources/<resourcepool>.
maxLength: 2048
minLength: 1
type: string
type: object
server:
description: server is the fully-qualified domain name or
the IP address of the vCenter server.
maxLength: 255
type: string
required:
- name
- placementConstraint
type: object
type: array
diskType:
description: DiskType is the name of the disk provisioning type,
valid values are thin, thick, and eagerZeroedThick. When not
Expand All @@ -2984,6 +3054,158 @@ spec:
- thick
- eagerZeroedThick
type: string
failureDomains:
description: failureDomains holds the VSpherePlatformFailureDomainSpec
which contains the definition of region, zone and the vCenter
topology. If this is omitted failure domains (regions and zones)
will not be used.
items:
description: FailureDomain holds the region and zone failure
domain and the vCenter topology of that failure domain.
properties:
name:
description: name defines the name of the FailureDomain
This name is abritrary but will be used in VSpherePlatformDeploymentZone
for association.
maxLength: 256
minLength: 1
type: string
region:
description: region defines a FailureDomainCoordinate which
includes the name of the vCenter tag, the failure domain
type and the name of the vCenter tag category.
properties:
name:
description: name is the name of the vCenter tag that
represents this failure domain
maxLength: 80
minLength: 1
type: string
tagCategory:
description: tagCategory is the category used for the
tag
maxLength: 80
minLength: 1
type: string
type:
allOf:
- enum:
- HostGroup
- Datacenter
- ComputeCluster
- enum:
- HostGroup
- Datacenter
- ComputeCluster
description: type is the name of the failure domain
type, which includes Datacenter, ComputeCluster and
HostGroup
type: string
required:
- name
- tagCategory
- type
type: object
topology:
description: Topology describes a given failure domain using
vSphere constructs
properties:
computeCluster:
description: computeCluster as the failure domain This
is required to be a path
maxLength: 2048
minLength: 1
type: string
datacenter:
description: datacenter is the vCenter datacenter in
which virtual machines will be located and defined
as the failure domain.
maxLength: 80
minLength: 1
type: string
datastore:
description: datastore is the name or inventory path
of the datastore in which the virtual machine is created/located.
maxLength: 2048
minLength: 1
type: string
hosts:
description: Hosts has information required for placement
of machines on VSphere hosts.
properties:
hostGroupName:
description: hostGroupName is the Host Gorup name
configured within a vCenter cluster defining a
group of ESXi hosts.
maxLength: 80
minLength: 1
type: string
vmGroupName:
description: vmGroupName is the Virtual Machine
Group name configured within a vCenter cluster
that is associated with the corresponding Host
Group.
maxLength: 80
minLength: 1
type: string
required:
- hostGroupName
- vmGroupName
type: object
networks:
description: networks is the list of networks within
this failure domain
items:
type: string
type: array
required:
- computeCluster
- datacenter
- datastore
type: object
zone:
description: zone defines a VSpherePlatformFailureDomain
which includes the name of the vCenter tag, the failure
domain type and the name of the vCenter tag category.
properties:
name:
description: name is the name of the vCenter tag that
represents this failure domain
maxLength: 80
minLength: 1
type: string
tagCategory:
description: tagCategory is the category used for the
tag
maxLength: 80
minLength: 1
type: string
type:
allOf:
- enum:
- HostGroup
- Datacenter
- ComputeCluster
- enum:
- HostGroup
- Datacenter
- ComputeCluster
description: type is the name of the failure domain
type, which includes Datacenter, ComputeCluster and
HostGroup
type: string
required:
- name
- tagCategory
- type
type: object
required:
- name
- region
- topology
- zone
type: object
type: array
folder:
description: Folder is the absolute path of the folder that will
be used and/or created for virtual machines. The absolute path
Expand Down Expand Up @@ -3012,6 +3234,49 @@ spec:
vCenter:
description: VCenter is the domain name or IP address of the vCenter.
type: string
vcenters:
description: vcenters holds the connection details for services
to communicate with vCenter. Currently only a single vCenter
is supported.
items:
description: VCenter stores the vCenter connection fields https://github.com/kubernetes/cloud-provider-vsphere/blob/master/pkg/common/config/types_yaml.go
properties:
datacenters:
description: Datacenter in which VMs are located.
items:
type: string
minItems: 1
type: array
password:
description: Password is the password for the user to use
to connect to the vCenter.
type: string
port:
default: 443
description: port is the TCP port that will be used to communicate
to the vCenter endpoint. This is typically unchanged from
the default of HTTPS TCP/443.
maximum: 32767
minimum: 1
type: integer
server:
description: server is the fully-qualified domain name or
the IP address of the vCenter server.
maxLength: 255
type: string
user:
description: Username is the username that will be used
to connect to vCenter
type: string
required:
- datacenters
- password
- server
- user
type: object
maxItems: 1
minItems: 1
type: array
required:
- datacenter
- defaultDatastore
Expand Down
6 changes: 4 additions & 2 deletions data/data/manifests/openshift/cloud-creds-secret.yaml.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,10 @@ data:
clouds.yaml: {{.CloudCreds.OpenStack.Base64encodeCloudCreds}}
clouds.conf: {{.CloudCreds.OpenStack.Base64encodeCloudCredsINI}}
{{- else if .CloudCreds.VSphere}}
{{.CloudCreds.VSphere.VCenter}}.username: {{.CloudCreds.VSphere.Base64encodeUsername}}
{{.CloudCreds.VSphere.VCenter}}.password: {{.CloudCreds.VSphere.Base64encodePassword}}
{{- range .CloudCreds.VSphere}}
{{.VCenter}}.username: {{.Base64encodeUsername}}
Copy link
Contributor

@patrickdillon patrickdillon Jul 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The server name is not being included here... is it needed?

Copy link
Contributor Author

@rvanderp3 rvanderp3 Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.VCenter resolves to the server name. the goal here was to produce the credentials secret in such a way that would allow multiple vCenters to be defined.

{{.VCenter}}.password: {{.Base64encodePassword}}
{{- end}}
{{- else if .CloudCreds.Ovirt}}
ovirt_url: {{.CloudCreds.Ovirt.Base64encodeURL}}
ovirt_username: {{.CloudCreds.Ovirt.Base64encodeUsername}}
Expand Down
6 changes: 6 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ require (
github.com/form3tech-oss/jwt-go v3.2.3+incompatible
github.com/ghodss/yaml v1.0.1-0.20190212211648-25d852aebe32
github.com/go-playground/validator/v10 v10.2.0
github.com/go-yaml/yaml v2.1.0+incompatible
github.com/golang-jwt/jwt v3.2.2+incompatible
github.com/golang/mock v1.6.0
github.com/golang/protobuf v1.5.2
Expand Down Expand Up @@ -79,6 +80,7 @@ require (
k8s.io/apiextensions-apiserver v0.24.3
k8s.io/apimachinery v0.24.3
k8s.io/client-go v12.0.0+incompatible
k8s.io/cloud-provider-vsphere v0.0.0
k8s.io/klog v1.0.0
k8s.io/klog/v2 v2.70.1
k8s.io/utils v0.0.0-20220812165043-ad590609e2e5
Expand Down Expand Up @@ -194,13 +196,17 @@ require (
github.com/prometheus/procfs v0.7.3 // indirect
github.com/satori/go.uuid v1.2.0 // indirect
golang.org/x/net v0.0.0-20220812174116-3211cb980234 // indirect
gopkg.in/gcfg.v1 v1.2.3 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
)

// OpenShift Forks
replace (
github.com/metal3-io/baremetal-operator => github.com/openshift/baremetal-operator v0.0.0-20220128094204-28771f489634
github.com/metal3-io/baremetal-operator/apis => github.com/openshift/baremetal-operator/apis v0.0.0-20220128094204-28771f489634
github.com/metal3-io/baremetal-operator/pkg/hardwareutils => github.com/openshift/baremetal-operator/pkg/hardwareutils v0.0.0-20220128094204-28771f489634
k8s.io/cloud-provider-vsphere => github.com/openshift/cloud-provider-vsphere v1.19.1-0.20211222185833-7829863d0558
sigs.k8s.io/cluster-api => sigs.k8s.io/cluster-api v0.4.5
sigs.k8s.io/cluster-api-provider-azure => github.com/openshift/cluster-api-provider-azure v0.1.0-alpha.3.0.20210626224711-5d94c794092f // Indirect dependency through MAO from cluster API providers
sigs.k8s.io/cluster-api-provider-openstack => github.com/openshift/cluster-api-provider-openstack v0.0.0-20211111204942-611d320170af
)
Expand Down
Loading