Skip to content

Support patching installer manifests #2499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-bot merged 1 commit into from
Mar 11, 2025
Merged

Support patching installer manifests #2499

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions apis/hive/v1/clusterdeployment_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,13 @@ type Provisioning struct {
// +optional
InstallConfigSecretRef *corev1.LocalObjectReference `json:"installConfigSecretRef,omitempty"`

// CustomizationRef is a reference to a ClusterDeploymentCustomization containing
// InstallerManifestPatches to be applied to the manifests generated by openshift-install prior
// to starting the installation. (InstallConfigPatches will be ignored -- those changes should
// be made directly to the install-config.yaml referenced by InstallConfigSecretRef.)
Comment on lines +221 to +224
Copy link
Contributor

@suhanime suhanime Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the point of this note, but I feel that cdc related things should be added as a comment right before the definition of cdc struct, that way, in case the cdc is expanded in the future, the comment is more likely to be amended and not go out-of-date

Copy link
Member Author

@2uasimojo 2uasimojo Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't follow. This is CD's type file, not CDC's. InstallConfigPatches are not ignored when this is used in an inventory CDC.

Copy link
Contributor

@suhanime suhanime Mar 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eh it's fine, I'm not too attached to moving the comment either ways. When I commented on it, I was thinking about what if an option 3 is introduced in cdc def in the future, and we forget to edit the comment here to say 3 will also be ignored? But it's fine, you have a similar comment on clusterpool's cdc ref, so we can let it be.

// +optional
CustomizationRef *corev1.LocalObjectReference `json:"customizationRef,omitempty"`

// ReleaseImage is the image containing metadata for all components that run in the cluster, and
// is the primary and best way to specify what specific version of OpenShift you wish to install.
ReleaseImage string `json:"releaseImage,omitempty"`
Expand Down
23 changes: 22 additions & 1 deletion apis/hive/v1/clusterdeploymentcustomization_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ const (
// ClusterDeploymentCustomization is the Schema for clusterdeploymentcustomizations API.
// +kubebuilder:subresource:status
// +k8s:openapi-gen=true
// +kubebuilder:resource:scope=Namespaced
// +kubebuilder:resource:shortName=cdc,scope=Namespaced
type ClusterDeploymentCustomization struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Expand All @@ -42,6 +42,27 @@ type ClusterDeploymentCustomization struct {
type ClusterDeploymentCustomizationSpec struct {
// InstallConfigPatches is a list of patches to be applied to the install-config.
InstallConfigPatches []PatchEntity `json:"installConfigPatches,omitempty"`

// InstallerManifestPatches is a list of patches to be applied to installer-generated manifests.
InstallerManifestPatches []InstallerManifestPatch `json:"installerManifestPatches,omitempty"`
}

type InstallerManifestPatch struct {
// ManifestSelector identifies one or more manifests to patch
ManifestSelector ManifestSelector `json:"manifestSelector"`

// Patches is a list of RFC6902 patches to apply to manifests identified by manifestSelector.
Patches []PatchEntity `json:"patches"`
}

type ManifestSelector struct {
// Glob is a file glob (per https://pkg.go.dev/path/filepath#Glob) identifying one or more
// manifests. Paths should be relative to the installer's working directory. Examples:
// - openshift/99_role-cloud-creds-secret-reader.yaml
// - openshift/99_openshift-cluster-api_worker-machineset-*.yaml
// - */*secret*
// It is an error if a glob matches zero manifests.
Glob string `json:"glob"`
}

// PatchEntity represents a json patch (RFC 6902) to be applied
Expand Down
7 changes: 7 additions & 0 deletions apis/hive/v1/clusterpool_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,13 @@ type ClusterPoolSpec struct {
// additional features of the installer.
// +optional
InstallerEnv []corev1.EnvVar `json:"installerEnv,omitempty"`

// CustomizationRef refers to a ClusterDeploymentCustomization object whose InstallerManifestPatches should
// be applied to *all* ClusterDeployments created by this ClusterPool. This is in addition to any CDC from
// Inventory. The CDC must exist in the ClusterPool's namespace. It will be copied to the namespace of each
// ClusterDeployment generated by the ClusterPool.
// +optional
CustomizationRef *corev1.LocalObjectReference `json:"customizationRef,omitempty"`
}

type HibernationConfig struct {
Expand Down
55 changes: 55 additions & 0 deletions apis/hive/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

68 changes: 68 additions & 0 deletions config/crds/hive.openshift.io_clusterdeploymentcustomizations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ spec:
kind: ClusterDeploymentCustomization
listKind: ClusterDeploymentCustomizationList
plural: clusterdeploymentcustomizations
shortNames:
- cdc
singular: clusterdeploymentcustomization
scope: Namespaced
versions:
Expand Down Expand Up @@ -79,6 +81,72 @@ spec:
- path
type: object
type: array
installerManifestPatches:
description: InstallerManifestPatches is a list of patches to be applied
to installer-generated manifests.
items:
properties:
manifestSelector:
description: ManifestSelector identifies one or more manifests
to patch
properties:
glob:
description: |-
Glob is a file glob (per https://pkg.go.dev/path/filepath#Glob) identifying one or more
manifests. Paths should be relative to the installer's working directory. Examples:
- openshift/99_role-cloud-creds-secret-reader.yaml
- openshift/99_openshift-cluster-api_worker-machineset-*.yaml
- */*secret*
It is an error if a glob matches zero manifests.
type: string
required:
- glob
type: object
patches:
description: Patches is a list of RFC6902 patches to apply to
manifests identified by manifestSelector.
items:
description: PatchEntity represents a json patch (RFC 6902)
to be applied
properties:
from:
description: From is the json path to copy or move the
value from
type: string
op:
description: Op is the operation to perform.
enum:
- add
- remove
- replace
- move
- copy
- test
type: string
path:
description: Path is the json path to the value to be
modified
type: string
value:
description: |-
Value is the *string* value to be used in the operation. For more complex values, use
ValueJSON.
type: string
valueJSON:
description: |-
ValueJSON is a string representing a JSON object to be used in the operation. As such,
internal quotes must be escaped. If nonempty, Value is ignored.
type: string
required:
- op
- path
type: object
type: array
required:
- manifestSelector
- patches
type: object
type: array
type: object
status:
description: ClusterDeploymentCustomizationStatus defines the observed
Expand Down
18 changes: 18 additions & 0 deletions config/crds/hive.openshift.io_clusterdeployments.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1017,6 +1017,24 @@ spec:
Provisioning contains settings used only for initial cluster provisioning.
May be unset in the case of adopted clusters.
properties:
customizationRef:
description: |-
CustomizationRef is a reference to a ClusterDeploymentCustomization containing
InstallerManifestPatches to be applied to the manifests generated by openshift-install prior
to starting the installation. (InstallConfigPatches will be ignored -- those changes should
be made directly to the install-config.yaml referenced by InstallConfigSecretRef.)
properties:
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
type: object
x-kubernetes-map-type: atomic
imageSetRef:
description: |-
ImageSetRef is a reference to a ClusterImageSet. If a value is specified for ReleaseImage,
Expand Down
18 changes: 18 additions & 0 deletions config/crds/hive.openshift.io_clusterpools.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,24 @@ spec:
pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|ms|s|m|h))+$
type: string
type: object
customizationRef:
description: |-
CustomizationRef refers to a ClusterDeploymentCustomization object whose InstallerManifestPatches should
be applied to *all* ClusterDeployments created by this ClusterPool. This is in addition to any CDC from
Inventory. The CDC must exist in the ClusterPool's namespace. It will be copied to the namespace of each
ClusterDeployment generated by the ClusterPool.
properties:
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
type: object
x-kubernetes-map-type: atomic
hibernateAfter:
description: |-
HibernateAfter will be applied to new ClusterDeployments created for the pool. HibernateAfter will transition
Expand Down
53 changes: 26 additions & 27 deletions docs/enhancements/clusterpool-inventory.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,27 @@

[HIVE-1367](https://issues.redhat.com/browse/HIVE-1367)

- [Clusterpool for on-prem cloud providers](#clusterpool-for-on-prem-cloud-providers)
- [Summary](#summary)
- [Problem Statement](#problem-statement)
- [Proposal](#proposal)
- [Summary](#summary-1)
- [`ClusterPool.Spec.Inventory`](#clusterpoolspecinventory)
- [How To Use](#how-to-use)
- [Validation](#validation)
- [`Size` and `MaxSize`](#size-and-maxsize)
- [Pool Version](#pool-version)
- [Handling Inventory Updates](#handling-inventory-updates)
- [Adding An Inventory](#adding-an-inventory)
- [Adding An Entry to the Inventory](#adding-an-entry-to-the-inventory)
- [Removing An Entry from the Inventory](#removing-an-entry-from-the-inventory)
- [Deleting The Inventory](#deleting-the-inventory)
- [Maintaining the lease of the ClusterDeploymentCustomization](#maintaining-the-lease-of-the-clusterdeploymentcustomization)
- [Fairness](#fairness)
- [Future](#future)
- [Alternatives](#alternatives)
- [Bespoke Inventory Definition](#bespoke-inventory-definition)
- [Full Spec](#full-spec)
- [Hooks](#hooks)
- [Summary](#summary)
- [Problem Statement](#problem-statement)
- [Proposal](#proposal)
- [Summary](#summary-1)
- [`ClusterPool.Spec.Inventory`](#clusterpoolspecinventory)
- [How To Use](#how-to-use)
- [Validation](#validation)
- [`Size` and `MaxSize`](#size-and-maxsize)
- [Pool Version](#pool-version)
- [Handling Inventory Updates](#handling-inventory-updates)
- [Adding An Inventory](#adding-an-inventory)
- [Adding An Entry to the Inventory](#adding-an-entry-to-the-inventory)
- [Removing An Entry from the Inventory](#removing-an-entry-from-the-inventory)
- [Deleting The Inventory](#deleting-the-inventory)
- [Maintaining the lease of the ClusterDeploymentCustomization](#maintaining-the-lease-of-the-clusterdeploymentcustomization)
- [Fairness](#fairness)
- [Future](#future)
- [Alternatives](#alternatives)
- [Bespoke Inventory Definition](#bespoke-inventory-definition)
- [Full Spec](#full-spec)
- [Hooks](#hooks)

## Summary

Expand Down Expand Up @@ -59,7 +58,7 @@ spec:

and ClusterDeploymentCustomization CR will look like
```yaml
apiVersion: v1
apiVersion: hive.openshift.io/v1
kind: ClusterDeploymentCustomization
metadata:
name: foo-cluster-deployment-customization
Expand Down Expand Up @@ -115,10 +114,10 @@ For the VSphere case, this allows the administrator to:
- Create a ClusterDeploymentCustomization CR to patch `spec.metadata.name` field of the default install config generated by clusterpool controller. Please refer the section above of a sample CR. The content in `spec.installConfigPatches` field should be as follows
```yaml
spec:
installConfigPatches:
- op: replace
path: metadata/name
value: foo
installConfigPatches:
- op: replace
path: metadata/name
value: foo
```
- Add the name of ClusterDeploymentCustomization CR to `clusterPool.spec.inventory.ClusterDeploymentCustomizations` list. For ClusterDeploymentCustomization with a name `foo-cluster-deployment-customization` the clusterpool should be configured as follows
```yaml
Expand Down
Loading