Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion api/hypershift/v1beta1/hostedcluster_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -1953,7 +1953,8 @@ type ManagedEtcdStorageSpec struct {
// +kubebuilder:validation:MaxItems=1
// +kubebuilder:validation:items:MaxLength=1024
// +kubebuilder:validation:XValidation:rule="self.size() <= 1", message="RestoreSnapshotURL shouldn't contain more than 1 entry"
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="restoreSnapshotURL is immutable"
// +kubebuilder:validation:XValidation:rule="self == oldSelf", message="restoreSnapshotURL is immutable"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not in this PR, but can please follow up to do the same for the etcdbackup API

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we please include a test / suite for that covers all cases for this field immutability https://github.com/openshift/hypershift/tree/main/cmd/install/assets/crds/hypershift-operator/tests/hostedclusters.hypershift.openshift.io

This is done in #8186

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created follow-up ticket: CNTRLPLANE-3214 — "Add CEL immutability validation and CRD tests for HCPEtcdBackup API"

// +kubebuilder:validation:XValidation:rule="self.size() == 0 || self[0].matches('^(https|s3)://.*')", message="restoreSnapshotURL must be a valid URL with scheme https or s3"
RestoreSnapshotURL []string `json:"restoreSnapshotURL,omitempty"`
}

Expand Down Expand Up @@ -2180,6 +2181,16 @@ type HostedClusterStatus struct {
// configuration contains the cluster configuration status of the HostedCluster
// +optional
Configuration *ConfigurationStatus `json:"configuration,omitempty"`

// lastSuccessfulEtcdBackupURL is the cloud storage URL of the most recent
// successful etcd backup snapshot. Persisted here because HCPEtcdBackup CRs
// are ephemeral and may be deleted by retention policies.
Comment on lines +2185 to +2187

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General questions for context:

  • Where do these HCPEtcdBackup CRs generally exist?
  • Who manages and enforces the retention policies of these CRs?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good questions!

  • Where do HCPEtcdBackup CRs exist? They are created in the HCP (HostedControlPlane) namespace — the same namespace where the etcd StatefulSet runs. The controller reconciles them from the HyperShift Operator and creates Jobs in the HO namespace.

  • Who manages retention? The controller itself enforces retention via enforceRetention() — it deletes the oldest completed HCPEtcdBackup CRs when the count exceeds MaxBackupCount (configurable). This runs after every backup reaches a terminal state.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, instead of adding a new field to the status have you considered having a different semantic for retention enforcement such that retention enforcement doesn't ever remove the most recently successful backup?

When it comes to status fields, I try to evaluate whether or not this information is inherently useful to the consumer of the field - which on the HostedCluster type I suspect that is likely an end-user. Is that accurate?

// +openshift:enable:FeatureGate=HCPEtcdBackup
// +optional
// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=2048
// +kubebuilder:validation:XValidation:rule="self.matches('^(https|s3)://.*')",message="lastSuccessfulEtcdBackupURL must be a valid URL with scheme https or s3"
LastSuccessfulEtcdBackupURL string `json:"lastSuccessfulEtcdBackupURL,omitempty"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URLs often have a formatting constraint to ensure that only valid URLs are accepted as input.

I double checked that the earliest supported version for running HyperShift is OCP 4.14 which runs k8s 1.27 so we should be OK to use the URL CEL library (https://kubernetes.io/docs/reference/using-api/cel/#kubernetes-url-library) to validate that the input parses as a valid URL.

Are there any additional constraints on the URL being provided as an input? For example, can it contain query parameters, fragments, userinfo? Does the scheme have to be http(s) only?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch on the CEL URL library availability.

The URL here is a cloud storage object URL like s3://bucket/prefix/snapshot.db or an Azure Blob URL. The scheme could be s3://, https://, or Azure-specific formats. It cannot contain query parameters, fragments, or userinfo — it's strictly a storage object path.

I'll look into adding CEL validation with url() to enforce it parses as a valid URL. Since the field is status-only (set by the controller, not user input), the risk of invalid URLs is low, but the validation would be a good safeguard.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this please have the same validation added to HCPEtcdBackupStatus for the same field

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and we can include this as well in the envtest etcd test suite

@jparrill jparrill Apr 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — added the same CEL URL scheme validation to lastSuccessfulEtcdBackupURL in HostedClusterStatus:

+kubebuilder:validation:XValidation:rule="self.matches('^(https|s3)://.*')",message="lastSuccessfulEtcdBackupURL must be a valid URL with scheme https or s3"

CRDs regenerated.

and we can include this as well in the envtest etcd test suite

Tracked in CNTRLPLANE-3214 as a follow-up.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this a follow up, can we please include here as we are introducing a new field

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Added struct-level CEL rule on ManagedEtcdStorageSpec to prevent restoreSnapshotURL removal (field-level self == oldSelf doesn't fire when the optional field is absent in the update)
  • Added CEL URL scheme validation (https or s3) on restoreSnapshotURL
  • Added CRD envtest suite (stable.hostedclusters.etcd.testsuite.yaml) with:
    • onCreate: invalid scheme (http) should fail, https should pass, s3 should pass
    • onUpdate: value change should fail, removal should fail, unchanged should pass

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the new field introduced by this PR LastSuccessfulEtcdBackupURL still doesn't seem to have coverage in envtest

}

// AutoNodeStatus contains the observed state of the AutoNode provisioner.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2529,6 +2529,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2656,6 +2656,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2520,6 +2520,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2520,6 +2520,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2853,6 +2853,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2993,6 +2993,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2974,6 +2974,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2520,6 +2520,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2585,6 +2585,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down Expand Up @@ -6688,6 +6691,18 @@ spec:
type: string
type: object
x-kubernetes-map-type: atomic
lastSuccessfulEtcdBackupURL:
description: |-
lastSuccessfulEtcdBackupURL is the cloud storage URL of the most recent
successful etcd backup snapshot. Persisted here because HCPEtcdBackup CRs
are ephemeral and may be deleted by retention policies.
maxLength: 2048
minLength: 1
type: string
x-kubernetes-validations:
- message: lastSuccessfulEtcdBackupURL must be a valid URL with scheme
https or s3
rule: self.matches('^(https|s3)://.*')
oauthCallbackURLTemplate:
description: |-
oauthCallbackURLTemplate contains a template for the URL to use as a callback
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2542,6 +2542,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2538,6 +2538,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2596,6 +2596,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2520,6 +2520,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2446,6 +2446,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2575,6 +2575,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2437,6 +2437,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2437,6 +2437,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2770,6 +2770,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2910,6 +2910,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2891,6 +2891,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2437,6 +2437,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2502,6 +2502,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2459,6 +2459,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2455,6 +2455,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2513,6 +2513,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2437,6 +2437,9 @@ spec:
rule: self.size() <= 1
- message: restoreSnapshotURL is immutable
rule: self == oldSelf
- message: restoreSnapshotURL must be a valid URL with
scheme https or s3
rule: self.size() == 0 || self[0].matches('^(https|s3)://.*')
type:
description: |-
type is the kind of persistent storage implementation to use for etcd.
Expand Down
Loading
Loading