Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api/core/v1beta2/machine_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -556,7 +556,7 @@ type MachineReadinessGate struct {
type MachineStatus struct {
// conditions represents the observations of a Machine's current state.
// Known condition types are Available, Ready, UpToDate, BootstrapConfigReady, InfrastructureReady, NodeReady,
// NodeHealthy, Deleting, Paused.
// NodeHealthy, Updating, Deleting, Paused.
// If a MachineHealthCheck is targeting this machine, also HealthCheckSucceeded, OwnerRemediated conditions are added.
// Additionally control plane Machines controlled by KubeadmControlPlane will have following additional conditions:
// APIServerPodHealthy, ControllerManagerPodHealthy, SchedulerPodHealthy, EtcdPodHealthy, EtcdMemberHealthy.
Expand Down
2 changes: 1 addition & 1 deletion api/core/v1beta2/zz_generated.openapi.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion config/crd/bases/cluster.x-k8s.io_machines.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

206 changes: 185 additions & 21 deletions docs/book/src/developer/providers/migrations/v1.11-to-v1.12.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,34 @@
# Cluster API v1.11 compared to v1.12

This document provides an overview over relevant changes between Cluster API v1.11 and v1.12 for
maintainers of providers and consumers of our Go API.
maintainers of providers and consumers of our Go API.

Any feedback or contributions to improve following documentation is welcome!

<!-- TOC -->
* [Cluster API v1.11 compared to v1.12](#cluster-api-v111-compared-to-v112)
* [Go version](#go-version)
* [Dependencies](#dependencies)
* [Implemented proposal](#implemented-proposal)
* [API Changes](#api-changes)
* [Cluster](#cluster-)
* [Machine](#machine)
* [MachineDeployment](#machinedeployment)
* [MachineSet](#machineset)
* [MachinePool](#machinepool)
* [MachineHealthCheck](#machinehealthcheck)
* [ClusterClass](#clusterclass)
* [KubeadmConfig](#kubeadmconfig)
* [KubeadmConfigTemplate](#kubeadmconfigtemplate)
* [KubeadmControlPlane](#kubeadmcontrolplane)
* [KubeadmControlPlaneTemplate](#kubeadmcontrolplanetemplate)
* [Runtime hooks Changes](#runtime-hooks-changes)
* [Cluster API Contract changes](#cluster-api-contract-changes)
* [Deprecation](#deprecation)
* [Removals](#removals)
* [Suggested changes for providers](#suggested-changes-for-providers)
* [Removals scheduled for future releases](#removals-scheduled-for-future-releases)
<!-- TOC -->

## Go version

Expand All @@ -13,36 +40,173 @@ maintainers of providers and consumers of our Go API.
- The Controller Runtime version used by Cluster API is v0.22.x
- The version of the Kubernetes libraries used by Cluster API is v1.34.x

## Changes by Kind
## Implemented proposal

### Deprecation
The following proposal have been implemented in the Cluster API v1.12 release:
- [In-place updates](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20240807-in-place-updates.md)
- [Chained and efficient upgrades](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20250513-chained-and-efficient-upgrades-for-clusters-with-managed-topologies.md)
- [Taint propagation](https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20250513-propogate-taints.md) (Only phase 1)

### Removals
## API Changes

* The `controlplane.cluster.x-k8s.io/kubeadm-cluster-configuration` annotation that KCP was previously setting on Machines has been removed.
KCP used this annotation to detect if a Machine requires a rollout, this is now done via the KubeadmConfig of the Machine instead.
### Cluster

### API Changes
- The new `spec.topology.controlPlane.healthCheck.checks.unhealthyMachineConditions` field has been added
- The new `spec.topology.workers.machineDeployments[].healthCheck.checks.unhealthyMachineConditions` field has been added
- The `TopologyReconciled` condition reports new `ClusterCreating` and `ClusterUpgrading` reasons
- The following reasons for the `TopologyReconciled` condition are now deprecated
- `ControlPlaneUpgradePending`
- `MachineDeploymentsCreatePending`
- `MachineDeploymentsUpgradePending`
- `MachinePoolsUpgradePending`
- `MachinePoolsCreatePending`
- `LifecycleHookBlocking`

### Machine

### Other
- The new `spec.taint` field has been added.
- The `HealthCheckSucceeded` condition reports a new `UnhealthyMachine` reason when `unhealthyMachineConditions` checks fail
- The `UpToDate` condition reports a new `Updating` reason when a machine is performing in-place updates
- The new `Updating` condition has been added. It reports status `True` when a machine is performing in-place updates
- Supported reasons are `NotUpdating`, `InPlaceUpdating`, `InPlaceUpdateFailed`
- The field `status.phases` reports a new `Updating` phase when a machine is performing in-place updates

* `util.IsOwnedByObject`, `util.IsControlledBy` and `collections.OwnedMachines` now also require `schema.GroupKind` as input parameter.
`schema.GroupKind` is needed for cases where typed objects are passed in because controller-runtime does not guarantee that GVK is set on typed objects.
* Various Cluster API e2e tests with Kubernetes upgrades now use the `wait-control-plane-upgrade` and `wait-machine-deployment-upgrade` timeouts.
If you are running Cluster API e2e tests with upgrades you have to configure the timeouts in your e2e test configuration file, otherwise
the e2e tests will use `1s` timeout which will lead to test failures. Example:
```yaml
default/wait-control-plane-upgrade: ["15m", "10s"]
default/wait-machine-deployment-upgrade: ["10m", "10s"]
```
### MachineDeployment

MachineDeployment `spec.template.spec` has been aligned to changes in the [Machine](#machine) `spec` struct

### MachineSet

MachineSet `spec.template.spec` has been aligned to changes in the [Machine](#machine) `spec` struct

### MachinePool

MachinePool `spec.template.spec` has been aligned to changes in the [Machine](#machine) `spec` struct

### MachineHealthCheck

- The new `spec.checks.unhealthyMachineConditions` field has been added

### ClusterClass

- The new `spec.controlPlane.healthCheck.checks.unhealthyMachineConditions` field has been added
- The new `spec.workers.machineDeployments[].healthCheck.checks.unhealthyMachineConditions` field has been added
- The new `spec.upgrade` field has been added
- The new `spec.kubernetesVersions` field has been added

### KubeadmConfig

- The new `spec.clusterConfiguration.encryptionAlgorithm` field has been added

### KubeadmConfigTemplate

KubeadmConfigTemplate `spec.template.spec` has been aligned to changes in the [KubeadmConfig](#kubeadmconfig) `spec` struct

### KubeadmControlPlane

KubeadmControlPlane `spec.kubeadmConfigSpec` has been aligned to changes in the [KubeadmConfig](#kubeadmconfig) `spec` struct

### KubeadmControlPlaneTemplate

### Suggested changes for providers
KubeadmControlPlaneTemplate `spec.template.spec` has been aligned to changes in the [KubeadmControlPlane](#kubeadmcontrolplane) `spec` struct

* For providers that copied the core Cluster API v1beta1 `APIEndpoint` struct and used it in their InfraCluster Go type
it is recommended to now make the `Host` and `Port` fields optional (they already have been made optional in
## Runtime hooks Changes

- Lifecycle hooks
- The request message for the `BeforeClusterUpgrade` hook has been extended to include info about the upgrade plan
- The new `BeforeControlPlaneUpgrade` hook has been added; the hook is called before each "upgrade control plane" step
- The request message for the `AfterControlPlaneUpgrade` hook have been extended to include info about the upgrade plan
- The `AfterControlPlaneUpgrade` hook is now called after each "upgrade control plane" step
- The new `BeforeWorkersUpgrade`, `AfterWorkersUpgrade` hooks have been added; the hooks are called before/after each "upgrade workers" step
- The `AfterClusterUpgrade` has been transformed into a blocking hook (it blocks next upgrade)
- Upgrade plan hooks (new!)
- The new `GenerateUpgradePlan` has been added; the hook is called by the topology controller when it is required
to compute the upgrade plan for a cluster
- In-place extension hooks (new!)
- the new `CanUpdateMachine` hook has been added; the hook is called by the KubeadmControlPlane controller to perform the
"can update in-place" decision for a machine that is not up-to-date
- the new `CanUpdateMachineSet` hook has been added; the hook is called by the MachineDeployment controller to perform the
"can update in-place" decision for a MachineSet that is not up-to-date
- the new `UpdateMachine` hook has been added; the hook is called by the Machine controller to perform in-place upgrades
- Some optimization have been implemented to ensure that hooks are called only when necessary
- e.g. if a lifecycle hook response, e.g. the `BeforeClusterUpgradeResponse`, informs CAPI to retry after X seconds,
CAPI is going to not call the runtime extension until X expires, no matter if the underlying controller reconciles
one or more times in the between
- `GenerateUpgradePlan`, `CanUpdateMachine`, `CanUpdateMachineSet` hooks will use slightly different caching strategy

## Cluster API Contract changes

- Documentation for the [MachinePool](../contracts/infra-machinepool.md) contract has been aligned to the documentation
of other contracts.
- We strongly recommend all the MachinePool providers to read carefully this document and check compliance with
the updated definition of the contract rules.

- A new, optional rule has been added to the control plane contract, defining what is required for implementing support
for in-place updates.
Comment on lines +144 to +145
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes for the control plane contract are included in #13088


## Deprecation

- The following reasons for the `TopologyReconciled` condition on the `Cluster` object are now deprecated
- `ControlPlaneUpgradePending`
- `MachineDeploymentsCreatePending`
- `MachineDeploymentsUpgradePending`
- `MachinePoolsUpgradePending`
- `MachinePoolsCreatePending`
- `LifecycleHookBlocking`

## Removals

- The `controlplane.cluster.x-k8s.io/kubeadm-cluster-configuration` annotation that KCP was previously setting on Machines has been removed.
KCP used this annotation to detect if a Machine requires a rollout, this is now done via the KubeadmConfig of the Machine instead.

## Suggested changes for providers

- For providers that copied the core Cluster API v1beta1 `APIEndpoint` struct and used it in their InfraCluster
or ControlPlane Go type it is recommended to now make the `Host` and `Port` fields optional (they already have been made optional in
`clusterv1beta1.APIEndpoint` in Cluster API v1.12).
tl;dr The fields were previously required, but due to side effects that validation was never enforced. These side
effects might go away which then makes the fields suddenly required. To avoid issues we recommend making the fields
optional. Similar to how they are optional in the v1beta2 `APIEndpoint` struct and v1beta2 InfraCluster contract.
For more details, please see: https://github.com/kubernetes-sigs/cluster-api/pull/12634#discussion_r2275468291.
But this might look differently depending on how your InfraCluster Go type and corresponding mutating webhook evolved over time.
But this might look differently depending on how your InfraCluster/ControlPlane Go type and corresponding
mutating webhook evolved over time.

- We strongly recommend all the MachinePool providers to read carefully the [MachinePool](../contracts/infra-machinepool.md) contract
and check compliance with the updated definition of the contract rules.

- We strongly recommend providers to start moving to the new v1beta2 version of the Cluster API contract as soon as possible.
- v1beta1 version of the Cluster API contract is now deprecated and it will be removed tentatively in August 2026:
- In order to ease the transition to the new v1beta2 version of the Cluster API contract, v1beta2 version
will implement temporarily compatibility with the deprecated v1beta1 version of the Cluster API contract
- Compatibility is only intended to ease the transition for providers, and it has some limitations; please
read details in following paragraphs.
- Compatibility support for the v1beta1 version of the Cluster API contract will be removed tentatively in August 2026.
- After compatibility support for the v1beta1 version of the Cluster API contract is removed, providers
which are implementing the v1beta1 contract will stop to work (they will work only with older versions of Cluster API).

- Various Cluster API e2e tests with Kubernetes upgrades now use the `wait-control-plane-upgrade` and `wait-machine-deployment-upgrade` timeouts.
If you are running Cluster API e2e tests with upgrades you have to configure the timeouts in your e2e test configuration file, otherwise
the e2e tests will use `1s` timeout which will lead to test failures. Example:
```yaml
default/wait-control-plane-upgrade: ["15m", "10s"]
default/wait-machine-deployment-upgrade: ["10m", "10s"]
```

- Reconciler rate limiting feature has been introduced ([#13006](https://github.com/kubernetes-sigs/cluster-api/pull/13006))
- Can be enabled with the new `ReconcilerRateLimiting` feature gate
- It rate-limits all reconcilers to at most 1 request per second

- `util.IsOwnedByObject`, `util.IsControlledBy` and `collections.OwnedMachines` now also require `schema.GroupKind` as input parameter.
`schema.GroupKind` is needed for cases where typed objects are passed in because controller-runtime does not guarantee that GVK is set on typed objects.

## Removals scheduled for future releases

As documented in [Suggested changes for providers](#suggested-changes-for-providers), it is highly recommended to start planning for future removals:

- v1beta1 API version will be removed tentatively in August 2026
- Starting from the CAPI release when v1beta1 removal will happen (tentative Aug 2026), the Cluster API project
will remove the Cluster API condition type, the `util/conditions/deprecated/v1beta1` package,
the `util/deprecated/v1beta1` package, the code handling old conditions in `util/patch.Helper`
and everything related to the custom Cluster API custom condition type.
- All the `status.deprecated` fields will be removed tentatively in August 2026.
- Compatibility support for the v1beta1 version of the Cluster API contract will be removed tentatively in August 2026