✨ ROSA: Add ROSA-HCP AutoNode for karpenter auto scale #5686

serngawy · 2025-09-26T16:17:08Z

What type of PR is this?

What this PR does / why we need it:

Adding auto node config for karpenter feature

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Checklist:

Release note:

Add ROSA-HCP auto node feature for karpenter support

robpblake · 2025-09-26T16:21:38Z

config/crd/bases/controlplane.cluster.x-k8s.io_rosacontrolplanes.yaml

+                    default: disable
+                    description: AutoNode mode allowed values are enable & disable
+                    enum:
+                    - enable


@serngawy enabled and disabled are the allowed values.

the enum values are enabled & disabled. Changed it to match the values.

robpblake · 2025-09-26T16:22:17Z

config/crd/bases/controlplane.cluster.x-k8s.io_rosacontrolplanes.yaml

+                    description: |-
+                      AutoNode role ARN that has the IAM Policy and cluster-specific Role that gives permissions to the Karpenter controller.
+                      IAM policy should be as below AND the role must be attached with the same OIDC-ID that is used with the ROSA-HCP cluster.
+                      {


@serngawy The Policy for the IAM role is highly likely to change. For the private preview we are giving this to customers via documentation, so I would suggest not to embed it here.

robpblake · 2025-09-26T16:22:58Z

controlplane/rosa/api/v1beta2/rosacontrolplane_types.go

+	AutoNode *AutoNode `json:"autoNode,omitempty"`
+}
+
+// AutoNode set the auto nde mode and auto node role ARN.


@serngawy set the AutoNode mode and AutoNode role ARN

robpblake · 2025-09-29T16:04:50Z

config/crd/bases/controlplane.cluster.x-k8s.io_rosacontrolplanes.yaml

                  If not set, audit log forwarding is disabled.
                type: string
+              autoNode:
+                description: AutoNode set the autoNode mode and autoNode role ARN.


@serngawy Minor typo here "A"utoNode rather than "a"utoNode.

done, thanks

damdo · 2025-09-30T09:44:43Z

/label tide/merge-method-squash

damdo

Left some API suggestions. Thanks

controlplane/rosa/api/v1beta2/rosacontrolplane_types.go

damdo · 2025-09-30T09:56:31Z

controlplane/rosa/api/v1beta2/rosacontrolplane_types.go

+	// AutoNode set the AutoNode mode and AutoNode role ARN.
+	// +optional
+	AutoNode *AutoNode `json:"autoNode,omitempty"`


I think this doesn't need to be a pointer right?

pointer is easier to check struct value.

controlplane/rosa/api/v1beta2/rosacontrolplane_types.go

damdo · 2025-09-30T09:59:32Z

controlplane/rosa/controllers/rosacontrolplane_controller_test.go

+	// Test case 7: AutoNode update
+	t.Run("Update Auto Node", func(t *testing.T) {
+		rosaControlPlane := &rosacontrolplanev1.ROSAControlPlane{
+			Spec: rosacontrolplanev1.RosaControlPlaneSpec{
+				AutoNode: &rosacontrolplanev1.AutoNode{
+					Mode:    rosacontrolplanev1.Enabled,
+					RoleARN: "autoNodeARN",
+				},
+			},
+		}
+
+		mockCluster, _ := v1.NewCluster().
+			AutoNode(v1.NewClusterAutoNode().Mode("disabled")).
+			AWS(v1.NewAWS().AutoNode(v1.NewAwsAutoNode().RoleArn("anyARN"))).
+			Build()
+
+		expectedOCMSpec := ocm.Spec{
+			AutoNodeMode:    "enabled",
+			AutoNodeRoleARN: "autoNodeARN",
+		}
+
+		reconciler := &ROSAControlPlaneReconciler{}
+		ocmSpec, updated := reconciler.updateOCMClusterSpec(rosaControlPlane, mockCluster)
+
+		g.Expect(updated).To(BeTrue())
+		g.Expect(ocmSpec).To(Equal(expectedOCMSpec))
+	})


Once we have polished the API above let's define unit tests for the various cases where things are left empty.

add more test for ValidateControlPlaneSpec below

damdo · 2025-09-30T10:00:16Z

/assign @nrb @damdo @richardcase

damdo · 2025-10-02T16:19:36Z

/cherry-pick release-2.9

k8s-infra-cherrypick-robot · 2025-10-02T16:19:39Z

@damdo: once the present PR merges, I will cherry-pick it on top of release-2.9 in a new PR and assign it to you.

In response to this:

/cherry-pick release-2.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

serngawy · 2025-10-02T17:25:33Z

/test pull-cluster-api-provider-aws-test

nrb

I ran KAL against this PR and it found some issues that I've marked inline.

I used the CAPI KAL configuration for the analysis.

nrb · 2025-10-02T20:06:27Z

controlplane/rosa/api/v1beta2/rosacontrolplane_types.go

 	// +optional
 	ClusterRegistryConfig *RegistryConfig `json:"clusterRegistryConfig,omitempty"`
+
+	// AutoNode set the AutoNode mode and AutoNode role ARN.


Since this will be exposed as a YAML field, the doc should start with autoNodeMode, rather than the usual Go convention.

controlplane/rosa/api/v1beta2/rosacontrolplane_types.go

Signed-off-by: serngawy <[email protected]>

damdo

/lgtm

Thanks @serngawy

k8s-ci-robot · 2025-10-06T09:22:28Z

LGTM label has been added.

Git tree hash: ad84c2beaeb2f73718631eee048733984c56952b

damdo · 2025-10-07T07:40:19Z

@richardcase @nrb if you are happy with the latest changes, let's try and get this in :) TY

richardcase · 2025-10-07T12:54:30Z

/approve

k8s-ci-robot · 2025-10-07T12:54:38Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: richardcase

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [richardcase]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-infra-cherrypick-robot · 2025-10-07T13:31:39Z

@damdo: #5686 failed to apply on top of branch "release-2.9":

Applying: Add ROSA-HCP AutoNode for karpenter auto scale
Using index info to reconstruct a base tree...
M	config/crd/bases/controlplane.cluster.x-k8s.io_rosacontrolplanes.yaml
M	controlplane/rosa/api/v1beta2/rosacontrolplane_types.go
M	controlplane/rosa/api/v1beta2/zz_generated.deepcopy.go
M	controlplane/rosa/controllers/rosacontrolplane_controller.go
M	controlplane/rosa/controllers/rosacontrolplane_controller_test.go
M	go.mod
M	go.sum
Falling back to patching base and 3-way merge...
Auto-merging go.sum
CONFLICT (content): Merge conflict in go.sum
Auto-merging go.mod
CONFLICT (content): Merge conflict in go.mod
Auto-merging controlplane/rosa/controllers/rosacontrolplane_controller_test.go
Auto-merging controlplane/rosa/controllers/rosacontrolplane_controller.go
CONFLICT (content): Merge conflict in controlplane/rosa/controllers/rosacontrolplane_controller.go
Auto-merging controlplane/rosa/api/v1beta2/zz_generated.deepcopy.go
CONFLICT (content): Merge conflict in controlplane/rosa/api/v1beta2/zz_generated.deepcopy.go
Auto-merging controlplane/rosa/api/v1beta2/rosacontrolplane_types.go
Auto-merging config/crd/bases/controlplane.cluster.x-k8s.io_rosacontrolplanes.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 Add ROSA-HCP AutoNode for karpenter auto scale

In response to this:

/cherry-pick release-2.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

serngawy · 2025-10-07T19:18:01Z

/cherry-pick release-2.9

k8s-infra-cherrypick-robot · 2025-10-07T19:18:38Z

@serngawy: new pull request created: #5699

In response to this:

/cherry-pick release-2.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-priority labels Sep 26, 2025

k8s-ci-robot requested review from damdo and luthermonson September 26, 2025 16:17

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Sep 26, 2025

robpblake reviewed Sep 26, 2025

View reviewed changes

serngawy force-pushed the karpenter branch from a63c5f5 to 6ea0a20 Compare September 26, 2025 17:39

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 26, 2025

serngawy force-pushed the karpenter branch from 6ea0a20 to db356d0 Compare September 26, 2025 17:53

robpblake reviewed Sep 29, 2025

View reviewed changes

serngawy force-pushed the karpenter branch from db356d0 to 91ca7bf Compare September 29, 2025 16:46

tinaafitz mentioned this pull request Sep 29, 2025

Add AutoNode testing framework for ROSA HCP clusters (PR #5686) tinaafitz/automation-capi#39

Open

serngawy force-pushed the karpenter branch from 91ca7bf to 33b7718 Compare September 29, 2025 17:47

k8s-ci-robot added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Sep 30, 2025

damdo reviewed Sep 30, 2025

View reviewed changes

k8s-ci-robot assigned damdo, nrb and richardcase Sep 30, 2025

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 30, 2025

serngawy force-pushed the karpenter branch from 33b7718 to 14fec18 Compare September 30, 2025 16:57

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 30, 2025

robpblake approved these changes Oct 1, 2025

View reviewed changes

serngawy force-pushed the karpenter branch from 14fec18 to 505e087 Compare October 2, 2025 16:25

nrb requested changes Oct 2, 2025

View reviewed changes

serngawy force-pushed the karpenter branch from 505e087 to a4b91ff Compare October 4, 2025 10:12

Add ROSA-HCP AutoNode for karpenter auto scale

a4b91ff

Signed-off-by: serngawy <[email protected]>

damdo reviewed Oct 6, 2025

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 6, 2025

damdo requested a review from nrb October 7, 2025 07:39

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 7, 2025

k8s-ci-robot merged commit 213faa0 into kubernetes-sigs:main Oct 7, 2025
18 checks passed

k8s-infra-cherrypick-robot mentioned this pull request Oct 7, 2025

[release-2.9] ✨ ROSA: Add ROSA-HCP AutoNode for karpenter auto scale #5699

Merged

✨ ROSA: Add ROSA-HCP AutoNode for karpenter auto scale #5686

✨ ROSA: Add ROSA-HCP AutoNode for karpenter auto scale #5686

Conversation

serngawy commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serngawy Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

robpblake Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

damdo commented Sep 30, 2025

Uh oh!

damdo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

damdo commented Sep 30, 2025

Uh oh!

damdo commented Oct 2, 2025

Uh oh!

k8s-infra-cherrypick-robot commented Oct 2, 2025

Uh oh!

serngawy commented Oct 2, 2025

Uh oh!

nrb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

damdo left a comment

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Oct 6, 2025

Uh oh!

damdo commented Oct 7, 2025

Uh oh!

richardcase commented Oct 7, 2025

Uh oh!

k8s-ci-robot commented Oct 7, 2025

Uh oh!

Uh oh!

k8s-infra-cherrypick-robot commented Oct 7, 2025

Uh oh!

serngawy commented Oct 7, 2025

Uh oh!

k8s-infra-cherrypick-robot commented Oct 7, 2025

Uh oh!

Uh oh!

serngawy commented Sep 26, 2025 •

edited

Loading

serngawy Sep 26, 2025 •

edited

Loading

robpblake Sep 26, 2025 •

edited

Loading