Skip to content

🌱 feat: implements nodeadm bootstrapping type #5700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

🌱 feat: implements nodeadm bootstrapping type #5700

wants to merge 4 commits into from

Conversation

@faiq
Copy link
Contributor

@faiq faiq commented Oct 7, 2025 •
edited
Loading

What type of PR is this?
/kind feature

What this PR does / why we need it:

This PR implements the nodeadm config type outlined by the KEP #5678

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Checklist:

  • squashed commits
  • includes documentation
  • includes emoji in title
  • adds unit tests
  • adds or updates e2e tests

Release note:

Adds Nodeadm bootstrapping type

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. release-note Denotes a PR that will be considered when it comes time to generate release notes. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 7, 2025
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Oct 7, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign neolit123 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested review from AndiDog and nrb October 7, 2025 20:30
@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Oct 7, 2025
@faiq
Copy link
Contributor Author

faiq commented Oct 7, 2025

testing with this manifest

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: default
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - 192.168.0.0/16
    services:
      cidrBlocks:
      - 10.96.0.0/12
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta2
    kind: AWSManagedControlPlane
    name: default-control-plane
  infrastructureRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1beta2
    kind: AWSManagedControlPlane
    name: default-control-plane
---
apiVersion: controlplane.cluster.x-k8s.io/v1beta2
kind: AWSManagedControlPlane
metadata:
  name: default-control-plane
spec:
  addons:
  - name: kube-proxy
    version: v1.32.0-eksbuild.2
  network:
    cni:
      cniIngressRules:
      - description: kube-proxy metrics
        fromPort: 10249
        protocol: tcp
        toPort: 10249
      - description: NVIDIA Data Center GPU Manager metrics
        fromPort: 9400
        protocol: tcp
        toPort: 9400
      - description: Prometheus node exporter metrics
        fromPort: 9100
        protocol: tcp
        toPort: 9100
  region: us-west-2
  sshKeyName: ""
  version: v1.33.0
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
metadata:
  name: default
spec:
  template:
    spec:
      cloudInit:
        insecureSkipSecretsManager: true
      ami:
        eksLookupType: AmazonLinux2023
      instanceMetadataOptions:
        httpTokens: required
        httpPutResponseHopLimit: 2
      iamInstanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io
      instanceType: m5a.16xlarge
      rootVolume:
        size: 80
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta2
kind: NodeadmConfigTemplate
metadata:
  name: default
spec:
  template:
    spec: {}
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: default
spec:
  clusterName: default
  replicas: 1
  template:
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1beta2
          kind: NodeadmConfigTemplate
          name: default
      clusterName: default
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
        kind: AWSMachineTemplate
        name: default
      version: v1.33.0

@faiq
Copy link
Contributor Author

faiq commented Oct 7, 2025

/retest

2 similar comments
@faiq
Copy link
Contributor Author

faiq commented Oct 8, 2025

/retest

@faiq
Copy link
Contributor Author

faiq commented Oct 8, 2025

/retest

@faiq faiq force-pushed the faiq/nodeadm-upstream branch 2 times, most recently from 8f854bd to 81f3664 Compare October 8, 2025 20:43
@faiq
Copy link
Contributor Author

faiq commented Oct 8, 2025

/test ?

@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Oct 8, 2025

@faiq: The following commands are available to trigger required jobs:

/test pull-cluster-api-provider-aws-build

/test pull-cluster-api-provider-aws-build-docker

/test pull-cluster-api-provider-aws-e2e-blocking

/test pull-cluster-api-provider-aws-test

/test pull-cluster-api-provider-aws-verify

The following commands are available to trigger optional jobs:

/test pull-cluster-api-provider-aws-apidiff-main

/test pull-cluster-api-provider-aws-e2e

/test pull-cluster-api-provider-aws-e2e-clusterclass

/test pull-cluster-api-provider-aws-e2e-conformance

/test pull-cluster-api-provider-aws-e2e-conformance-with-ci-artifacts

/test pull-cluster-api-provider-aws-e2e-eks

/test pull-cluster-api-provider-aws-e2e-eks-gc

/test pull-cluster-api-provider-aws-e2e-eks-testing

Use /test all to run the following jobs that were automatically triggered:

pull-cluster-api-provider-aws-apidiff-main

pull-cluster-api-provider-aws-build

pull-cluster-api-provider-aws-build-docker

pull-cluster-api-provider-aws-e2e-blocking

pull-cluster-api-provider-aws-test

pull-cluster-api-provider-aws-verify

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@faiq
Copy link
Contributor Author

faiq commented Oct 8, 2025

/test /test pull-cluster-api-provider-aws-e2e-eks

@dsanders1234
Copy link

dsanders1234 commented Oct 9, 2025

Does this work with AWSManagedMachinePool ?

@faiq faiq changed the title 🌱 feat: starts bootstrapping new type with kubebuilder 🌱 feat: implements nodeadm bootstrapping type Oct 9, 2025
@faiq faiq force-pushed the faiq/nodeadm-upstream branch from 81f3664 to 59ecae0 Compare October 9, 2025 19:01
@faiq
Copy link
Contributor Author

faiq commented Oct 9, 2025

@dsanders1234 try this

 apiVersion: cluster.x-k8s.io/v1beta1
 kind: MachinePool
 metadata:
   name: default
 spec:
   clusterName: default
   template:
     spec:
       bootstrap:
         #dataSecretName: ""
         configRef:
           apiVersion: bootstrap.cluster.x-k8s.io/v1beta2
           kind: NodeadmConfig
           name: default
       clusterName: default
       infrastructureRef:
         apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
         kind: AWSManagedMachinePool
         name: default
       version: v1.33.0
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSManagedMachinePool
metadata:
  name: default
spec:
  roleName: "nodes.cluster-api-provider-aws.sigs.k8s.io"
  scaling:
    minSize: 1
    maxSize: 3
  amiType: CUSTOM
  awsLaunchTemplate:
    ami:
      eksLookupType: AmazonLinux2023
    instanceMetadataOptions:
      httpTokens: required
      httpPutResponseHopLimit: 2
    instanceType: "m5a.16xlarge"
    rootVolume:
      size: 80
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta2
kind: NodeadmConfig
metadata:
  name: default
spec:
  kubelet:
    config:
      evictionHard:
        memory.available: "2000Mi"

@faiq
Copy link
Contributor Author

faiq commented Oct 9, 2025

/test pull-cluster-api-provider-aws-e2e-eks

@faiq faiq force-pushed the faiq/nodeadm-upstream branch 2 times, most recently from 10503ae to 476eb43 Compare October 9, 2025 20:20
@faiq
Copy link
Contributor Author

faiq commented Oct 9, 2025

/test pull-cluster-api-provider-aws-e2e-eks

@faiq
Copy link
Contributor Author

faiq commented Oct 9, 2025

/retest

3 similar comments
@faiq
Copy link
Contributor Author

faiq commented Oct 9, 2025

/retest

@faiq
Copy link
Contributor Author

faiq commented Oct 9, 2025

/retest

@faiq
Copy link
Contributor Author

faiq commented Oct 9, 2025

/retest

@faiq faiq force-pushed the faiq/nodeadm-upstream branch from 476eb43 to 234d905 Compare October 10, 2025 02:57
@faiq
Copy link
Contributor Author

faiq commented Nov 4, 2025

/retest

3 similar comments
@faiq
Copy link
Contributor Author

faiq commented Nov 4, 2025

/retest

@faiq
Copy link
Contributor Author

faiq commented Nov 4, 2025

/retest

@faiq
Copy link
Contributor Author

faiq commented Nov 4, 2025

/retest

@faiq faiq force-pushed the faiq/nodeadm-upstream branch from 9282218 to 6b78f52 Compare November 4, 2025 14:48
@faiq
Copy link
Contributor Author

faiq commented Nov 4, 2025

/test pull-cluster-api-provider-aws-e2e-eks

@faiq faiq force-pushed the faiq/nodeadm-upstream branch from 6b78f52 to 81960e5 Compare November 4, 2025 16:45
@faiq
Copy link
Contributor Author

faiq commented Nov 4, 2025

/test pull-cluster-api-provider-aws-e2e-eks

AmitSahastra
AmitSahastra reviewed Nov 5, 2025
View reviewed changes
@AmitSahastra
Copy link
Contributor

AmitSahastra commented Nov 5, 2025 •
edited
Loading

@faiq Thanks for this! 🙌 I tested EKS with the MP + Launch Template + AL2023 + NodeadmConfig and it worked as expected.
One note: I had to manually add an MachinePool OwnerReference to NodeadmConfig (CAPI didn’t set it automatically in this scenario). Is that expected?

Also, is there a tentative merge timeline or any remaining blockers.

@faiq
Copy link
Contributor Author

faiq commented Nov 5, 2025

One note: I had to manually add an MachinePool OwnerReference to NodeadmConfig (CAPI didn’t set it automatically in this scenario). Is that expected?

thats strange. i feel like this is a bug in the machinepool controller. shouldn't it be setting the owner ref?

@faiq faiq force-pushed the faiq/nodeadm-upstream branch from 81960e5 to a10f273 Compare November 5, 2025 18:41
@faiq
Copy link
Contributor Author

faiq commented Nov 7, 2025

/test pull-cluster-api-provider-aws-e2e-eks

1 similar comment
@faiq
Copy link
Contributor Author

faiq commented Nov 10, 2025

/test pull-cluster-api-provider-aws-e2e-eks

@faiq faiq force-pushed the faiq/nodeadm-upstream branch from ac77773 to a4a638e Compare November 10, 2025 19:27
@faiq
Copy link
Contributor Author

faiq commented Nov 10, 2025

/test pull-cluster-api-provider-aws-e2e-eks

@faiq faiq force-pushed the faiq/nodeadm-upstream branch from a4a638e to 6f464e0 Compare November 10, 2025 20:01
@faiq
Copy link
Contributor Author

faiq commented Nov 10, 2025

/test pull-cluster-api-provider-aws-e2e-eks

@faiq faiq force-pushed the faiq/nodeadm-upstream branch from 6f464e0 to e89afb2 Compare November 10, 2025 21:32
@faiq
Copy link
Contributor Author

faiq commented Nov 10, 2025

/test pull-cluster-api-provider-aws-e2e-eks

@faiq faiq force-pushed the faiq/nodeadm-upstream branch from e89afb2 to a9bf155 Compare November 10, 2025 22:50
@faiq
Copy link
Contributor Author

faiq commented Nov 10, 2025

/test pull-cluster-api-provider-aws-e2e-eks

@faiq faiq force-pushed the faiq/nodeadm-upstream branch from a9bf155 to d657f69 Compare November 11, 2025 00:25
@faiq
Copy link
Contributor Author

faiq commented Nov 11, 2025

/test pull-cluster-api-provider-aws-e2e-eks

@faiq
Copy link
Contributor Author

faiq commented Nov 11, 2025 •
edited
Loading

The last test failed in deleting the aws managed control plane, which is entirely unrelated to these changes.

EDITS:

see run here: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cluster-api-provider-aws/5700/pull-cluster-api-provider-aws-e2e-eks/1988044875567730688

i'll run the tests once again to get that sweet ✔️

@faiq
Copy link
Contributor Author

faiq commented Nov 11, 2025

/test pull-cluster-api-provider-aws-e2e-eks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AndiDog AndiDog Awaiting requested review from AndiDog

@nrb nrb Awaiting requested review from nrb

1 more reviewer

@AmitSahastra AmitSahastra AmitSahastra left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. needs-priority release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@faiq @k8s-ci-robot @dsanders1234 @AmitSahastra