Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add k8s 1.24 variants #2437

Merged
merged 9 commits into from
Oct 7, 2022
Merged

Conversation

arnaldo2792
Copy link
Contributor

@arnaldo2792 arnaldo2792 commented Sep 19, 2022

Issue number:

Closes #2374

Description of changes:

sources: add symlink for metal-k8s-1.23 variant
sources: add symlink for vmware-k8s-1.23 variant
sources: add symlink for aws-k8s-1.23 variant
sources,variants: add metal-k8s-1.24 variant
sources,variants: add vmware-k8s-1.24 variant
sources,variants: add aws-k8s-1.24 variant
packages: add kubernetes 1.24

As part of this PR, the k8s 1.23 variants directories in sources/model/src/* were replaced with a symlink to the new k8s-1.24 variant for each platform.

Starting with k8s 1.24, all command-line parameters to manage CNI plugins with the kubelet were removed, as part of the efforts to clean up any remaining work related to the dockershim.

Testing done:

  • Test aws variant: the nodes joined a 1.23/1.24 cluster , and pods were successfully deployed to them.
[ec2-user@admin]$ apiclient get os
{
  "os": {
    "arch": "x86_64",
    "build_id": "e08be563",
    "pretty_name": "Bottlerocket OS 1.10.0 (aws-k8s-1.24)",
    "variant_id": "aws-k8s-1.24",
    "version_id": "1.10.0"
  }
}
[ec2-user@admin]$ uname -a
Linux <ip>.us-west-2.compute.internal 5.15.54 #1 SMP Mon Sep 19 18:51:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  • Test vmware variant (Note: tested in a 1.23 cluster)
  • Test metal variant (Note: tested in a 1.23 cluster)
  • Build 1.23 variants
  • Test ipv6 clusters

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@etungsten
Copy link
Contributor

Remember to add and *-k8s.1.24* to the list of github actions workflows variant matrix as part of this PR.

@markusboehme
Copy link
Member

There is a dependency between this PR and #2460. Whichever merges last will have to update the kubelet config template for k8s 1.24.

@stmcginnis
Copy link
Contributor

Thanks for pointing that out. There's also work in #2377 that will need to be adjusted for this PR. I'll watch for this PR to go through and update accordingly, otherwise will make sure to help out here with any other changes.

@arnaldo2792
Copy link
Contributor Author

(Forced push due to rebase onto develop)

@arnaldo2792
Copy link
Contributor Author

Forced push includes:

@arnaldo2792
Copy link
Contributor Author

(Rebase onto develop)

@kschumy
Copy link
Contributor

kschumy commented Oct 5, 2022

The changes I suggested above are for the prod release we just cut. Conformance tests are running now. They're expected to pass in a few hours. I'll update y'all either way in the morning

@kschumy
Copy link
Contributor

kschumy commented Oct 5, 2022

Conformance tests passed for EKS-D

@arnaldo2792
Copy link
Contributor Author

( Forced push updates documentation and kubelet sources )

@arnaldo2792
Copy link
Contributor Author

(Forced push fixes compatibility list in new k8s variants)

Copy link
Contributor

@jpmcb jpmcb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏🏼

Testing done:

  • Built the aws-k8s-1.24 variant with cargo make -e BUILDSYS_VARIANT=aws-k8s-1.24 and deployed an ami with cargo make -e BUILDSYS_VARIANT=aws-k8s-1.24 ami
  • Deployed a k8s 1.23 cluster with the above ami via eksctl:
#!/usr/bin/env bash

eksctl create cluster \
    --name br-test \
    --version 1.23 \
    --region us-west-2 \
    --nodegroup-name br-ng \
    --node-type c5.xlarge \
    --node-ami ami-{ID}
  • Nodes are up look good and via ssm I'm able to see the 1.24 version. I deployed the prometheues stack and see pods scheduled on the nodes.

Comment on lines 1 to 7
# Don't generate debug packages because we are compiling without CGO,
# and the `gc` compiler doesn't append the the ".note.gnu.build-id" section
# https://fedoraproject.org/wiki/PackagingDrafts/Go#Build_ID
%global debug_package %{nil}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem correct - if this is happening we might not be building kubelet with the flags we expect.

{{/unless}}
--config /etc/kubernetes/kubelet/config \
--container-runtime=remote \
--container-runtime-endpoint=unix:///run/containerd/containerd.sock \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--network-plugin cni is no longer supported?

It'd be good to cover changes like this in the commit message, since the assumption otherwise is that nothing changed beyond the obvious stuff like versions.

The [aws-k8s-1.24-nvidia](aws-k8s-1.24-nvidia/Cargo.toml) variant includes the packages needed to run a Kubernetes node in AWS.
It also includes the required packages to configure containers to leverage NVIDIA GPUs.
It supports self-hosted clusters and clusters managed by [EKS](https://aws.amazon.com/eks/).
This variant is compatible with Kubernetes 1.23, 1.24, and 1.25 clusters.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be "1.24, 1.25, and 1.26"

@arnaldo2792
Copy link
Contributor Author

( forced push adds more context to the message of the commit to add the new kubelet sources )

@arnaldo2792
Copy link
Contributor Author

( forced push fixes docs and adds missing github actions for NVIDIA variant )

This version is compiled without support for debug symbols, since the
way the Makefile is written make optimizations and debug symbols
mutually exclusive.

Starting with Kubernetes 1.24, all command-line parameters to manage
CNI plugins with the kubelet were removed, that includes the flag
`--network-plugin` previously used in older versions. No additional
changes were required in this version to support CNI plugins.

Signed-off-by: Arnaldo Garcia Rincon <[email protected]>
Signed-off-by: Arnaldo Garcia Rincon <[email protected]>
The symlink points to the newer aws-k8s-1.24 variant

Signed-off-by: Arnaldo Garcia Rincon <[email protected]>
The symlink points to the newer vmware-k8s-1.24 variant

Signed-off-by: Arnaldo Garcia Rincon <[email protected]>
The symlink points to the newer metal-k8s-1.24 variant

Signed-off-by: Arnaldo Garcia Rincon <[email protected]>
@arnaldo2792
Copy link
Contributor Author

( forced push fixes comment about the missing debug symbols in generated binaries )

@arnaldo2792 arnaldo2792 merged commit d1e1658 into bottlerocket-os:develop Oct 7, 2022
@arnaldo2792 arnaldo2792 deleted the k8s-1.24 branch October 26, 2022 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kubernetes 1.24 support
7 participants