-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✨ Add bootCommands to cloud-init file generation #11271
base: main
Are you sure you want to change the base?
Changes from all commits
31406da
97311d6
4367074
0c23af8
504dfd5
19d52df
51068a2
8ed0171
332e78d
28edd6c
8918ff3
94835dc
84e06c6
bd63a05
a4f3bdb
4673fb0
c059138
6fb468d
810f7d6
03cbf01
b7c575a
0484cdc
3db48fa
23a608c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
/* | ||
Copyright 2025 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package cloudinit | ||
|
||
const ( | ||
bootCommandsTemplate = `{{ define "boot_commands" -}} | ||
{{- if . }} | ||
bootcmd:{{ range . }} | ||
- {{ range . }}- {{ . }} | ||
{{ end -}} | ||
{{- end -}} | ||
{{- end -}} | ||
{{- end -}} | ||
` | ||
) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -34,6 +34,7 @@ func TestNewInitControlPlaneAdditionalFileEncodings(t *testing.T) { | |
cpinput := &ControlPlaneInput{ | ||
BaseUserData: BaseUserData{ | ||
Header: "test", | ||
BootCommands: nil, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we have a test for BootCommands not nil for init, join CP, join workers? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I added non-nil values for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. WRT to changes in TestNewInitControlPlaneCommands, TestNewJoinControlPlaneAdditionalFileEncodings I have dedicated comments. We still need test coverage for join and join CP (if they are missing, kindly add them so we pay down some tech debt before adding more on top) |
||
PreKubeadmCommands: nil, | ||
PostKubeadmCommands: nil, | ||
AdditionalFiles: []bootstrapv1.File{ | ||
|
@@ -94,9 +95,12 @@ func TestNewInitControlPlaneCommands(t *testing.T) { | |
|
||
cpinput := &ControlPlaneInput{ | ||
BaseUserData: BaseUserData{ | ||
BootCommands: []bootstrapv1.BootCommand{ | ||
{"echo", "$(date) hello BootCommands!"}, | ||
}, | ||
Header: "test", | ||
PreKubeadmCommands: []string{`"echo $(date) ': hello world!'"`}, | ||
PostKubeadmCommands: []string{"echo $(date) ': hello world!'"}, | ||
PreKubeadmCommands: []string{`"echo $(date) ': hello PreKubeadmCommands!'"`}, | ||
PostKubeadmCommands: []string{"echo $(date) ': hello PostKubeadmCommands!'"}, | ||
AdditionalFiles: nil, | ||
WriteFiles: nil, | ||
Users: nil, | ||
|
@@ -117,13 +121,18 @@ func TestNewInitControlPlaneCommands(t *testing.T) { | |
out, err := NewInitControlPlane(cpinput) | ||
g.Expect(err).ToNot(HaveOccurred()) | ||
|
||
expectedCommands := []string{ | ||
`"\"echo $(date) ': hello world!'\""`, | ||
`"echo $(date) ': hello world!'"`, | ||
} | ||
for _, f := range expectedCommands { | ||
g.Expect(out).To(ContainSubstring(f)) | ||
} | ||
expectedBootCmd := `bootcmd: | ||
- - echo | ||
- $(date) hello BootCommands!` | ||
|
||
g.Expect(out).To(ContainSubstring(expectedBootCmd)) | ||
|
||
expectedRunCmd := `runcmd: | ||
- "\"echo $(date) ': hello PreKubeadmCommands!'\"" | ||
- 'kubeadm init --config /run/kubeadm/kubeadm.yaml && echo success > /run/cluster-api/bootstrap-success.complete' | ||
- "echo $(date) ': hello PostKubeadmCommands!'"` | ||
|
||
g.Expect(out).To(ContainSubstring(expectedRunCmd)) | ||
} | ||
|
||
func TestNewInitControlPlaneDiskMounts(t *testing.T) { | ||
|
@@ -132,6 +141,7 @@ func TestNewInitControlPlaneDiskMounts(t *testing.T) { | |
cpinput := &ControlPlaneInput{ | ||
BaseUserData: BaseUserData{ | ||
Header: "test", | ||
BootCommands: nil, | ||
PreKubeadmCommands: nil, | ||
PostKubeadmCommands: nil, | ||
WriteFiles: nil, | ||
|
@@ -194,6 +204,7 @@ func TestNewJoinControlPlaneAdditionalFileEncodings(t *testing.T) { | |
|
||
cpinput := &ControlPlaneJoinInput{ | ||
BaseUserData: BaseUserData{ | ||
BootCommands: nil, | ||
Header: "test", | ||
PreKubeadmCommands: nil, | ||
PostKubeadmCommands: nil, | ||
|
@@ -247,6 +258,7 @@ func TestNewJoinControlPlaneExperimentalRetry(t *testing.T) { | |
cpinput := &ControlPlaneJoinInput{ | ||
BaseUserData: BaseUserData{ | ||
Header: "test", | ||
BootCommands: nil, | ||
PreKubeadmCommands: nil, | ||
PostKubeadmCommands: nil, | ||
UseExperimentalRetry: true, | ||
|
@@ -315,3 +327,83 @@ func Test_useKubeadmBootstrapScriptPre1_31(t *testing.T) { | |
}) | ||
} | ||
} | ||
|
||
func TestNewJoinControlPlaneCommands(t *testing.T) { | ||
g := NewWithT(t) | ||
|
||
cpinput := &ControlPlaneJoinInput{ | ||
BaseUserData: BaseUserData{ | ||
BootCommands: []bootstrapv1.BootCommand{ | ||
{"echo", "$(date) hello BootCommands!"}, | ||
}, | ||
Header: "test", | ||
PreKubeadmCommands: []string{`"echo $(date) ': hello PreKubeadmCommands!'"`}, | ||
PostKubeadmCommands: []string{"echo $(date) ': hello PostKubeadmCommands!'"}, | ||
AdditionalFiles: nil, | ||
WriteFiles: nil, | ||
Users: nil, | ||
NTP: nil, | ||
}, | ||
Certificates: secret.Certificates{}, | ||
JoinConfiguration: "my-join-config", | ||
} | ||
|
||
for _, certificate := range cpinput.Certificates { | ||
certificate.KeyPair = &certs.KeyPair{ | ||
Cert: []byte("some certificate"), | ||
Key: []byte("some key"), | ||
} | ||
} | ||
|
||
out, err := NewJoinControlPlane(cpinput) | ||
g.Expect(err).ToNot(HaveOccurred()) | ||
|
||
expectedBootCmd := `bootcmd: | ||
- - echo | ||
- $(date) hello BootCommands!` | ||
|
||
g.Expect(out).To(ContainSubstring(expectedBootCmd)) | ||
|
||
expectedRunCmd := `runcmd: | ||
- "\"echo $(date) ': hello PreKubeadmCommands!'\"" | ||
- kubeadm join --config /run/kubeadm/kubeadm-join-config.yaml && echo success > /run/cluster-api/bootstrap-success.complete | ||
- "echo $(date) ': hello PostKubeadmCommands!'"` | ||
|
||
g.Expect(out).To(ContainSubstring(expectedRunCmd)) | ||
} | ||
|
||
func TestNewJoinNodeCommands(t *testing.T) { | ||
g := NewWithT(t) | ||
|
||
nodeinput := &NodeInput{ | ||
BaseUserData: BaseUserData{ | ||
BootCommands: []bootstrapv1.BootCommand{ | ||
{"echo", "$(date) hello BootCommands!"}, | ||
}, | ||
Header: "test", | ||
PreKubeadmCommands: []string{`"echo $(date) ': hello PreKubeadmCommands!'"`}, | ||
PostKubeadmCommands: []string{"echo $(date) ': hello PostKubeadmCommands!'"}, | ||
AdditionalFiles: nil, | ||
WriteFiles: nil, | ||
Users: nil, | ||
NTP: nil, | ||
}, | ||
JoinConfiguration: "my-join-config", | ||
} | ||
|
||
out, err := NewNode(nodeinput) | ||
g.Expect(err).ToNot(HaveOccurred()) | ||
|
||
expectedBootCmd := `bootcmd: | ||
- - echo | ||
- $(date) hello BootCommands!` | ||
|
||
g.Expect(out).To(ContainSubstring(expectedBootCmd)) | ||
|
||
expectedRunCmd := `runcmd: | ||
- "\"echo $(date) ': hello PreKubeadmCommands!'\"" | ||
- kubeadm join --config /run/kubeadm/kubeadm-join-config.yaml && echo success > /run/cluster-api/bootstrap-success.complete | ||
- "echo $(date) ': hello PostKubeadmCommands!'"` | ||
|
||
g.Expect(out).To(ContainSubstring(expectedRunCmd)) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there is no equivalent for the ignition format, it is good to prevent user confusion by introducing a validation webhook check on
kubeadmConfig.spec.format == “cloud-init”
for this field to be set.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm unfamiliar with ignition so I don't know if there is an equivalent format 😕. Where would you suggest to put this validation webhook check?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bootstrap/kubeadm/internal/webhooks/kubeadmconfig.go
seems to be the place for validation webhooks codeThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean to validate that
format == "clout-init"
only if ignition is not used? Is that based onfeature.KubeadmBootstrapFormatIgnition
or is there another way of checking if ignition should be used?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Format field on the config designates the type of configuration used. If it is cloud-init, then it is not ignition :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay thanks, is this how you meant? b7c575a
Also, I believe the default value for the format is "cloud-config", but please let me know if I misunderstood and it should be "cloud-init".