Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow cluster-dns-ip to be set to a list of IP Addresses #2176

Merged
merged 2 commits into from
Jun 6, 2022

Conversation

cbgbt
Copy link
Contributor

@cbgbt cbgbt commented Jun 3, 2022

Issue number: #2132

Description of changes:
This allows settings.kubernetes.cluster-dns-ip to be set to a list. The value may also remain as a string, for backwards compatibility. The model will store the data using the type submitted, so existing code expecting string types will continue to work.

If a list is present, downwards migrations will truncate the list, using the first element as the configured value.

Testing done:

  • Ensure that /etc/kubernetes/kubelet/config contains clusterDNS settings, regardless of if the setting is a string or a list
  • Push the new migrations to a TUF repo and ensure they work as expected
  • Unit tests
  • Test on all modified k8s versions (1.19, 1.20, 1.21, 1.22)
  • Launch a pod with multiple DNS IPs configured and assert that /etc/resolv.conf contains them

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@cbgbt cbgbt force-pushed the clusterdnsip branch 2 times, most recently from c5aacb6 to c8ba9db Compare June 3, 2022 19:15
@cbgbt
Copy link
Contributor Author

cbgbt commented Jun 3, 2022

Fixed formatting. I always forget to run check-fmt 😮‍💨

@webern webern self-requested a review June 3, 2022 19:46
Comment on lines +32 to +36
{{#each settings.kubernetes.cluster-dns-ip}}
- {{this}}
{{else}}
- {{settings.kubernetes.cluster-dns-ip}}
{{/each}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to make sure this ends up in 1.23 as well, if #2140 lands first.

/// for backwards compatibility.
#[derive(Debug, Clone, Eq, PartialEq, Hash, Serialize, Deserialize)]
#[serde(untagged)]
pub enum KubernetesClusterDNSIp {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this instead?

Suggested change
pub enum KubernetesClusterDNSIp {
pub enum KubernetesClusterDnsIp {


/// KubernetesClusterDNSIp represents the --cluster-dns settings for kubelet.
///
/// The default is determined via information from IMDS in AWS.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to mention where the default comes from? Feels like a different concern.

#[test]
fn test_parse_cluster_dns_ip_from_list() {
assert_eq!(
serde_json::from_str::<KubernetesClusterDNSIp>(r#"[]"#).unwrap(),
KubernetesClusterDNSIp::Vector(vec![])
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't allow an empty string, should we allow an empty list?

I am OK either way as long as this doesn't cause trouble with kubelet when rendering the template:

[settings.kubernetes]
cluster-dns-ip = []

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guard in the kubelet configs protects the config from an empty list, as empty lists are considered "falsey" by handlebars.

I've tested empirically and confirmed that an empty list results in clusterDNS being absent from the kubelet config.

I think an empty string is "less correct" model-wise than an empty list. I'd like the consistency of disallowing both, but I'm not sure it's worth writing the custom deserializer since there's not an easy way to enforce the constraint in serde otherwise.

Comment on lines 1029 to 1031
pub fn first(&self) -> Option<&IpAddr> {
self.iter().next()
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why add a special first method vs. making callers write iter().next() ? It seems like giving them the iterator and letting them use any of the trait methods is more powerful.

Comment on lines +47 to +48
.and_then(|ip_array| ip_array.iter().next())
.map(|ip_value| ip_value.clone());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking my understanding here also -

If we have a value and it's the empty list, we'll have a None here, and remove the setting from the datastore. The previous template had a guard for whether the setting was defined, so this is safe - we'll render a config file without that section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's right.

packages/kubernetes-1.20/kubelet-config Show resolved Hide resolved
@cbgbt
Copy link
Contributor Author

cbgbt commented Jun 6, 2022

Force push to address feedback from @bcressey.

}
}

#[cfg(test)]
Copy link
Contributor

@webern webern Jun 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice tests.

Edit: somehow my other comment on the migration tests was lost. Basically, awesome that we have migration tests! 🚀

let cluster_dns_ip = if let Some(ip) = aws_k8s_info.cluster_dns_ip {
let configured_cluster_dns_ip = aws_k8s_info
.cluster_dns_ip
.and_then(|cluster_ip| cluster_ip.iter().next().cloned());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe check clippy on this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It didn't have any tips here, but it did suggest some others in pluto, so I pushed some small changes.

cbgbt added 2 commits June 6, 2022 20:52
While clusterDNS is modeled as a list in the kubelet config, we allow
the settings to be given as either a string or list in the API for
backwards compatibility. The API will store and return the data
as-provided, and the model provides a unified iterator view over both
models.
@cbgbt
Copy link
Contributor Author

cbgbt commented Jun 6, 2022

Rebase on develop. Fix two clippy warnings in pluto.

@cbgbt cbgbt merged commit 5d616e8 into bottlerocket-os:develop Jun 6, 2022
Copy link
Contributor

@zmrow zmrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

🧑‍🔧

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants