Add expanded DNS configuration KEP #2596

gjkim42 · 2021-04-02T03:16:52Z

Enhancement issue: #2595

previous discussion: kubernetes/kubernetes#100583

gjkim42 · 2021-04-02T11:35:50Z

/cc @aojea @thockin @liggitt

keps/sig-network/2595-expanded-dns-config/kep.yaml

keps/sig-network/2595-expanded-dns-config/README.md

aojea · 2021-04-02T12:54:58Z

keps/sig-network/2595-expanded-dns-config/README.md

+#### Alpha -> Beta Graduation
+
+- Address feedback from alpha
+- Sufficient testing
+
+#### Beta -> GA Graduation


should we go to beta directly and enable it by default?

I think it would be ok to enable it by default.

the relaxed validation cannot allow new expanded data into objects by default in the first release, since it would allow data the previous API server version would choke on

@liggitt
Isn't it enough to wrap the feature into the feature gate to make the user can disable it if there is such a case?

no. that means that, by default, the first version of the API server including this feature allows storing unsafe data. we must default safe.

Ah... I think you are right. Let's go to alpha and disable the feature by default.

Agree. We can very likely skip beta, but we can't skip alpha

aojea · 2021-04-02T12:55:58Z

keps/sig-network/2595-expanded-dns-config/kep.yaml

+status: implementable
+creation-date: 2021-04-02
+reviewers:
+  - TBD


@claudiubelu do you know the implications of this resolv.conf changes in windows?

keps/sig-network/2595-expanded-dns-config/README.md

aojea · 2021-04-02T13:04:33Z

keps/sig-network/2595-expanded-dns-config/README.md

+-->
+
+In HA(highly-available) clusters, all kube-apiservers should enable or disable
+expanded DNS configuration feature.


there should be no problem meanwhile they don't create pods with an expanded DNSConfig

I think we should avoid the situation that there is a HA cluster with 1.22 apiserver and 1.21 apiserver having a pod with expanded DNS configuration due to partial roll-back.
If a user updates the pod, it can be updated by 1.22 apiserver or blocked by 1.21 apiserver. That will be an inconsistent behavior.

aojea · 2021-04-02T13:07:07Z

LGTM, this was previously discussed here

the problem was discussed here kubernetes/kubernetes#100583
and the implementation is not a big change
kubernetes/kubernetes#100622

the KEP and the feature gate will help to document the feature and give us the possibility to avoid surprises :)

thockin · 2021-04-30T20:31:43Z

What does this need in terms of docs?

gjkim42 · 2021-04-30T21:43:37Z

I think we need some documentation updates like

adding a description of the feature gate
(maybe) updating https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues

Linux's libc (a.k.a. glibc) has a limit for the DNS nameserver records to 3 by default. What's more, for the glibc versions which are older than glibc-2.17-222 (the new versions update see this issue), the allowed number of DNS search records has been limited to 6 (see this bug from 2005). Kubernetes needs to consume 1 nameserver record and 3 search records. This means that if a local installation already uses 3 nameservers or uses more than 3 searches while your glibc version is in the affected list, some of those settings will be lost. To work around the DNS nameserver records limit, the node can run dnsmasq, which will provide more nameserver entries. You can also use kubelet's --resolv-conf flag. To fix the DNS search records limit, consider upgrading your linux distribution or upgrading to an unaffected version of glibc.

gjkim42 · 2021-05-01T05:42:33Z

Only updated the release signoff checklist to the desired state.

thockin · 2021-05-03T15:53:39Z

keps/sig-network/2595-expanded-dns-config/README.md

+
+### Version Skew Strategy
+
+In clusters with a replicated control plane, all kube-apiservers should enable


This is more about kubelets. It is valid to have kubelets be a (slightly) lower version than apiserver. Looking at kubelet code I can see this will throw warnings (pkg/kubelet/network/dns/dns.go Configurer.CheckLimitsForResolvConf()) but that does not hard-fail. Still worth noting here.

In the same file, Configurer.formDNSSearchFitsLimits(), it does truncate. That is what is important to note here (and to test-confirm) -- back-rev kubelet should be fine, it will just truncate the results.

Updated.

thanks @thockin

gjkim42 · 2021-05-07T10:27:34Z

@johnbelamaric
Would you please take a look and give a PRR? I hope we can catch the v1.22 enhancement freeze date (May 13th).
Thanks :)

keps/sig-network/2595-expanded-dns-config/README.md

gjkim42 · 2021-05-08T03:17:11Z

@johnbelamaric
Thanks!
Updated the PRR questionnaire section.

gjkim42 · 2021-05-10T04:47:58Z

@thockin @johnbelamaric
Added graduation criteria for alpha.
Could you PTAL? Thanks

johnbelamaric · 2021-05-10T16:43:53Z

/approve

k8s-ci-robot · 2021-05-10T16:44:06Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gjkim42, johnbelamaric, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~keps/prod-readiness/OWNERS~~ [johnbelamaric]
~~keps/sig-network/OWNERS~~ [johnbelamaric,thockin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

johnbelamaric · 2021-05-10T16:45:13Z

/lgtm

dcbw · 2021-05-13T20:39:30Z

keps/sig-network/2595-expanded-dns-config/README.md

+
+Yes, the feature can be disabled by disabling the feature gate.
+
+Once the feature is disabled, kube-apiserver will reject the pod having expanded


@gjkim42 what happens to existing pods in the API when the feature gate is disabled, and then something tries to update non-DNS podSpec fields? Do those things just their update denied, even though they didn't change any relevant fields?

what happens to existing pods in the API when the feature gate is disabled

The existing pods will be running with the expanded DNS config.

and then something tries to update non-DNS podSpec fields? Do those things just their update denied, even though they didn't change any relevant fields?

In v1.22 or higher, you can update non-DNSConfig podSpec fields, because we will add the exception.
(See the PR in progress.)
In v1.21 or lower(downgrade), any update requests will be denied, if the pod already had expanded DNS config.

ritpanjw · 2021-05-18T01:09:51Z

Hello @gjkim42 👋 , 1.22 Docs Shadow here.

This enhancement is marked as Needs Docs for 1.22 release.
Please follow the steps detailed in the documentation to open a PR against dev-1.22 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Fri July 9, 11:59 PM PDT.
Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Thank you!

k8s-ci-robot requested review from dcbw and thockin April 2, 2021 03:17

gjkim42 mentioned this pull request Apr 2, 2021

Expanded DNS configuration #2595

Closed

12 tasks

gjkim42 changed the title ~~Add expanded DNS configuration KEP~~ WIP: Add expanded DNS configuration KEP Apr 2, 2021

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 2, 2021

gjkim42 force-pushed the expanded-dns-config branch 8 times, most recently from a346c44 to e78ec0f Compare April 2, 2021 11:12

gjkim42 changed the title ~~WIP: Add expanded DNS configuration KEP~~ Add expanded DNS configuration KEP Apr 2, 2021

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 2, 2021

k8s-ci-robot requested review from aojea and liggitt April 2, 2021 11:35

aojea reviewed Apr 2, 2021

View reviewed changes

keps/sig-network/2595-expanded-dns-config/kep.yaml Outdated Show resolved Hide resolved

aojea reviewed Apr 2, 2021

View reviewed changes

keps/sig-network/2595-expanded-dns-config/README.md Outdated Show resolved Hide resolved

aojea reviewed Apr 2, 2021

View reviewed changes

keps/sig-network/2595-expanded-dns-config/README.md Show resolved Hide resolved

aojea reviewed Apr 2, 2021

View reviewed changes

gjkim42 force-pushed the expanded-dns-config branch from e78ec0f to 8b37167 Compare April 2, 2021 15:11

gjkim42 force-pushed the expanded-dns-config branch from fedcf67 to fce09c5 Compare May 1, 2021 05:37

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 1, 2021

thockin reviewed May 3, 2021

View reviewed changes

johnbelamaric reviewed May 8, 2021

View reviewed changes

gjkim42 force-pushed the expanded-dns-config branch 2 times, most recently from 2ca6725 to d7713c8 Compare May 8, 2021 15:33

gjkim42 added 3 commits May 9, 2021 00:35

Add expanded DNS configuration KEP#2595

421cbfe

Mark expanded DNS configuration as implementable

5ca1202

Add the version skew strategy of older kubelet

d03b93b

gjkim42 force-pushed the expanded-dns-config branch 2 times, most recently from 4852c0e to 19a2876 Compare May 10, 2021 04:43

gjkim42 added 2 commits May 10, 2021 13:51

Answer the PRR questionnaire

4ff65d3

Add graduation criteria for alpha

ce72cc7

gjkim42 force-pushed the expanded-dns-config branch from 19a2876 to ce72cc7 Compare May 10, 2021 04:52

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 10, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 10, 2021

k8s-ci-robot merged commit 5422920 into kubernetes:master May 10, 2021

k8s-ci-robot added this to the v1.22 milestone May 10, 2021

gjkim42 deleted the expanded-dns-config branch May 10, 2021 17:02

dcbw reviewed May 13, 2021

View reviewed changes


		### Version Skew Strategy

		In clusters with a replicated control plane, all kube-apiservers should enable


		Yes, the feature can be disabled by disabling the feature gate.

		Once the feature is disabled, kube-apiserver will reject the pod having expanded

Add expanded DNS configuration KEP #2596

Add expanded DNS configuration KEP #2596

Uh oh!

Conversation

gjkim42 commented Apr 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gjkim42 commented Apr 2, 2021

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gjkim42 Apr 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aojea commented Apr 2, 2021

Uh oh!

thockin commented Apr 30, 2021

Uh oh!

gjkim42 commented Apr 30, 2021

Uh oh!

gjkim42 commented May 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gjkim42 commented May 7, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gjkim42 commented May 8, 2021

Uh oh!

gjkim42 commented May 10, 2021

Uh oh!

johnbelamaric commented May 10, 2021

Uh oh!

k8s-ci-robot commented May 10, 2021

Uh oh!

johnbelamaric commented May 10, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gjkim42 May 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ritpanjw commented May 18, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

gjkim42 commented Apr 2, 2021 •

edited

Loading

gjkim42 Apr 2, 2021 •

edited

Loading

gjkim42 commented May 1, 2021 •

edited

Loading

gjkim42 May 14, 2021 •

edited

Loading