Taints/tolerations

[davidopp]

Description

Design doc discussion in kubernetes/kubernetes#18263, checked in here.

Implementation in kubernetes/kubernetes#24134

This feature first appeared in Kubernetes 1.4. I'm filing the issue to track moving it to Beta in 1.5 (kubernetes/kubernetes#25320).

See also #61

Progress Tracker

• Before Alpha
  • Write and maintain draft quality doc
    • During development keep a doc up-to-date about the desired experience of the feature and how someone can try the feature in its current state. Think of it as the README of your new feature and a skeleton for the docs to be written before the Kubernetes release. Paste link to Google Doc: DOC-LINK
  • Design Approval
    • Design Proposal. This goes under docs/proposals. Doing a proposal as a PR allows line-by-line commenting from community, and creates the basis for later design documentation. Paste link to merged design proposal here: PROPOSAL-NUMBER
    • Decide which repo this feature's code will be checked into. Not everything needs to land in the core kubernetes repo. REPO-NAME
    • Initial API review (if API). Maybe same PR as design doc. PR-NUMBER
      • Any code that changes an API (/pkg/apis/...)
      • cc @kubernetes/api
    • Identify shepherd (your SIG lead and/or [email protected] will be able to help you). My Shepherd is: [email protected] (and/or GH Handle)
      • A shepherd is an individual who will help acquaint you with the process of getting your feature into the repo, identify reviewers and provide feedback on the feature. They are not (necessarily) the code reviewer of the feature, or tech lead for the area.
      • The shepherd is not responsible for showing up to Kubernetes-PM meetings and/or communicating if the feature is on-track to make the release goals. That is still your responsibility.
    • Identify secondary/backup contact point. My Secondary Contact Point is: [email protected] (and/or GH Handle)
  • Write (code + tests + docs) then get them merged. ALL-PR-NUMBERS
    • Code needs to be disabled by default. Verified by code OWNERS
    • Minimal testing
    • Minimal docs
      • cc @kubernetes/docs on docs PR
      • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
      • New apis: Glossary Section Item in the docs repo: kubernetes/kubernetes.github.io
    • Update release notes
• Before Beta
  • Testing is sufficient for beta
  • User docs with tutorials
    • Updated walkthrough / tutorial in the docs repo: kubernetes/kubernetes.github.io
    • cc @kubernetes/docs on docs PR
    • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
  • Thorough API review
    • cc @kubernetes/api
• Before Stable
  • docs/proposals/foo.md moved to docs/design/foo.md
    • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off
  • Soak, load testing
  • detailed user docs and examples
    • cc @kubernetes/docs
    • cc @kubernetes/feature-reviewers on this issue to get approval before checking this off

FEATURE_STATUS is used for feature tracking and to be updated by @kubernetes/feature-reviewers.
FEATURE_STATUS: IN_DEVELOPMENT

More advice:

Design

• Once you get LGTM from a @kubernetes/feature-reviewers member, you can check this checkbox, and the reviewer will apply the "design-complete" label.

Coding

• Use as many PRs as you need. Write tests in the same or different PRs, as is convenient for you.
• As each PR is merged, add a comment to this issue referencing the PRs. Code goes in the https://github.com/kubernetes/kubernetes repository,
  and sometimes https://github.com/kubernetes/contrib, or other repos.
• When you are done with the code, apply the "code-complete" label.
• When the feature has user docs, please add a comment mentioning @kubernetes/feature-reviewers and they will
  check that the code matches the proposed feature and design, and that everything is done, and that there is adequate
  testing. They won't do detailed code review: that already happened when your PRs were reviewed.
  When that is done, you can check this box and the reviewer will apply the "code-complete" label.

Docs

• Write user docs and get them merged in.
• User docs go into https://github.com/kubernetes/kubernetes.github.io.
• When the feature has user docs, please add a comment mentioning @kubernetes/docs.
• When you get LGTM, you can check this checkbox, and the reviewer will apply the "docs-complete" label.

[davidopp]

We will be moving this feature to beta in 1.6. Tracking issue is
kubernetes/kubernetes#25320

[idvoretskyi]

@davidopp could you please update the documentation status of the feature.

[davidopp]

This has been moved to beta, still needs documentation.

[davidopp]

In addition to documentation, we need to update the rescheduler and cluster autoscaler to use the beta fields instead of alpha annotations. I've filed issues for this:
kubernetes-retired/contrib#2381
kubernetes-retired/contrib#2382

[erictune]

These aren't guaranteed to be at the same version as the cluster's master, right? So you need to support both for a while.

…

On Wed, Mar 1, 2017 at 11:41 PM, David Oppenheimer ***@***.*** > wrote: In addition to documentation, we need to update the rescheduler and cluster autoscaler to use the beta fields instead of alpha annotations. I've filed issues for this: kubernetes-retired/contrib#2381 <kubernetes-retired/contrib#2381> kubernetes-retired/contrib#2382 <kubernetes-retired/contrib#2382> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#108 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHuudopYSH4A-zwQzrhTw9nAOjSrzB4vks5rhnK0gaJpZM4KLtdy> .

[davidopp]

Most deployment platforms will upgrade all the master components at approximately the same time, so it just needs to not do anything bad during the short window when the versions mismatch. But we're also going to have instructions on what order to upgrade the components if they don't upgrade at approximately the same time (in particular, upgrade the rescheduler first). @piosz is working on all of this.

[idvoretskyi]

@davidopp any update on this feature? Docs and release notes are required (please, provide them to the features spreadsheet.

[davidopp]

Speadsheet updated. Docs not done yet.

[davidopp]

Documentation PR out for review: kubernetes/website#2774

[davidopp]

@aveshagarwal noted that in addition to the above doc PR, we also need the following doc updates:

• documentation for --register-with-taints.
  doc link: https://kubernetes.io/docs/admin/node/#self-registration-of-nodes

• a one-sentence pointer from NodeController documentation to the last section of the above PR, about using taint-based evictions for node problems
  doc link: https://kubernetes.io/docs/admin/node/#node-controller

• updating existing daemonsets doc for explaining behavior when a node becomes unreachable/notReady (TaintNodeUnreachable, TaintNodeNotReady)
  doc link: https://kubernetes.io/docs/admin/daemons/

• updating kubectl taint for NoExecute
  doc link: https://kubernetes.io/docs/user-guide/kubectl/kubectl_taint/

[davidopp]

Oh, and also, I have text for the "Notable Changes to Existing Behavior" and "Action Required Before Upgrading" sections of the Release Notes ready to go, but won't be able to create a PR for that until https://github.com/kubernetes/features/blob/master/release-1.6/release-notes-draft.md is created.

(BTW I'd like to change "Action Required Before Upgrading" to "Action Required Before or After Upgrading" for this release, since the scheduler features we moved from alpha to beta require using fields that only exist after you upgrade. So the action is actually taken after upgrade, not before.)

[davidopp]

kubernetes/website#2737

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close