Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Topology Manager Policy: single-numa-node #82099

Merged
merged 3 commits into from
Aug 30, 2019
Merged

Topology Manager Policy: single-numa-node #82099

merged 3 commits into from
Aug 30, 2019

Conversation

lmdaly
Copy link
Contributor

@lmdaly lmdaly commented Aug 28, 2019

What type of PR is this?
/kind feature

What this PR does / why we need it:
Issue for tracking PRs: #72828

This PR introduces a new Topology Manager policy which aligns resources on a single NUMA Node or fails the pod if this is not possible. The restricted policy previously implicitly did this. Based on discussions with @ConnorDoyle & @klueska, it was determined that a specific policy to cater for this would be more clear for the users.

The restricted policy will fail pods that have the preferred policy set to false, this can occur when a sub-optimal NUMA affinity is determined for the pod.

Does this PR introduce a user-facing change?:
This adds an additional option to the --topology-manager-policy called single-numa-node.
The topology-manager-policy has not been used in previous releases and is being introduced in the 1.16 release.

@k8s-ci-robot
Copy link
Contributor

@lmdaly: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/kubelet kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API sig/node Categorizes an issue or PR as relevant to SIG Node. and removed needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Aug 28, 2019
@lmdaly lmdaly mentioned this pull request Aug 28, 2019
Closed
@klueska
Copy link
Contributor

klueska commented Aug 28, 2019

/release-note-none

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Aug 28, 2019
klueska
klueska reviewed Aug 28, 2019
View reviewed changes
pkg/kubelet/apis/config/types.go
RestrictedTopologyManagerPolicy = "restricted"
// BestEffortTopologyManagerPolicy is a mode in which kubelet will favour
// pods with NUMA alignment of CPU and device resources.
BestEffortTopologyManagerPolicy = "best-effort"
// NoneTopologyManager Policy is a mode in which kubelet has no knowledge
// of NUMA alignment of a pod's CPU and device resources.
NoneTopologyManagerPolicy = "none"
// SingleNumaNodeTopologyManager Policy iis a mode in which kubelet only allows
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: misspelling of is

@fejta-bot
Copy link

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

@klueska
Copy link
Contributor

klueska commented Aug 28, 2019

/cc @erictune @liggitt @bgrant0607 @lavalamp @smarterclayton @thockin

This does not introduce any user-facing API change because this is a new feature for 1.16

@klueska
Copy link
Contributor

klueska commented Aug 28, 2019

/assign @derekwaynecarr @ConnorDoyle

@klueska
Copy link
Contributor

klueska commented Aug 28, 2019

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 28, 2019
@derekwaynecarr
Copy link
Member

the code changes look fine, and the policy is clear. before this graduates in future releases beyond alpha, we need to clarify the strategy to test this with mock device plugins.

/approve
/lgtm

will need an api review for new token , but looks fine to me for sig-node.

@klueska
Copy link
Contributor

klueska commented Aug 28, 2019

/label api-review

@k8s-ci-robot k8s-ci-robot added the api-review Categorizes an issue or PR as actively needing an API review. label Aug 28, 2019
@liggitt
Copy link
Member

liggitt commented Aug 28, 2019

/approve

field is still alpha gated and is new in 1.16

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: derekwaynecarr, liggitt, lmdaly

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 28, 2019
@adrianchiris
Copy link
Contributor

PR #81586 "strict" policy already deals with aligning resources to a single NUMA node.
just to make sure i understand, this PR will affect #81586 in a way that:

  1. will require a rename of the policy to singleNumaPolicy
  2. an additional policy called restrictedPolicy which is identical to the bestEffortPolicy but will fail if a hint is not preferred (or a configuration of the existing policy)

is my understanding correct ?

@ConnorDoyle
Copy link
Contributor

@adrianchiris yes your description is accurate. We decided to make this change to meet the needs of everyone involved in the design discussions and to make the policy semantics as easy to understand as possible.

@klueska
Copy link
Contributor

klueska commented Aug 29, 2019

PR #81586 "strict" policy already deals with aligning resources to a single NUMA node.
just to make sure i understand, this PR will affect #81586 in a way that:

  1. will require a rename of the policy to singleNumaPolicy
  2. an additional policy called restrictedPolicy which is identical to the bestEffortPolicy but will fail if a hint is not preferred (or a configuration of the existing policy)

is my understanding correct ?

@adrianchiris The long term plan is still to add the Merge() method you have in your / @moshe010's PR. This is a short term fix to unblock the release of these policies.

@liggitt
Copy link
Member

liggitt commented Aug 29, 2019

/priority important-soon
/milestone v1.16

important to include this before shipping the config field

@k8s-ci-robot k8s-ci-robot added the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Aug 29, 2019
@k8s-ci-robot k8s-ci-robot added this to the v1.16 milestone Aug 29, 2019
@k8s-ci-robot k8s-ci-robot removed the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label Aug 29, 2019
klueska and others added 3 commits August 30, 2019 07:17
@klueska @lmdaly
Update CanAdmitPodResult() in TopologyManager to take a TopologyHint
5ed80da 
Previously it only took a bool, which limited the logic it could perform
to determine if a pod should be admitted or not based on the merged hint
from the policy.
@lmdaly @klueska
Added Single NUMA Node Policy which ensure resource are
f6c085f 
aligned on a single NUMA node

Co-authored-by: Kevin Klues <[email protected]>
@lmdaly @klueska
Single-numa-node Topology Manager bug fix
8ad1b5b 
Added one off fix for single-numa-node policy to correctly
reject pod admission on a resource allocation that spans
NUMA nodes

Co-authored-by: Kevin Klues <[email protected]>
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 30, 2019
@klueska
Copy link
Contributor

klueska commented Aug 30, 2019

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 30, 2019
@klueska
Copy link
Contributor

klueska commented Aug 30, 2019

/retest

@k8s-ci-robot k8s-ci-robot merged commit 887edd2 into kubernetes:master Aug 30, 2019
@pacoxu pacoxu mentioned this pull request Jan 18, 2024
Closed
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@klueska klueska klueska left review comments

@dashpole dashpole Awaiting requested review from dashpole

@dims dims Awaiting requested review from dims

@bgrant0607 bgrant0607 Awaiting requested review from bgrant0607

@erictune erictune Awaiting requested review from erictune

@lavalamp lavalamp Awaiting requested review from lavalamp

@liggitt liggitt Awaiting requested review from liggitt

@smarterclayton smarterclayton Awaiting requested review from smarterclayton

@thockin thockin Awaiting requested review from thockin

Assignees

@klueska klueska

@ConnorDoyle ConnorDoyle

@derekwaynecarr derekwaynecarr

Labels
api-review Categorizes an issue or PR as actively needing an API review. approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubelet cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. sig/node Categorizes an issue or PR as relevant to SIG Node. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
v1.16
Development

Successfully merging this pull request may close these issues.
8 participants
@lmdaly @k8s-ci-robot @klueska @fejta-bot @derekwaynecarr @liggitt @adrianchiris @ConnorDoyle