Skip to content

[release-4.17] OCPBUGS-43820: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup #2123

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
openshift-cherrypick-robot wants to merge 4 commits into from
Closed

[release-4.17] OCPBUGS-43820: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup #2123

openshift-cherrypick-robot wants to merge 4 commits into from

Conversation

@openshift-cherrypick-robot
Copy link

@openshift-cherrypick-robot openshift-cherrypick-robot commented Oct 25, 2024

This is an automated cherry-pick of #2031

/assign haircommander

haircommander and others added 4 commits October 25, 2024 05:14
@haircommander
UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from mi…
65330c1 
…ssing cpuset cgroup

on None cpumanager policy, cgroupv2, and systemd cgroup manager, kubelet
could get into a situation where it believes the cpuset cgroup was created
(by libcontainer in the cgroupfs) but systemd has deleted it, as it wasn't requested
to create it. This causes one unnecessary restart, as kubelet fails with

`failed to initialize top level QOS containers: root container [kubepods] doesn't exist.`

This only causes one restart because the kubelet skips recreating the cgroup
if it already exists, but it's still a bother and is fixed this way

adapted version of kubernetes@c51195d, avoid merge conflict
this can be dropped in the 4.19 rebase

Signed-off-by: Peter Hunt <[email protected]>
@haircommander
UPSTREAM: <carry>: kubelet/cm: move CPU reading from cm to cm/cpumanager
40151eb 
Adapted from kubernetes@77d03e4 to avoid merge conflict
This can be dropped in the 4.19 rebase

Authored-by: Francesco Romani <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
@ffromani
UPSTREAM: 125923: e2e_node: add a test to verify the kubelet starts
c1ce74e 
with systemd cgroup driver and cpumanager none policy.

This was originally planned to be a correctness check for
https://issues.k8s.io/125923, but it was difficult to reproduce the bug,
so it's now a regression test against it.

Signed-off-by: Francesco Romani <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
@haircommander
UPSTREAM: 125923: e2e_node: use restart instead of start stop
abbe934 
Signed-off-by: Peter Hunt <[email protected]>
@openshift-cherrypick-robot openshift-cherrypick-robot mentioned this pull request Oct 25, 2024
Merged
@openshift-ci-robot openshift-ci-robot added the backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. label Oct 25, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 25, 2024

@openshift-cherrypick-robot: Jira Issue OCPBUGS-28812 has been cloned as Jira Issue OCPBUGS-43820. Will retitle bug to link to clone.
/retitle [release-4.17] OCPBUGS-43820: UPSTREAM: : kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup

Details

In response to this:

This is an automated cherry-pick of #2031

/assign haircommander

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 25, 2024

@openshift-cherrypick-robot: the contents of this pull request could not be automatically validated.

The following commits are valid:

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@openshift-ci openshift-ci bot changed the title [release-4.17] OCPBUGS-28812: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup [release-4.17] OCPBUGS-43820: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup Oct 25, 2024
@openshift-ci-robot openshift-ci-robot added jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. labels Oct 25, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 25, 2024

@openshift-cherrypick-robot: This pull request references Jira Issue OCPBUGS-43820, which is invalid:

  • release note text must be set and not match the template OR release note type must be set to "Release Note Not Required". For more information you can reference the OpenShift Bug Process.
  • expected dependent Jira Issue OCPBUGS-28812 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is MODIFIED instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

This is an automated cherry-pick of #2031

/assign haircommander

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Oct 25, 2024
@openshift-ci openshift-ci bot requested review from deads2k and mrunalp October 25, 2024 05:16
@haircommander
Copy link
Member

haircommander commented Nov 1, 2024

/retest

@harche
Copy link

harche commented Nov 1, 2024

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 1, 2024
@mrunalp mrunalp added approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. labels Nov 1, 2024
@openshift-ci
Copy link

openshift-ci bot commented Nov 1, 2024

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: harche, mrunalp, openshift-cherrypick-robot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
Copy link

openshift-ci bot commented Nov 1, 2024

@openshift-cherrypick-robot: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/verify abbe934 link true /test verify

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@haircommander haircommander mentioned this pull request Nov 1, 2024
Merged
@haircommander
Copy link
Member

haircommander commented Nov 1, 2024

replaced by #2126 which fixes the verify job
/close

@openshift-ci openshift-ci bot closed this Nov 1, 2024
@openshift-ci
Copy link

openshift-ci bot commented Nov 1, 2024

@haircommander: Closed this PR.

Details

In response to this:

replaced by #2126 which fixes the verify job
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Nov 1, 2024

@openshift-cherrypick-robot: This pull request references Jira Issue OCPBUGS-43820. The bug has been updated to no longer refer to the pull request using the external bug tracker.

Details

In response to this:

This is an automated cherry-pick of #2031

/assign haircommander

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mrunalp mrunalp mrunalp approved these changes

@deads2k deads2k Awaiting requested review from deads2k

Assignees

@haircommander haircommander

@harche harche

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@openshift-cherrypick-robot @openshift-ci-robot @haircommander @harche @mrunalp @ffromani