Skip to content

[release-4.16] OCPBUGS-45931: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup #2158 #2200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
openshift-merge-bot merged 4 commits into from
Feb 7, 2025
Merged

[release-4.16] OCPBUGS-45931: UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from missing cpuset cgroup #2158 #2200

openshift-merge-bot merged 4 commits into from
Feb 7, 2025

Conversation

@haircommander
Copy link
Member

@haircommander haircommander commented Feb 6, 2025

manual cherry-pick of automated cherry-pick #2126

NONE

haircommander and others added 3 commits February 6, 2025 10:26
@haircommander
UPSTREAM: <carry>: kubelet/cm: fix bug where kubelet restarts from mi…
287ad89 
…ssing cpuset cgroup

on None cpumanager policy, cgroupv2, and systemd cgroup manager, kubelet
could get into a situation where it believes the cpuset cgroup was created
(by libcontainer in the cgroupfs) but systemd has deleted it, as it wasn't requested
to create it. This causes one unnecessary restart, as kubelet fails with

`failed to initialize top level QOS containers: root container [kubepods] doesn't exist.`

This only causes one restart because the kubelet skips recreating the cgroup
if it already exists, but it's still a bother and is fixed this way

adapted version of kubernetes@c51195d, avoid merge conflict
this can be dropped in the 4.19 rebase

Signed-off-by: Peter Hunt <[email protected]>
@haircommander
UPSTREAM: <carry>: kubelet/cm: move CPU reading from cm to cm/cpumanager
e4fe3b1 
Adapted from kubernetes@77d03e4 to avoid merge conflict
This can be dropped in the 4.19 rebase

Authored-by: Francesco Romani <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
@ffromani @haircommander
UPSTREAM: 125923: e2e_node: add a test to verify the kubelet starts
3b7c061 
with systemd cgroup driver and cpumanager none policy.

This was originally planned to be a correctness check for
https://issues.k8s.io/125923, but it was difficult to reproduce the bug,
so it's now a regression test against it.

Signed-off-by: Francesco Romani <[email protected]>
Signed-off-by: Peter Hunt <[email protected]>
@openshift-ci-robot openshift-ci-robot added backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. labels Feb 6, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 6, 2025

@haircommander: the contents of this pull request could not be automatically validated.

The following commits are valid:

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 6, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 6, 2025

@haircommander: This pull request references Jira Issue OCPBUGS-45931, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.16.z) matches configured target version for branch (4.16.z)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note type set to "Release Note Not Required"
  • dependent bug Jira Issue OCPBUGS-43820 is in the state Closed (Done-Errata), which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-43820 targets the "4.17.z" version, which is one of the valid target versions: 4.17.0, 4.17.z
  • bug has dependents

Requesting review from QA contact:
/cc @cpmeadors

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

manual cherry-pick of automated cherry-pick #2126

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Feb 6, 2025
@haircommander haircommander force-pushed the cherry-pick-2126-to-release-4.16 branch from 25f8ec1 to 66b084b Compare February 6, 2025 19:41
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 6, 2025

@haircommander: the contents of this pull request could not be automatically validated.

The following commits are valid:

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@haircommander
Copy link
Member Author

haircommander commented Feb 6, 2025

/retest

@haircommander haircommander mentioned this pull request Feb 6, 2025
Closed
@haircommander
Copy link
Member Author

haircommander commented Feb 6, 2025

/skip

@mrunalp mrunalp removed the backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. label Feb 6, 2025
mrunalp
mrunalp approved these changes Feb 6, 2025
View reviewed changes
Copy link
Member

@mrunalp mrunalp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 6, 2025
@mrunalp mrunalp added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 6, 2025
@openshift-ci
Copy link

openshift-ci bot commented Feb 6, 2025

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: haircommander, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mrunalp mrunalp added cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. labels Feb 6, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 6, 2025

/retest-required

Remaining retests: 0 against base HEAD ef2a55c and 2 for PR HEAD 66b084b in total

@openshift-ci
Copy link

openshift-ci bot commented Feb 7, 2025

@haircommander: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-agnostic-ovn-cmd 66b084b link false /test e2e-agnostic-ovn-cmd

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit 148a389 into openshift:release-4.16 Feb 7, 2025
20 checks passed
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 7, 2025

@haircommander: Jira Issue OCPBUGS-45931: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-45931 has been moved to the MODIFIED state.

Details

In response to this:

manual cherry-pick of automated cherry-pick #2126

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-bot
Copy link

openshift-bot commented Feb 7, 2025

[ART PR BUILD NOTIFIER]

Distgit: openshift-enterprise-pod
This PR has been included in build openshift-enterprise-pod-container-v4.16.0-202502070105.p0.g148a389.assembly.stream.el9.
All builds following this will include this PR.

@openshift-bot
Copy link

openshift-bot commented Feb 7, 2025

[ART PR BUILD NOTIFIER]

Distgit: openshift-enterprise-hyperkube
This PR has been included in build openshift-enterprise-hyperkube-container-v4.16.0-202502070105.p0.g148a389.assembly.stream.el9.
All builds following this will include this PR.

@openshift-bot
Copy link

openshift-bot commented Feb 7, 2025

[ART PR BUILD NOTIFIER]

Distgit: ose-installer-kube-apiserver-artifacts
This PR has been included in build ose-installer-kube-apiserver-artifacts-container-v4.16.0-202502070105.p0.g148a389.assembly.stream.el9.
All builds following this will include this PR.

@openshift-merge-robot
Copy link

openshift-merge-robot commented Feb 9, 2025

Fix included in accepted release 4.16.0-0.nightly-2025-02-09-045321

@haircommander
Copy link
Member Author

haircommander commented Feb 28, 2025

/cherry-pick release-4.15

@openshift-cherrypick-robot
Copy link

openshift-cherrypick-robot commented Feb 28, 2025

@haircommander: new pull request created: #2226

Details

In response to this:

/cherry-pick release-4.15

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-cherrypick-robot openshift-cherrypick-robot mentioned this pull request Feb 28, 2025
Closed
@haircommander haircommander mentioned this pull request Feb 28, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mrunalp mrunalp mrunalp approved these changes

@cpmeadors cpmeadors Awaiting requested review from cpmeadors

@deads2k deads2k Awaiting requested review from deads2k

Assignees

@mrunalp mrunalp

@kasturinarra kasturinarra

@zhouying7780 zhouying7780

@xingxingxia xingxingxia

@duanwei33 duanwei33

@wangke19 wangke19

@gangwgr gangwgr

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.