[release-4.15] [manual] OCPBUGS-30507: Add performance real time tuned template #984

rbaturov · 2024-03-07T08:25:15Z

This is a manual backport of PR #954 and #1005
This PR includes both of these PR's together, because #1005 introducing a fix for the tuned profile degradation test added in #954.

Add performance real time tuned template

Kernel parameters that are not supported on RT kernel systems are being applied, causing errors to be logged to the tuned daemon, resulting in a degraded profile state. Therefore, I added the openshift-node-performance-rt profile that would be included if an RT kernel is detected, thereby dropping the unsupported kernel parameters before they are applied.

Added a test that make sure the tuned profile is not degraded

This could be great to have a generic test that make sure that after performance profile is applied the tuned profile is not degraded. This would prevent future issues like OCPBUGS-23167.
The test will fail only when the tuned profile was found degraded on BM host.
In case of tuned profile found degraded on a VM, only a warning would be reported in the logs.
The reason for that is that the fact CI is using VMs - and some
configurations like certain kernel args can't be applied and therby lead to
the tuned profile being degraded.

render-sync update

Updated render-sync to include artifacts that are needed for the e2e tests. Committing the rendered items to this commit.

rbaturov · 2024-03-07T08:27:49Z

/jira cherrypick OCPBUGS-23167

openshift-ci-robot · 2024-03-07T08:27:56Z

@rbaturov: Jira Issue OCPBUGS-23167 has been cloned as Jira Issue OCPBUGS-30507. Will retitle bug to link to clone.
/retitle OCPBUGS-30507: Backport: Add performance real time tuned template (#954)

Details

In response to this:

/jira cherrypick OCPBUGS-23167

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2024-03-07T08:28:04Z

@rbaturov: This pull request references Jira Issue OCPBUGS-30507, which is invalid:

expected dependent Jira Issue OCPBUGS-23167 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Add performance real time tuned template

Kernel parameters that are not supported on RT kernel systems are being applied, causing errors to be logged to the tuned daemon, resulting in a degraded profile state. Therefore, I added the openshift-node-performance-rt profile that would be included if an RT kernel is detected, thereby dropping the unsupported kernel parameters before they are applied.

Added a test that make sure the tuned profile is not degraded

This could be great to have a generic test that make sure that after performance profile is applied the tuned profile is not degraded. This would prevent future issues like OCPBUGS-23167.

render-sync update

Updated render-sync to include artifacts that are needed for the e2e tests. Committing the rendered items to this commit.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

rbaturov · 2024-03-07T12:41:08Z

/hold

yanirq · 2024-03-10T17:18:36Z

/hold

@rbaturov can you add the reason for the hold?

rbaturov · 2024-03-11T08:58:00Z

/unhold

yanirq · 2024-03-11T09:31:55Z

/cc @MarSik
/uncc @dagrayvid

yanirq · 2024-03-11T11:22:46Z

/approve

hack/render-sync.sh

yanirq · 2024-03-11T15:56:36Z

/label backport-risk-assessed

yanirq · 2024-03-11T20:14:35Z

/approve cancel

yanirq · 2024-03-12T16:28:02Z

/approve
/lgtm

yanirq · 2024-03-12T16:28:22Z

cc @mrniranjan

rbaturov · 2024-03-27T21:33:45Z

/retest-required

* Added isVM func to util Signed-off-by: Ronny Baturov <[email protected]> * Update tuned profile degraded test Updated the tuned degradation test to fail only when the tuned profile was found degraded on BM host. In case of tuned profile found degraded on a VM, only a warning would be reported in the logs. The reason for that is that the fact CI is using VMs - and some configurations like certain kernel args can't be applied and therby lead to the tuned profile being degraded. Signed-off-by: Ronny Baturov <[email protected]> --------- Signed-off-by: Ronny Baturov <[email protected]>

rbaturov · 2024-03-28T20:39:16Z

/retest-required

rbaturov · 2024-03-31T06:35:21Z

/retest-required

Tal-or · 2024-04-03T07:53:43Z

/retitle [release-4.15] [manual] OCPBUGS-30507: Add performance real time tuned template

Tal-or · 2024-04-03T07:54:01Z

/lgtm

Tal-or · 2024-04-03T07:54:59Z

/jira refresh

openshift-ci-robot · 2024-04-03T07:55:04Z

@Tal-or: This pull request references Jira Issue OCPBUGS-30507, which is invalid:

expected dependent Jira Issue OCPBUGS-23167 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

yanirq · 2024-04-03T10:53:45Z

/cc @shajmakh @mrniranjan

shajmakh · 2024-04-03T10:58:05Z

/label cherry-pick-approved

MarSik · 2024-04-04T09:17:00Z

/jira refresh

openshift-ci-robot · 2024-04-04T09:17:05Z

@MarSik: This pull request references Jira Issue OCPBUGS-30507, which is valid.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.15.z) matches configured target version for branch (4.15.z)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
dependent bug Jira Issue OCPBUGS-23167 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
dependent Jira Issue OCPBUGS-23167 targets the "4.16.0" version, which is one of the valid target versions: 4.16.0
bug has dependents

Requesting review from QA contact:
/cc @shajmakh

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2024-04-04T14:13:49Z

@rbaturov: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot · 2024-04-04T14:17:24Z

@rbaturov: Jira Issue OCPBUGS-30507: All pull requests linked via external trackers have merged:

openshift/cluster-node-tuning-operator#984

Jira Issue OCPBUGS-30507 has been moved to the MODIFIED state.

Details

In response to this:

This is a manual backport of PR #954 and #1005
This PR includes both of these PR's together, because #1005 introducing a fix for the tuned profile degradation test added in #954.

Add performance real time tuned template

Kernel parameters that are not supported on RT kernel systems are being applied, causing errors to be logged to the tuned daemon, resulting in a degraded profile state. Therefore, I added the openshift-node-performance-rt profile that would be included if an RT kernel is detected, thereby dropping the unsupported kernel parameters before they are applied.

Added a test that make sure the tuned profile is not degraded

This could be great to have a generic test that make sure that after performance profile is applied the tuned profile is not degraded. This would prevent future issues like OCPBUGS-23167.
The test will fail only when the tuned profile was found degraded on BM host.
In case of tuned profile found degraded on a VM, only a warning would be reported in the logs.
The reason for that is that the fact CI is using VMs - and some
configurations like certain kernel args can't be applied and therby lead to
the tuned profile being degraded.

render-sync update

Updated render-sync to include artifacts that are needed for the e2e tests. Committing the rendered items to this commit.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-bot · 2024-04-04T15:44:30Z

[ART PR BUILD NOTIFIER]

This PR has been included in build cluster-node-tuning-operator-container-v4.15.0-202404041043.p0.g621e6ee.assembly.stream.el9 for distgit cluster-node-tuning-operator.
All builds following this will include this PR.

openshift-merge-robot · 2024-04-07T18:55:17Z

Fix included in accepted release 4.15.0-0.nightly-2024-04-07-120427

…d template (openshift#984) * Backport: Add performance real time tuned template (openshift#954) * Add performance real time tuned template Kernel parameters that are not supported on RT kernel systems are being applied, causing errors to be logged to the tuned daemon, resulting in a degraded profile state. Therefore, I added the openshift-node-performance-rt profile that would be included if an RT kernel is detected, thereby dropping the unsupported kernel parameters before they are applied. * Added a test that make sure the tuned profile is not degraded This could be great to have a generic test that make sure that after performance profile is applied the tuned profile is not degraded. This would prevent future issues like OCPBUGS-23167. * render-sync update Updated render-sync to include artifacts that are needed for the e2e tests. Committing the rendered items to this commit. Signed-off-by: rbaturov <[email protected]> * NO-JIRA: Update tuned profile degraded test (openshift#1005) * Added isVM func to util Signed-off-by: Ronny Baturov <[email protected]> * Update tuned profile degraded test Updated the tuned degradation test to fail only when the tuned profile was found degraded on BM host. In case of tuned profile found degraded on a VM, only a warning would be reported in the logs. The reason for that is that the fact CI is using VMs - and some configurations like certain kernel args can't be applied and therby lead to the tuned profile being degraded. Signed-off-by: Ronny Baturov <[email protected]> --------- Signed-off-by: Ronny Baturov <[email protected]> --------- Signed-off-by: rbaturov <[email protected]> Signed-off-by: Ronny Baturov <[email protected]>

…d template (#984) (#1025) * Backport: Add performance real time tuned template (#954) * Add performance real time tuned template Kernel parameters that are not supported on RT kernel systems are being applied, causing errors to be logged to the tuned daemon, resulting in a degraded profile state. Therefore, I added the openshift-node-performance-rt profile that would be included if an RT kernel is detected, thereby dropping the unsupported kernel parameters before they are applied. * Added a test that make sure the tuned profile is not degraded This could be great to have a generic test that make sure that after performance profile is applied the tuned profile is not degraded. This would prevent future issues like OCPBUGS-23167. * render-sync update Updated render-sync to include artifacts that are needed for the e2e tests. Committing the rendered items to this commit. * NO-JIRA: Update tuned profile degraded test (#1005) * Added isVM func to util * Update tuned profile degraded test Updated the tuned degradation test to fail only when the tuned profile was found degraded on BM host. In case of tuned profile found degraded on a VM, only a warning would be reported in the logs. The reason for that is that the fact CI is using VMs - and some configurations like certain kernel args can't be applied and therby lead to the tuned profile being degraded. --------- --------- Signed-off-by: rbaturov <[email protected]> Signed-off-by: Ronny Baturov <[email protected]>

openshift-ci bot requested review from dagrayvid and ffromani March 7, 2024 08:26

openshift-ci bot changed the title ~~Backport: Add performance real time tuned template (#954)~~ OCPBUGS-30507: Backport: Add performance real time tuned template (#954) Mar 7, 2024

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 7, 2024

rbaturov changed the title ~~OCPBUGS-30507: Backport: Add performance real time tuned template (#954)~~ OCPBUGS-30507: [release-4.15] : Add performance real time tuned template (#954) Mar 7, 2024

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 11, 2024

openshift-ci bot requested review from MarSik and removed request for dagrayvid March 11, 2024 09:31

rbaturov changed the title ~~OCPBUGS-30507: [release-4.15] : Add performance real time tuned template (#954)~~ [release-4.15] : OCPBUGS-30507: Add performance real time tuned template (#954) Mar 11, 2024

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 11, 2024

yanirq reviewed Mar 11, 2024

View reviewed changes

hack/render-sync.sh Show resolved Hide resolved

openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Mar 11, 2024

openshift-ci bot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 11, 2024

openshift-ci bot assigned yanirq Mar 12, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 12, 2024

rbaturov force-pushed the add-performance-rt-profile-4.15 branch from 72aae39 to 030d3a6 Compare March 28, 2024 07:20

openshift-ci bot changed the title ~~[release-4.15] OCPBUGS-30507: Add performance real time tuned template~~ [release-4.15] [manual] OCPBUGS-30507: Add performance real time tuned template Apr 3, 2024

openshift-ci bot assigned Tal-or Apr 3, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 3, 2024

openshift-ci bot requested review from mrniranjan and shajmakh April 3, 2024 10:53

openshift-ci bot added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Apr 3, 2024

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 4, 2024

openshift-merge-bot bot merged commit 621e6ee into openshift:release-4.15 Apr 4, 2024

rbaturov mentioned this pull request Apr 9, 2024

OCPBUGS-31905: [release-4.14] [manual] : Add performance real time tuned template #1025

Merged

rbaturov deleted the add-performance-rt-profile-4.15 branch May 20, 2024 09:56

[release-4.15] [manual] OCPBUGS-30507: Add performance real time tuned template #984

[release-4.15] [manual] OCPBUGS-30507: Add performance real time tuned template #984

Uh oh!

Conversation

rbaturov commented Mar 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rbaturov commented Mar 7, 2024

Uh oh!

openshift-ci-robot commented Mar 7, 2024

Uh oh!

openshift-ci-robot commented Mar 7, 2024

Uh oh!

rbaturov commented Mar 7, 2024

Uh oh!

yanirq commented Mar 10, 2024

Uh oh!

rbaturov commented Mar 11, 2024

Uh oh!

yanirq commented Mar 11, 2024

Uh oh!

yanirq commented Mar 11, 2024

Uh oh!

Uh oh!

yanirq commented Mar 11, 2024

Uh oh!

yanirq commented Mar 11, 2024

Uh oh!

yanirq commented Mar 12, 2024

Uh oh!

yanirq commented Mar 12, 2024

Uh oh!

rbaturov commented Mar 27, 2024

Uh oh!

rbaturov commented Mar 28, 2024

Uh oh!

rbaturov commented Mar 31, 2024

Uh oh!

Tal-or commented Apr 3, 2024

Uh oh!

Tal-or commented Apr 3, 2024

Uh oh!

Tal-or commented Apr 3, 2024

Uh oh!

openshift-ci-robot commented Apr 3, 2024

Uh oh!

yanirq commented Apr 3, 2024

Uh oh!

shajmakh commented Apr 3, 2024

Uh oh!

MarSik commented Apr 4, 2024

Uh oh!

openshift-ci-robot commented Apr 4, 2024

Uh oh!

openshift-ci bot commented Apr 4, 2024

Uh oh!

openshift-ci-robot commented Apr 4, 2024

Uh oh!

openshift-bot commented Apr 4, 2024

Uh oh!

openshift-merge-robot commented Apr 7, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

rbaturov commented Mar 7, 2024 •

edited

Loading