e2e: stalld and sysctl_d_override e2e test changes by jmencak · Pull Request #298 · openshift/cluster-node-tuning-operator

jmencak · 2021-12-13T07:19:36Z

Simplify the e2e stalld test. It is unnecessarily complex and only adds
to the time it takes to pass all e2e tests by approximately 1 node
reboot. Add more debugging for any future troubleshooting.

Adjust the sysctl_d_override golang e2e test to show the contents of the
sysctl override file for debugging and synchronize [fsync()] it to
persistent storage after write.

Simplify the e2e stalld test. It is unnecessarily complex and only adds to the time it takes to pass all e2e tests by approximately 1 node reboot. Add more debugging for any future troubleshooting. Adjust the sysctl_d_override golang e2e test to show the contents of the sysctl override file for debugging and synchronize [fsync()] it to persistent storage after write.

openshift-ci · 2021-12-13T07:20:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jmencak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [jmencak]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jmencak · 2021-12-13T08:22:35Z

sigh
/retest verify

openshift-ci · 2021-12-13T08:22:36Z

@jmencak: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

/test e2e-aws
/test e2e-aws-operator
/test e2e-upgrade
/test images
/test unit
/test verify

Use /test all to run all jobs.

Details

In response to this:

sigh
/retest verify

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jmencak · 2021-12-13T08:22:47Z

/test verify

jmencak · 2021-12-13T09:07:18Z

A note to reviewers. I've tested this by running ~500 e2e tests on 4.10 with no issues, so hopefully the most prominent operator e2e tests issues are resolved now. (Famous last words...).

jmencak · 2021-12-13T10:14:55Z

/retest

jmencak · 2021-12-13T10:29:58Z

OK, time to look into this again:

$ git clone https://github.com/openshift/cluster-node-tuning-operator
Cloning into 'cluster-node-tuning-operator'...
remote: Enumerating objects: 27663, done.
remote: Counting objects: 100% (4050/4050), done.
remote: Compressing objects: 100% (2057/2057), done.
remote: Total 27663 (delta 1959), reused 3472 (delta 1836), pack-reused 23613
Receiving objects: 100% (27663/27663), 31.51 MiB | 12.71 MiB/s, done.
Resolving deltas: 100% (12726/12726), done.

$ git fetch origin pull/298/head:298
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 8 (delta 5), reused 8 (delta 5), pack-reused 0
Unpacking objects: 100% (8/8), 1013 bytes | 202.00 KiB/s, done.
From https://github.com/openshift/cluster-node-tuning-operator
 * [new ref]           refs/pull/298/head -> 298

$ git checkout 298
Switched to branch '298'

$ make verify
Installing controller-gen into '_output/tools/bin/controller-gen-v0.6.0'
mkdir -p '_output/tools/bin/'
curl -s -f -L https://github.com/openshift/kubernetes-sigs-controller-tools/releases/download/v0.6.0/controller-gen-linux-amd64 -o '_output/tools/bin/controller-gen-v0.6.0'
chmod +x '_output/tools/bin/controller-gen-v0.6.0';
Installing yq into '_output/tools/bin/yq-2.4.0'
mkdir -p '_output/tools/bin/'
curl -s -f -L https://github.com/mikefarah/yq/releases/download/2.4.0/yq_linux_amd64 -o '_output/tools/bin/yq-2.4.0'
chmod +x '_output/tools/bin/yq-2.4.0';
Installing yaml-patch into '_output/tools/bin/yaml-patch-v0.0.10'
mkdir -p '_output/tools/bin/'
curl -s -f -L https://github.com/krishicks/yaml-patch/releases/download/v0.0.10/yaml_patch_linux -o '_output/tools/bin/yaml-patch-v0.0.10'
make: *** [vendor/github.com/openshift/build-machinery-go/make/targets/openshift/operator/../yaml-patch.mk:18: ensure-yaml-patch] Error 22

$ wget https://github.com/krishicks/yaml-patch/releases/download/v0.0.10/yaml_patch_linux
--2021-12-13 11:25:11--  https://github.com/krishicks/yaml-patch/releases/download/v0.0.10/yaml_patch_linux
Resolving github.com (github.com)... 140.82.121.3
Connecting to github.com (github.com)|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-12-13 11:25:11 ERROR 404: Not Found.

$ wget https://github.com/krishicks/yaml-patch/
--2021-12-13 11:29:02--  https://github.com/krishicks/yaml-patch/
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-12-13 11:29:03 ERROR 404: Not Found.

Awesome... As the tests pass with my local copy
/override verify
for the time being.

jmencak · 2021-12-13T10:31:51Z

/override ci/prow/verify

openshift-ci · 2021-12-13T10:34:24Z

@jmencak: Overrode contexts on behalf of jmencak: ci/prow/verify

Details

In response to this:

/override ci/prow/verify

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci · 2021-12-13T10:34:39Z

@jmencak: /override requires a failed status context or a job name to operate on.
The following unknown contexts were given:

verify

Only the following contexts were expected:

ci/prow/e2e-aws
ci/prow/e2e-aws-operator
ci/prow/e2e-upgrade
ci/prow/images
ci/prow/unit
ci/prow/verify
pull-ci-openshift-cluster-node-tuning-operator-release-4.1-e2e-aws
pull-ci-openshift-cluster-node-tuning-operator-release-4.1-images
pull-ci-openshift-cluster-node-tuning-operator-release-4.1-unit
pull-ci-openshift-cluster-node-tuning-operator-release-4.1-verify
pull-ci-openshift-cluster-node-tuning-operator-release-4.10-e2e-aws-operator
pull-ci-openshift-cluster-node-tuning-operator-release-4.10-e2e-upgrade
tide

Details

In response to this:

OK, time to look into this again:

$ git clone https://github.com/openshift/cluster-node-tuning-operator
Cloning into 'cluster-node-tuning-operator'...
remote: Enumerating objects: 27663, done.
remote: Counting objects: 100% (4050/4050), done.
remote: Compressing objects: 100% (2057/2057), done.
remote: Total 27663 (delta 1959), reused 3472 (delta 1836), pack-reused 23613
Receiving objects: 100% (27663/27663), 31.51 MiB | 12.71 MiB/s, done.
Resolving deltas: 100% (12726/12726), done.

$ git fetch origin pull/298/head:298
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 8 (delta 5), reused 8 (delta 5), pack-reused 0
Unpacking objects: 100% (8/8), 1013 bytes | 202.00 KiB/s, done.
From https://github.com/openshift/cluster-node-tuning-operator
* [new ref]           refs/pull/298/head -> 298

$ git checkout 298
Switched to branch '298'

$ make verify
Installing controller-gen into '_output/tools/bin/controller-gen-v0.6.0'
mkdir -p '_output/tools/bin/'
curl -s -f -L https://github.com/openshift/kubernetes-sigs-controller-tools/releases/download/v0.6.0/controller-gen-linux-amd64 -o '_output/tools/bin/controller-gen-v0.6.0'
chmod +x '_output/tools/bin/controller-gen-v0.6.0';
Installing yq into '_output/tools/bin/yq-2.4.0'
mkdir -p '_output/tools/bin/'
curl -s -f -L https://github.com/mikefarah/yq/releases/download/2.4.0/yq_linux_amd64 -o '_output/tools/bin/yq-2.4.0'
chmod +x '_output/tools/bin/yq-2.4.0';
Installing yaml-patch into '_output/tools/bin/yaml-patch-v0.0.10'
mkdir -p '_output/tools/bin/'
curl -s -f -L https://github.com/krishicks/yaml-patch/releases/download/v0.0.10/yaml_patch_linux -o '_output/tools/bin/yaml-patch-v0.0.10'
make: *** [vendor/github.com/openshift/build-machinery-go/make/targets/openshift/operator/../yaml-patch.mk:18: ensure-yaml-patch] Error 22

$ wget https://github.com/krishicks/yaml-patch/releases/download/v0.0.10/yaml_patch_linux
--2021-12-13 11:25:11--  https://github.com/krishicks/yaml-patch/releases/download/v0.0.10/yaml_patch_linux
Resolving github.com (github.com)... 140.82.121.3
Connecting to github.com (github.com)|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-12-13 11:25:11 ERROR 404: Not Found.

$ wget https://github.com/krishicks/yaml-patch/
--2021-12-13 11:29:02--  https://github.com/krishicks/yaml-patch/
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-12-13 11:29:03 ERROR 404: Not Found.

Awesome... As the tests pass with my local copy
/override verify
for the time being.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jmencak · 2021-12-13T11:00:38Z

The verify issue is fixed by:
openshift/build-machinery-go#58
we need to vendor it in.

jmencak · 2021-12-13T11:36:52Z

#299 should fix the verify step.

kpouget · 2021-12-13T16:02:44Z

/test verify

kpouget · 2021-12-13T19:02:45Z

thanks @jmencak, the PR looks good to me

/lgtm

/hold
I let you unhold to merge when you want

jmencak · 2021-12-13T19:22:08Z

Thank you Kevin!
/hold cancel

openshift-bot · 2021-12-13T22:05:19Z

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-12-13T22:29:21Z

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-ci · 2021-12-13T23:30:56Z

@jmencak: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Simplify the e2e stalld test. It is unnecessarily complex and only adds to the time it takes to pass all e2e tests by approximately 1 node reboot. Add more debugging for any future troubleshooting. Adjust the sysctl_d_override golang e2e test to show the contents of the sysctl override file for debugging and synchronize [fsync()] it to persistent storage after write. Co-authored-by: Jiri Mencak <jmencak@users.noreply.github.com>

openshift-ci Bot requested review from dagrayvid and kpouget December 13, 2021 07:20

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 13, 2021

openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 13, 2021

openshift-ci Bot assigned kpouget Dec 13, 2021

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Dec 13, 2021

openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 13, 2021

openshift-merge-robot merged commit bc9e58a into openshift:master Dec 13, 2021

jmencak deleted the 4.10-e2e-fixes branch December 14, 2021 07:02

Conversation

jmencak commented Dec 13, 2021

Uh oh!

openshift-ci Bot commented Dec 13, 2021

Uh oh!

jmencak commented Dec 13, 2021

Uh oh!

openshift-ci Bot commented Dec 13, 2021

Uh oh!

jmencak commented Dec 13, 2021

Uh oh!

jmencak commented Dec 13, 2021

Uh oh!

jmencak commented Dec 13, 2021

Uh oh!

jmencak commented Dec 13, 2021

Uh oh!

jmencak commented Dec 13, 2021

Uh oh!

openshift-ci Bot commented Dec 13, 2021

Uh oh!

openshift-ci Bot commented Dec 13, 2021

Uh oh!

jmencak commented Dec 13, 2021

Uh oh!

jmencak commented Dec 13, 2021

Uh oh!

kpouget commented Dec 13, 2021

Uh oh!

kpouget commented Dec 13, 2021

Uh oh!

jmencak commented Dec 13, 2021

Uh oh!

openshift-bot commented Dec 13, 2021

Uh oh!

openshift-bot commented Dec 13, 2021

Uh oh!

openshift-ci Bot commented Dec 13, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants