Bug 2038481: Flake failed sandboxes from bug in new guard pods

[jluhrsen]

more info in https://bugzilla.redhat.com/show_bug.cgi?id=2038481

essentially, two new guard pods are being started in 4.10+ and
are incorrectly being restarted on a cordoned node and when the
node is rebooted those pods fail to set up sandboxes right away
before the underlying network config file is present. This can
be reverted when the PR to fix this is merged:
openshift/library-go#1287

Signed-off-by: Jamo Luhrsen jluhrsen@gmail.com

[openshift-ci]

@jluhrsen: This pull request references Bugzilla bug 2038481, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target release (4.10.0) matches configured target release for branch (4.10.0)
• bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Requesting review from QA contact:
/cc @kasturinarra

Details

In response to this:

Bug 2038481: Flake failed sandboxes from bug in new guard pods

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[dgoodwin]

/lgtm

[jluhrsen]

/assign @spadgett

[jluhrsen]

/hold
looks like the possible real fix for this was merged. I'll check the results of that and get back to this if needed.

[vrutkovs]

Still happens fairly often:

• 1
• 2
• 3
• 4

/retest

[openshift-ci]

@jluhrsen: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-gcp-csi	d4a1a28	link	false	/test e2e-gcp-csi
ci/prow/e2e-aws-single-node	d4a1a28	link	false	/test e2e-aws-single-node
ci/prow/verify	d4a1a28	link	true	/test verify

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[jluhrsen]

Still happens fairly often:

• 1
• 2
• 3
• 4

/retest

right, I had assumed the "real fix" was all that was needed. Actually it was just the foundation in library-go that needed to go
in before the 3 real fixes could land. those are here:
openshift/cluster-kube-apiserver-operator#1295
openshift/cluster-kube-scheduler-operator#397
openshift/cluster-kube-controller-manager-operator#591

I will check in on those and if they don't look like they'll be merged in a timely fashion (I get the sense they will be),
then we can see about merging this origin PR.

[deads2k]

The explanation in the bugs makes sense.

/lgtm
/approve
/hold

verify failure is real.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, dgoodwin, jluhrsen

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• pkg/OWNERS [deads2k]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@jluhrsen: This pull request references Bugzilla bug 2038481. The bug has been updated to no longer refer to the pull request using the external bug tracker.

Details

In response to this:

Bug 2038481: Flake failed sandboxes from bug in new guard pods

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[deads2k]

merged in #26776