Bug 1901208: Nodes goes into NotReady state (VMware) by p0lyn0mial · Pull Request #466 · openshift/kubernetes

p0lyn0mial · 2020-11-19T10:38:50Z

we will carry a patch that allows the clients (Kubelet, KCM, etc.) to detect broken connections to the api server more quickly than the default TCP timeout of 15 minutes.

note: this patch will be dropped in future releases once http2/ping is introduced.

the first commit uses openshift/library-go#937

the second commit was created as follows:
manually changed go.mod to pin to github.com/openshift/library-go v0.0.0-20201123125610-83d6d67a1e98 followed by ./hack/update-vendor.sh

it allows the client to detect broken connections to the api server more quickly than the default TCP timeout of 15 minutes. note that it will be replaced by an HTTP ping mechanism in the future releases.

openshift-ci-robot · 2020-11-19T10:38:52Z

@p0lyn0mial: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

release-4-6: UPSTREAM: : enable TCP_USER_TIMEOUT on client connections

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

p0lyn0mial · 2020-11-19T10:39:02Z

/assign @sttts

p0lyn0mial · 2020-11-19T11:00:57Z

/test integration

p0lyn0mial · 2020-11-19T13:09:15Z

/retest

p0lyn0mial · 2020-11-19T13:37:38Z

Both integration test (TestCertRotation and TestCertRotationContinuousRequests) seems to be permanently failing. Investigating.

rphillips · 2020-11-19T19:40:35Z

/retest

p0lyn0mial · 2020-11-20T16:35:26Z

vendor/github.com/openshift/library-go/pkg/network/dialer_linux.go

it gets overwritten here https://github.com/golang/go/blob/master/src/net/dial.go#L431

p0lyn0mial · 2020-11-20T16:35:47Z

vendor/github.com/openshift/library-go/pkg/network/dialer_linux.go

now we are using the original file descriptor not a copy

p0lyn0mial · 2020-11-20T16:47:58Z

the last commit requires performing manual tests - will do that on Monday

p0lyn0mial · 2020-11-20T21:05:22Z

/retest

p0lyn0mial · 2020-11-21T13:40:17Z

/test e2e-agnostic-cmd

p0lyn0mial · 2020-11-23T08:53:50Z

/test e2e-agnostic-cmd

openshift-ci-robot · 2020-11-23T13:15:28Z

@p0lyn0mial: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

release-4-6: UPSTREAM: : enable TCP_USER_TIMEOUT on client connections

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

p0lyn0mial · 2020-11-23T13:19:59Z

@rphillips I did manual tests this morning and haven't found any issues (openshift/library-go#944 (comment)). Note that I tested it against my custom applications not kubelet.

@rphillips feel free to retitle so that it points to the BZ you want.

p0lyn0mial · 2020-11-23T13:20:12Z

/assign @rphillips

p0lyn0mial · 2020-11-23T13:29:51Z

/retest

p0lyn0mial · 2020-11-23T15:53:43Z

/retest

rphillips · 2020-11-23T17:00:20Z

/lgtm

openshift-ci-robot · 2020-11-23T17:04:38Z

@p0lyn0mial: This pull request references Bugzilla bug 1873114, which is invalid:

expected the bug to target the "4.6.z" release, but it targets "4.7.0" instead
expected dependent Bugzilla bug 1857446 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is ASSIGNED instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

Bug 1873114: Nodes goes into NotReady state (VMware)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-12-02T15:15:33Z

@nee1esh: This pull request references Bugzilla bug 1901208, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.6.z) matches configured target release for branch (4.6.z)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 1873114 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA))
dependent Bugzilla bug 1873114 targets the "4.7.0" release, which is one of the valid target releases: 4.7.0
bug has dependents

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

nee1esh · 2020-12-02T20:47:30Z

/retest

nee1esh · 2020-12-03T13:46:34Z

/retest

nee1esh · 2020-12-03T20:22:02Z

/retest

openshift-bot · 2020-12-04T01:18:05Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-04T03:41:09Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-04T04:46:06Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-04T08:27:26Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-04T08:40:24Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-04T10:50:24Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-04T11:29:28Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-04T15:23:37Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-04T16:28:26Z

/retest

Please review the full test history for this PR and help us cut down flakes.

nee1esh · 2020-12-04T21:05:11Z

/retest

sdodson · 2020-12-05T01:49:13Z

/retest

openshift-bot · 2020-12-05T02:25:58Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-05T05:40:57Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-05T05:54:00Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-05T08:16:55Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-05T08:42:56Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-05T10:13:57Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-12-05T10:26:55Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot · 2020-12-05T12:01:15Z

@p0lyn0mial: All pull requests linked via external trackers have merged:

openshift/kubernetes#466

Bugzilla bug 1901208 has been moved to the MODIFIED state.

Details

In response to this:

Bug 1901208: Nodes goes into NotReady state (VMware)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

cuppett · 2020-12-15T14:47:24Z

/cherry-pick release-4.5

openshift-cherrypick-robot · 2020-12-15T14:49:40Z

@cuppett: cannot checkout release-4.5: error checking out release-4.5: exit status 1. output: error: pathspec 'release-4.5' did not match any file(s) known to git

Details

In response to this:

/cherry-pick release-4.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

sttts · 2020-12-15T14:54:29Z

@cuppett for 4.5 you have to bring the change into openshift/origin.

UPSTREAM: <drop>: enable TCP_USER_TIMEOUT on client connections

c085042

it allows the client to detect broken connections to the api server more quickly than the default TCP timeout of 15 minutes. note that it will be replaced by an HTTP ping mechanism in the future releases.

openshift-ci-robot assigned sttts Nov 19, 2020

openshift-ci-robot added the vendor-update Touching vendor dir or related files label Nov 19, 2020

openshift-ci-robot requested review from bparees and damemi November 19, 2020 10:39

p0lyn0mial commented Nov 20, 2020

View reviewed changes

UPSTREAM: <drop>: bump(*)

a5615fa

p0lyn0mial force-pushed the release-4-6-tcp-usr-timeout-dialer branch from 2782777 to a5615fa Compare November 23, 2020 13:11

openshift-ci-robot assigned rphillips Nov 23, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 23, 2020

p0lyn0mial changed the title ~~release-4-6: UPSTREAM: <drop>: enable TCP_USER_TIMEOUT on client connections~~ Bug 1873114: Nodes goes into NotReady state (VMware) Nov 23, 2020

openshift-ci-robot added the bugzilla/severity-urgent Referenced Bugzilla bug's severity is urgent for the branch this PR is targeting. label Nov 23, 2020

openshift-ci-robot added bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Dec 2, 2020

sdodson added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Dec 2, 2020

openshift-merge-robot merged commit 7070803 into openshift:release-4.6 Dec 5, 2020

Comments

Conversation

p0lyn0mial commented Nov 19, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Nov 19, 2020

Uh oh!

p0lyn0mial commented Nov 19, 2020

Uh oh!

p0lyn0mial commented Nov 19, 2020

Uh oh!

p0lyn0mial commented Nov 19, 2020

Uh oh!

p0lyn0mial commented Nov 19, 2020

Uh oh!

rphillips commented Nov 19, 2020

Uh oh!

p0lyn0mial Nov 20, 2020

Choose a reason for hiding this comment

Uh oh!

p0lyn0mial Nov 20, 2020

Choose a reason for hiding this comment

Uh oh!

p0lyn0mial commented Nov 20, 2020

Uh oh!

p0lyn0mial commented Nov 20, 2020

Uh oh!

p0lyn0mial commented Nov 21, 2020

Uh oh!

p0lyn0mial commented Nov 23, 2020

Uh oh!

openshift-ci-robot commented Nov 23, 2020

Uh oh!

p0lyn0mial commented Nov 23, 2020

Uh oh!

p0lyn0mial commented Nov 23, 2020

Uh oh!

p0lyn0mial commented Nov 23, 2020

Uh oh!

p0lyn0mial commented Nov 23, 2020

Uh oh!

rphillips commented Nov 23, 2020

Uh oh!

openshift-ci-robot commented Nov 23, 2020

Uh oh!

openshift-ci-robot commented Dec 2, 2020

Uh oh!

nee1esh commented Dec 2, 2020

Uh oh!

nee1esh commented Dec 3, 2020

Uh oh!

nee1esh commented Dec 3, 2020

Uh oh!

openshift-bot commented Dec 4, 2020

Uh oh!

openshift-bot commented Dec 4, 2020

Uh oh!

openshift-bot commented Dec 4, 2020

Uh oh!

openshift-bot commented Dec 4, 2020

Uh oh!

openshift-bot commented Dec 4, 2020

Uh oh!

openshift-bot commented Dec 4, 2020

Uh oh!

openshift-bot commented Dec 4, 2020

Uh oh!

openshift-bot commented Dec 4, 2020

Uh oh!

openshift-bot commented Dec 4, 2020

Uh oh!

nee1esh commented Dec 4, 2020

Uh oh!

sdodson commented Dec 5, 2020

Uh oh!

openshift-bot commented Dec 5, 2020

Uh oh!

openshift-bot commented Dec 5, 2020

Uh oh!

openshift-bot commented Dec 5, 2020

p0lyn0mial commented Nov 19, 2020 •

edited

Loading