Bug 1803697: Changing OVN DB readinessProbe to check membership status instead of DB status by alexanderConstantinescu · Pull Request #466 · openshift/cluster-network-operator

alexanderConstantinescu · 2020-02-04T17:00:55Z

This PR fixes the recently changed readinessProbe for nbdb and sbdb to check each pod's membership status instead of the general cluster status.

Some background concerning this change (as to understand the implemented check better):

We should check raft cluster status using ovs-appctl -t /var/run/openvswitch/ovnnb_db.ctl cluster/status OVN_Northbound, which outputs

sh-4.2#  ovs-appctl -t /var/run/openvswitch/ovnnb_db.ctl  cluster/status OVN_Northbound       
f782
Name: OVN_Northbound
Cluster ID: f72d (f72d04b2-f90e-4ce0-98a4-3636e668d1e7)
Server ID: f782 (f7826705-a5a9-4e4c-b2a0-935da3d72743)
Address: ssl:10.0.146.224:9643
Status: cluster member
Role: follower
Term: 3
Leader: 1b1b
Vote: 1b1b

Election timer: 1000
Log: [2, 1699]
Entries not yet committed: 0
Entries not yet applied: 0
Connections: ->0368 ->1b1b <-1b1b <-0368
Servers:
    0368 (0368 at ssl:10.0.137.205:9643)
    f782 (f782 at ssl:10.0.146.224:9643) (self)
    1b1b (1b1b at ssl:10.0.134.97:9643)

The following will be returned in the case that the pod is not a member of the cluster (has left or otherwise):

sh-4.2#  ovs-appctl -t /var/run/openvswitch/ovnnb_db.ctl  cluster/status OVN_Northbound
unknown cluster
ovs-appctl: /var/run/openvswitch/ovnnb_db.ctl: server returned an error

/assign @dcbw

…cluster status

pecameron · 2020-02-04T17:04:38Z

/test e2e-aws-ovn
/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-04T18:43:40Z

/retest

alexanderConstantinescu · 2020-02-05T13:20:38Z

/test e2e-aws-ovn

alexanderConstantinescu · 2020-02-05T15:17:04Z

/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-07T12:03:26Z

/test e2e-gcp
/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-10T14:17:02Z

/test e2e-gcp-ovn

dcbw · 2020-02-12T13:15:08Z

Terraform GCP failures....

/test e2e-gcp-ovn

dcbw · 2020-02-12T13:15:29Z

/lgtm

dcbw · 2020-02-12T13:15:52Z

@alexanderConstantinescu this needs a Bugzilla bug with a 4.4.0 target...

openshift-ci-robot · 2020-02-12T13:15:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alexanderConstantinescu, dcbw

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [dcbw]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2020-02-12T14:58:52Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-12T16:55:12Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-12T18:13:01Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-12T19:44:10Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-12T21:29:44Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-12T22:50:50Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-12T22:58:56Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-12T23:11:53Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-12T23:38:06Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-13T00:04:02Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-13T00:29:46Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-13T01:34:51Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-13T02:39:54Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-13T13:46:24Z

/retest

Please review the full test history for this PR and help us cut down flakes.

danwinship · 2020-02-13T13:52:34Z

/hold
to stop the bot while CI is broken, and this needs a bugzilla bug before it can merge anyway

alexanderConstantinescu · 2020-02-17T09:37:48Z

/retitle Bug 1803697: Changing OVN DB readinessProbe to check membership status instead of DB status

/bugzilla refresh

openshift-ci-robot · 2020-02-17T09:37:49Z

@alexanderConstantinescu: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

/retitle Bug 1803697: Changing OVN DB readinessProbe to check membership status instead of DB status

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-02-17T09:37:56Z

@alexanderConstantinescu: This pull request references Bugzilla bug 1803697, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Bug 1803697: Changing OVN DB readinessProbe to check membership status instead of DB status

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

alexanderConstantinescu · 2020-02-17T09:38:33Z

/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-17T12:51:29Z

/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-18T08:42:00Z

/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-18T14:16:42Z

/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-19T09:32:53Z

/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-19T10:59:45Z

/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-19T12:26:26Z

/test e2e-gcp-ovn

alexanderConstantinescu · 2020-02-19T14:18:48Z

And what if this works?

/test e2e-aws-ovn

danwinship · 2020-02-24T12:17:30Z

/hold cancel

openshift-bot · 2020-02-24T12:26:10Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-24T13:44:02Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-24T13:56:47Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-24T15:42:42Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-02-24T16:29:56Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot · 2020-02-24T17:01:21Z

@alexanderConstantinescu: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-gcp-ovn	`31476cc`	link	`/test e2e-gcp-ovn`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot · 2020-02-24T17:04:50Z

@alexanderConstantinescu: All pull requests linked via external trackers have merged. Bugzilla bug 1803697 has been moved to the MODIFIED state.

Details

In response to this:

Bug 1803697: Changing OVN DB readinessProbe to check membership status instead of DB status

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Changing OVN DB readinessProbe to check membership status instead of …

31476cc

…cluster status

openshift-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Feb 4, 2020

openshift-ci-robot assigned dcbw Feb 4, 2020

openshift-ci-robot requested review from danwinship and dcbw February 4, 2020 17:49

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 12, 2020

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 12, 2020

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 13, 2020

openshift-ci-robot changed the title ~~Changing OVN DB readinessProbe to check membership status instead of DB status~~ Bug 1803697: Changing OVN DB readinessProbe to check membership status instead of DB status Feb 17, 2020

openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Feb 17, 2020

alexanderConstantinescu mentioned this pull request Feb 17, 2020

Bug 1803701: Back-port changing OVN DB readinessProbe to check membership status instead of … #478

Closed

danwinship mentioned this pull request Feb 18, 2020

prow retitle appends "\r" to the bug title kubernetes/test-infra#16319

Closed

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 24, 2020

openshift-merge-robot merged commit 3bda376 into openshift:master Feb 24, 2020

This was referenced Apr 13, 2020

Bug 1823557: backport fixes to support upstream->downstream merges #589

Closed

Bug 1823560: backport fixes to support upstream->downstream merges #590

Closed

Conversation

alexanderConstantinescu commented Feb 4, 2020

Uh oh!

pecameron commented Feb 4, 2020

Uh oh!

alexanderConstantinescu commented Feb 4, 2020

Uh oh!

alexanderConstantinescu commented Feb 5, 2020

Uh oh!

alexanderConstantinescu commented Feb 5, 2020

Uh oh!

alexanderConstantinescu commented Feb 7, 2020

Uh oh!

alexanderConstantinescu commented Feb 10, 2020

Uh oh!

dcbw commented Feb 12, 2020

Uh oh!

dcbw commented Feb 12, 2020

Uh oh!

dcbw commented Feb 12, 2020

Uh oh!

openshift-ci-robot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 12, 2020

Uh oh!

openshift-bot commented Feb 13, 2020

Uh oh!

openshift-bot commented Feb 13, 2020

Uh oh!

openshift-bot commented Feb 13, 2020

Uh oh!

openshift-bot commented Feb 13, 2020

Uh oh!

openshift-bot commented Feb 13, 2020

Uh oh!

danwinship commented Feb 13, 2020

Uh oh!

alexanderConstantinescu commented Feb 17, 2020

Uh oh!

openshift-ci-robot commented Feb 17, 2020

Uh oh!

openshift-ci-robot commented Feb 17, 2020

Uh oh!

alexanderConstantinescu commented Feb 17, 2020

Uh oh!

alexanderConstantinescu commented Feb 17, 2020

Uh oh!

alexanderConstantinescu commented Feb 18, 2020

Uh oh!

alexanderConstantinescu commented Feb 18, 2020

Uh oh!

alexanderConstantinescu commented Feb 19, 2020

Uh oh!

alexanderConstantinescu commented Feb 19, 2020

Uh oh!

alexanderConstantinescu commented Feb 19, 2020

Uh oh!

alexanderConstantinescu commented Feb 19, 2020

Uh oh!

danwinship commented Feb 24, 2020

Uh oh!

openshift-bot commented Feb 24, 2020

Uh oh!

openshift-bot commented Feb 24, 2020

openshift-ci-robot commented Feb 24, 2020 •

edited

Loading