Skip to content

SDN-662: OVN raft followups#410

Closed
alexanderConstantinescu wants to merge 1 commit intoopenshift:masterfrom
alexanderConstantinescu:feature/SDN-662
Closed

SDN-662: OVN raft followups#410
alexanderConstantinescu wants to merge 1 commit intoopenshift:masterfrom
alexanderConstantinescu:feature/SDN-662

Conversation

@alexanderConstantinescu
Copy link
Contributor

This PR adds a PodDisruptionBudget for maintaining concensus for the raft cluster + a readinessProbe for the sbdb and nbdb checking that all members have joined the cluster, and if they have: waits 30 seconds according to what has been mentioned here: https://jira.coreos.com/browse/SDN-662

/assign @squeed

@openshift-ci-robot openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Dec 2, 2019
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: alexanderConstantinescu
To complete the pull request process, please assign squeed
You can assign the PR to them by writing /assign @squeed in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@alexanderConstantinescu
Copy link
Contributor Author

/test e2e-gcp-upgrade
/test e2e-gcp

@alexanderConstantinescu alexanderConstantinescu force-pushed the feature/SDN-662 branch 2 times, most recently from a0b5d0a to b1943ab Compare December 3, 2019 16:16
@alexanderConstantinescu
Copy link
Contributor Author

@squeed: I added a livenessProbe. This one checks that the node itself is up and able to respond, as supposed to the readinessProbe which checks the general cluster member status.

@danwinship
Copy link
Contributor

/test e2e-gcp-ovn-upgrade

@alexanderConstantinescu
Copy link
Contributor Author

/test e2e-gcp

@alexanderConstantinescu
Copy link
Contributor Author

/test e2e-gcp

@squeed
Copy link
Contributor

squeed commented Dec 4, 2019

/test e2e-ovn-aws

@squeed
Copy link
Contributor

squeed commented Dec 4, 2019

/test e2e-aws-ovn

AVAILABLE_NODES=0
for node in "${OVN_NODES_ARRAY[@]}"; do
node_ip=$(getent ahostsv4 "${node}" | grep RAW | awk '{print $1}')
if ovs-appctl -t /var/run/openvswitch/ovnnb_db.ctl cluster/status OVN_Northbound | grep -q $node_ip; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tested this, and it turns out we don't ever remove servers if we're not available. So we need to parse the "connections" line. I'll give you a sample in a bit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, all we need to do is check that the DB port is open. it turns out ovsdb takes care of this for us and only opens the port when consensus is achieved.

@openshift-ci-robot
Copy link
Contributor

@alexanderConstantinescu: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/prow/e2e-gcp ee14aed link /test e2e-gcp
ci/prow/e2e-aws-ovn ee14aed link /test e2e-aws-ovn

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@squeed
Copy link
Contributor

squeed commented Dec 4, 2019

Alexander is looking in to something else, he asked me to get this over the line.
Continued in #414

@squeed squeed closed this Dec 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments