cmd/machine-config-operator/bootstrap: add cluster-etcd-operator flag #1224

hexfusion · 2019-10-29T21:23:32Z

This PR adds a flag to bootstrap allowing the cluster-etcd-operator image to be be used in the etcd-member spec. We are also using the value of the flag as conditional logic for the spec. Because we need to balance that with what is put in etcd toggling CEO will involve unsetting the flag and the value of the configmap.

We also add a readinessProbe to etcd-member container. etcd should not show ready in service until it can serve traffic.

kikisdeliveryservice

A few comments

cmd/machine-config-operator/bootstrap.go

cmd/setup-etcd-environment/run.go

pkg/operator/render_test.go

pkg/server/server_test.go

hexfusion · 2019-10-30T13:22:15Z

using for test release for debug

/test e2e-aws

hexfusion · 2019-10-30T15:03:47Z

FTR the failure has been isolated and will be resolved shortly

kikisdeliveryservice · 2019-10-30T20:36:00Z

fyi: that aws-scaleup-rhel7 is basically broken. i filed a bz last night. so you can just:

/skip

hexfusion · 2019-11-01T02:46:52Z

level=error msg="Error: Error creating DNS RecordSet: googleapi: Error 503: Backend Error, backendError"
level=error
level=error msg="  on ../tmp/openshift-install-025962353/dns/base.tf line 39, in resource \"google_dns_record_set\" \"etcd_a_nodes\":"
level=error msg="  39: resource \"google_dns_record_set\" \"etcd_a_nodes\" {"

/test e2e-gcp-upgrade

An empty image field will mean CEO is not enabled and cluster is bootstrapping 4.2 style. A lot left to do for making etcd-member manifest file boot without 4.2 but this is a start.

hexfusion · 2019-11-01T14:11:17Z

/cc @runcom @kikisdeliveryservice PTAL

openshift-ci-robot · 2019-11-01T14:11:19Z

@hexfusion: GitHub didn't allow me to request PR reviews from the following users: PTAL.

Note that only openshift members and repo collaborators can review this PR, and authors cannot review their own PRs.

Details

In response to this:

/cc @runcom @kikisdeliveryservice PTAL

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

hexfusion · 2019-11-01T14:42:47Z

install/0000_80_machine-config-operator_02_images.configmap.yaml

      "etcd": "registry.svc.ci.openshift.org/openshift:etcd",
      "infraImage": "registry.svc.ci.openshift.org/openshift:pod",
      "kubeClientAgentImage": "registry.svc.ci.openshift.org/openshift:kube-client-agent",
+      "clusterEtcdOperatorImage": "",


to be clear this is intentional and will be added in a separate PR when we enable the operator.

hexfusion · 2019-11-01T15:16:29Z

mcd_test.go:555: pool worker didn't report updated to rendered-worker-a4935ca22d7431a704c741eba9e4d563: timed out waiting for the condition

Doesn't feel related as the error is reporting from worker which we should not of changed.

/retest

alaypatel07 · 2019-11-01T15:34:31Z

/lgtm

hexfusion · 2019-11-01T16:31:37Z

level=info msg="Cluster operator authentication Progressing is Unknown with NoData: "

/test e2e-gcp-op

kikisdeliveryservice

tests are kind of broken looking into that now..

but this looks good!

kikisdeliveryservice · 2019-11-01T17:31:58Z

/skip

hexfusion · 2019-11-01T18:47:07Z

#1232 (comment)

FIPS is broken I guess?

hexfusion · 2019-11-01T18:54:43Z

cc @cgwalters

kikisdeliveryservice · 2019-11-01T19:23:35Z

hitting same error as here: #1232 (comment)

we are trying to get a fix

kikisdeliveryservice · 2019-11-01T20:05:51Z

@cgwalters do you think we can do an override for this as it's blocking other PRs in the bunch?

cgwalters · 2019-11-01T20:11:16Z

Yeah,
/approve
And I'm OK overriding, you want to pull the trigger?

openshift-ci-robot · 2019-11-01T20:11:32Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alaypatel07, cgwalters, hexfusion, kikisdeliveryservice

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [cgwalters,kikisdeliveryservice]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kikisdeliveryservice · 2019-11-01T20:14:23Z

/override ci/prow/e2e-gcp-op

openshift-ci-robot · 2019-11-01T20:14:39Z

@kikisdeliveryservice: Overrode contexts on behalf of kikisdeliveryservice: ci/prow/e2e-gcp-op

Details

In response to this:

/override ci/prow/e2e-gcp-op

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kikisdeliveryservice · 2019-11-01T22:21:13Z

thanks bot for re-running all the tests again, including the one we told you not to run...

/override ci/prow/e2e-gcp-op

kikisdeliveryservice · 2019-11-01T22:23:06Z

i think it has to fail before i can override, ill check in later

openshift-ci-robot · 2019-11-02T00:09:18Z

@hexfusion: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/prow/e2e-aws-scaleup-rhel7	`492fac6`	link	`/test e2e-aws-scaleup-rhel7`
ci/prow/e2e-gcp-op	`492fac6`	link	`/test e2e-gcp-op`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

kikisdeliveryservice · 2019-11-02T00:25:38Z

/override ci/prow/e2e-gcp-op

openshift-ci-robot · 2019-11-02T00:25:53Z

@kikisdeliveryservice: Overrode contexts on behalf of kikisdeliveryservice: ci/prow/e2e-gcp-op

Details

In response to this:

/override ci/prow/e2e-gcp-op

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 29, 2019

openshift-ci-robot requested review from ashcrow and cgwalters October 29, 2019 21:23

hexfusion force-pushed the add_ceo_flag branch from 87644b5 to eda3678 Compare October 29, 2019 21:35

hexfusion mentioned this pull request Oct 29, 2019

setup-etcd-environment: support scaling from cluster-etcd-operator #1221

Merged

kikisdeliveryservice reviewed Oct 29, 2019

View reviewed changes

cmd/machine-config-operator/bootstrap.go Show resolved Hide resolved

cmd/setup-etcd-environment/run.go Outdated Show resolved Hide resolved

pkg/operator/render_test.go Outdated Show resolved Hide resolved

pkg/server/server_test.go Outdated Show resolved Hide resolved

openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 29, 2019

hexfusion force-pushed the add_ceo_flag branch 2 times, most recently from 1bf123b to bd9fc6f Compare October 30, 2019 00:32

hexfusion force-pushed the add_ceo_flag branch 7 times, most recently from 1feecc2 to bb2ca60 Compare November 1, 2019 02:10

hexfusion changed the title ~~WIP cmd/machine-config-operator: add cluster-etcd-operator flag to bootstrap cmd.~~ cmd/machine-config-operator/bootstrap: add cluster-etcd-operator flag Nov 1, 2019

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 1, 2019

hexfusion force-pushed the add_ceo_flag branch 5 times, most recently from e0f6f00 to 5ade6a6 Compare November 1, 2019 12:42

*: Add cluster etcd operator image in MCO

492fac6

An empty image field will mean CEO is not enabled and cluster is bootstrapping 4.2 style. A lot left to do for making etcd-member manifest file boot without 4.2 but this is a start.

openshift-ci-robot requested review from kikisdeliveryservice and runcom November 1, 2019 14:11

hexfusion commented Nov 1, 2019

View reviewed changes

hexfusion mentioned this pull request Nov 1, 2019

[WIP] *: add logic for cluster-etcd-operator toggle openshift/installer#2608

Closed

3 tasks

openshift-ci-robot assigned alaypatel07 Nov 1, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 1, 2019

kikisdeliveryservice approved these changes Nov 1, 2019

View reviewed changes

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 1, 2019

openshift-merge-robot merged commit bd84695 into openshift:master Nov 2, 2019

hexfusion mentioned this pull request Nov 27, 2019

*: add support for cluster-etcd-operator openshift/installer#2730

Merged

3 tasks

alaypatel07 mentioned this pull request Dec 11, 2019

*: conditionally populate the images configmap with cluster-etcd-operator image #1288

Merged

cmd/machine-config-operator/bootstrap: add cluster-etcd-operator flag #1224

cmd/machine-config-operator/bootstrap: add cluster-etcd-operator flag #1224

Uh oh!

Conversation

hexfusion commented Oct 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kikisdeliveryservice left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hexfusion commented Oct 30, 2019

Uh oh!

hexfusion commented Oct 30, 2019

Uh oh!

kikisdeliveryservice commented Oct 30, 2019

Uh oh!

hexfusion commented Nov 1, 2019

Uh oh!

hexfusion commented Nov 1, 2019

Uh oh!

openshift-ci-robot commented Nov 1, 2019

Uh oh!

hexfusion Nov 1, 2019

Choose a reason for hiding this comment

Uh oh!

hexfusion commented Nov 1, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alaypatel07 commented Nov 1, 2019

Uh oh!

hexfusion commented Nov 1, 2019

Uh oh!

kikisdeliveryservice left a comment

Choose a reason for hiding this comment

Uh oh!

kikisdeliveryservice commented Nov 1, 2019

Uh oh!

hexfusion commented Nov 1, 2019

Uh oh!

hexfusion commented Nov 1, 2019

Uh oh!

kikisdeliveryservice commented Nov 1, 2019

Uh oh!

kikisdeliveryservice commented Nov 1, 2019

Uh oh!

cgwalters commented Nov 1, 2019

Uh oh!

openshift-ci-robot commented Nov 1, 2019

Uh oh!

kikisdeliveryservice commented Nov 1, 2019

Uh oh!

openshift-ci-robot commented Nov 1, 2019

Uh oh!

kikisdeliveryservice commented Nov 1, 2019

Uh oh!

kikisdeliveryservice commented Nov 1, 2019

Uh oh!

openshift-ci-robot commented Nov 2, 2019

Uh oh!

kikisdeliveryservice commented Nov 2, 2019

Uh oh!

openshift-ci-robot commented Nov 2, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

hexfusion commented Oct 29, 2019 •

edited

Loading

hexfusion commented Nov 1, 2019 •

edited

Loading