*: conditionally populate the images configmap with cluster-etcd-operator image #1288

alaypatel07 · 2019-11-24T17:19:43Z

This adds a flag to MCO for cluster-etcd-operator image and hardcodes
the value to "".

This PR conditionally populates the cluster-etcd-operator image only
if the etcd custom resource exists and if it has a spec field set as
managementState: Managed. It is the same mechanism used
in the installer. This will make sure that the MC rendered during the initial.
boot with the value coming in from the installer flag is always same
as one generate in the sync loop.

With this, we can use a single knob in cluster-etcd-operator
project to test the bootstrap process in CI with cluster-etcd-operator
enabled.

The existence of cluster-etcd-operator image is used to render static
etcd pod yaml. This PR will be followed by another PR which introduces
a boolean flag to be used for conditional rendering of the static pod image.
It will be a more readable way of achieving the same means.

alaypatel07 · 2019-11-24T17:25:33Z

/hold testing this manually

kikisdeliveryservice · 2019-11-27T20:01:01Z

/skip

hexfusion · 2019-12-05T22:29:32Z

/retest

kikisdeliveryservice · 2019-12-06T00:00:41Z

at a quick glance this seems to be failing on masters only so prob the pr? seems etcd server is taking too long: https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/1288/pull-ci-openshift-machine-config-operator-master-e2e-aws/5908/artifacts/e2e-aws/pods/openshift-etcd_etcd-member-ip-10-0-137-114.ec2.internal_etcd-member.log

alaypatel07 · 2019-12-07T17:35:20Z

clusterrolebinding.rbac.authorization.k8s.io/system:openshift:controller:resourcequota-controller
E1207 16:34:33.205040       1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted
E1207 16:36:17.108350       1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted
E1207 16:36:36.023379       1 watcher.go:214] watch chan error: etcdserver: mvcc: required revision has been compacted

Etcd was up but openshift-apiserver had the above error messages, compacted data store, trying again

/test e2e-aws

openshift-bot · 2020-01-13T12:45:25Z

/retest

Please review the full test history for this PR and help us cut down flakes.

This is necessary to prevent races

hexfusion · 2020-01-13T16:05:06Z

level=error msg="Error: Error waiting for AMI (ami-0123e5f5f322fe54f) to be ready: timeout while waiting for state to become 'available' (last state: 'pending', timeout: 40m0s)"

/retest

hexfusion · 2020-01-13T17:38:34Z

level=fatal msg="failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply using Terraform"

/test e2e-aws-scaleup-rhel7

alaypatel07 · 2020-01-13T18:01:15Z

digging into the error e2e-gcp-op

/hold
/wip

alaypatel07 · 2020-01-17T20:01:51Z

/test e2e-gcp-op

hexfusion · 2020-01-18T15:12:19Z

level=error msg="Error: Error applying IAM policy for project "openshift-gce-devel-ci": Error setting IAM policy for project "openshift-gce-devel-ci": googleapi: Error 400: The number of members in the policy (1,501) is larger than the maximum allowed size 1,500., badRequest"

/retest

EtcdInformer is only used by the MCO pod to reconcile image names. This pulls out the logic of creating etcd informer from the generic controller context to MCO pod start method.

hexfusion · 2020-01-22T03:39:04Z

https://prow.svc.ci.openshift.org/job-history/origin-ci-test/pr-logs/directory/pull-ci-openshift-machine-config-operator-master-e2e-gcp-op

Looks like CI is up again.

/retest

hexfusion · 2020-01-22T03:39:17Z

/joke

openshift-ci-robot · 2020-01-22T03:39:18Z

@hexfusion: A red and a blue ship have just collided in the Caribbean. Apparently the survivors are marooned.

Details

In response to this:

/joke

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

hexfusion · 2020-01-22T11:02:30Z

/lgtm

openshift-ci-robot · 2020-01-22T11:02:56Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alaypatel07, hexfusion, kikisdeliveryservice

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [kikisdeliveryservice]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

hexfusion · 2020-01-22T11:07:11Z

/hold cancel

hexfusion · 2020-01-22T12:26:54Z

--- FAIL: TestKernelArguments (448.05s)
mcd_test.go:259: Created kargs-1ac503cd-b15f-43e7-9316-403fe39db164
mcd_test.go:115: Pool worker has rendered config kargs-1ac503cd-b15f-43e7-9316-403fe39db164 with rendered-worker-98c738161d53257c99ec59c65bcaa218 (waited 4.007773128s)
mcd_test.go:263: pool worker didn't report updated to rendered-worker-98c738161d53257c99ec59c65bcaa218: Get https://api.ci-op-7jwmbk77-1354f.origin-ci-int-gce.dev.openshift.com:6443/apis/machineconfiguration.openshift.io/v1/machineconfigpools/worker: dial tcp 35.185.78.239:6443: connect: connection refused

New, I have not see this... last round passed

/test e2e-gcp-op

openshift-ci-robot · 2020-01-22T13:33:08Z

@alaypatel07: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-vsphere	6f850a18762ee5a1c6a4d1d709fc47938faec5b8	link	`/test e2e-vsphere`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-bot · 2020-01-22T13:48:18Z

/retest

Please review the full test history for this PR and help us cut down flakes.

vrutkovs · 2020-01-28T19:40:38Z

/cherrypick fcos

openshift-cherrypick-robot · 2020-01-28T19:40:48Z

@vrutkovs: new pull request created: #1413

Details

In response to this:

/cherrypick fcos

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Nov 24, 2019

alaypatel07 changed the title *: install *: enable cluster-etcd-operator Nov 24, 2019

openshift-ci-robot requested review from ericavonb and sinnykumari November 24, 2019 17:20

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 24, 2019

alaypatel07 force-pushed the enable_ceo branch from 6f850a1 to b3235f8 Compare December 7, 2019 01:19

openshift-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Dec 7, 2019

alaypatel07 force-pushed the enable_ceo branch from b3235f8 to 370c2c7 Compare December 7, 2019 01:21

alaypatel07 changed the title *: enable cluster-etcd-operator [WIP]: *: enable cluster-etcd-operator Dec 7, 2019

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 7, 2019

alaypatel07 force-pushed the enable_ceo branch from 415d39c to e3caad8 Compare December 7, 2019 11:25

alaypatel07 changed the title ~~[WIP]: *: enable cluster-etcd-operator~~ *: enable cluster-etcd-operator Dec 7, 2019

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 7, 2019

alaypatel07 force-pushed the enable_ceo branch from e3caad8 to ad0c14d Compare December 7, 2019 11:44

openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 7, 2019

alaypatel07 force-pushed the enable_ceo branch 2 times, most recently from da8bbf6 to 8d9a25c Compare December 7, 2019 16:02

openshift-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 7, 2019

alaypatel07 force-pushed the enable_ceo branch from 8d9a25c to 4da3c5a Compare December 9, 2019 02:52

openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 9, 2019

alaypatel07 force-pushed the enable_ceo branch from 0893291 to bd48395 Compare December 9, 2019 21:06

alaypatel07 added 3 commits January 13, 2020 20:09

install: seed configmap and image-references with ceo images

72efda9

This is necessary to prevent races

*: get etcd CR from kube api to check if it managed/unmanaged

fa67bac

controller_context.go: terminate MCO pod for transient apiserver errors

91ab6c5

alaypatel07 force-pushed the enable_ceo branch from 213919b to 91ab6c5 Compare January 13, 2020 14:39

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Jan 13, 2020

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 13, 2020

*: undo the change to controller_context

c29ebe9

EtcdInformer is only used by the MCO pod to reconcile image names. This pulls out the logic of creating etcd informer from the generic controller context to MCO pod start method.

alaypatel07 force-pushed the enable_ceo branch from ee499f2 to c29ebe9 Compare January 19, 2020 14:11

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 22, 2020

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 22, 2020

openshift-merge-robot merged commit d7e308d into openshift:master Jan 22, 2020

openshift-cherrypick-robot mentioned this pull request Jan 28, 2020

[fcos] *: conditionally populate the images configmap with cluster-etcd-operator image #1413

Closed

vrutkovs mentioned this pull request Jan 28, 2020

[fcos] Cherry-pick commits to enable cluster-etcd-operator #1415

Merged

*: conditionally populate the images configmap with cluster-etcd-operator image #1288

*: conditionally populate the images configmap with cluster-etcd-operator image #1288

Uh oh!

Conversation

alaypatel07 commented Nov 24, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alaypatel07 commented Nov 24, 2019

Uh oh!

kikisdeliveryservice commented Nov 27, 2019

Uh oh!

hexfusion commented Dec 5, 2019

Uh oh!

kikisdeliveryservice commented Dec 6, 2019

Uh oh!

alaypatel07 commented Dec 7, 2019

Uh oh!

openshift-bot commented Jan 13, 2020

Uh oh!

hexfusion commented Jan 13, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hexfusion commented Jan 13, 2020

Uh oh!

alaypatel07 commented Jan 13, 2020

Uh oh!

alaypatel07 commented Jan 17, 2020

Uh oh!

hexfusion commented Jan 18, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hexfusion commented Jan 22, 2020

Uh oh!

hexfusion commented Jan 22, 2020

Uh oh!

openshift-ci-robot commented Jan 22, 2020

Uh oh!

hexfusion commented Jan 22, 2020

Uh oh!

openshift-ci-robot commented Jan 22, 2020

Uh oh!

hexfusion commented Jan 22, 2020

Uh oh!

hexfusion commented Jan 22, 2020

Uh oh!

openshift-ci-robot commented Jan 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-bot commented Jan 22, 2020

Uh oh!

vrutkovs commented Jan 28, 2020

Uh oh!

openshift-cherrypick-robot commented Jan 28, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

alaypatel07 commented Nov 24, 2019 •

edited

Loading

hexfusion commented Jan 13, 2020 •

edited

Loading

hexfusion commented Jan 18, 2020 •

edited

Loading

openshift-ci-robot commented Jan 22, 2020 •

edited

Loading