Bug 1737806: Update MCO image name and entrypoint #11801

vrutkovs · 2019-08-05T07:33:38Z

MCO image names have been updated, see openshift/machine-config-operator#1028

mtnbikenc

/lgtm

vrutkovs · 2019-08-05T16:38:10Z

/retest

vrutkovs · 2019-08-05T19:04:45Z


Failing tests:
[Feature:Prometheus][Conformance] Prometheus when installed on the cluster should report less than two alerts in firing or pending state [Suite:openshift/conformance/parallel/minimal]
[sig-autoscaling] [HPA] Horizontal pod autoscaling (scale resource: CPU) [sig-autoscaling] ReplicationController light Should scale from 2 pods to 1 pod [Suite:openshift/conformance/parallel] [Suite:k8s]

/retest

openshift-bot · 2019-08-05T20:48:53Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2019-08-06T01:21:48Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2019-08-06T03:05:46Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2019-08-06T04:23:07Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2019-08-06T05:41:10Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2019-08-06T07:38:08Z

/retest

Please review the full test history for this PR and help us cut down flakes.

vrutkovs · 2019-08-06T08:49:48Z

Alerts: machine ci-op-mfwy9hl8-716a3-p8hcp-worker-us-east-1a-centos-x5ffc does not have valid node reference

@runcom any ideas why is happening to RHEL7 nodes?

vrutkovs · 2019-08-06T08:50:41Z

/hold

until machine link alert is resolved

openshift-ci-robot · 2019-08-06T12:38:58Z

@vrutkovs: This pull request references a valid Bugzilla bug. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Bug 1737806: Update MCO image name and entrypoint

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

runcom · 2019-08-06T12:42:11Z

roles/openshift_node/tasks/apply_machine_config.yml

+- name: Get machine-config-operator image
  command: >
-    oc get daemonset machine-config-daemon
+    oc get daemonset machine-config-operator


this looks wrong:

12:37:51 [github.com/openshift/installer] ‹master*› oc get daemonset machine-config-daemon NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE machine-config-daemon 6 6 6 6 6 kubernetes.io/os=linux 16h 14:41:52 [github.com/openshift/installer] ‹master*› oc get daemonset machine-config-operator Error from server (NotFound): daemonsets.extensions "machine-config-operator" not found

Ah, right, the daemonset name didn't change

runcom · 2019-08-06T12:42:27Z

roles/openshift_node/tasks/apply_machine_config.yml

    --config={{ openshift_node_kubeconfig_path }}
    --namespace=openshift-machine-config-operator
-    --output=jsonpath='{.spec.template.spec.containers[?(@.name=="machine-config-daemon")].image}'
+    --output=jsonpath='{.spec.template.spec.containers[?(@.name=="machine-config-operator")].image}'


this as well (wrong)

mtnbikenc · 2019-08-07T13:24:02Z

Failing tests:

[Feature:Prometheus][Conformance] Prometheus when installed on the cluster should report less than two alerts in firing or pending state [Suite:openshift/conformance/parallel/minimal]

Opened bug for failing test:
https://bugzilla.redhat.com/show_bug.cgi?id=1738291

mtnbikenc · 2019-08-07T19:03:18Z

I tested this PR manually and found that the nodes are not coming up.

On the new hosts I found:
machine-config-daemon-host.service failing with Failed to start Machine Config Daemon Initial.
network.service failing with Failed to start LSB: Bring up/down networking.

cgwalters · 2019-08-07T20:55:19Z

I tested this PR manually and found that the nodes are not coming up.

Can you be more precise about "not coming up"? Like, kubelet.service failing, nodes not joining the cluster, something else?

mtnbikenc · 2019-08-07T20:59:54Z

nodes are not joining the cluster
kubelet.service is running but unable to find the node
I'll get kubelet logs tomorrow

mtnbikenc · 2019-08-08T14:34:26Z

Manually tested with the most recent build and nodes joined the cluster.
/retest

mtnbikenc · 2019-08-08T17:07:57Z

Alert issue resolved now that nodes are joining the cluster.
/hold cancel
/lgtm

openshift-ci-robot · 2019-08-08T17:08:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mtnbikenc, vrutkovs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [mtnbikenc,vrutkovs]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mtnbikenc · 2019-08-08T17:56:38Z

ssh load balancer was not deployed
/retest

openshift-ci-robot · 2019-08-08T19:33:47Z

@vrutkovs: All pull requests linked via external trackers have merged. The Bugzilla bug has been moved to the MODIFIED state.

Details

In response to this:

Bug 1737806: Update MCO image name and entrypoint

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Aug 5, 2019

openshift-ci-robot requested review from jcpowermac and mtnbikenc August 5, 2019 07:33

openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Aug 5, 2019

vrutkovs force-pushed the mco-image-name branch 2 times, most recently from d5af79e to 6d01847 Compare August 5, 2019 10:54

vrutkovs changed the title ~~Update MCO image name~~ Update MCO image name and entrypoint Aug 5, 2019

mtnbikenc approved these changes Aug 5, 2019

View reviewed changes

openshift-ci-robot assigned mtnbikenc Aug 5, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Aug 5, 2019

vrutkovs mentioned this pull request Aug 5, 2019

rhel scaleup: update other repos openshift/release#4279

Merged

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 6, 2019

mtnbikenc changed the title ~~Update MCO image name and entrypoint~~ Bug 1737806: Update MCO image name and entrypoint Aug 6, 2019

openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Aug 6, 2019

runcom reviewed Aug 6, 2019

View reviewed changes

Update MCO image name

ae11e06

Run MCD in once-from mode via setting an entrypoint in MCO image

d28fb17

vrutkovs force-pushed the mco-image-name branch from 6d01847 to d28fb17 Compare August 6, 2019 14:54

openshift-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed lgtm Indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 6, 2019

cgwalters mentioned this pull request Aug 7, 2019

Fix byo machine-config-daemon-host.service openshift/machine-config-operator#1046

Closed

vrutkovs mentioned this pull request Aug 8, 2019

test/e2e/dr/quorum_restore: set MCO image for setup-etcd-environment image openshift/origin#23572

Merged

openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Aug 8, 2019

openshift-merge-robot merged commit 52e6680 into openshift:master Aug 8, 2019

mtnbikenc mentioned this pull request Aug 8, 2019

Update MCO entrypoint for upgrade #11807

Merged

Bug 1737806: Update MCO image name and entrypoint #11801

Bug 1737806: Update MCO image name and entrypoint #11801

Uh oh!

Conversation

vrutkovs commented Aug 5, 2019

Uh oh!

mtnbikenc left a comment

Choose a reason for hiding this comment

Uh oh!

vrutkovs commented Aug 5, 2019

Uh oh!

vrutkovs commented Aug 5, 2019

Uh oh!

openshift-bot commented Aug 5, 2019

Uh oh!

openshift-bot commented Aug 6, 2019

Uh oh!

openshift-bot commented Aug 6, 2019

Uh oh!

openshift-bot commented Aug 6, 2019

Uh oh!

openshift-bot commented Aug 6, 2019

Uh oh!

openshift-bot commented Aug 6, 2019

Uh oh!

vrutkovs commented Aug 6, 2019

Uh oh!

vrutkovs commented Aug 6, 2019

Uh oh!

openshift-ci-robot commented Aug 6, 2019

Uh oh!

runcom Aug 6, 2019

Choose a reason for hiding this comment

Uh oh!

vrutkovs Aug 6, 2019

Choose a reason for hiding this comment

Uh oh!

vrutkovs Aug 6, 2019

Choose a reason for hiding this comment

Uh oh!

runcom Aug 6, 2019

Choose a reason for hiding this comment

Uh oh!

vrutkovs Aug 6, 2019

Choose a reason for hiding this comment

Uh oh!

mtnbikenc commented Aug 7, 2019

Uh oh!

mtnbikenc commented Aug 7, 2019

Uh oh!

cgwalters commented Aug 7, 2019

Uh oh!

mtnbikenc commented Aug 7, 2019

Uh oh!

mtnbikenc commented Aug 8, 2019

Uh oh!

mtnbikenc commented Aug 8, 2019

Uh oh!

openshift-ci-robot commented Aug 8, 2019

Uh oh!

mtnbikenc commented Aug 8, 2019

Uh oh!

openshift-ci-robot commented Aug 8, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants