Bug 1926558: [release-4.4]: rhcos.json: Update bootimages for ppc64le/s390x to fix kubelet error #4628

Prashanth684 · 2021-02-09T03:27:00Z

Refer to https://bugzilla.redhat.com/show_bug.cgi?id=1926558. After #4604 landed in 4.4, the multi-arch CI runs started failing because the hyperkube package was old. This was not an issue for x86 as the rhcos images were bumped for a CVE (#3985), resulting in a newer hyperkube package which did not have this issue with the kubelet erroring out on anonymous-auth=false

Logs of error:

Feb 08 22:02:17 psundara-ocp-ptj44-bootstrap hyperkube[2888]: I0208 22:02:17.946044    2888 server.go:540] No cloud provider specified: "" from the config file: ""
Feb 08 22:02:17 psundara-ocp-ptj44-bootstrap hyperkube[2888]: W0208 22:02:17.946082    2888 server.go:563] standalone mode, no API client
Feb 08 22:02:17 psundara-ocp-ptj44-bootstrap hyperkube[2888]: F0208 22:02:17.946123    2888 server.go:273] failed to run Kubelet: No authentication method configured

These are the builds that were closest in parity to x86:
https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.4-s390x/44.82.202011122341-0/s390x/meta.json for s390x
https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.4-ppc64le/44.82.202012010439-0/ppc64le/meta.json for ppc64le

…ubelet erroring out with "No authentication method configured" Refer to https://bugzilla.redhat.com/show_bug.cgi?id=1926558. After openshift#4604 landed in 4.4, the multi-arch CI runs started failing because the hyperkube package was old. This was not an issue for x86 as the rhcos images were bumped for a CVE (openshift#3985), resulting in a newer hyperkube package which did not have this issue with the kubelet erroring out on `anonymous-auth=false` These are the builds that were closest in parity to x86: https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.4-s390x/44.82.202011122341-0/s390x/meta.json for s390x https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.4-ppc64le/44.82.202012010439-0/ppc64le/meta.json for ppc64le

openshift-ci-robot · 2021-02-09T03:27:03Z

@Prashanth684: This pull request references Bugzilla bug 1926558, which is invalid:

expected the bug to target the "4.4.z" release, but it targets "---" instead
expected Bugzilla bug 1926558 to depend on a bug targeting a release in 4.5.0, 4.5.z and in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but no dependents were found

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

Bug 1926558: rhcos.json: Update bootimages for ppc64le/s390x to fix kubelet error

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Prashanth684 · 2021-02-09T03:35:46Z

the package diffs were quite big but this is the diff of openshift-hyperkube:

 "openshift-hyperkube": {
            "s390x/44.81.202004252140-0": "openshift-hyperkube-4.4.0-202004241258.git.1.bfb96d1.el8.s390x",
            "s390x/44.82.202011122341-0": "openshift-hyperkube-4.4.0-202011122017.p0.git.0.149ca32.el8.s390x"
        },

        "openshift-hyperkube": {
            "ppc64le/44.81.202004252240-0": "openshift-hyperkube-4.4.0-202004241258.git.1.bfb96d1.el8.ppc64le",
            "ppc64le/44.82.202012010439-0": "openshift-hyperkube-4.4.0-202011130111.p0.git.0.4861dfa.el8.ppc64le"
        },

Prashanth684 · 2021-02-09T04:14:39Z

/bugzilla refresh

openshift-ci-robot · 2021-02-09T04:14:41Z

@Prashanth684: This pull request references Bugzilla bug 1926558, which is invalid:

expected the bug to target the "4.4.z" release, but it targets "---" instead
expected dependent Bugzilla bug 1926564 to target a release in 4.5.0, 4.5.z, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Prashanth684 · 2021-02-09T04:16:42Z

/bugzilla refresh

openshift-ci-robot · 2021-02-09T04:16:49Z

@Prashanth684: This pull request references Bugzilla bug 1926558, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.4.z) matches configured target release for branch (4.4.z)
bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 1926564 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA))
dependent Bugzilla bug 1926564 targets the "4.5.z" release, which is one of the valid target releases: 4.5.0, 4.5.z
bug has dependents

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Prashanth684 · 2021-02-09T13:46:20Z

/cc @miabbott @ashcrow

andymcc · 2021-02-09T14:06:45Z

/lgtm

Prashanth684 · 2021-02-09T15:42:03Z

/assign @staebler

staebler · 2021-02-10T16:01:15Z

/approve

openshift-bot · 2021-02-10T16:42:16Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-10T17:47:17Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-10T18:52:20Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-10T19:05:15Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-10T20:10:17Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-10T22:46:15Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T02:27:13Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T02:40:15Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T02:53:15Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T13:17:15Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T15:01:15Z

/retest

Please review the full test history for this PR and help us cut down flakes.

staebler · 2021-02-11T17:29:56Z

The e2e-aws-workers-rhel7 job is failing consistently with the following 2 test failures.

[Feature:Prometheus][Conformance] Prometheus when installed on the cluster [Top Level] [Feature:Prometheus][Conformance] Prometheus when installed on the cluster shouldn't have failing rules evaluation [Suite:openshift/conformance/parallel/minimal]
[Feature:Prometheus][Late] Alerts [Top Level] [Feature:Prometheus][Late] Alerts shouldn't report any alerts in firing state apart from Watchdog and AlertmanagerReceiversNotConfigured [Suite:openshift/conformance/parallel]

Those failures are not related to these changes.

/override ci/prow/e2e-aws-workers-rhel7

openshift-ci-robot · 2021-02-11T17:30:24Z

@staebler: Overrode contexts on behalf of staebler: ci/prow/e2e-aws-workers-rhel7

Details

In response to this:

The e2e-aws-workers-rhel7 job is failing consistently with the following 2 test failures.

[Feature:Prometheus][Conformance] Prometheus when installed on the cluster [Top Level] [Feature:Prometheus][Conformance] Prometheus when installed on the cluster shouldn't have failing rules evaluation [Suite:openshift/conformance/parallel/minimal]
[Feature:Prometheus][Late] Alerts [Top Level] [Feature:Prometheus][Late] Alerts shouldn't report any alerts in firing state apart from Watchdog and AlertmanagerReceiversNotConfigured [Suite:openshift/conformance/parallel]

Those failures are not related to these changes.

/override ci/prow/e2e-aws-workers-rhel7

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

miabbott · 2021-02-11T18:02:02Z

I trust Andy and Prashanth have this handled 👍

/approve

openshift-ci-robot · 2021-02-11T18:02:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: miabbott, staebler

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [staebler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2021-02-11T18:15:19Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T18:28:19Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T19:33:18Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T19:46:16Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T20:12:17Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-02-11T20:38:17Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci · 2021-02-11T20:41:32Z

@Prashanth684: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-openstack	`c3c83a5`	link	`/test e2e-openstack`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot · 2021-02-11T21:15:44Z

@Prashanth684: All pull requests linked via external trackers have merged:

openshift/installer#4628

Bugzilla bug 1926558 has been moved to the MODIFIED state.

Details

In response to this:

Bug 1926558: [release-4.4]: rhcos.json: Update bootimages for ppc64le/s390x to fix kubelet error

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Feb 9, 2021

openshift-ci-robot requested review from jcpowermac and mtnbikenc February 9, 2021 03:27

Prashanth684 changed the title ~~Bug 1926558: rhcos.json: Update bootimages for ppc64le/s390x to fix kubelet error~~ Bug 1926558: [release-4.4]: rhcos.json: Update bootimages for ppc64le/s390x to fix kubelet error Feb 9, 2021

openshift-ci-robot added bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Feb 9, 2021

openshift-ci-robot requested review from ashcrow and miabbott February 9, 2021 13:46

openshift-ci-robot assigned andymcc Feb 9, 2021

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 9, 2021

openshift-ci-robot assigned staebler Feb 9, 2021

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 10, 2021

bparees added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Feb 10, 2021

openshift-merge-robot merged commit eadd4ef into openshift:release-4.4 Feb 11, 2021

Prashanth684 deleted the bz1926558-ma branch February 12, 2021 14:13

Bug 1926558: [release-4.4]: rhcos.json: Update bootimages for ppc64le/s390x to fix kubelet error #4628

Bug 1926558: [release-4.4]: rhcos.json: Update bootimages for ppc64le/s390x to fix kubelet error #4628

Uh oh!

Conversation

Prashanth684 commented Feb 9, 2021

Uh oh!

openshift-ci-robot commented Feb 9, 2021

Uh oh!

Prashanth684 commented Feb 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Prashanth684 commented Feb 9, 2021

Uh oh!

openshift-ci-robot commented Feb 9, 2021

Uh oh!

Prashanth684 commented Feb 9, 2021

Uh oh!

openshift-ci-robot commented Feb 9, 2021

Uh oh!

Prashanth684 commented Feb 9, 2021

Uh oh!

andymcc commented Feb 9, 2021

Uh oh!

Prashanth684 commented Feb 9, 2021

Uh oh!

staebler commented Feb 10, 2021

Uh oh!

openshift-bot commented Feb 10, 2021

Uh oh!

openshift-bot commented Feb 10, 2021

Uh oh!

openshift-bot commented Feb 10, 2021

Uh oh!

openshift-bot commented Feb 10, 2021

Uh oh!

openshift-bot commented Feb 10, 2021

Uh oh!

openshift-bot commented Feb 10, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

staebler commented Feb 11, 2021

Uh oh!

openshift-ci-robot commented Feb 11, 2021

Uh oh!

miabbott commented Feb 11, 2021

Uh oh!

openshift-ci-robot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-bot commented Feb 11, 2021

Uh oh!

openshift-ci bot commented Feb 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Feb 11, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Prashanth684 commented Feb 9, 2021 •

edited

Loading

openshift-ci bot commented Feb 11, 2021 •

edited

Loading