Skip to content

Conversation

@wking
Copy link
Member

@wking wking commented Sep 29, 2021

Avoid:

alert ClusterNotUpgradeable fired for 1115 seconds with labels: {condition="Upgradeable", endpoint="metrics", name="version", severity="warning"}

for tests that install 4.8 and then sit on it for over an hour, now that openshift/cluster-version-operator#647 has landed in 4.8. openshift/origin#26446 will follow up with code to handle CI that installs an earlier release and updates into 4.8 or later.

…e-1.22-api-removals-in-4.9

Avoid [1]:

  alert ClusterNotUpgradeable fired for 1115 seconds with labels: {condition="Upgradeable", endpoint="metrics", name="version", severity="warning"}

for tests that install 4.8 and then sit on it for over an hour, now
that [2] has landed in 4.8.  [3] will follow up with code to handle CI
that installs an earlier release and updates into 4.8 or later.

[1]: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.8-e2e-aws-serial/1443018123379740672
[2]: openshift/cluster-version-operator#647
[3]: openshift/origin#26446
@openshift-ci openshift-ci bot requested review from csrwng and staebler September 29, 2021 20:54
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 29, 2021
@wking
Copy link
Member Author

wking commented Sep 29, 2021

Two jobs exercising both sides of this code: a 4.7 install and a 4.9 install. We'll see what they have to say about the patch and gathered ack-state when they complete. Until then:

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 29, 2021
@jottofar
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 29, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 29, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jottofar, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@wking
Copy link
Member Author

wking commented Sep 29, 2021

4.7 job is happy, and:

$ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_release/22368/rehearse-22368-pull-ci-operator-framework-operator-lifecycle-manager-release-4.7-e2e-gcp/1443318799435567104/artifacts/e2e-gcp/openshift-e2e-test/build-log.txt | grep -A1 admin-acks
Error from server (NotFound): configmaps "admin-acks" not found
failed to ack the 4.9 Kube v1beta1 removals; possibly API-server issue, or a pre-4.8 release image

is what we'd expect for 4.7.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 29, 2021

@wking: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/rehearse/openshift/cloud-credential-operator/release-4.9/e2e-aws-manual-oidc 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/cluster-network-operator/release-4.9/e2e-azure-ovn-dualstack 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/vmware-vsphere-csi-driver/release-4.9/e2e-vsphere 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/vmware-vsphere-csi-driver/release-4.9/e2e-vsphere-csi 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/azure-disk-csi-driver-operator/release-4.10/e2e-azurestack-csi 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/installer/release-4.10/e2e-azurestack 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/installer/release-4.10/e2e-azurestack-upi 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/installer/release-4.9/e2e-gcp-upi-xpn 656eda9 link unknown /test pj-rehearse
ci/rehearse/redhat-developer/jenkins-operator/main/e2e 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/installer/release-4.9/e2e-gcp-upi 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/cluster-logging-operator/tech-preview/e2e-operator 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/aws-efs-csi-driver-operator/release-4.9/operator-e2e 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/installer/release-4.9/e2e-gcp-shared-vpc 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/installer/release-4.9/e2e-kubevirt 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/sdn/release-4.9/e2e-aws-multitenant 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/openstack-cinder-csi-driver-operator/release-4.9/e2e-openstack-csi 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/vsphere-problem-detector/release-4.9/e2e-vsphere 656eda9 link unknown /test pj-rehearse
ci/rehearse/openshift/kubernetes/release-4.9/e2e-openstack-csi-manila 656eda9 link unknown /test pj-rehearse

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@wking
Copy link
Member Author

wking commented Sep 29, 2021

4.9 job failed on KubePersistentVolumeErrors firing, which is unrelated. It has the successful patch:

$ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_release/22368/rehearse-22368-pull-ci-openshift-router-release-4.9-e2e-agnostic/1443318801436250112/artifacts/e2e-agnostic/openshift-e2e-test/build-log.txt | grep admin-acks
configmap/admin-acks patched

Ack is in place:

$ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_release/22368/rehearse-22368-pull-ci-openshift-router-release-4.9-e2e-agnostic/1443318801436250112/artifacts/e2e-agnostic/gather-extra/artifacts/configmaps.json | jq '.items[] | select(.metadata.name == "admin-acks").data'
{
  "ack-4.8-kube-1.22-api-removals-in-4.9": "true"
}

And ClusterVersion conditions are expected (notably no Upgradeable=False):

$ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_release/22368/rehearse-22368-pull-ci-openshift-router-release-4.9-e2e-agnostic/1443318801436250112/artifacts/e2e-agnostic/gather-extra/artifacts/clusterversion.json | jq -r '.items[].status.conditions[] | .lastTransitionTime + " " + .ty
pe + "=" + .status + " " + (.reason // "-") + ": " + (.message // "-")'
2021-09-29T21:24:05Z RetrievedUpdates=False NoChannel: The update channel has not been configured.
2021-09-29T21:48:58Z Available=True -: Done applying 4.9.0-0.ci.test-2021-09-29-210457-ci-op-24bshkpi-latest
2021-09-29T21:48:58Z Failing=False -: -
2021-09-29T21:48:58Z Progressing=False -: Cluster version is 4.9.0-0.ci.test-2021-09-29-210457-ci-op-24bshkpi-latest

/hold cancel

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 29, 2021
@openshift-merge-robot openshift-merge-robot merged commit 7a34ad6 into openshift:master Sep 29, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 29, 2021

@wking: Updated the step-registry configmap in namespace ci at cluster app.ci using the following files:

  • key openshift-e2e-test-commands.sh using file ci-operator/step-registry/openshift/e2e/test/openshift-e2e-test-commands.sh
Details

In response to this:

Avoid:

alert ClusterNotUpgradeable fired for 1115 seconds with labels: {condition="Upgradeable", endpoint="metrics", name="version", severity="warning"}

for tests that install 4.8 and then sit on it for over an hour, now that openshift/cluster-version-operator#647 has landed in 4.8. openshift/origin#26446 will follow up with code to handle CI that installs an earlier release and updates into 4.8 or later.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants