Revert "OCPNODE-3611: manifests.rhel/0000_90_openshift-cluster-image-policy: Drop TechPreviewNoUpgrade limitation"#83
Conversation
…policy: Drop TechPreviewNoUpgrade limitation"
|
@wking: This pull request references OCPNODE-3611 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: stbenjam, wking The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
This is HEAD of the repo, we can safely force it in /override ci/prow/e2e-aws |
|
@stbenjam: Overrode contexts on behalf of stbenjam: ci/prow/e2e-aws, ci/prow/e2e-aws-upgrade, ci/prow/okd-scos-e2e-aws-ovn DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/label approved |
|
/label acknowledge-critical-fixes-only |
|
@wking: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
…-openshift-cip"" This reverts commit 7a5dcee. This one has taken us some time: * 2025-08-27, 94f7582, openshift#82 was our first attempt at enabling the ClusterImagePolicy. * ...but it tripped up the origin test suite, so it was reverted in 2025-08-28, c40e7b9, openshift#83. * Qi then hardened the test suite with openshift/origin@d3af51e4acb (not fail upgrade checks if all nodes are ready, 2025-09-29, openshift/origin#30318) and openshift/origin@2fd0d8e242 (Upgrade test add 2min grace period allow non-drain updates to complete, 2025-11-12, openshift/origin#30480). * With the tougher CI in place, we tried a second time with 2025-11-17, 1f89a67, openshift#85. * ...but still tripped up origin, with runs like [1] taking 2.25m (more than the 2m grace period): I1119 17:26:21.890667 1511 upgrade.go:629] Waiting on pools to be upgraded I1119 17:26:21.939178 1511 upgrade.go:792] Pool master is still reporting (Updated: false, Updating: true, Degraded: false) I1119 17:26:21.939259 1511 upgrade.go:666] Invariant violation detected: master pool requires update but nodes not ready. Waiting up to 2m0s for non-draining updates to complete I1119 17:26:31.984116 1511 upgrade.go:792] Pool master is still reporting (Updated: false, Updating: true, Degraded: false) ... I1119 17:28:21.981438 1511 upgrade.go:792] Pool master is still reporting (Updated: false, Updating: true, Degraded: false) I1119 17:28:21.981514 1511 upgrade.go:673] Invariant violation detected: the "master" pool should be updated before the CVO reports available at the new version and: $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/logs/periodic-ci-openshift-release-master-ci-4.21-upgrade-from-stable-4.20-e2e-gcp-ovn-rt-upgrade/1991158541779472384/artifacts/e2e-gcp-ovn-rt-upgrade/gather-extra/artifacts/inspect/cluster-scoped-resources/machineconfiguration.openshift.io/machineconfigpools/master.yaml | yaml2json | jq -r '.status.conditions[] | select(.type == "Updating") | .lastTransitionTime + " " + .status' 2025-11-19T17:28:36Z False 28:36 - 26:21 = 135s = 2.25m, which overshot the 2m grace period. The second attempt was reverted in 7a5dcee, openshift#87. * Qi then hardened the test suite further with openshift/origin@c17e560263 (Update grace period for cluster upgrade to 10 minutes, 2025-11-19, #openshift/origin#30506). * This commit is taking a third attempt at enabling the ClusterImagePolicy. [1]: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-ci-4.21-upgrade-from-stable-4.20-e2e-gcp-ovn-rt-upgrade/1991158541779472384
Reverts #82
This was tripping up CI by causing a new MachineConfigPool rollout late in the update. For example, this run, had:
0:55:22 UTC,
rendered-master-5d3ab69b0a03bc70fa37249f040c06b4MachineConfig created to push out the version bump.1:19:15 (and 1:19:20?) UTC,
OperatorVersionChangedEvents claimed themachine-configClusterOperator finished its transition.1:19:47 UTC, ClusterVersion claimed the update completed.
1:19:49 UTC,
rendered-master-975808f8abad8954df577f74c6809db3MachineConfig created.CI fails with:
Confirming that the difference between those MachineConfig is the
openshiftClusterImagePolicy:$ diff -U3 <(curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/logs/periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-upgrade-fips/1960838341934977024/artifacts/e2e-aws-ovn-upgrade-fips/gather-extra/artifacts/inspect/cluster-scoped-resources/machineconfiguration.openshift.io/machineconfigs/rendered-master-5d3ab69b0a03bc70fa37249f040c06b4.yaml | yaml2json | jq -r '.spec.config.storage.files[] | select(.path == "/etc/containers/policy.json").contents.source | split(",")[-1]' | python3 -c 'import urllib.parse, sys; print(urllib.parse.unquote(sys.stdin.read()))') <(curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/logs/periodic-ci-openshift-release-master-nightly-4.20-e2e-aws-ovn-upgrade-fips/1960838341934977024/artifacts/e2e-aws-ovn-upgrade-fips/gather-extra/artifacts/inspect/cluster-scoped-resources/machineconfiguration.openshift.io/machineconfigs/rendered-master-975808f8abad8954df577f74c6809db3.yaml | yaml2json | jq -r '.spec.config.storage.files[] | select(.path == "/etc/containers/policy.json").contents.source | split(",")[-1] | @base64d') --- /dev/fd/63 2025-08-28 00:44:33.542232745 -0700 +++ /dev/fd/62 2025-08-28 00:44:33.547232745 -0700 @@ -1,15 +1,38 @@ { - "default": [ + "default": [ + { + "type": "insecureAcceptAnything" + } + ], + "transports": { + "atomic": { + "quay.io/openshift-release-dev/ocp-release": [ { ...