OCPBUGS-33248: daemon: upgrade os image from local container storage if it exists

[hexfusion]

This PR add the ability to upgrade an os image if it has already been pulled into local container storage. This functionality is behind PinnedImage feature gate. The advantage to this path is that before this change skopeo by default will make a call to the registry to pull the image during upgrade. That does not work for disconnected clusters.

[openshift-ci-robot]

@hexfusion: This pull request references Jira Issue OCPBUGS-33248, which is invalid:

• expected the bug to target the "4.16.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

This PR add the ability to upgrade an os image if it has already been pulled into local container storage. This functionality is gated behind PinnedImage gates.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@hexfusion: This pull request references Jira Issue OCPBUGS-33248, which is invalid:

• expected the bug to target the "4.16.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[hexfusion]

/jira refresh

[openshift-ci-robot]

@hexfusion: This pull request references Jira Issue OCPBUGS-33248, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.16.0) matches configured target version for branch (4.16.0)
• bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @siserafin

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@hexfusion: This pull request references Jira Issue OCPBUGS-33248, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.16.0) matches configured target version for branch (4.16.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @siserafin

Details

In response to this:

This PR add the ability to upgrade an os image if it has already been pulled into local container storage. This functionality is gated behind PinnedImage gates. The advantage to this path is that prior to this change skopeo by default will make a call to the registry to pull the image during upgrade. That does not work for disconnected clusters.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[siserafin]

Verified in a disconnected cluster.

1. Mirror an os image
2. Pull the image from the node
3. Add the firewall rule to block the pulling from registry
   sudo iptables -A OUTPUT -p tcp --dport 443 -j DROP
4. Trigger the os upgrade

rpm-ostree --experimental rebase ostree-unverified-image:containers-storage:registry.build03.ci.openshift.org/ci-ln-gv2713b/stable@sha256:fda7f7db7bed8d6691acc2dd9870d492995b6bae6354b17edb3537f2637e113a
Pulling manifest: ostree-unverified-image:containers-storage:registry.build03.ci.openshift.org/ci-ln-gv2713b/stable@sha256:fda7f7db7bed8d6691acc2dd9870d492995b6bae6354b17edb3537f2637e113a
Importing: ostree-unverified-image:containers-storage:registry.build03.ci.openshift.org/ci-ln-gv2713b/stable@sha256:fda7f7db7bed8d6691acc2dd9870d492995b6bae6354b17edb3537f2637e113a (digest: sha256:fda7f7db7bed8d6691acc2dd9870d492995b6bae6354b17edb3537f2637e113a)
ostree chunk layers already present: 36
ostree chunk layers needed: 15 (284.5?MB)
⠁ Staging deployment...                                                                                                                                                                                                                                                                                                                                                                               Staging deployment... done
Downgraded:
  container-selinux 3:2.231.0-1.rhaos4.16.el9 -> 3:2.229.0-1.rhaos4.16.el9
  cri-o 1.29.4-6.rhaos4.16.git0e93ae2.el9 -> 1.29.4-5.rhaos4.16.gitec19fc9.el9
  openshift-kubelet 4.16.0-202405062116.p0.g681e46b.assembly.stream.el9 -> 4.16.0-202404251943.p0.gd1ec84a.assembly.stream.el9
Changes queued for next boot. Run "systemctl reboot" to start a reboot

/label qe-approved

[rioliu-rh]

/assign @sinnykumari

[sinnykumari]

one minor nit in case we want to address before merge, otherwise lgtm.

[hexfusion]

/retest-required

[hexfusion]

/retest-required

[sinnykumari]

/lgtm
/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hexfusion, sinnykumari

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [sinnykumari]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hexfusion]

/cherry-pick 4.16

[openshift-cherrypick-robot]

@hexfusion: once the present PR merges, I will cherry-pick it on top of 4.16 in a new PR and assign it to you.

Details

In response to this:

/cherry-pick 4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-cherrypick-robot]

@hexfusion: once the present PR merges, I will cherry-pick it on top of release-4.16 in a new PR and assign it to you.

Details

In response to this:

/cherrypick release-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[hexfusion]

/retest-required

[openshift-ci]

@hexfusion: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-azure-ovn-upgrade-out-of-change	9fcd4b0	link	false	/test e2e-azure-ovn-upgrade-out-of-change
ci/prow/e2e-aws-ovn-upgrade-out-of-change	9fcd4b0	link	false	/test e2e-aws-ovn-upgrade-out-of-change

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[sinnykumari]

/test e2e-gcp-op

[openshift-ci-robot]

@hexfusion: Jira Issue OCPBUGS-33248: All pull requests linked via external trackers have merged:

• openshift/machine-config-operator#4347

Jira Issue OCPBUGS-33248 has been moved to the MODIFIED state.

Details

In response to this:

This PR add the ability to upgrade an os image if it has already been pulled into local container storage. This functionality is behind PinnedImage feature gate. The advantage to this path is that before this change skopeo by default will make a call to the registry to pull the image during upgrade. That does not work for disconnected clusters.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-cherrypick-robot]

@hexfusion: cannot checkout 4.16: error checking out "4.16": exit status 1 error: pathspec '4.16' did not match any file(s) known to git

Details

In response to this:

/cherry-pick 4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[hexfusion]

/cherry-pick release-4.16

[openshift-cherrypick-robot]

@hexfusion: new pull request created: #4392

Details

In response to this:

/cherry-pick release-4.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-bot]

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-machine-config-operator-container-v4.17.0-202406052200.p0.g743162d.assembly.stream.el9 for distgit ose-machine-config-operator.
All builds following this will include this PR.

[sinnykumari]

Seeing if automation will kicks in for bug auto move
/jira refresh

[openshift-ci-robot]

@sinnykumari: Jira Issue OCPBUGS-33248: All pull requests linked via external trackers have merged:

• openshift/machine-config-operator#4347

Jira Issue OCPBUGS-33248 has been moved to the MODIFIED state.

Details

In response to this:

Seeing if automation will kicks in for bug auto move
/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.