Skip to content

USHIFT-2170: support upgrading 2 Y versions at a time#2952

Merged
openshift-merge-bot[bot] merged 9 commits intoopenshift:mainfrom
dhellmann:USHIFT-2081-eus-upgrade-spike
Mar 7, 2024
Merged

USHIFT-2170: support upgrading 2 Y versions at a time#2952
openshift-merge-bot[bot] merged 9 commits intoopenshift:mainfrom
dhellmann:USHIFT-2081-eus-upgrade-spike

Conversation

@dhellmann
Copy link
Contributor

/assign @pmtk

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jan 30, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jan 30, 2024

@dhellmann: This pull request references USHIFT-2170 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the spike to target the "4.16.0" version, but no target version was set.

Details

In response to this:

/assign @pmtk

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 30, 2024
@openshift-ci openshift-ci bot requested review from copejon and ggiguash January 30, 2024 19:31
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 30, 2024
@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch 2 times, most recently from 4171edc to e4c9cb8 Compare January 31, 2024 17:24
@dhellmann dhellmann changed the title WIP: USHIFT-2170: support upgrading 2 Y versions at a time USHIFT-2170: support upgrading 2 Y versions at a time Jan 31, 2024
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 31, 2024
@dhellmann
Copy link
Contributor Author

This should be rebased on #2950 before it is merged.

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 31, 2024
@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch from e4c9cb8 to 9117d79 Compare February 1, 2024 17:13
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 1, 2024
@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch from 9117d79 to 55d4aa0 Compare February 1, 2024 18:50
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 1, 2024
@dhellmann
Copy link
Contributor Author

repo: downloading from remote: rhocp-4.15-for-rhel-9-x86_64-rpms
error: Status code: 403 for https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/rhocp/4.15/os/repodata/repomd.xml (IP: 23.198.106.83) (https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/rhocp/4.15/os/repodata/repomd.xml).
error: Status code: 403 for https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/rhocp/4.15/os/repodata/repomd.xml (IP: 23.198.106.83) (https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/rhocp/4.15/os/repodata/repomd.xml).
error: Status code: 403 for https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/rhocp/4.15/os/repodata/repomd.xml (IP: 23.198.106.83) (https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/rhocp/4.15/os/repodata/repomd.xml).
error: Status code: 403 for https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/rhocp/4.15/os/repodata/repomd.xml (IP: 23.198.106.83) (https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/rhocp/4.15/os/repodata/repomd.xml).
Red Hat OpenShift Container Platform 4.15 for R 572  B/s | 378  B     00:00    
Errors during downloading metadata for repository 'rhocp-4.15-for-rhel-9-x86_64-rpms':
  - Status code: 403 for https://cdn.redhat.com/content/dist/layered/rhel9/x86_64/rhocp/4.15/os/repodata/repomd.xml (IP: 23.198.106.83)
Error: Failed to download metadata for repo 'rhocp-4.15-for-rhel-9-x86_64-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch from 55d4aa0 to 5aa8273 Compare February 1, 2024 21:09
@dhellmann
Copy link
Contributor Author

./bin/create_local_repo.sh: line 41: YPLUS2_REPO: unbound variable

@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch from 5aa8273 to 08c0abd Compare February 2, 2024 14:58
@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch 2 times, most recently from 2d50b02 to a9caa1f Compare February 7, 2024 16:16
@dhellmann
Copy link
Contributor Author

/hold cancel

rebased after #2950 merged

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 7, 2024
@dhellmann
Copy link
Contributor Author

The issue is a package dependency problem between selinux-policy-targeted and osbuild. https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_microshift/2952/pull-ci-openshift-microshift-main-microshift-metal-tests/1755264697868750848

Let's see if that's being caused by a RHEL CDN or cache problem.

/retest-required

@dhellmann
Copy link
Contributor Author

Same dependency error

19:51:31.930541286 microshift/scripts/image-builder/../dnf_retry.sh:26 	sudo dnf install -y osbuild osbuild-composer git composer-cli ostree rpm-ostree cockpit-composer bash-completion podman runc genisoimage createrepo yum-utils selinux-policy-devel jq wget lorax rpm-build containernetworking-plugins expect
Updating Subscription Management repositories.
Red Hat Enterprise Linux 9 for x86_64 - AppStre  46 MB/s |  29 MB     00:00    
Red Hat Enterprise Linux 9 for x86_64 - BaseOS   28 MB/s |  17 MB     00:00    
Fast Datapath for RHEL 9 x86_64 (RPMs)          408 kB/s | 195 kB     00:00    
Red Hat OpenShift Container Platform 4.14 for R  31 MB/s |  17 MB     00:00    
Red Hat Enterprise Linux 9 for x86_64 - AppStre 100 MB/s |  29 MB     00:00    
Red Hat Enterprise Linux 9 for x86_64 - BaseOS   76 MB/s |  17 MB     00:00    
Red Hat Enterprise Linux 9 Client Configuration  30 kB/s | 2.2 kB     00:00    
Package git-2.39.3-1.el9_2.x86_64 is already installed.
Package bash-completion-1:2.11-4.el9.noarch is already installed.
Package yum-utils-4.3.0-11.el9_3.noarch is already installed.
Package selinux-policy-devel-38.1.23-1.el9_3.1.noarch is already installed.
Package jq-1.6-15.el9.x86_64 is already installed.
Package rpm-build-4.16.1.3-27.el9_3.x86_64 is already installed.
Error: 
 Problem: problem with installed package selinux-policy-targeted-38.1.23-1.el9_3.1.noarch
  - package osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms requires (osbuild-selinux if selinux-policy-targeted), but none of the providers can be installed
  - package osbuild-selinux-53-1.el9_0.noarch from rhel-9-for-x86_64-appstream-rpms requires osbuild = 53-1.el9_0, but none of the providers can be installed
  - package osbuild-selinux-53-1.el9_0.noarch from rhel-9-appstream-rhui-rpms requires osbuild = 53-1.el9_0, but none of the providers can be installed
  - package osbuild-selinux-53.1-1.el9_0.noarch from rhel-9-for-x86_64-appstream-rpms requires osbuild = 53.1-1.el9_0, but none of the providers can be installed
  - package osbuild-selinux-53.1-1.el9_0.noarch from rhel-9-appstream-rhui-rpms requires osbuild = 53.1-1.el9_0, but none of the providers can be installed
  - package osbuild-selinux-65-1.el9.noarch from rhel-9-for-x86_64-appstream-rpms requires osbuild = 65-1.el9, but none of the providers can be installed
  - package osbuild-selinux-65-1.el9.noarch from rhel-9-appstream-rhui-rpms requires osbuild = 65-1.el9, but none of the providers can be installed
  - package osbuild-selinux-81-1.el9.noarch from rhel-9-for-x86_64-appstream-rpms requires osbuild = 81-1.el9, but none of the providers can be installed
  - package osbuild-selinux-81-1.el9.noarch from rhel-9-appstream-rhui-rpms requires osbuild = 81-1.el9, but none of the providers can be installed
  - package osbuild-selinux-81-1.el9_2.1.noarch from rhel-9-for-x86_64-appstream-rpms requires osbuild = 81-1.el9_2.1, but none of the providers can be installed
  - package osbuild-selinux-81-1.el9_2.1.noarch from rhel-9-appstream-rhui-rpms requires osbuild = 81-1.el9_2.1, but none of the providers can be installed
  - package osbuild-selinux-93-1.el9.noarch from rhel-9-for-x86_64-appstream-rpms requires osbuild = 93-1.el9, but none of the providers can be installed
  - package osbuild-selinux-93-1.el9.noarch from rhel-9-appstream-rhui-rpms requires osbuild = 93-1.el9, but none of the providers can be installed
  - cannot install both osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms and osbuild-53-1.el9_0.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms and osbuild-53.1-1.el9_0.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms and osbuild-65-1.el9.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms and osbuild-81-1.el9.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms and osbuild-81-1.el9_2.1.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms and osbuild-93-1.el9.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-53-1.el9_0.noarch from rhel-9-appstream-rhui-rpms and osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-53.1-1.el9_0.noarch from rhel-9-appstream-rhui-rpms and osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-65-1.el9.noarch from rhel-9-appstream-rhui-rpms and osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-81-1.el9.noarch from rhel-9-appstream-rhui-rpms and osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-81-1.el9_2.1.noarch from rhel-9-appstream-rhui-rpms and osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install both osbuild-93-1.el9.noarch from rhel-9-appstream-rhui-rpms and osbuild-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms
  - cannot install the best candidate for the job
  - nothing provides selinux-policy >= 38.1.23-1.el9_3.2 needed by osbuild-selinux-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms
  - nothing provides selinux-policy-base >= 38.1.23-1.el9_3.2 needed by osbuild-selinux-93-1.el9_3.1.noarch from rhel-9-for-x86_64-appstream-rpms
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
+ 19:51:55.850737209 microshift/scripts/image-builder/../dnf_retry.sh:28 	rc=1
+ 19:51:55.852799889 microshift/scripts/image-builder/../dnf_retry.sh:30 	sudo dnf clean -y all
Updating Subscription Management repositories.
58 files removed
+ 19:51:57.299222175 microshift/scripts/image-builder/../dnf_retry.sh:33 	exit 1

@dhellmann
Copy link
Contributor Author

More background in openshift/enhancements#1562

@dhellmann
Copy link
Contributor Author

/retest-required

@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch from 2d0bf81 to d0b62f6 Compare March 4, 2024 22:21
@dhellmann
Copy link
Contributor Author

Building edge-commit from rhel-9.2-microshift-4.14 --parent rhel-9.2 --url http://10.192.10.207:8080/repo
+ 22:48:19.125119040 ./bin/build_images.sh:340 	build_cmd='sudo composer-cli compose start-ostree --parent rhel-9.2 --url http://10.192.10.207:8080/repo --ref rhel-9.2-microshift-4.14 rhel-9.2-microshift-4.14 edge-commit'
++ 22:48:19.126649880 ./bin/build_images.sh:341 	seq 3
+ 22:48:19.128846588 ./bin/build_images.sh:341 	for _ in $(seq 3)
+ 22:48:19.130049451 ./bin/build_images.sh:342 	set +e
++ 22:48:19.131572494 ./bin/build_images.sh:343 	sudo composer-cli compose start-ostree --parent rhel-9.2 --url http://10.192.10.207:8080/repo --ref rhel-9.2-microshift-4.14 rhel-9.2-microshift-4.14 edge-commit
ERROR: DepsolveError: DNF error occurred: MarkingErrors: Error occurred when marking packages for installation: Problems in request:
missing packages: microshift-4.14.0~rc.7-202310241737.p0.g1586504.assembly.rc.7.el9*, microshift-greenboot-4.14.0~rc.7-202310241737.p0.g1586504.assembly.rc.7.el9*, microshift-networking-4.14.0~rc.7-202310241737.p0.g1586504.assembly.rc.7.el9*, microshift-selinux-4.14.0~rc.7-202310241737.p0.g1586504.assembly.rc.7.el9*

The `dnf repoquery` command does not require a repository to be
enabled in order to check its contents. Since we use the command to
look for old versions of MicroShift, in repositories that might not be
enabled, we should not require the repo to be enabled before
performing the query.
@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch from a642ca7 to f16d284 Compare March 6, 2024 16:35
newest=$(sudo dnf repoquery microshift --quiet --queryformat '%{version}-%{release}' --repo "${rhsm_repo}" | sort --version-sort | tail -n1)
if [ -n "${newest}" ]; then
echo "${newest}"
return
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pmtk I think this is the change you've mentioned that we need, please take a look when you have a few minutes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so.
BTW. command can be changed to sudo dnf repoquery --quiet --latest-limit 1 --qf "%{version}-%{release}" --repo "${rhsm_repo}" microshift - --latest-limit instead of | sort | tail

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that function were to return "version,repo", then YMINUS2_RELEASE_REPO="$(get_ocp_repo_name_for_version ${YMINUS2_MINOR_VERSION})" could be cut -d, -f2 but I'm fine with either

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about that. I liked the simplicity of that repo being explicitly known to exist in the caller, but maybe the symmetry of always having some repo reference returned is better? I'll defer to you, let me know if I should change it.

@dhellmann
Copy link
Contributor Author

/retest-required

@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch from e7140db to 65552c9 Compare March 6, 2024 21:27
@dhellmann
Copy link
Contributor Author

/hold cancel

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 6, 2024
dhellmann and others added 2 commits March 7, 2024 11:20
…active

make container extraction more flexible about handling repositories by
handling repo names, paths, and URLs
Co-authored-by: Gregory Giguashvili <85498834+ggiguash@users.noreply.github.com>
@dhellmann dhellmann force-pushed the USHIFT-2081-eus-upgrade-spike branch from b66273e to 9d43c9d Compare March 7, 2024 16:20
@@ -0,0 +1,10 @@
{{- if env.Getenv "RHOCP_MINOR_Y2" "" -}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comment said RHOCP for y-2 will always be available - do we need if here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I can remove that.

@pmtk
Copy link
Member

pmtk commented Mar 7, 2024

/lgtm

@pmtk
Copy link
Member

pmtk commented Mar 7, 2024

/hold

@openshift-ci openshift-ci bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm Indicates that a PR is ready to be merged. labels Mar 7, 2024
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 7, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dhellmann, ggiguash, pmtk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [dhellmann,ggiguash,pmtk]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@dhellmann
Copy link
Contributor Author

/hold cancel

I'll take care of the suggested cleanup in a follow-up PR.

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 7, 2024
@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 0bdc4cc and 2 for PR HEAD 9d43c9d in total

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 7, 2024

@dhellmann: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit 80c3847 into openshift:main Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants