Add option to mirror openshift images #850

honza · 2019-11-18T21:48:19Z

We add a new MIRROR_IMAGES option. When non-empty, it will use oc adm to mirror openshift images to our local registry. It will also
modify install-config.yaml to add the new mirror.

04_setup_ironic.sh

utils.sh

metal3ci · 2019-11-18T23:51:10Z

Build FAILURE, see build http://10.8.144.11:8080/job/dev-tools/1308/

04_setup_ironic.sh

honza · 2019-11-19T18:25:03Z

I just had a successful deployment using the latest version of this patch.

metal3ci · 2019-11-19T20:56:47Z

Build SUCCESS, see build http://10.8.144.11:8080/job/dev-tools/1313/

04_setup_ironic.sh

common.sh

hardys · 2019-11-20T13:54:24Z

This isn't working for me:

level=fatal msg="failed to fetch Master Machines: failed to load asset \"Install Config\": invalid \"install-config.yaml\" file: imageContentSources[0].source: Invalid value: \"registry.svc.ci.openshift.org/ocp/release:4.3.0-0.ci-2019-11-20-103944\": must be repository--not reference"
make: *** [Makefile:21: ocp_run] Error 1

real	6m54.557s
user	2m14.596s
sys	0m43.079s
provisioning-host> tail ocp/install-config.yaml
- mirrors:
    - 192.168.111.1:5000/localimages/local-release-image
  source: registry.svc.ci.openshift.org/ocp/release:4.3.0-0.ci-2019-11-20-103944
- mirrors:
    - 192.168.111.1:5000/localimages/local-release-image
  source: registry.svc.ci.openshift.org/ocp/4.3.0-0.ci-2019-11-20-103944

Maybe push a WIP commit that flips the default to true for MIRROR_IMAGES to true so we can see this work in CI when that's fixed?

metal3ci · 2019-11-20T15:06:54Z

Build FAILURE, see build http://10.8.144.11:8080/job/dev-tools/1319/

metal3ci · 2019-11-20T15:27:29Z

Build FAILURE, see build http://10.8.144.11:8080/job/dev-tools/1320/

metal3ci · 2019-11-20T15:39:17Z

Build FAILURE, see build http://10.8.144.11:8080/job/dev-tools/1321/

honza · 2019-11-20T15:39:20Z

Temporarily rebasing on top of #854 because the installer patch doesn't seem to apply correctly in CI

metal3ci · 2019-11-20T17:27:05Z

Build SUCCESS, see build http://10.8.144.11:8080/job/dev-tools/1322/

We add a new `MIRROR_IMAGES` option. When non-empty, it will use `oc adm` to mirror openshift images to our local registry. It will also modify `install-config.yaml` to add the new mirror.

metal3ci · 2019-11-21T14:30:32Z

Build SUCCESS, see build http://10.8.144.11:8080/job/dev-tools/1324/

honza · 2019-11-21T15:38:42Z

WIP commit removed, ready to merge

metal3ci · 2019-11-21T17:34:01Z

Build FAILURE, see build http://10.8.144.11:8080/job/dev-tools/1326/

metal3ci · 2019-11-25T10:36:38Z

Build SUCCESS, see build http://10.8.144.11:8080/job/dev-tools/1330/

hardys · 2019-11-25T11:36:37Z

CI passed but I'm not getting a working deploy with export MIRROR_IMAGES=true - looking into why, could be specific to my environment I guess @yprokule is this latest revision working for you with mirroring enabled?

hardys · 2019-11-25T13:52:24Z

Ok I now get a working deployment but AFAICS the images used aren't from the mirror - do we need #856 to make this work?

yprokule · 2019-11-25T14:01:27Z

CI passed but I'm not getting a working deploy with export MIRROR_IMAGES=true - looking into why, could be specific to my environment I guess @yprokule is this latest revision working for you with mirroring enabled?

Deployment passed but cvo doesn't report mirrored image being used

        "desired": {
            "force": false,
            "image": "registry.svc.ci.openshift.org/ocp/release@sha256:b1d66f12a1496be5e591674594c96985b516bdc01d697e557dc2ab2a5e3b0216",
            "version": "4.3.0-0.nightly-2019-11-25-022933"
        },
        "history": [
            {
                "completionTime": "2019-11-25T13:52:28Z",
                "image": "registry.svc.ci.openshift.org/ocp/release@sha256:b1d66f12a1496be5e591674594c96985b516bdc01d697e557dc2ab2a5e3b0216",
                "startedTime": "2019-11-25T13:31:54Z",
                "state": "Completed",
                "verified": false,
                "version": "4.3.0-0.nightly-2019-11-25-022933"
            }
        ],

yprokule · 2019-11-26T12:20:37Z

utils.sh

+        TAGGED=$(echo $OPENSHIFT_RELEASE_IMAGE | sed -e 's/release://')
+        RELEASE=$(echo $OPENSHIFT_RELEASE_IMAGE | grep -o 'registry.svc.ci.openshift.org[^":]\+')


This doesn't always work, that's what I got while using nightly

Success Update image: 192.168.123.141:5000/localimages/local-release-image:4.3.0-0.nightly-2019-11-25-022933 Mirror prefix: 192.168.123.141:5000/localimages/local-release-image To use the new mirrored repository to install, add the following section to the install-config.yaml: imageContentSources: - mirrors: - 192.168.123.141:5000/localimages/local-release-image source: quay.io/openshift-release-dev/ocp-v4.0-art-dev - mirrors: - 192.168.123.141:5000/localimages/local-release-image source: registry.svc.ci.openshift.org/ocp/release

Maybe if the user wants to mirror images, we should ask them for the exact sources? What do you think?

I can confirm that was the behavior I got as well using 4.3 nightly's on an environment where I was doing disconnected.

I don't like how the data structure is laid out at all... What we have here is a hodge podge of array and string objects. It will be mind bending to code against this.

How about ...

imageContentSources: - target: <url>:<port> user: <user ie. openshift-release-dev or localimages> release: <release ie. ocp-v4.0-art-dev or release> - target: <url>:<port> user: <user ie. openshift-release-dev or localimages> release: <release ie. ocp-v4.0-art-dev or release> - target: <url>:<port> user: <user ie. openshift-release-dev or localimages> release: <release ie. ocp-v4.0-art-dev or release>

This is an API defined by OpenShift, it's not something we can revisit.

@mazzystr I'm not sure what you mean. We're simply using openshift-installer-provided values. We don't define any data structures.

stbenjam · 2019-12-09T17:41:24Z

The work on this has moved to #856

stbenjam reviewed Nov 18, 2019

View reviewed changes

04_setup_ironic.sh Outdated Show resolved Hide resolved

honza force-pushed the offline-mirror branch from 1111a03 to 704039e Compare November 18, 2019 22:22

stbenjam added the CI check this PR with CI label Nov 18, 2019

stbenjam reviewed Nov 18, 2019

View reviewed changes

utils.sh Outdated Show resolved Hide resolved

yprokule reviewed Nov 19, 2019

View reviewed changes

04_setup_ironic.sh Outdated Show resolved Hide resolved

honza force-pushed the offline-mirror branch from 704039e to 868e1d9 Compare November 19, 2019 18:20

yprokule mentioned this pull request Nov 20, 2019

Make local registry address configurable #853

Merged

honza force-pushed the offline-mirror branch from 868e1d9 to feef57b Compare November 20, 2019 13:29

hardys reviewed Nov 20, 2019

View reviewed changes

04_setup_ironic.sh Show resolved Hide resolved

honza force-pushed the offline-mirror branch from feef57b to 6a1cd06 Compare November 20, 2019 13:42

hardys reviewed Nov 20, 2019

View reviewed changes

common.sh Outdated Show resolved Hide resolved

honza force-pushed the offline-mirror branch from 6a1cd06 to ba7de32 Compare November 20, 2019 13:51

honza force-pushed the offline-mirror branch from c2ac194 to 6b9e61f Compare November 20, 2019 15:09

honza force-pushed the offline-mirror branch from 6b9e61f to 06e08eb Compare November 20, 2019 15:37

honza force-pushed the offline-mirror branch from 06e08eb to af29404 Compare November 20, 2019 15:44

Add option to mirror openshift images

aa6e914

We add a new `MIRROR_IMAGES` option. When non-empty, it will use `oc adm` to mirror openshift images to our local registry. It will also modify `install-config.yaml` to add the new mirror.

honza force-pushed the offline-mirror branch from af29404 to 34d0710 Compare November 21, 2019 13:01

honza force-pushed the offline-mirror branch from 34d0710 to aa6e914 Compare November 21, 2019 15:38

yprokule reviewed Nov 26, 2019

View reviewed changes

stbenjam closed this Dec 9, 2019

		TAGGED=$(echo $OPENSHIFT_RELEASE_IMAGE \| sed -e 's/release://')
		RELEASE=$(echo $OPENSHIFT_RELEASE_IMAGE \| grep -o 'registry.svc.ci.openshift.org[^":]\+')

Add option to mirror openshift images #850

Add option to mirror openshift images #850

Uh oh!

Conversation

honza commented Nov 18, 2019

Uh oh!

Uh oh!

Uh oh!

metal3ci commented Nov 18, 2019

Uh oh!

Uh oh!

honza commented Nov 19, 2019

Uh oh!

metal3ci commented Nov 19, 2019

Uh oh!

Uh oh!

Uh oh!

hardys commented Nov 20, 2019

Uh oh!

metal3ci commented Nov 20, 2019

Uh oh!

metal3ci commented Nov 20, 2019

Uh oh!

metal3ci commented Nov 20, 2019

Uh oh!

honza commented Nov 20, 2019

Uh oh!

metal3ci commented Nov 20, 2019

Uh oh!

metal3ci commented Nov 21, 2019

Uh oh!

honza commented Nov 21, 2019

Uh oh!

metal3ci commented Nov 21, 2019

Uh oh!

metal3ci commented Nov 25, 2019

Uh oh!

hardys commented Nov 25, 2019

Uh oh!

hardys commented Nov 25, 2019

Uh oh!

yprokule commented Nov 25, 2019

Uh oh!

yprokule Nov 26, 2019

Choose a reason for hiding this comment

Uh oh!

honza Nov 26, 2019

Choose a reason for hiding this comment

Uh oh!

schmaustech Dec 7, 2019

Choose a reason for hiding this comment

Uh oh!

mazzystr Dec 9, 2019

Choose a reason for hiding this comment

Uh oh!

stbenjam Dec 9, 2019

Choose a reason for hiding this comment

Uh oh!

honza Dec 9, 2019

Choose a reason for hiding this comment

Uh oh!

stbenjam commented Dec 9, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants