Skip to content

OSDOCS-628 updating restricted network installation #16894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kalexand-rh wants to merge 3 commits into from
Closed

OSDOCS-628 updating restricted network installation #16894

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5de3002
OSDOCS-628 updating restricted network installation
kalexand-rh Sep 26, 2019
c7f88e4
updates per Trevor
kalexand-rh Nov 7, 2019
2e5060e
more updates per Trevor
kalexand-rh Nov 8, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions _topic_map.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,8 +181,8 @@ Topics:
File: updating-cluster-cli
- Name: Updating a cluster that includes RHEL compute machines
File: updating-cluster-rhel-compute
#- Name: Updating a disconnected cluster
# File: updating-disconnected-cluster
- Name: Updating a restricted network cluster
File: updating-restricted-network
# - Name: Troubleshooting an update
# File: updating-troubleshooting
---
Expand Down
4 changes: 2 additions & 2 deletions ...installing_restricted_networks/installing-restricted-networks-preparations.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ include::modules/cli-installing-cli.adoc[leveloffset=+2]

include::modules/installation-creating-mirror-registry.adoc[leveloffset=+1]

include::modules/installation-local-registry-pull-secret.adoc[leveloffset=+1]
//include::modules/installation-local-registry-pull-secret.adoc[leveloffset=+1]

//include::modules/installation-adding-registry-pull-secret.adoc[leveloffset=+1]
include::modules/installation-adding-registry-pull-secret.adoc[leveloffset=+1]

include::modules/installation-mirror-repository.adoc[leveloffset=+1]

Expand Down
37 changes: 21 additions & 16 deletions modules/installation-adding-registry-pull-secret.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,21 +17,30 @@ restricted network.

Complete the following steps on the bastion host:

. Download your `registry.redhat.io` pull secret from the
link:https://cloud.redhat.com/openshift/install/pull-secret[Pull Secret] page on the {cloud-redhat-com} site.
. Generate the pull secret for your registry:
+
----
$ podman login --authfile ~/pullsecret_config.json <local_registry_host_name>:<local_registry_host_port> <1>
----
<1> For `<local_registry_host_name>`, specify the registry domain name
for your mirror registry, such as `registry.example.com`. For
`<local_registry_host_port>`, specify the port that your mirror registry uses to
serve content.
+
Provide your credentials for the mirror registry at the prompts.

. Generate the base64-encoded user name and password or token for your mirror
registry:
. View the pull secret that you created and record the pull secret value:
+
----
$ echo -n '<user_name>:<password>' | base64 -w0 <1>
# cat ~/pullsecret_config.json

BGVtbYk3ZHAtqXs=
{ "auths": { "<local_registry_host_name>:<local_registry_host_port>": { "auth": "ZHVtbXk6ZHVtbXk=" } } }
----
<1> For `<user_name>` and `<password>`, specify the user name and password that
you configured for your registry.

. Make a copy of your pull secret in JSON format:
. Download your pull secret from the
link:https://cloud.redhat.com/openshift/install/pull-secret[Pull Secret] page on the {cloud-redhat-com} site.

. Make a copy of the {cloud-redhat-com} link:https://cloud.redhat.com/openshift/install/pull-secret[Pull Secret] that you downloaded in JSON format:
+
----
$ cat ./pull-secret.text | jq . > <path>/<pull-secret-file><1>
Expand Down Expand Up @@ -64,22 +73,18 @@ The contents of the file resemble the following example:
}
----

. Edit the new file and add a section that describes your registry to it:
. Edit the {cloud-redhat-com} link:https://cloud.redhat.com/openshift/install/pull-secret[Pull Secret] file and add a section that describes your registry to it:
+
----
"auths": {
"auths": { <1>
...
"<local_registry_host_name>:<local_registry_host_port>": { <1>
"auth": "<credentials>", <2>
"email": "you@example.com"
},
...
----
<1> For `bastion_host_name`, specify the registry domain name
that you specified in your certificate, and for `<local_registry_host_port>`,
specify the port that your mirror registry uses to serve content.
<2> For `<credentials>`, specify the base64-encoded user name and password for
the mirror registry that you generated.
<1> Paste the contents of the `pullsecret_config.json` file that you created.
+
The file resembles the following example:
+
Expand Down
144 changes: 137 additions & 7 deletions modules/installation-mirror-repository.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,27 +1,58 @@
// Module included in the following assemblies:
//
// * installing/installing_restricted_networks/installing-restricted-networks-preparations.adoc
// * updating/updating-restricted-network.adoc

ifeval::["{context}" == "installing-restricted-networks-preparations"]
:restricted:
:install:
endif::[]
ifeval::["{context}" == "updating-restricted-network"]
:restricted:
:update:
endif::[]

ifdef::install[]
[id="installation-mirror-repository_{context}"]
= Mirroring the {product-title} image repository

Mirror the {product-title} image repository to use during cluster installation
or upgrade.
Mirror the {product-title} image repository to use during cluster installation.

.Prerequisites

* You configured a mirror registry to use in your restricted network and
can access the certificate and credentials that you configured.
* You downloaded the pull secret from the
link:https://cloud.redhat.com/openshift/install/pull-secret[Pull Secret] page on the {cloud-redhat-com} site and modified it to include authentication to your mirror repository.
endif::install[]

ifdef::update[]
[id="update-mirror-image-repository_{context}"]
= Update the contents of the {product-title} image repository

Update the contents of the image repository that hosts the mirrored content that
you require for installing {product-title}. You must update the mirror registry
to update {product-title} to a new version.

.Prerequisites

* You have access to the mirror registry that you used to store the images that
you used to install {product-title}.
endif::update[]

.Procedure

Complete the following steps on the bastion host:

. Review the
link:https://access.redhat.com/downloads/content/290/[{product-title} downloads page]
to determine the version of {product-title} that you want to install.
to determine the version of {product-title} that you want to
ifdef::install[]
install.
endif::install[]
ifdef::update[]
update to.
endif::update[]

. Set the required environment variables:
+
Expand All @@ -34,7 +65,11 @@ $ export LOCAL_SECRET_JSON='<path_to_pull_secret>' <5>
$ export RELEASE_NAME="ocp-release" <6>
----
<1> For `<release_version>`, specify the version number of {product-title} to
install, such as `4.2.0`.
install, such as `4.2.1`.
ifdef::update[]
When you update {product-title}, you must specify a version number that is
higher than the version that is installed.
endif::update[]
<2> For `<local_registry_host_name>`, specify the registry domain name for your mirror
repository, and for `<local_registry_host_port>`, specify the port that it
serves content on.
Expand All @@ -57,11 +92,105 @@ $ oc adm -a ${LOCAL_SECRET_JSON} release mirror \
----
+
This command pulls the release information as a digest, and its output includes
the `imageContentSources` data that you require when you install your cluster.
text that resembles the following sample:

. Record the
ifdef::install[]
`imageContentSources`
endif::install[]
ifdef::update[]
`ImageContentSourcePolicy`
endif::update[]
section from the output of the previous
command. This information is required
ifdef::install[]
during {product-title} installation.
endif::install[]
ifdef::update[]
when you update your {product-title} cluster.
endif::update[]

. Build the signature for the content that you mirrored and verify that it is
signed by an official Red Hat key:
.. Set the digest for the version to
ifdef::install[]
install:
endif::install[]
ifdef::update[]
update to:
endif::update[]
+
----
$ DIGEST="$(oc adm release info quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE} | sed -n 's/Pull From: .*@//p')" <1>
----
<1> For `<release_version>`, specify the version number of {product-title} to
that you mirrored content for.

.. Build a signature URI:
+
----
$ URI="https://mirror.openshift.com/pub/openshift-v4/signatures/openshift/release/${DIGEST/:/=}/signature-<n>" <1>
----
<1> Specify the name of the signature to store. The signature name must be in the format of `signature-<n>`, where `<n>` is a sequential series of numbers that starts with `1`. The signature verification process checks all valid signatures by incrementing `<n>` until it finds a valid signature or the requested `signature-<n>` does not exist.

. Record the entire `imageContentSources` section from the output of the previous
command. The information about your mirrors is unique to your mirrored repository, and you must add the `imageContentSources` section to the `install-config.yaml` file during installation.
.. Download the signature:
+
----
$ wget "${URI}"
----

.. Download the keys that were used to sign the signature:
+
----
$ wget https://www.redhat.com/security/data/f21541eb.txt
$ wget https://www.redhat.com/security/data/fd431d51.txt
----

.. Import the keys:
+
----
$ gpg --no-default-keyring --keyring ./temp.keyring --import ./f21541eb.txt
$ gpg --no-default-keyring --keyring ./temp.keyring --import ./fd431d51.txt
----

.. Verify the signature:
+
----
$ gpg --no-default-keyring --keyring ./temp.keyring --verify <signature_name> <1>
Copy link
Member

@wking wking Nov 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed that this is checking for a valid sig, but not checking the signed content (so a single valid sig could be applied to multiple images). I'll get instructions for checking content when I get in.

Copy link
Member

@wking wking Nov 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so what we want is --decrypt, not --verify.

$ gpg --no-default-keyring --keyring ./temp.keyring --decrypt signature-1 
{"critical": {"image": {"docker-manifest-digest": "sha256:a7e97365d16d8d920fedd3684b018b780337e069deb1dd8500e866c0d6110334"}, "type": "atomic container signature", "identity": {"docker-reference": "quay.io/openshift-release-dev/ocp-release:4.1.20"}}}gpg: Signature made Tue 15 Oct 2019 02:21:18 PM PDT using RSA key ID F21541EB
gpg: Good signature from "Red Hat, Inc. (beta key 2) <security@redhat.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: B08B 659E E86A F623 BC90  E8DB 938A 80CA F215 41EB

We want:

$ gpg --no-default-keyring --keyring ./temp.keyring --decrypt signature-1 | jq -r '.critical.image["docker-manifest-digest"]'
gpg: Signature made Tue 15 Oct 2019 02:21:18 PM PDT using RSA key ID F21541EB
gpg: Good signature from "Red Hat, Inc. (beta key 2) <security@redhat.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: B08B 659E E86A F623 BC90  E8DB 938A 80CA F215 41EB
sha256:a7e97365d16d8d920fedd3684b018b780337e069deb1dd8500e866c0d6110334

to match ${DIGEST}. Docs on the full structure here, altough I dunno how formal we want to get. I'll work on oc tooling for this next week so I'm ok cutting some corners in the docs here in the meantime, as long as we're confirming that the expected digest is in the signed content.

Copy link
Contributor

@mtrmac mtrmac Nov 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not even that.

This code, which resolves ${DIGEST} from the :${OCP_RELEASE} tag above, and then uses :${OCP_RELEASE} again in oc adm release mirror, is INSECURE and MUST NOT be used; the registry can maliciously modify the :${OCP_RELEASE} tag in the meantime. (Alternatively, if you trust the registry never to do that, and to always point at legitimate OCP images, you don’t need to bother with signatures, by definition.)

Copy link
Contributor

@mtrmac mtrmac Nov 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs on the full structure here, altough I dunno how formal we want to get. I'll work on oc tooling for this next week

Please use c/image/copy.Image, to do copying with signature verification integrated, if at all possible. Feel free to ping me anytime if you need help with that.

I don’t want copy&pasted partial implementations of that format all over OCP if I can help it.

gpg: Signature made Tue 24 Sep 2019 09:38:24 AM PDT using RSA key ID F21541EB
gpg: Good signature from "Red Hat, Inc. (beta key 2) <security@redhat.com>"
gpg: aka "Mark Cox Internal RSA 4096 test key <mjc@redhat.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: B08B 659E E86A F623 BC90 E8DB 938A 80CA F215 41EB
----
<1> Specify the name of the signature that you stored, which is in the format `signature-<n>`.

.. Confirm that the `Primary key fingerprint` value from the signature output is
is listed on the
link:https://access.redhat.com/security/team/key[Product Signing Keys]
page on the Red Hat Customer Portal.

.. If you cannot verify the signature, repeat this process to create another one. Name the new signature `signature-<n+1>`, where `<n>` is the number that you used in the last signature name.

. Mirror the repository:
+
----
$ oc adm -a ${LOCAL_SECRET_JSON} release mirror \
--from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE} \
--to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
--to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}
----
+
This command pulls the release information as a digest, and its output includes the
ifdef::install[]
`imageContentSources` data that you require when you install your cluster.
endif::install[]
ifdef::update[]
`ImageContentSourcePolicy` data that you require when you update your cluster.
endif::update[]

ifdef::install[]
. To create the installation program that is based on the content that you
mirrored, extract it and pin it to the release:
+
Expand All @@ -75,3 +204,4 @@ To ensure that you use the correct images for the version of {product-title}
that you selected, you must extract the installation program from the mirrored
content.
====
endif::install[]
Loading