Skip to content

bug 1670732 need to update master-config.yaml for a cert change #13616

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kalexand-rh merged 1 commit into from
Feb 19, 2019
Merged

bug 1670732 need to update master-config.yaml for a cert change #13616

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 52 additions & 42 deletions install_config/certificate_customization.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ openshift_hosted_router_certificate={"certfile": "/path/on/host/to/app-crt-file"
----

<1> If you provide a value for the `openshift_master_named_certificates` parameter, set this parameter to `true`.
<2> Provisions a xref:configuring-custom-certificates-master[master API certificate].
<2> Provisions a xref:configuring-custom-certificates-master[master API certificate].
<3> Provisions a xref:configuring-custom-certificates-wildcard[wildcard API certificate].

Example parameters for a master API certificate:
Expand All @@ -57,10 +57,10 @@ You can specify custom certificates for the web console and for the CLI through
`servingInfo` section of the xref:../install_config/master_node_configuration.adoc#master-configuration-files[master
configuration file]:

* The `servingInfo.namedCertificates` section serves up custom certificates for the web console.
* The `servingInfo` section serves up custom certificates for the CLI and other API calls.
* The `servingInfo.namedCertificates` section serves up custom certificates for the web console.
* The `servingInfo` section serves up custom certificates for the CLI and other API calls.

You can configure multiple certificates this way, and each certificate can be associated with
You can configure multiple certificates this way, and each certificate can be associated with
xref:configuring-custom-certificates-master[multiple host names], xref:configuring-custom-certificates-master[multiple routers], or the xref:configuring-custom-certificates-registry[{product-title} image registry].

A default certificate must be configured in the `servingInfo.certFile` and
Expand All @@ -71,7 +71,7 @@ A default certificate must be configured in the `servingInfo.certFile` and
====
The `namedCertificates` section should be configured only for the host name
associated with the `masterPublicURL` and
`oauthConfig.assetPublicURL` settings n the *_/etc/origin/master/master-config.yaml_* file.
`oauthConfig.assetPublicURL` settings n the *_/etc/origin/master/master-config.yaml_* file.
Using a custom serving certificate for
the host name associated with the `masterURL` will result in TLS errors as
infrastructure components will attempt to contact the master API using the
Expand Down Expand Up @@ -104,15 +104,15 @@ servingInfo:
<1> Path to certificate and key files for the CLI and other API calls.
<2> Path to certificate and key files for the web console.

The `openshift_master_cluster_public_hostname` and `openshift_master_cluster_hostname` parameters in the xref:../install/configuring_inventory_file.adoc#configuring-ansible[Ansible inventory file], by default `/etc/ansible/hosts`, must be different. If they are the same, the named certificates will fail and you will need to re-install them.
The `openshift_master_cluster_public_hostname` and `openshift_master_cluster_hostname` parameters in the xref:../install/configuring_inventory_file.adoc#configuring-ansible[Ansible inventory file], by default `/etc/ansible/hosts`, must be different. If they are the same, the named certificates will fail and you will need to re-install them.

----
# Native HA with External LB VIPs
openshift_master_cluster_hostname=internal.paas.example.com
openshift_master_cluster_public_hostname=external.paas.example.com
----

For more information on
For more information on
using DNS with {product-title}, see the xref:../install/prerequisites.adoc#prereq-dns[DNS installation prerequisites].

This approach allows you to take advantage of the self-signed certificates generated by {product-title} and add custom trusted certificates to individual components as needed.
Expand All @@ -125,17 +125,17 @@ the server to pick up the configuration changes.
[[configuring-custom-certificates-master]]
== Configuring a Custom Master Host Certificate

In order to facilitate trusted connections with external users of {product-title}, you can provision a named certificate that matches the domain name provided in the `openshift_master_cluster_public_hostname` paramater in the xref:../install/configuring_inventory_file.adoc#configuring-ansible[Ansible inventory file],
by default `/etc/ansible/hosts`.
In order to facilitate trusted connections with external users of {product-title}, you can provision a named certificate that matches the domain name provided in the `openshift_master_cluster_public_hostname` paramater in the xref:../install/configuring_inventory_file.adoc#configuring-ansible[Ansible inventory file],
by default `/etc/ansible/hosts`.

You must place this certificate in a directory accessible to Ansible and add the path in the
You must place this certificate in a directory accessible to Ansible and add the path in the
Ansible inventory file, as follows:

----
openshift_master_named_certificates=[{"certfile": "/path/to/console.ocp-c1.myorg.com.crt", "keyfile": "/path/to/console.ocp-c1.myorg.com.key", "names": ["console.ocp-c1.myorg.com"]}]
----
----

Where the parameter values are:
Where the parameter values are:

* *certfile* is the path to the file that contains the {product-title} custom certificate.

Expand All @@ -150,54 +150,54 @@ The file paths must be local to the system where Ansible runs. Certificates
are copied to master hosts and are deployed within the
*_/etc/origin/master_* directory.

When securing the registry, add the service hostnames and IP addresses to the server certificate for the registry.
When securing the registry, add the service hostnames and IP addresses to the server certificate for the registry.
The Subject Alternative Names (SAN) must contain the following.

* Two service hostnames:
+
----
docker-registry.default.svc.cluster.local
docker-registry.default.svc
docker-registry.default.svc
----

* Service IP address.
* Service IP address.
+
For example:
+
----
172.30.252.46
----
+
Use the following command to get the container image registry service IP address:
Use the following command to get the container image registry service IP address:
+
----
oc get service docker-registry --template='{{.spec.clusterIP}}'
----

* Public hostname.
* Public hostname.
+
----
docker-registry-default.apps.example.com
----
+
Use the following command to get the container image registry public hostname:
+
----
----
oc get route docker-registry --template '{{.spec.host}}'
----

For example, the server certificate should contain SAN details similar to the following:
For example, the server certificate should contain SAN details similar to the following:

----
X509v3 Subject Alternative Name:
X509v3 Subject Alternative Name:
DNS:docker-registry-public.openshift.com, DNS:docker-registry.default.svc, DNS:docker-registry.default.svc.cluster.local, DNS:172.30.2.98, IP Address:172.30.2.98
----

[[configuring-custom-certificates-wildcard]]
== Configuring a Custom Wildcard Certificate for the Default Router

You can configure the {product-title} default router with a default wildcard certificate. A default wildcard certificate provides a convenient way for
applications that are deployed in {product-title} to use default encryption without needing custom certificates.
You can configure the {product-title} default router with a default wildcard certificate. A default wildcard certificate provides a convenient way for
applications that are deployed in {product-title} to use default encryption without needing custom certificates.

[NOTE]
====
Expand All @@ -216,15 +216,15 @@ For example:
openshift_hosted_router_certificate={"certfile": "/home/cloud-user/star-apps.148.251.233.173.nip.io.cert.pem", "keyfile": "/home/cloud-user/star-apps.148.251.233.173.nip.io.key.pem", "cafile": "/home/cloud-user/ca-chain.cert.pem"}
----

Where the parameter values are:
Where the parameter values are:

* *certfile* is the path to the file that contains the {product-title} router certificate.

* *keyfile* is the path to the file that contains the {product-title} wildcard key.

* *cafile* is the path to the file that contains the root CA for this key and certificate. If an intermediate CA is in use, the file should contain both the intermediate and root CA.

If these certificate files are new to your {product-title} cluster, change to the playbook directory and run the Ansible *_deploy_router.yml_* playbook to add these files to the {product-title} configuration files.
If these certificate files are new to your {product-title} cluster, change to the playbook directory and run the Ansible *_deploy_router.yml_* playbook to add these files to the {product-title} configuration files.
The playbook adds the certificate files to the *_/etc/origin/master/_* directory.

----
Expand All @@ -238,23 +238,23 @@ ifdef::openshift-origin[]
endif::[]
----

If xref:../install_config/redeploying_certificates.adoc#redeploying-all-certificates-current-ca[the certificates are not new],
If xref:../install_config/redeploying_certificates.adoc#redeploying-all-certificates-current-ca[the certificates are not new],
for example, you want to change existing certificates or replace expired certificates, change to the playbook directory and run the following playbook:

----
ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/redeploy-certificates.yml
----

[NOTE]
[NOTE]
For this playbook to run, the certificate names must not change. If the certificate names change, rerun the Ansible *_deploy_cluster.yml_* playbook
as if the certificates were new.

[[configuring-custom-certificates-registry]]
== Configuring a Custom Certificate for the Image Registry

The {product-title} image registry is an internal service that facilitates builds and deployments. Most of the communication with the registry is handled by internal components in {product-title}. As such, you should not need to replace the certificate used by the registry service itself.
The {product-title} image registry is an internal service that facilitates builds and deployments. Most of the communication with the registry is handled by internal components in {product-title}. As such, you should not need to replace the certificate used by the registry service itself.

However, by default, the registry uses routes to allow external systems and users to do pulls and pushes of images. You can use a _re-encrypt route_ with a custom certificate that is presented to external users instead of using the internal, self-signed certificate.
However, by default, the registry uses routes to allow external systems and users to do pulls and pushes of images. You can use a _re-encrypt route_ with a custom certificate that is presented to external users instead of using the internal, self-signed certificate.

To configure this, add the
xref:../install/configuring_inventory_file.adoc#advanced-install-configuring-docker-route[following lines]
Expand Down Expand Up @@ -286,8 +286,8 @@ openshift_hosted_registry_routetermination=reencrypt <3>
[[configuring-custom-certificates-lb]]
== Configuring a Custom Certificate for a Load Balancer

If your {product-title} cluster uses the default load balancer or an enterprise-level load balancer,
you can use custom certificates to make the web console and API available externally using a publicly-signed custom certificate. leaving the existing internal certificates for
If your {product-title} cluster uses the default load balancer or an enterprise-level load balancer,
you can use custom certificates to make the web console and API available externally using a publicly-signed custom certificate. leaving the existing internal certificates for
the internal endpoints.

To configure {product-title} to use custom certificates in this way:
Expand Down Expand Up @@ -320,12 +320,12 @@ servingInfo:
+
[NOTE]
====
Configure the `namedCertificates` section for only the host name associated with the `masterPublicURL` and `oauthConfig.assetPublicURL` settings.
Using a custom serving certificate for the host name associated with the `masterURL` causes in TLS errors as infrastructure components
Configure the `namedCertificates` section for only the host name associated with the `masterPublicURL` and `oauthConfig.assetPublicURL` settings.
Using a custom serving certificate for the host name associated with the `masterURL` causes in TLS errors as infrastructure components
attempt to contact the master API using the internal masterURL host.
====

. Specify the `openshift_master_cluster_public_hostname` and `openshift_master_cluster_hostname` paramaters in the Ansible inventory file, by default *_/etc/ansible/hosts_*.
. Specify the `openshift_master_cluster_public_hostname` and `openshift_master_cluster_hostname` paramaters in the Ansible inventory file, by default *_/etc/ansible/hosts_*.
These values must be different. If they are the same, the named certificates will fail.
+
----
Expand All @@ -343,38 +343,48 @@ For information specific to your load balancer environment, refer to link:https:
[[configuring-custom-certificates-retrofit]]
== Retrofit Custom Certificates into a Cluster

You can retrofit custom master and custom router certificates into an existing {product-title} cluster by editing the
the Ansible inventory file, by default *_/etc/ansible/hosts_*, and running a playbook.
You can retrofit custom master and custom router certificates into an existing
{product-title} cluster.

[[configuring-custom-certificates-retrofit-master]]
=== Retrofit Custom Master Certificates into a Cluster

To retrofit custom certificates:
To retrofit custom certificates:

. Edit the Ansible inventory file to set the `openshift_master_overwrite_named_certificates=true`.
. Edit the Ansible inventory file to set the `openshift_master_overwrite_named_certificates=true`.

. Specify the path to the certificate using the `openshift_master_named_certificates` parameter.
. Specify the path to the certificate using the `openshift_master_named_certificates` parameter.
+
[source,yaml]
----
openshift_master_overwrite_named_certificates=true
openshift_master_named_certificates=[{"certfile": "/path/on/host/to/crt-file", "keyfile": "/path/on/host/to/key-file", "names": ["public-master-host.com"], "cafile": "/path/on/host/to/ca-file"}] <1>
----
+
<1> Path to a xref:configuring-custom-certificates-master[master API certificate].
<1> Path to a xref:configuring-custom-certificates-master[master API certificate].

. Change to the playbook directory and run the following playbook:
+
----
ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/redeploy-certificates.yml
----

. If you use xref:configuring-custom-certificates[named certificates]:
Copy link
Contributor Author

@kalexand-rh kalexand-rh Feb 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jianlinliu, will you PTAL? (The other changes to the file are whitespace trimming.)

Copy link

@gpei gpei Feb 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @kalexand-rh I think we need to add some explanation here that the above steps are actually not working for now, so have to manually update the certificate parameters in master-config.yaml. Or else it looks like all of these steps are required.

Copy link
Contributor Author

@kalexand-rh kalexand-rh Feb 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gpei, I thought you did have to do both. Scott said that "The playbook will ensure that the certificate files are distributed to the hosts but won't update master-config.yaml." I interpreted that to mean that you had to run the playbook to distribute the certs and then separately add the parameters to the master-config.yaml file.

Is that right?

Copy link

@gpei gpei Feb 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, I forgot the step to distribute the certs, you're correct.

.. Update the certificate parameters in the *_master-config.yaml_* file on each
master node.
.. Restart the {product-title} master service to apply the changes.
+
----
# master-restart api
# master-restart controllers
----

[[configuring-custom-certificates-retrofit-router]]
=== Retrofit Custom Router Certificates into a Cluster

To retrofit custom router certificates:
To retrofit custom router certificates:

. Edit the Ansible inventory file to set the `openshift_master_overwrite_named_certificates=true`.
. Edit the Ansible inventory file to set the `openshift_master_overwrite_named_certificates=true`.

. Specify the path to the certificate using the `openshift_hosted_router_certificate` parameter.
+
Expand All @@ -396,5 +406,5 @@ $ ansible-playbook playbooks/openshift-hosted/redeploy-router-certificates.yml
[[ansible-configuring-custom-certificates-other]]
== Using Custom Certificates with Other Components

For information on how other components, such as Logging & Metrics, use custom certificates, see
For information on how other components, such as Logging & Metrics, use custom certificates, see
xref:../day_two_guide/certificate_management.adoc#admin-solutions-certificate-management[Certificate Management].