Skip to content

BZ#1886450: Added information for Keepalived router id check for RHV/VMware IPI #35316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dcdacosta wants to merge 1 commit into from
Closed

BZ#1886450: Added information for Keepalived router id check for RHV/VMware IPI #35316

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions installing/installing_rhv/installing-rhv-customizations.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ include::modules/ssh-agent-using.adoc[leveloffset=+1]

include::modules/installation-obtaining-installer.adoc[leveloffset=+1]

include::modules/keepalived-limitation-for-ipi.adoc[leveloffset=+2]

include::modules/installation-initializing.adoc[leveloffset=+1]

include::modules/installing-rhv-example-install-config-yaml.adoc[leveloffset=+2]
Expand Down
2 changes: 2 additions & 0 deletions installing/installing_rhv/installing-rhv-default.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ include::modules/ssh-agent-using.adoc[leveloffset=+1]

include::modules/installation-obtaining-installer.adoc[leveloffset=+1]

include::modules/keepalived-limitation-for-ipi.adoc[leveloffset=+2]

include::modules/installation-launching-installer.adoc[leveloffset=+1]

[IMPORTANT]
Expand Down
2 changes: 2 additions & 0 deletions ...lling_vsphere/installing-restricted-networks-installer-provisioned-vsphere.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ include::modules/vmware-csi-driver-reqs.adoc[leveloffset=+1]

include::modules/installation-vsphere-installer-infra-requirements.adoc[leveloffset=+1]

include::keepalived-limitation-for-ipi.adoc[leveloffset=+2]

include::modules/ssh-agent-using.adoc[leveloffset=+1]

include::modules/installation-adding-vcenter-root-certificates.adoc[leveloffset=+1]
Expand Down
2 changes: 2 additions & 0 deletions ...installing_vsphere/installing-vsphere-installer-provisioned-customizations.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ include::modules/vmware-csi-driver-reqs.adoc[leveloffset=+1]

include::modules/installation-vsphere-installer-infra-requirements.adoc[leveloffset=+1]

include::keepalived-limitation-for-ipi.adoc[leveloffset=+2]

include::modules/ssh-agent-using.adoc[leveloffset=+1]

include::modules/installation-obtaining-installer.adoc[leveloffset=+1]
Expand Down
2 changes: 2 additions & 0 deletions ...ng_vsphere/installing-vsphere-installer-provisioned-network-customizations.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ include::modules/vmware-csi-driver-reqs.adoc[leveloffset=+1]

include::modules/installation-vsphere-installer-infra-requirements.adoc[leveloffset=+1]

include::keepalived-limitation-for-ipi.adoc[leveloffset=+2]

include::modules/ssh-agent-using.adoc[leveloffset=+1]

include::modules/installation-obtaining-installer.adoc[leveloffset=+1]
Expand Down
2 changes: 2 additions & 0 deletions installing/installing_vsphere/installing-vsphere-installer-provisioned.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ include::modules/vmware-csi-driver-reqs.adoc[leveloffset=+1]

include::modules/installation-vsphere-installer-infra-requirements.adoc[leveloffset=+1]

include::keepalived-limitation-for-ipi.adoc[leveloffset=+2]

include::modules/ssh-agent-using.adoc[leveloffset=+1]

include::modules/installation-obtaining-installer.adoc[leveloffset=+1]
Expand Down
1 change: 0 additions & 1 deletion installing/installing_vsphere/preparing-to-install-on-vsphere.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ See the xref:../../architecture/architecture-installation.html#installation-proc
The steps for performing a user-provisioned infrastructure installation are provided as an example only. Installing a cluster with infrastructure you provide requires knowledge of the vSphere platform and the installation process of {product-title}. Use the user-provisioned infrastructure installation instructions as a guide; you are free to create the required resources through other methods.
====


=== Installer-provisioned infrastructure installation of {product-title} on vSphere

Installer-provisioned infrastructure allows the installation program to pre-configure and automate the provisioning of resources required by {product-title}.
Expand Down
5 changes: 5 additions & 0 deletions modules/installation-configuration-parameters.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1249,6 +1249,11 @@ Additional {rh-virtualization} configuration parameters are described in the fol
|Integer. Example: `3`
|====

[NOTE]
====
To avoid virtual router ID conflicts, see "Identifying virtual router conflicts".
====

[id="installation-configuration-parameters-additional-machine_{context}"]
== Additional {rh-virtualization} parameters for machine pools

Expand Down
80 changes: 80 additions & 0 deletions modules/keepalived-limitation-for-ipi.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// Module included in the following assemblies:
//
// * installing/installing_rhv/installing-rhv-customizations.adoc
// * installing/installing_rhv/installing-rhv-default.adoc
[id="keepalived-limitation-for-ipi_{context}"]
= Identifying virtual router conflicts

The Openshift cluster Virtual IPs are managed using multicast (VRRPv2 or VRRPv3), and there is a limitation of 255 unique virtual routers per multicast domain. Each VIP has a virtual router-id associated with it based on the cluster name. Duplication of these ids can result in a conflict, preventing the Virtual IPs from being assigned to the node. Therefore, if you have multiple clusters using the same network, check to see which VIPs the installation will choose to avoid conflicts.
Copy link
Member

@EmilienM EmilienM Jun 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in 4.11, this has switched to Unicast for all on-prem. see openshift/machine-config-operator#3016


[NOTE]
====
This procedure is only required if you have multiple clusters using the same network.
====

To discover the VIPs that will be chosen, you can use the `runtimecfg` utility.

.Procedure

. Identify the release being used by the installer:
+
[source, terminal]
----
$ ./openshift-install version|grep 'release image'
----
+
.Example output
[source, terminal]
----
release image registry.ci.openshift.org/ocp/release@sha256:9aa0dce0b119d1d3f284df2cd30394a23e2d97f4b3e3853503a655caf90219a5
----
+
. Display information about the release:
Copy link
Contributor

@bobfuru bobfuru Oct 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider writing "Display information about the release, for example:" to indicate that the user will not use the exact same image ID (if my understanding is correct) about this command.

+
[source, terminal]
----
$ oc adm release info --registry-config pull-secret.txt registry.ci.openshift.org/ocp/release@sha256:9aa0dce0b119d1d3f284df2cd30394a23e2d97f4b3e3853503a655caf90219a5 -o json | jq -r '.references.spec.tags[] | select(.name == "baremetal-runtimecfg") | .from.name'
Copy link
Contributor

@bobfuru bobfuru Oct 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few considerations:

  • It's not a hard rule at the moment, but we are trying to get away from using jq in the docs and using jsonpath to accomplish the same thing (mostly because RHEL 7 does not explicitly support jq, I believe). If it's possible for Eng to suggest an alternative, please consider using that instead.
  • This command is rather long and could be split into multiple lines by escaping with a backslash (\) and/or splitting after a pipe (|), for example:
$ oc adm release info --registry-config pull-secret.txt \ 
registry.ci.openshift.org/ocp/release@sha256:9aa0dce0b119d1d3f284df2cd30394a23e2d97f4b3e3853503a655caf90219a5 -o json | 
jq -r '.references.spec.tags[] | 
select(.name == "baremetal-runtimecfg") | 
.from.name'

Also consider breaking up similarly long commands in the subsequent steps of this procedure.

----
+
.Example output
[source, terminal]
----
quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b717191f780ebc5a9cce440993f51e0a2b6ef3eab3c450185996b4c79189506b
----
+
+
. Run the runtimecfg utility with the cluster name, in this example `cnf10`:
Copy link
Contributor

@bobfuru bobfuru Oct 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/runtimecfg utility/runtimecfg utility

Copy link
Contributor

@bobfuru bobfuru Nov 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runtimecfg should be wrapped in backticks here, if you wouldn't mind adding

+
[source, terminal]
----
$ podman run --authfile /<path>/<to>/<pull-secret> quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b717191f780ebc5a9cce440993f51e0a2b6ef3eab3c450185996b4c79189506b vr-ids cnf10
----
+
.Example output
[source, terminal]
----
APIVirtualRouterID: 147
DNSVirtualRouterID: 158
IngressVirtualRouterID: 2
----
+
. Run the utility again using a different cluster name, in this example `cnf11`:
Copy link
Contributor

@bobfuru bobfuru Oct 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as previous comment, should they be using runtimecfg or podman run?

+
[source, terminal]
----
$ podman run --authfile /<path>/<to>/<pull-secret> quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b717191f780ebc5a9cce440993f51e0a2b6ef3eab3c450185996b4c79189506b vr-ids cnf11
----
+
.Example output
[source, terminal]
----
APIVirtualRouterID: 228
DNSVirtualRouterID: 239
IngressVirtualRouterID: 147
----
In the prevous example output, you can see that installing two clusters in the same multicast domain with names `cnf10` and `cnf11` will lead to a conflict with the `APIVirtualRouterID` and the `IngressVirtrualRouterID` IDs. To resolve this conflict, rename one of the clusters and repeat this procedure until the conflict is eliminated.
+
[NOTE]
====
Be sure that none of the IDs are assigned to other independent Virtual Router Redundancy Protocol (VRRP) virtual routers running in the same broadcast domain.
====