OpenStack: Add scaling information for API and Ingress

modules/installation-osp-kuryr-api-scaling.adoc

@@ -0,0 +1,86 @@
+// Module included in the following assemblies:
+//
+// * networking/openstack/load-balancing-openstack.adoc
+
+[id="installation-osp-kuryr-octavia-configure{context}"]
+= Scaling API using {rh-openstack} Octavia
+
+API can be scaled on {product-title} cluster running on {rh-openstack} with
+Octavia support
+Octavia is a requirement to deploy {product-title} with Kuryr, as Kuryr uses it
+to create the services, but when installing without Kuryr it is not. So first
+step is to ensure Octavia is available.
+If it is, it can be leveraged to create extra API IPs and better scale with the
+load. This allows to remove the API bottleneck as currently the traffics enters
+through the master node that holds the API VIP.
+
+If {product-title} cluster was installed with Kuryr, the Cluster Network
+Operator already creates an internal Octavia Load Balancer (Amphora driver) used
+for by the pods that wants to reach the kubernetes service IP (e.g.,
+172.30.0.1). Note: components using the api-int DNS record do not get to the API
+through this load balancer.
+
+As this loadbalancer already exists, we can simply associate the API Floating IP
+to it:
++
+----
+$ openstack floating ip unset $API_FIP
+$ openstack floating ip set  --port $(openstack loadbalancer show -c vip_port_id
+-f value ${OCP_CLUSTER}-kuryr-api-loadbalancer) $API_FIP
+----
+
+
+On the other hand, if {product-title} cluster was installed without kuryr, or an
+extra load balancer(s) wants to be created, then the next steps can be used:
+
+. Create an Octavia Load Balancer (Amphora as it requires L7 capabilities). Note
+the subnet selected is the same where the worker node VMs are running.
++
+----
+$ openstack loadbalancer create --name API_OCP_CLUSTER --vip-subnet-id WORKER_VMS_SUBNET_ID
+----
+
+. Wait for the loadbalancer to be in ACTIVE status, and then create the listeners
++
+----
+$ openstack loadbalancer listener create --name API_OCP_CLUSTER_6443 --protocol HTTPS--protocol-port 6443 API_OCP_CLUSTER
+----
+
+. Create the pool. Note the load balancer algorithm selected is ROUND_ROBIN and
+the pool session persistence is enabled:
++
+----
+$ openstack loadbalancer pool create --name API_OCP_CLUSTER_pool_6443 --lb-algorithm ROUND_ROBIN --session-persistence type=SOURCE_IP --listener API_OCP_CLUSTER_6443 --protocol HTTPS
+----
+
+. Create a health monitor to ensure masters nodes are available (node, if Octavia
+version supports HTTP monitors it can be used to check /healthz/readyz instead):
++
+----
+$ openstack loadbalancer healthmonitor create --delay 5 --max-retries 4 --timeout 10 --type TCP API_OCP_CLUSTER_pool_6443
+----
+
+. Add the master nodes as members of the load balancer pool:
++
+----
+$ for SERVER in $(MASTER-0-IP MASTER-1-IP MASTER-2-IP)
+do
+  openstack loadbalancer member create --address $SERVER  --protocol-port 6443 API_OCP_CLUSTER_pool_6443
+done
+----
+
+. Remove the API_FIP from the current port if the same wants to be used (otherwise a new one can be created):
++
+----
+$ openstack floating ip unset $API_FIP
+----
+
+. Add the API_FIP to the created Load Balancer VIP:
++
+----
+$ openstack floating ip set  --port $(openstack loadbalancer show -c vip_port_id -f value API_OCP_CLUSTER) $API_FIP
+----
+
+Note that with Octavia Amphora driver the traffic will still go through a single VM (in this case the Amphora VM).
+So, to avoid this the above steps can be replicated as many times as desired, creating several API load balancers to achieve better performance at scale.
+

modules/installation-osp-kuryr-ingress-scaling.adoc

@@ -0,0 +1,95 @@
+// Module included in the following assemblies:
+//
+// * networking/openstack/load-balancing-openstack.adoc
+
+[id="installation-osp-kuryr-octavia-configure{context}"]
+= Scaling Ingress using {rh-openstack} Octavia on Kuryr deployments
+
+Ingress can be scaled on {product-title} cluster that uses Kuryr by using
+Octavia Load Balancers.
+Octavia is a requirement to deploy {product-title} with Kuryr, as Kuryr uses it
+to create the services.
+This can be leveraged to create extra Ingress APIs and better scale with the
+load.
+This allows to remove the Ingress bottleneck as currently the traffics enters
+through the worker node that holds the Ingress VIP.
+
+The steps to follow are the next:
+
+. Copy the current internal router service:
++
+----
+$ oc -n openshift-ingress get svc router-internal-default -o yaml > external_router.yaml
+----
+
+. Change the `name` (e.g., `router-external-default`) and the `type` to
+`LoadBalancer` so that Kuryr associates a Floating IP to the Load Balancer VIP.
+Also remove all the extra information not needed such as timeStamps, IDs, ...
++
+----
+$ cat external_router.yaml
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    ingresscontroller.operator.openshift.io/owning-ingresscontroller: default
+  name: router-external-default
+  namespace: openshift-ingress
+spec:
+  ports:
+  - name: http
+    port: 80
+    protocol: TCP
+    targetPort: http
+  - name: https
+    port: 443
+    protocol: TCP
+    targetPort: https
+  - name: metrics
+    port: 1936
+    protocol: TCP
+    targetPort: 1936
+  selector:
+    ingresscontroller.operator.openshift.io/deployment-ingresscontroller: default
+  sessionAffinity: None
+  type: LoadBalancer
+----
+
+. And create the above service:
++
+----
+$ oc apply -f external_router.yaml
+----
+
+. Check the `external-ip` that the service got, and ensure it is the same as the one associated to the created load balancer:
++
+----
+$ oc -n openshift-ingress get svc
+NAME                      TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                                     AGE
+router-external-default   LoadBalancer   172.30.235.33    <pending>     80:30112/TCP,443:32359/TCP,1936:30317/TCP   10s
+router-internal-default   ClusterIP      172.30.115.123   <none>        80/TCP,443/TCP,1936/TCP                     22h
+
+$ oc -n openshift-ingress get svc
+NAME                      TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)                                     AGE
+router-external-default   LoadBalancer   172.30.235.33    10.46.22.161   80:30112/TCP,443:32359/TCP,1936:30317/TCP   3m38s
+router-internal-default   ClusterIP      172.30.115.123   <none>         80/TCP,443/TCP,1936/TCP                     22h
+
+$ openstack loadbalancer list | grep router-external
+| 21bf6afe-b498-4a16-a958-3229e83c002c | openshift-ingress/router-external-default                                   | 66f3816acf1b431691b8d132cc9d793c | 172.30.235.33  | ACTIVE              | octavia  |
+
+$ openstack floating ip list | grep 172.30.235.33
+| e2f80e97-8266-4b69-8636-e58bacf1879e | 10.46.22.161        | 172.30.235.33    | 655e7122-806a-4e0a-a104-220c6e17bda6 | a565e55a-99e7-4d15-b4df-f9d7ee8c9deb | 66f3816acf1b431691b8d132cc9d793c |
+----
+
+The `external-ip` is the new Ingress IP to use externally.
+
+Note that if Kuryr uses Octavia Amphora driver, still the traffic will go
+through a single VM (in this case the Amphora VM). So, to avoid this the above
+steps can be replicated as many times as desired, creating different Ingress
+Floating IPs that can be used to achieve better performance.
+
+By contrast, if Octavia OVN driver is used, the traffic is already distributed
+and it will enter the cluster though the OVN gateway nodes (same as any other
+N/S traffic). Anyway, it is also possible to create several external
+(LoadBalancer type) services with it too.
+

networking/load-balancing-openstack.adoc

@@ -5,4 +5,6 @@ include::modules/common-attributes.adoc[]
 
 toc::[]
 
-include::modules/installation-osp-kuryr-octavia-upgrade.adoc[leveloffset=+1]
+include::modules/installation-osp-kuryr-octavia-upgrade.adoc[leveloffset=+1]
+include::modules/installation-osp-kuryr-api-scaling.adoc[leveloffset=+1]
+include::modules/installation-osp-kuryr-ingress-scaling.adoc[leveloffset=+1]