diff --git a/_topic_map.yml b/_topic_map.yml index 61f6ed29c5ad..d9ccdf16bb43 100644 --- a/_topic_map.yml +++ b/_topic_map.yml @@ -890,6 +890,10 @@ Topics: File: configuring-egress-ips-ovn - Name: Assigning an egress IP address File: assigning-egress-ips-ovn + - Name: Considerations for the use of an egress router pod + File: using-an-egress-router-ovn + - Name: Deploying an egress router pod in redirect mode + File: deploying-egress-router-ovn-redirection - Name: Enabling multicast for a project File: enabling-multicast - Name: Disabling multicast for a project diff --git a/modules/nw-egress-router-about.adoc b/modules/nw-egress-router-about.adoc index acc6e632823d..75b8d0414bd6 100644 --- a/modules/nw-egress-router-about.adoc +++ b/modules/nw-egress-router-about.adoc @@ -1,12 +1,27 @@ // Module included in the following assemblies: // // * networking/openshift_sdn/using-an-egress-router.adoc +// * networking/ovn_kubernetes_network_provider/using-an-egress-router-ovn.adoc + +ifeval::["{context}" == "using-an-egress-router-ovn"] +:ovn: +:egress-pod-image-name: registry.redhat.io/openshift3/ose-pod + +// Image names are different for OKD +ifdef::openshift-origin[] +:egress-pod-image-name: quay.io/openshift/origin-pod +endif::[] + +endif::[] + +ifeval::["{context}" == "using-an-egress-router"] +:openshift-sdn: +endif::[] [id="nw-egress-router-about_{context}"] = About an egress router pod -The {product-title} egress router pod redirects traffic to a specified remote server, using a private source IP address that is not used for any other purpose. -This allows you to send network traffic to servers that are set up to allow access only from specific IP addresses. +The {product-title} egress router pod redirects traffic to a specified remote server from a private source IP address that is not used for any other purpose. An egress router pod enables you to send network traffic to servers that are set up to allow access only from specific IP addresses. [NOTE] ==== @@ -21,26 +36,44 @@ The egress router image is not compatible with Amazon AWS, Azure Cloud, or any o [id="nw-egress-router-about-modes_{context}"] == Egress router modes -In _redirect mode_, an egress router pod sets up iptables rules to redirect traffic from its own IP address to one or more destination IP addresses. Client pods that need to use the reserved source IP address must be modified to connect to the egress router rather than connecting directly to the destination IP. +In _redirect mode_, an egress router pod configures `iptables` rules to redirect traffic from its own IP address to one or more destination IP addresses. Client pods that need to use the reserved source IP address must be modified to connect to the egress router rather than connecting directly to the destination IP. +ifdef::openshift-sdn[] In _HTTP proxy mode_, an egress router pod runs as an HTTP proxy on port `8080`. This mode only works for clients that are connecting to HTTP-based or HTTPS-based services, but usually requires fewer changes to the client pods to get them to work. Many programs can be told to use an HTTP proxy by setting an environment variable. In _DNS proxy mode_, an egress router pod runs as a DNS proxy for TCP-based services from its own IP address to one or more destination IP addresses. To make use of the reserved, source IP address, client pods must be modified to connect to the egress router pod rather than connecting directly to the destination IP address. This modification ensures that external destinations treat traffic as though it were coming from a known source. Redirect mode works for all services except for HTTP and HTTPS. For HTTP and HTTPS services, use HTTP proxy mode. For TCP-based services with IP addresses or domain names, use DNS proxy mode. +endif::openshift-sdn[] + +ifdef::ovn[] +[NOTE] +==== +The egress router CNI plug-in supports redirect mode only. This is a difference with the egress router implementation that you can deploy with OpenShift SDN. Unlike the egress router for OpenShift SDN, the egress router CNI plug-in does not support _HTTP proxy mode_ or _DNS proxy mode_. +==== +endif::ovn[] [id="nw-egress-router-about-router-pod-implementation_{context}"] == Egress router pod implementation -The egress router pod setup is performed by an initialization container. That container runs in a privileged context so that it can configure the macvlan interface and set up `iptables` rules. -After the initialization container finishes setting up the `iptables` rules, it exits. -Next the egress router pod executes the container to handle the egress router traffic. The image used varies depending on the egress router mode. +ifdef::openshift-sdn[] +The egress router pod setup is performed by an initialization container. That container runs in a privileged context so that it can configure the macvlan interface and set up `iptables` rules. After the initialization container finishes setting up the `iptables` rules, it exits. Next the egress router pod executes the container to handle the egress router traffic. The image used varies depending on the egress router mode. The environment variables determine which addresses the egress-router image uses. The image configures the macvlan interface to use `EGRESS_SOURCE` as its IP address, with `EGRESS_GATEWAY` as the IP address for the gateway. Network Address Translation (NAT) rules are set up so that connections to the cluster IP address of the pod on any TCP or UDP port are redirected to the same port on IP address specified by the `EGRESS_DESTINATION` variable. -If only some of the nodes in your cluster are capable of claiming the specified source IP address and using the specified gateway, you can specify a `nodeName` or `nodeSelector` indicating which nodes are acceptable. +If only some of the nodes in your cluster are capable of claiming the specified source IP address and using the specified gateway, you can specify a `nodeName` or `nodeSelector` to identify which nodes are acceptable. +endif::openshift-sdn[] + +ifdef::ovn[] +The egress router implementation uses the egress router Container Network Interface (CNI) plug-in. The plug-in adds a secondary network interface to a pod. + +An egress router is a pod that has two network interfaces. For example, the pod can have `eth0` and `net1` network interfaces. The `eth0` interface is on the cluster network and the pod continues to use the interface for ordinary cluster-related network traffic. The `net1` interface is on a secondary network and has an IP address and gateway for that network. Other pods in the {product-title} cluster can access the egress router service and the service enables the pods to access external services. The egress router acts as a bridge between pods and an external system. + +Traffic that leaves the egress router exits through a node, but the packets +have the MAC address of the `net1` interface from the egress router pod. +endif::ovn[] [id="nw-egress-router-about-deployments_{context}"] == Deployment considerations @@ -49,7 +82,7 @@ An egress router pod adds an additional IP address and MAC address to the primar {rh-openstack-first}:: -If you are deploying {product-title} on {rh-openstack}, you must whitelist the IP and MAC addresses on your OpenStack environment, otherwise link:https://access.redhat.com/solutions/2803331[communication will fail]: +If you deploy {product-title} on {rh-openstack}, you must allow traffic from the IP and MAC addresses of the egress router pod on your OpenStack environment. If you do not allow the traffic, then link:https://access.redhat.com/solutions/2803331[communication will fail]: + [source,terminal] ---- @@ -76,16 +109,18 @@ Specifically, ensure that the following are enabled: To avoid downtime, you can deploy an egress router pod with a `Deployment` resource, as in the following example. To create a new `Service` object for the example deployment, use the `oc expose deployment/egress-demo-controller` command. +ifdef::openshift-sdn[] [source,yaml,subs="attributes+"] ---- -apiVersion: v1 +apiVersion: apps/v1 kind: Deployment metadata: name: egress-demo-controller spec: replicas: 1 <1> selector: - name: egress-router + matchLabels: + name: egress-router template: metadata: name: egress-router @@ -99,5 +134,43 @@ spec: containers: ... ---- +endif::openshift-sdn[] + +ifdef::ovn[] +[source,yaml,subs="attributes+"] +---- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: egress-demo-controller +spec: + replicas: 1 <1> + selector: + matchLabels: + name: egress-router + template: + metadata: + name: egress-router + labels: + name: egress-router + annotations: + k8s.v1.cni.cncf.io/networks: egress-router-redirect + spec: <2> + containers: + - name: egress-router-redirect + image: {egress-pod-image-name} +---- +endif::ovn[] <1> Ensure that replicas is set to `1`, because only one pod can use a given egress source IP address at any time. This means that only a single copy of the router runs on a node. + <2> Specify the `Pod` object template for the egress router pod. + +// Clear temporary attributes +ifdef::openshift-sdn[] +:!openshift-sdn: +endif::[] + +ifdef::ovn[] +:!ovn: +:!egress-pod-image-name: +endif::[] \ No newline at end of file diff --git a/modules/nw-egress-router-network-attachment.adoc b/modules/nw-egress-router-network-attachment.adoc new file mode 100644 index 000000000000..019a913aa41f --- /dev/null +++ b/modules/nw-egress-router-network-attachment.adoc @@ -0,0 +1,50 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/deploying-egress-router-ovn-redirection.adoc + +ifeval::["{context}" == "deploying-egress-router-ovn-redirection"] +:redirect: +:router-type: redirect +endif::[] + +[id="nw-egress-router-network-attachment_{context}"] += Network attachment definition for an egress router in {router-type} mode + +Before a pod can act as an egress router, you must specify the network interface configuration as a `NetworkAttachmentDefinition` object. The object specifies information such as the IP address to attach to the egress router pod, the network destinations, and a network gateway. As the pod for the egress router starts, Multus uses the network attachment definition to add a network interface with the specified properties to the pod. + +.Example network attachment definition +[source,yaml] +---- +apiVersion: "k8s.cni.cncf.io/v1" +kind: NetworkAttachmentDefinition +metadata: + name: egress-router-redirect <.> +spec: + config: '{ + "cniVersion": "0.4.0", + "type": "egress-router", + "name": "egress-router", + "ip": { + "addresses": [ + "192.168.12.99/24" <.> + ], + "destinations": [ + "192.168.12.91/32" <.> + ], + "gateway": "192.168.12.1" <.> + } + }' +---- +<.> The name of the network attachment definition is used later in the specification for the egress router pod. + +<.> The `addresses` key specifies the reserved source IP address to use with the additional network interface. Specify a single IP address in CIDR notation, such as `192.168.12.99/24`. + +<.> The `destinations` key specifies a single IP address in CIDR notation that the egress router sends packets to. The network address translation (NAT) tables for the egress router pod are configured so that connections to the cluster IP address of the pod are redirected to the same port on the destination IP address. Using this example, connections to the pod are redirected to `192.168.12.91`, with a source IP address of `192.168.12.99`. + +<.> The `gateway` key specifies the IP address for the network gateway. + +// clear temporary attributes +:!router-type: +ifdef::redirect[] +:!redirect: +endif::[] diff --git a/modules/nw-egress-router-pod-ovn.adoc b/modules/nw-egress-router-pod-ovn.adoc new file mode 100644 index 000000000000..c0c248c81278 --- /dev/null +++ b/modules/nw-egress-router-pod-ovn.adoc @@ -0,0 +1,47 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/deploying-egress-router-ovn-redirection.adoc + +ifeval::["{context}" == "deploying-egress-router-ovn-redirection"] +:redirect: +:router-type: redirect +// Like nw-egress-router-pod, monitor bz-1896170 +:egress-pod-image-name: registry.redhat.com/openshift3/ose-pod +endif::[] + +// Images are different for OKD +ifdef::openshift-origin[] + +ifdef::redirect[] +:egress-pod-image-name: quay.io/openshift/origin-pod +endif::[] + +endif::[] + +[id="nw-egress-router-cni-pod_{context}"] += Egress router pod specification for {router-type} mode + +After you create a network attachment definition, you add a pod that references the definition. + +.Example egress router pod specification +[source,yaml,subs="attributes+"] +---- +apiVersion: v1 +kind: Pod +metadata: + name: egress-router-pod + annotations: + k8s.v1.cni.cncf.io/networks: egress-router-redirect <.> +spec: + containers: + - name: egress-router-pod + image: {egress-pod-image-name} +---- +<.> The specified network must match the name of the network attachment definition. You can specify a namespace, interface name, or both, by replacing the values in the following pattern: `/@`. By default, Multus adds a secondary network interface to the pod with a name such as `net1`, `net2`, and so on. + +// Clear temporary attributes +:!router-type: +:!egress-pod-image-name: +ifdef::redirect[] +:!redirect: +endif::[] diff --git a/modules/nw-egress-router-redirect-mode-ovn.adoc b/modules/nw-egress-router-redirect-mode-ovn.adoc new file mode 100644 index 000000000000..6c217fa3a8db --- /dev/null +++ b/modules/nw-egress-router-redirect-mode-ovn.adoc @@ -0,0 +1,139 @@ +// Module included in the following assemblies: +// +// * networking/ovn_kubernetes_network_provider/deploying-egress-router-ovn-redirection.adoc + +[id="nw-egress-router-redirect-mode-ovn_{context}"] += Deploying an egress router pod in redirect mode + +You can deploy an egress router pod to redirect traffic from its own reserved source IP address to one or more destination IP addresses. + +After you add an egress router pod, the client pods that need to use the reserved source IP address must be modified to connect to the egress router rather than connecting directly to the destination IP. + +.Prerequisites + +* Install the OpenShift CLI (`oc`). +* Log in as a user with `cluster-admin` privileges. + +.Procedure + +. Create a network attachment definition. + +. Create an egress router pod. + +. To ensure that other pods can find the IP address of the egress router pod, create a service that uses the egress router pod, as in the following example: ++ +[source,yaml] +---- +apiVersion: v1 +kind: Service +metadata: + name: egress-1 +spec: + ports: + - name: database + protocol: TCP + port: 3306 + type: ClusterIP + selector: + name: egress-router-pod +---- ++ +After you create the service, your pods can connect to the service. The egress router pod redirects the connection to the corresponding port on the destination IP address. The connections originate from the reserved source IP address. + +.Verification steps + +To verify that the egress router pod started and has the secondary network interface, complete the following procedure: + +. View the events for the egress router pod: ++ +[source,terminal] +---- +$ oc get events --field-selector involvedObject.name=egress-router-pod +---- ++ +If the pod references the network attachment definition, the previous command returns output that is similar to the following: ++ +.Example output ++ +[source,terminal] +---- +LAST SEEN TYPE REASON OBJECT MESSAGE +5m4s Normal Scheduled pod/egress-router-pod Successfully assigned default/egress-router-pod to ci-ln-9x2bnsk-f76d1-j2v6g-worker-c-24g65 +5m3s Normal AddedInterface pod/egress-router-pod Add eth0 [10.129.2.31/23] +5m3s Normal AddedInterface pod/egress-router-pod Add net1 [192.168.12.99/24] from default/egress-router-redirect +---- + +. Optional: View the routing table for the egress router pod. +// Terminology from support-collecting-network-trace.adoc +.. Get the node name for the egress router pod: ++ +[source,terminal] +---- +$ POD_NODENAME=$(oc get pod egress-router-pod -o jsonpath="{.spec.nodeName}") +---- + +.. Enter into a debug session on the target node. This step instantiates a debug pod called `-debug`: ++ +[source,terminal] +---- +$ oc debug node/$POD_NODENAME +---- + +.. Set `/host` as the root directory within the debug shell. The debug pod mounts the root file system of the host in `/host` within the pod. By changing the root directory to `/host`, you can run binaries from the executable paths of the host: ++ +[source,terminal] +---- +# chroot /host +---- + +.. From within the `chroot` environment console, get the container ID: ++ +[source,terminal] +---- +# crictl ps --name egress-router-redirect | awk '{print $1}' +---- ++ +.Example output +[source,terminal] +---- +CONTAINER +bac9fae69ddb6 +---- + +.. Determine the process ID of the container. In this example, the container ID is `bac9fae69ddb6`: ++ +[source,terminal] +---- +# crictl inspect -o yaml bac9fae69ddb6 | grep 'pid:' | awk '{print $2}' +---- ++ +.Example output +[source,terminal] +---- +68857 +---- + +.. Enter the network namespace of the container: ++ +[source,terminal] +---- +# nsenter -n -t 68857 +---- + +.. Display the routing table: ++ +[source,terminal] +---- +# ip route +---- ++ +In the following example output, the `net1` network interface is the default route. Traffic for the cluster network uses the `eth0` network interface. Traffic for the `192.168.12.0/24` network uses the `net1` network interface and originates from the reserved source IP address `192.168.12.99`. The pod routes all other traffic to the gateway at IP address `192.168.12.1`. Routing for the service network is not shown. ++ +.Example output +[source,terminal] +---- +default via 192.168.12.1 dev net1 +10.129.2.0/23 dev eth0 proto kernel scope link src 10.129.2.31 +192.168.12.0/24 dev net1 proto kernel scope link src 192.168.12.99 +192.168.12.1 dev net1 +---- diff --git a/networking/ovn_kubernetes_network_provider/deploying-egress-router-ovn-redirection.adoc b/networking/ovn_kubernetes_network_provider/deploying-egress-router-ovn-redirection.adoc new file mode 100644 index 000000000000..a8b807c732fe --- /dev/null +++ b/networking/ovn_kubernetes_network_provider/deploying-egress-router-ovn-redirection.adoc @@ -0,0 +1,23 @@ +[id="deploying-egress-router-ovn-redirection"] += Deploying an egress router pod in redirect mode +include::modules/common-attributes.adoc[] +:context: deploying-egress-router-ovn-redirection + +toc::[] + +As a cluster administrator, you can deploy an egress router pod to redirect traffic to specified destination IP addresses from a reserved source IP address. + +The egress router implementation uses the egress router Container Network Interface (CNI) plug-in. + +:FeatureName: The egress router CNI plug-in +include::modules/technology-preview.adoc[leveloffset=+1] +:!FeatureName: + +// Network attachment definition for egress router in {router-type} mode +include::modules/nw-egress-router-network-attachment.adoc[leveloffset=+1] + +// Egress router pod specification for {router-type} mode +include::modules/nw-egress-router-pod-ovn.adoc[leveloffset=+1] + +// Deploying an egress router pod in {router-type} mode +include::modules/nw-egress-router-redirect-mode-ovn.adoc[leveloffset=+1] diff --git a/networking/ovn_kubernetes_network_provider/using-an-egress-router-ovn.adoc b/networking/ovn_kubernetes_network_provider/using-an-egress-router-ovn.adoc new file mode 100644 index 000000000000..39035d9a69e3 --- /dev/null +++ b/networking/ovn_kubernetes_network_provider/using-an-egress-router-ovn.adoc @@ -0,0 +1,14 @@ +[id="using-an-egress-router-ovn"] += Considerations for the use of an egress router pod +include::modules/common-attributes.adoc[] +:context: using-an-egress-router-ovn + +toc::[] + +// About an egress router pod +include::modules/nw-egress-router-about.adoc[leveloffset=+1] + +[id="using-an-egress-router-ovn-additional-resources"] +== Additional resources + +- xref:../../networking/ovn_kubernetes_network_provider/deploying-egress-router-ovn-redirection.adoc#deploying-egress-router-ovn-redirection[Deploying an egress router in redirection mode]