Skip to content

Commit

Permalink
Add IP to TransportServer Listener (#6367)
Browse files Browse the repository at this point in the history 
* add ip to transport listener

* fix ci

* add template test

* remove space

* add example readme to transportserver ip

* add ipv6

* change ipv4ip to ipv4

* add back in nosec

* python test transport server custom ip listener

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add global config event check to test

* fix ip4vip to ipv4 in tests

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
@pre-commit-ci
Jim Ryan and pre-commit-ci[bot] authored Sep 17, 2024
1 parent ea56c16 commit b091bad
Show file tree
Hide file tree
Showing 23 changed files with 701 additions and 60 deletions.
4 changes: 2 additions & 2 deletions charts/nginx-ingress/values.schema.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -995,15 +995,15 @@
"dns-tcp"
]
},
"ipv4ip": {
"ipv4": {
"type": "string",
"default": "",
"title": "The ipv4 ip",
"examples": [
"127.0.0.1"
]
},
"ipv6ip": {
"ipv6": {
"type": "string",
"default": "",
"title": "The ipv6 ip",
Expand Down
4 changes: 2 additions & 2 deletions docs/content/configuration/global-configuration/globalconfiguration-resource.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,8 @@ The `listeners:` key defines a listener (a combination of a protocol and a port)
| *port* | The port of the listener. The port must fall into the range ``1..65535`` with the following exceptions: ``80``, ``443``, the [status port](/nginx-ingress-controller/logging-and-monitoring/status-page), the [Prometheus metrics port](/nginx-ingress-controller/logging-and-monitoring/prometheus). Among all listeners, only a single combination of a port-protocol is allowed. | *int* | Yes |
| *protocol* | The protocol of the listener. Supported values: ``TCP``, ``UDP`` and ``HTTP``. | *string* | Yes |
| *ssl* | Configures the listener with SSL. This is currently only supported for ``HTTP`` listeners. Default value is ``false`` | *bool* | No |
| *ipv4* | Specifies the IPv4 address to listen on. This is currently only supported for ``HTTP`` or ``HTTPS`` listeners. | *string* | No |
| *ipv6* | Specifies the IPv6 address to listen on. This is currently only supported for ``HTTP`` or ``HTTPS`` listeners. | *string* | No |
| *ipv4* | Specifies the IPv4 address to listen on. | *string* | No |
| *ipv6* | Specifies the IPv6 address to listen on. | *string* | No |

{{</bootstrap-table>}}

Expand Down
215 changes: 215 additions & 0 deletions examples/custom-resources/custom-ip-listeners/transportserver/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
# Custom IPv4 and IPv6 Address Listeners

In this example, we will configure a TransportServer resource with custom IPv4 and IPv6 Address using TCP/UDP listeners.

## Prerequisites

1. Follow the [installation](https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/)
instructions to deploy the Ingress Controller with custom resources enabled.
2. Ensure the Ingress Controller is configured with the `-global-configuration` argument:

```console
args:
- -global-configuration=$(POD_NAMESPACE)/nginx-configuration
```

**Note:**

- **No Updates for GC:** If a GlobalConfiguration resource already exists, delete the previous one before applying the new configuration.
- **Single Replica:** Only one replica is allowed when using this configuration.

## Step 1 - Deploy the GlobalConfiguration resource

Similar to how listeners are configured in our [custom-listeners](../../custom-listeners) examples,
here we deploy a GlobalConfiguration resource with the listeners we want to use in our VirtualServer.

```yaml
apiVersion: k8s.nginx.org/v1
kind: GlobalConfiguration
metadata:
name: nginx-configuration
namespace: nginx-ingress
spec:
listeners:
- name: tcp-ip-dns-listener
port: 5353
protocol: TCP
ipv4: 127.0.0.1
- name: udp-ip-dns-listener
port: 5252
protocol: UDP
ipv4: 127.0.0.2
ipv6: ::1
```
```console
kubectl create -f global-configuration.yaml
```

## Step 2 - Deploy the DNS Application

Create the dns deployment and service:

```console
kubectl create -f dns.yaml
```

## Step 3 - Deploy the TransportServers with custom listeners

The first TransportServer is set to use the udp listener defined in the GlobalConfiguration resource
that was deployed in Step 1. Below is the yaml of this example TransportServer:

```yaml
apiVersion: k8s.nginx.org/v1
kind: TransportServer
metadata:
name: dns-udp
spec:
listener:
name: udp-ip-dns-listener
protocol: UDP
upstreams:
- name: dns-app
service: coredns
port: 5252
upstreamParameters:
udpRequests: 1
udpResponses: 1
action:
pass: dns-app
```
Create the TransportServer resource:
```console
kubectl create -f udp-transport-server.yaml
```

The second TransportServer is set to use the tcp listener defined in the GlobalConfiguration resource.

```yaml
apiVersion: k8s.nginx.org/v1
kind: TransportServer
metadata:
name: tcp-dns
spec:
listener:
name: tcp-ip-dns-listener
protocol: TCP
upstreams:
- name: dns-app
service: coredns
port: 5353
action:
pass: dns-app
```
Create the TransportServer resource:
```console
kubectl create -f tcp-transport-server.yaml
```

## Step 4 - Test the Configuration

1. Check that the configuration has been successfully applied by inspecting the events of the TransportServer and the GlobalConfiguration:

```console
kubectl describe ts udp-dns
```

Below you will see the events as well as the new `Listeners` field

```console
. . .
Spec:
Listener:
name: udp-ip-dns-listener
protocol: UDP
. . .
Routes:
. . .
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal AddedOrUpdated 1s nginx-ingress-controller Configuration for default/udp-dns was added or updated
```

```console
kubectl describe globalconfiguration nginx-configuration -n nginx-ingress
```

```console
. . .
Spec:
Listeners:
ipv4: 127.0.0.1
Name: tcp-ip-dns-listener
Port: 5353
Protocol: TCP
ipv4: 127.0.0.2
ipv6: ::1
Name: udp-ip-dns-listener
Port: 5252
Protocol: UDP

Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Updated 10s nginx-ingress-controller GlobalConfiguration nginx-ingress/nginx-configuration was added or updated
```

2. Since the deployed TransportServer is using port `5252` this example. you can see that the specific ips and ports
are set and listening by using the below commands:

Access the NGINX Pod:

```console
kubectl get pods -n nginx-ingress
```

```text
NAME READY STATUS RESTARTS AGE
nginx-ingress-5cc9c8f66-4dg2t 1/1 Running 0 50s
```

```console
kubectl debug -it nginx-ingress-5cc9c8f66-4dg2t --image=busybox:1.28 --target=nginx-ingress
```

```console
/ # netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:5353 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN -
tcp 0 0 :::9113 :::* LISTEN -
tcp 0 0 :::80 :::* LISTEN -
tcp 0 0 :::443 :::* LISTEN -
tcp 0 0 :::8080 :::* LISTEN -
tcp 0 0 :::8081 :::* LISTEN -
udp 0 0 127.0.0.2:5252 0.0.0.0:* -
udp 0 0 ::1:5252 :::* -
/ #
```

We can see here that the two IPv4 addresses (`127.0.0.1:5353` and `127.0.0.2:5252`) and the one IPv6 address (`::1:5252`) are listed.

3. Examine the NGINX config using the following command:

```console
kubectl exec -it nginx-ingress-5cc9c8f66-4dg2t -n nginx-ingress -- cat /etc/nginx/stream-conf.d/ts_default_dns-udp.conf
```

```console
...
server {
listen 127.0.0.2:5252 udp;
listen [::1]:5252 udp;

...
}
```
64 changes: 64 additions & 0 deletions examples/custom-resources/custom-ip-listeners/transportserver/dns.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns
data:
Corefile: |
.:5353 {
forward . 8.8.8.8:53
log
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: coredns
spec:
replicas: 2
selector:
matchLabels:
app: coredns
template:
metadata:
labels:
app: coredns
spec:
containers:
- name: coredns
image: coredns/coredns:1.10.0
args: [ "-conf", "/etc/coredns/Corefile" ]
volumeMounts:
- name: config-volume
mountPath: /etc/coredns
readOnly: true
ports:
- containerPort: 5353
name: dns
protocol: UDP
- containerPort: 5353
name: dns-tcp
protocol: TCP
securityContext:
readOnlyRootFilesystem: true
volumes:
- name: config-volume
configMap:
name: coredns
items:
- key: Corefile
path: Corefile
---
apiVersion: v1
kind: Service
metadata:
name: coredns
spec:
selector:
app: coredns
ports:
- name: dns
port: 5353
protocol: UDP
- name: dns-tcp
port: 5353
protocol: TCP
16 changes: 16 additions & 0 deletions examples/custom-resources/custom-ip-listeners/transportserver/global-configuration.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: k8s.nginx.org/v1
kind: GlobalConfiguration
metadata:
name: my-release-nginx-ingress-controller
spec:
listeners:
- name: tcp-ip-dns-listener
port: 5353
protocol: TCP
ipv4: 127.0.0.1
ipv6: ::1
- name: udp-ip-dns-listener
port: 5252
protocol: UDP
ipv4: 127.0.0.2
ipv6: ::1
14 changes: 14 additions & 0 deletions examples/custom-resources/custom-ip-listeners/transportserver/tcp-transport-server.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: k8s.nginx.org/v1
kind: TransportServer
metadata:
name: tcp-dns
spec:
listener:
name: tcp-ip-dns-listener
protocol: TCP
upstreams:
- name: dns-app
service: coredns
port: 5353
action:
pass: dns-app
17 changes: 17 additions & 0 deletions examples/custom-resources/custom-ip-listeners/transportserver/udp-transport-server.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
apiVersion: k8s.nginx.org/v1
kind: TransportServer
metadata:
name: dns-udp
spec:
listener:
name: udp-ip-dns-listener
protocol: UDP
upstreams:
- name: dns-app
service: coredns
port: 5353
upstreamParameters:
udpRequests: 1
udpResponses: 1
action:
pass: dns-app
4 changes: 4 additions & 0 deletions internal/configs/transportserver.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ type TransportServerEx struct {
ExternalNameSvcs map[string]bool
DisableIPV6 bool
SecretRefs map[string]*secrets.SecretReference
IPv4 string
IPv6 string
}

func (tsEx *TransportServerEx) String() string {
Expand Down Expand Up @@ -118,6 +120,8 @@ func generateTransportServerConfig(p transportServerConfigParams) (*version2.Tra
ServerSnippets: serverSnippets,
DisableIPV6: p.transportServerEx.DisableIPV6,
SSL: sslConfig,
IPv4: p.transportServerEx.IPv4,
IPv6: p.transportServerEx.IPv6,
},
Match: match,
Upstreams: upstreams,
Expand Down
Loading

0 comments on commit b091bad

Please sign in to comment.