Skip to content

Commit 219c973

Browse files
kate-osbornkate-osborn
authored
Support cross-namespace BackendRefs in HTTPRoutes (#806)
Allow HTTPRoutes to reference Backends in different namespaces if a ReferenceGrant permits it. Problem: NKG does not allow HTTPRoutes to reference Backends in different namespaces. Solution: Allow HTTPRoutes to reference Backends in different namespaces if a ReferenceGrant permits it. Both SecretObjectReferences and BackendObjectReferences are resolved through the referenceGrantResolver. The referenceGrantResolver transforms the map of ReferenceGrants into a structure that allows for O(n) lookup of references.
1 parent 8ca16d2 commit 219c973

18 files changed

+760
-243
lines changed

‎docs/gateway-api-compatibility.md

+5-6
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ This document describes which Gateway API resources NGINX Kubernetes Gateway sup
1212
| [TLSRoute](#tlsroute) | Not supported |
1313
| [TCPRoute](#tcproute) | Not supported |
1414
| [UDPRoute](#udproute) | Not supported |
15-
| [ReferenceGrant](#referencegrant) | Partially supported |
15+
| [ReferenceGrant](#referencegrant) | Supported |
1616
| [Custom policies](#custom-policies) | Not supported |
1717

1818
## Terminology
@@ -152,19 +152,18 @@ Fields:
152152
153153
### ReferenceGrant
154154

155-
> Status: Partially supported.
156-
157-
NKG only supports ReferenceGrants that permit Gateways to reference Secrets.
155+
> Status: Supported.
156+
> Support Level: Core
158157
159158
Fields:
160159
* `spec`
161160
* `to`
162161
* `group` - supported.
163-
* `kind` - partially supported. Only `Secret`.
162+
* `kind` - supports `Secret` and `Service`.
164163
* `name`- supported.
165164
* `from`
166165
* `group` - supported.
167-
* `kind` - partially supported. Only `Gateway`.
166+
* `kind` - supports `Gateway` and `HTTPRoute`.
168167
* `namespace`- supported.
169168

170169
### Custom Policies

‎examples/cross-namespace-routing/README.md

+128
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# Example
2+
3+
In this example, we expand on the simple [cafe-example](../cafe-example) by using a ReferenceGrant to route to backends
4+
in a different namespace from our HTTPRoutes.
5+
6+
## Running the Example
7+
8+
## 1. Deploy NGINX Kubernetes Gateway
9+
10+
1. Follow the [installation instructions](/docs/installation.md) to deploy NGINX Gateway.
11+
12+
1. Save the public IP address of NGINX Kubernetes Gateway into a shell variable:
13+
14+
```
15+
GW_IP=XXX.YYY.ZZZ.III
16+
```
17+
18+
1. Save the port of NGINX Kubernetes Gateway:
19+
20+
```
21+
GW_PORT=<port number>
22+
```
23+
24+
## 2. Deploy the Cafe Application
25+
26+
1. Create the cafe namespace and cafe application:
27+
28+
```
29+
kubectl apply -f cafe-ns-and-app.yaml
30+
```
31+
32+
1. Check that the Pods are running in the `cafe` namespace:
33+
34+
```
35+
kubectl -n cafe get pods
36+
NAME READY STATUS RESTARTS AGE
37+
coffee-6f4b79b975-2sb28 1/1 Running 0 12s
38+
tea-6fb46d899f-fm7zr 1/1 Running 0 12s
39+
```
40+
41+
## 3. Configure Routing
42+
43+
1. Create the `Gateway`:
44+
45+
```
46+
kubectl apply -f gateway.yaml
47+
```
48+
49+
1. Create the `HTTPRoute` resources:
50+
51+
```
52+
kubectl apply -f cafe-routes.yaml
53+
```
54+
1. Create the `ReferenceGrant`:
55+
56+
```
57+
kubectl apply -f reference-grant.yaml
58+
```
59+
This ReferenceGrant allows all HTTPRoutes in the `default` namespace to reference all Services in the `cafe`
60+
namespace.
61+
62+
## 4. Test the Application
63+
64+
To access the application, we will use `curl` to send requests to the `coffee` and `tea` Services.
65+
66+
To get coffee:
67+
68+
```
69+
curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/coffee
70+
Server address: 10.12.0.18:80
71+
Server name: coffee-7586895968-r26zn
72+
```
73+
74+
To get tea:
75+
76+
```
77+
curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea
78+
Server address: 10.12.0.19:80
79+
Server name: tea-7cd44fcb4d-xfw2x
80+
```
81+
82+
## 5. Remove the ReferenceGrant
83+
84+
To restrict access to Services in the `cafe` Namespace, we can delete the ReferenceGrant we created in
85+
Step 3:
86+
87+
```
88+
kubectl delete -f reference-grant.yaml
89+
```
90+
91+
Now, if we try to access the application over HTTP, we will get an internal server error:
92+
```
93+
curl --resolve cafe.example.com:$GW_PORT:$GW_IP http://cafe.example.com:$GW_PORT/tea
94+
95+
<html>
96+
<head><title>500 Internal Server Error</title></head>
97+
<body>
98+
<center><h1>500 Internal Server Error</h1></center>
99+
<hr><center>nginx/1.25.1</center>
100+
</body>
101+
</html>
102+
```
103+
104+
You can also check the conditions of the HTTPRoutes `coffee` and `tea` to verify that the reference is not permitted:
105+
106+
```
107+
kubectl describe httproute coffee
108+
109+
Condtions:
110+
Message: Backend ref to Service cafe/coffee not permitted by any ReferenceGrant
111+
Observed Generation: 1
112+
Reason: RefNotPermitted
113+
Status: False
114+
Type: ResolvedRefs
115+
Controller Name: k8s-gateway.nginx.org/nginx-gateway-controller
116+
```
117+
118+
```
119+
kubectl describe httproute tea
120+
121+
Condtions:
122+
Message: Backend ref to Service cafe/tea not permitted by any ReferenceGrant
123+
Observed Generation: 1
124+
Reason: RefNotPermitted
125+
Status: False
126+
Type: ResolvedRefs
127+
Controller Name: k8s-gateway.nginx.org/nginx-gateway-controller
128+
```

‎examples/cross-namespace-routing/cafe-ns-and-app.yaml

+74
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
apiVersion: v1
2+
kind: Namespace
3+
metadata:
4+
name: cafe
5+
---
6+
apiVersion: apps/v1
7+
kind: Deployment
8+
metadata:
9+
name: coffee
10+
namespace: cafe
11+
spec:
12+
replicas: 1
13+
selector:
14+
matchLabels:
15+
app: coffee
16+
template:
17+
metadata:
18+
labels:
19+
app: coffee
20+
spec:
21+
containers:
22+
- name: coffee
23+
image: nginxdemos/nginx-hello:plain-text
24+
ports:
25+
- containerPort: 8080
26+
---
27+
apiVersion: v1
28+
kind: Service
29+
metadata:
30+
name: coffee
31+
namespace: cafe
32+
spec:
33+
ports:
34+
- port: 80
35+
targetPort: 8080
36+
protocol: TCP
37+
name: http
38+
selector:
39+
app: coffee
40+
---
41+
apiVersion: apps/v1
42+
kind: Deployment
43+
metadata:
44+
name: tea
45+
namespace: cafe
46+
spec:
47+
replicas: 1
48+
selector:
49+
matchLabels:
50+
app: tea
51+
template:
52+
metadata:
53+
labels:
54+
app: tea
55+
spec:
56+
containers:
57+
- name: tea
58+
image: nginxdemos/nginx-hello:plain-text
59+
ports:
60+
- containerPort: 8080
61+
---
62+
apiVersion: v1
63+
kind: Service
64+
metadata:
65+
name: tea
66+
namespace: cafe
67+
spec:
68+
ports:
69+
- port: 80
70+
targetPort: 8080
71+
protocol: TCP
72+
name: http
73+
selector:
74+
app: tea

‎examples/cross-namespace-routing/cafe-routes.yaml

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
apiVersion: gateway.networking.k8s.io/v1beta1
2+
kind: HTTPRoute
3+
metadata:
4+
name: coffee
5+
spec:
6+
parentRefs:
7+
- name: gateway
8+
sectionName: http
9+
hostnames:
10+
- "cafe.example.com"
11+
rules:
12+
- matches:
13+
- path:
14+
type: PathPrefix
15+
value: /coffee
16+
backendRefs:
17+
- name: coffee
18+
namespace: cafe
19+
port: 80
20+
---
21+
apiVersion: gateway.networking.k8s.io/v1beta1
22+
kind: HTTPRoute
23+
metadata:
24+
name: tea
25+
spec:
26+
parentRefs:
27+
- name: gateway
28+
sectionName: http
29+
hostnames:
30+
- "cafe.example.com"
31+
rules:
32+
- matches:
33+
- path:
34+
type: Exact
35+
value: /tea
36+
backendRefs:
37+
- name: tea
38+
namespace: cafe
39+
port: 80

‎examples/cross-namespace-routing/gateway.yaml

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: gateway.networking.k8s.io/v1beta1
2+
kind: Gateway
3+
metadata:
4+
name: gateway
5+
labels:
6+
domain: k8s-gateway.nginx.org
7+
spec:
8+
gatewayClassName: nginx
9+
listeners:
10+
- name: http
11+
port: 80
12+
protocol: HTTP
13+
hostname: "*.example.com"

‎examples/cross-namespace-routing/reference-grant.yaml

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: gateway.networking.k8s.io/v1beta1
2+
kind: ReferenceGrant
3+
metadata:
4+
name: access-to-cafe-services
5+
namespace: cafe
6+
spec:
7+
to:
8+
- group: ""
9+
kind: Service
10+
from:
11+
- group: gateway.networking.k8s.io
12+
kind: HTTPRoute
13+
namespace: default

‎examples/https-termination/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,8 @@ curl: (7) Failed to connect to cafe.example.com port 443 after 0 ms: Connection
164164
You can also check the conditions of the Gateway `https` Listener to verify the that the reference is not permitted:
165165

166166
```
167+
kubectl describe gateway gateway
168+
167169
Name: https
168170
Conditions:
169171
Last Transition Time: 2023-06-26T20:23:56Z

‎examples/https-termination/reference-grant.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
apiVersion: gateway.networking.k8s.io/v1beta1
22
kind: ReferenceGrant
33
metadata:
4-
name: allow-default-to-cafe-secret
4+
name: access-to-cafe-secret
55
namespace: certificate
66
spec:
77
to:

0 commit comments

Comments
 (0)