Skip to content

Commit 0deece6

Browse files
authored
Update documents to cover CEL validation on CRDs (#1212)
Update resource validation docs to cover CEL validation. Problem: Gateway API 0.8.0 introduced CEL based-validation and the shift away from webhook validation. Solution: Updated the resource validation docs to cover CEL validation for supported Kubernetes versions and kept webhook validation for older versions.
1 parent fbc7183 commit 0deece6

File tree

2 files changed

+63
-14
lines changed

2 files changed

+63
-14
lines changed

design/resource-validation.md

+29-6
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,20 @@ To help the implementations with the validation, the Gateway API already include
3030
X must be a string) and the contents of the fields (for example, field Y only allows values 'a' and 'b').
3131
Additionally, it enforces the limits like max lengths on field values. Note:
3232
Kubernetes API server enforces this validation. To bypass it, a user needs to change the CRDs.
33-
- *The webhook validation*. This validation is written in go and run as part of the webhook, which is included in the
34-
Gateway API installation files. The validation covers additional logic, not possible to implement in the CRDs. It does
35-
not repeat the validation from the CRDs. Note: a user can bypass this validation if the webhook is not installed.
33+
34+
#### For Kubernetes 1.25+
35+
36+
- *CEL Validation*. This validation is embedded in the Gateway API CRDs and covers additional logic not available in the
37+
OpenAPI schema validation. For example, the field X must be specified when type is set to Y; or X must be nil if
38+
Y is not Z. Note: Kubernetes API server enforces this validation. To bypass it, a user needs to change the CRDs.
39+
40+
#### For Kubernetes 1.23 and 1.24
41+
42+
- *The webhook validation*. This validation is written in go and ran as part of the webhook, which is included in the
43+
Gateway API installation files. The validation covers additional logic, not possible to implement in the OpenAPI
44+
schema validation.
45+
It does not repeat the OpenAPI schema validation from the CRDs. Note: a user can bypass this validation if the webhook
46+
is not installed.
3647

3748
However, the built-in validation rules do not cover all validation needs of NGF:
3849

@@ -109,7 +120,7 @@ If a resource is invalid:
109120

110121
### NGF-specific validation
111122

112-
After re-running the webhook validation, NGF will run NGF-specific validation, written in go.
123+
After re-running the webhook validation, NGF will run NGF-specific validation written in go.
113124

114125
NGF-specific validation will:
115126

@@ -127,16 +138,28 @@ If a resource is invalid, NGF will report the error in its status.
127138

128139
### Summary of Validation
129140

130-
The table below summarizes the validation methods NGF will use. Any Gateway API resource will be validated by the
141+
The tables below summarize the validation methods NGF will use. Any Gateway API resource will be validated by the
131142
following methods in order of their appearance in the table.
132143

144+
#### For Kubernetes 1.25+
145+
146+
| Name | Type | Component | Scope | Feedback loop for errors | Can be bypassed? |
147+
|------------------------------|----------------------------|-----------------------|-------------------------|----------------------------------------------------------------------------------|--------------------------------|
148+
| CRD validation | OpenAPI and CEL validation | Kubernetes API server | Structure, field values | Kubernetes API server returns any errors a response for an API call. | Yes, if the CRDs are modified. |
149+
| Re-run of webhook validation | Go code | NGF control plane | Field values | Errors are reported as Event for the resource. | No |
150+
| NGF-specific validation | Go code | NGF control plane | Field values | Errors are reported in the status of a resource after its creation/modification. | No |
151+
152+
153+
#### For Kubernetes 1.23 and 1.24
154+
133155
| Name | Type | Component | Scope | Feedback loop for errors | Can be bypassed? |
134156
|------------------------------|---------|-----------------------|-------------------------|----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|
135157
| CRD validation | OpenAPI | Kubernetes API server | Structure, field values | Kubernetes API server returns any errors a response for an API call. | Yes, if the CRDs are modified. |
136158
| Webhook validation | Go code | Gateway API webhook | Field values | Kubernetes API server returns any errors a response for an API call. | Yes, if the webhook is not installed, misconfigured, or running a different version. |
137159
| Re-run of webhook validation | Go code | NGF control plane | Field values | Errors are reported as Event for the resource. | No |
138160
| NGF-specific validation | Go code | NGF control plane | Field values | Errors are reported in the status of a resource after its creation/modification. | No |
139161

162+
140163
Notes:
141164

142165
- The amount and the extent of the validation should allow multiple application developers to share a single NGF (User
@@ -148,7 +171,7 @@ Notes:
148171

149172
NGF will support more resources:
150173

151-
- More Gateway API resources. For those, NGF will use the four validation methods from the table in the previous
174+
- More Gateway API resources. For those, NGF will use the validation methods from the table in the previous
152175
section.
153176
- Introduce NGF resources. For those, NGF will use CRD validation (the rules of which are fully controlled by us). The
154177
CRD validation will include the validation to prevent invalid NGINX configuration values and malicious values. Because

docs/resource-validation.md

+34-8
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,15 @@ created.
1515

1616
A Gateway API resource (a new resource or an update for the existing one) is validated by the following steps:
1717

18+
### For Kubernetes 1.25+
19+
20+
1. OpenAPI schema validation by the Kubernetes API server.
21+
2. CEL validation by the Kubernetes API server.
22+
3. Webhook validation by NGF.
23+
4. Validation by NGF.
24+
25+
### For Kubernetes 1.23 and 1.24
26+
1827
1. OpenAPI schema validation by the Kubernetes API server.
1928
2. Webhook validation by the Gateway API webhook.
2029
3. Webhook validation by NGF.
@@ -64,35 +73,52 @@ The HTTPRoute "coffee" is invalid: spec.hostnames[0]: Invalid value: "cafe.!@#$%
6473
> While unlikely, bypassing this validation step is possible if the Gateway API CRDs are modified to remove the validation.
6574
> If this happens, Step 4 will reject any invalid values (from NGINX perspective).
6675
67-
### Step 2 - Webhook Validation by Gateway API Webhook
76+
### Step 2 - For Kubernetes 1.25+ - CEL Validation by Kubernetes API Server
77+
78+
The Kubernetes API server validates Gateway API resources using CEL validation embedded in the Gateway API CRDs.
79+
It validates Gateway API resources using advanced rules unavailable in the OpenAPI schema validation.
80+
For example, if you create a Gateway resource with a TCP listener that configures a hostname, the CEL validation will
81+
reject it with the following error:
82+
83+
84+
```shell
85+
kubectl apply -f some-gateway.yaml
86+
```
87+
88+
```text
89+
The Gateway "some-gateway" is invalid: spec.listeners: Invalid value: "array": hostname must not be specified for protocols ['TCP', 'UDP']
90+
```
91+
92+
More information on CEL in Kubernetes can be found [here](https://kubernetes.io/docs/reference/using-api/cel/).
93+
94+
### Step 2 - For Kubernetes 1.23 and 1.24 - Webhook Validation by Gateway API Webhook
6895

6996
The Gateway API comes with a validating webhook which is enabled by default in the Gateway API installation manifests.
7097
It validates Gateway API resources using advanced rules unavailable in the OpenAPI schema validation. For example, if
7198
you create a Gateway resource with a TCP listener that configures a hostname, the webhook will reject it with the
7299
following error:
73100

74101
```shell
75-
kubectl apply -f prod-gateway.yaml
102+
kubectl apply -f some-gateway.yaml
76103
```
77104

78105
```text
79-
Error from server: error when creating "prod-gateway.yaml": admission webhook "validate.gateway.networking.k8s.io" denied the request: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP
106+
Error from server: error when creating "some-gateway.yaml": admission webhook "validate.gateway.networking.k8s.io" denied the request: spec.listeners[1].hostname: Forbidden: should be empty for protocol TCP
80107
```
81108

82109
> Bypassing this validation step is possible if the webhook is not running in the cluster.
83110
> If this happens, Step 3 will reject the invalid values.
84111
85112
### Step 3 - Webhook validation by NGF
86-
87-
The previous step relies on the Gateway API webhook running in the cluster. To ensure that the resources are validated
88-
with the webhook validation rules, even if the webhook is not running, NGF performs the same validation. However, NGF
89-
performs the validation *after* the Kubernetes API server accepts the resource.
113+
To ensure that the resources are validated with the webhook validation rules, even if the webhook is not running,
114+
NGF performs the same validation. However, NGF performs the validation *after* the Kubernetes API server accepts
115+
the resource.
90116

91117
Below is an example of how NGF rejects an invalid resource (a Gateway resource with a TCP listener that configures a
92118
hostname) with a Kubernetes event:
93119

94120
```shell
95-
kubectl describe gateway prod-gateway
121+
kubectl describe gateway some-gateway
96122
```
97123

98124
```text

0 commit comments

Comments
 (0)