Skip to content

Commit 9ed0d7f

Browse files
authored
Separate third party NGINX configuration (#10470)
* Document container separation * Separate configurations
1 parent d6a0f46 commit 9ed0d7f

File tree

19 files changed

+158
-36
lines changed

19 files changed

+158
-36
lines changed
Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Proposal to split containers
2+
3+
* All the NGINX files should live on one container
4+
* No file other than NGINX files should exist on this container
5+
* This includes not mounting the service account
6+
* All the controller files should live on a different container
7+
* Controller container should have bare minimum to work (just go program)
8+
* ServiceAccount should be mounted just on controller
9+
10+
* Inside nginx container, there should be a really small http listener just able
11+
to start, stop and reload NGINX
12+
13+
## Roadmap (what needs to be done)
14+
* Map what needs to be done to mount the SA just on controller container
15+
* Map all the required files for NGINX to work
16+
* Map all the required network calls between controller and NGINX
17+
* eg.: Dynamic lua reconfiguration
18+
* Map problematic features that will need attention
19+
* SSLPassthrough today happens on controller process and needs to happen on NGINX
20+
21+
### Ports and endpoints on NGINX container
22+
* Public HTTP/HTTPs port - 80 and 443
23+
* Lua configuration port - 10246 (HTTP) and 10247 (Stream)
24+
* 3333 (temp) - Dataplane controller http server
25+
* /reload - (POST) Reloads the configuration.
26+
* "config" argument is the location of temporary file that should be used / moved to nginx.conf
27+
* /test - (POST) Test the configuration of a given file location
28+
* "config" argument is the location of temporary file that should be tested
29+
30+
### Mounting empty SA on controller container
31+
32+
```yaml
33+
kind: Pod
34+
apiVersion: v1
35+
metadata:
36+
name: test
37+
spec:
38+
containers:
39+
- name: nginx
40+
image: nginx:latest
41+
ports:
42+
- containerPort: 80
43+
- name: othernginx
44+
image: alpine:latest
45+
command: ["/bin/sh"]
46+
args: ["-c", "while true; do date; sleep 3; done"]
47+
volumeMounts:
48+
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
49+
name: emptysecret
50+
volumes:
51+
- name: emptysecret
52+
emptyDir:
53+
sizeLimit: 1Mi
54+
```
55+
56+
### Mapped folders on NGINX configuration
57+
**WARNING** We need to be aware of inter mount containers and inode problems. If we
58+
mount a file instead of a directory, it may take time to reflect the file value on
59+
the target container
60+
61+
* "/etc/nginx/lua/?.lua;/etc/nginx/lua/vendor/?.lua;;"; - Lua scripts
62+
* "/var/log/nginx" - NGINX logs
63+
* "/tmp/nginx (nginx.pid)" - NGINX pid directory / file, fcgi socket, etc
64+
* " /etc/nginx/geoip" - GeoIP database directory - OK - /etc/ingress-controller/geoip
65+
* /etc/nginx/mime.types - Mime types
66+
* /etc/ingress-controller/ssl - SSL directory (fake cert, auth cert)
67+
* /etc/ingress-controller/auth - Authentication files
68+
* /etc/nginx/modsecurity - Modsecurity configuration
69+
* /etc/nginx/owasp-modsecurity-crs - Modsecurity rules
70+
* /etc/nginx/tickets.key - SSL tickets - OK - /etc/ingress-controller/tickets.key
71+
* /etc/nginx/opentelemetry.toml - OTEL config - OK - /etc/ingress-controller/telemetry
72+
* /etc/nginx/opentracing.json - Opentracing config - OK - /etc/ingress-controller/telemetry
73+
* /etc/nginx/modules - NGINX modules
74+
* /etc/nginx/fastcgi_params (maybe) - fcgi params
75+
* /etc/nginx/template - Template, may be used by controller only
76+
77+
##### List of modules
78+
```
79+
ngx_http_auth_digest_module.so ngx_http_modsecurity_module.so
80+
ngx_http_brotli_filter_module.so ngx_http_opentracing_module.so
81+
ngx_http_brotli_static_module.so ngx_stream_geoip2_module.so
82+
ngx_http_geoip2_module.so
83+
```
84+
85+
##### List of files that may be removed
86+
```
87+
-rw-r--r-- 1 www-data www-data 1077 Jun 23 19:44 fastcgi.conf
88+
-rw-r--r-- 1 www-data www-data 1077 Jun 23 19:44 fastcgi.conf.default
89+
-rw-r--r-- 1 www-data www-data 1007 Jun 23 19:44 fastcgi_params
90+
-rw-r--r-- 1 www-data www-data 1007 Jun 23 19:44 fastcgi_params.default
91+
drwxr-xr-x 2 www-data www-data 4096 Jun 23 19:34 geoip
92+
-rw-r--r-- 1 www-data www-data 2837 Jun 23 19:44 koi-utf
93+
-rw-r--r-- 1 www-data www-data 2223 Jun 23 19:44 koi-win
94+
drwxr-xr-x 6 www-data www-data 4096 Sep 19 14:13 lua
95+
-rw-r--r-- 1 www-data www-data 5349 Jun 23 19:44 mime.types
96+
-rw-r--r-- 1 www-data www-data 5349 Jun 23 19:44 mime.types.default
97+
drwxr-xr-x 2 www-data www-data 4096 Jun 23 19:44 modsecurity
98+
drwxr-xr-x 2 www-data www-data 4096 Jun 23 19:44 modules
99+
-rw-r--r-- 1 www-data www-data 18275 Oct 1 21:28 nginx.conf
100+
-rw-r--r-- 1 www-data www-data 2656 Jun 23 19:44 nginx.conf.default
101+
-rwx------ 1 www-data www-data 420 Oct 1 21:28 opentelemetry.toml
102+
-rw-r--r-- 1 www-data www-data 2 Oct 1 21:28 opentracing.json
103+
drwxr-xr-x 7 www-data www-data 4096 Jun 23 19:44 owasp-modsecurity-crs
104+
-rw-r--r-- 1 www-data www-data 636 Jun 23 19:44 scgi_params
105+
-rw-r--r-- 1 www-data www-data 636 Jun 23 19:44 scgi_params.default
106+
drwxr-xr-x 2 www-data www-data 4096 Sep 19 14:13 template
107+
-rw-r--r-- 1 www-data www-data 664 Jun 23 19:44 uwsgi_params
108+
-rw-r--r-- 1 www-data www-data 664 Jun 23 19:44 uwsgi_params.default
109+
-rw-r--r-- 1 www-data www-data 3610 Jun 23 19:44 win-utf
110+
```

docs/user-guide/nginx-configuration/configmap.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,7 @@ The following table shows a configuration option's name, type, and the default v
164164
|[enable-opentelemetry](#enable-opentelemetry)|bool|"false"||
165165
|[opentelemetry-trust-incoming-span](#opentelemetry-trust-incoming-span)|bool|"true"||
166166
|[opentelemetry-operation-name](#opentelemetry-operation-name)|string|""||
167-
|[opentelemetry-config](#/etc/nginx/opentelemetry.toml)|string|"/etc/nginx/opentelemetry.toml"||
167+
|[opentelemetry-config](#/etc/ingress-controller/telemetry/opentelemetry.toml)|string|"/etc/ingress-controller/telemetry/opentelemetry.toml"||
168168
|[otlp-collector-host](#otlp-collector-host)|string|""||
169169
|[otlp-collector-port](#otlp-collector-port)|int|4317||
170170
|[otel-max-queuesize](#otel-max-queuesize)|int|||

docs/user-guide/third-party-addons/opentelemetry.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,7 @@ To install the example and collectors run:
165165
kind: ConfigMap
166166
data:
167167
enable-opentelemetry: "true"
168-
opentelemetry-config: "/etc/nginx/opentelemetry.toml"
168+
opentelemetry-config: "/etc/ingress-controller/telemetry/opentelemetry.toml"
169169
opentelemetry-operation-name: "HTTP $request_method $service_name $uri"
170170
opentelemetry-trust-incoming-span: "true"
171171
otlp-collector-host: "otel-coll-collector.otel.svc"

go.work.sum

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,7 @@ github.com/klauspost/asmfmt v1.3.2/go.mod h1:AG8TuvYojzulgDAMCnYn50l/5QV3Bs/tp6j
186186
github.com/klauspost/compress v1.15.9/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
187187
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
188188
github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80/go.mod h1:imJHygn/1yfhB7XSJJKlFZKl/J+dCPAknuiaGOshXAs=
189+
github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
189190
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8/go.mod h1:mC1jAcsrzbxHt8iiaC+zU4b1ylILSosueou12R++wfY=
190191
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3/go.mod h1:RagcQ7I8IeTMnF8JTXieKnO4Z6JCsikNEzj0DwauVzE=
191192
github.com/moby/term v0.0.0-20221205130635-1aeaba878587/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
@@ -199,6 +200,7 @@ github.com/onsi/gomega v1.27.4/go.mod h1:riYq/GJKh8hhoM01HN6Vmuy93AarCXCBGpvFDK3
199200
github.com/opencontainers/selinux v1.10.0/go.mod h1:2i0OySw99QjzBBQByd1Gr9gSjvuho1lHsJxIJ3gGbJI=
200201
github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
201202
github.com/pierrec/lz4/v4 v4.1.15/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
203+
github.com/pquerna/cachecontrol v0.1.0/go.mod h1:NrUG3Z7Rdu85UNR3vm7SOsl1nFIeSiQnrHV5K9mBcUI=
202204
github.com/seccomp/libseccomp-golang v0.9.2-0.20220502022130-f33da4d89646/go.mod h1:JA8cRccbGaA1s33RQf7Y1+q9gHmZX1yB/z9WDN1C6fg=
203205
github.com/spf13/afero v1.3.3/go.mod h1:5KUK8ByomD5Ti5Artl0RtHeI5pTF7MIDuXL3yY520V4=
204206
github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww=
@@ -254,6 +256,7 @@ golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBc
254256
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
255257
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
256258
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
259+
golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
257260
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
258261
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
259262
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -284,6 +287,7 @@ google.golang.org/genproto v0.0.0-20230530153820-e85fd2cbaebc/go.mod h1:xZnkP7mR
284287
google.golang.org/genproto v0.0.0-20230706204954-ccb25ca9f130/go.mod h1:O9kGHb51iE/nOGvQaDUuadVYqovW56s5emA88lQnj6Y=
285288
google.golang.org/genproto v0.0.0-20230711160842-782d3b101e98 h1:Z0hjGZePRE0ZBWotvtrwxFNrNE9CUAGtplaDK5NNI/g=
286289
google.golang.org/genproto v0.0.0-20230711160842-782d3b101e98/go.mod h1:S7mY02OqCJTD0E1OiQy1F72PWFB4bZJ87cAtLPYgDR0=
290+
google.golang.org/genproto v0.0.0-20230822172742-b8732ec3820d h1:VBu5YqKPv6XiJ199exd8Br+Aetz+o08F+PLMnwJQHAY=
287291
google.golang.org/genproto/googleapis/api v0.0.0-20230530153820-e85fd2cbaebc/go.mod h1:vHYtlOoi6TsQ3Uk2yxR7NI5z8uoV+3pZtR4jmHIkRig=
288292
google.golang.org/genproto/googleapis/api v0.0.0-20230706204954-ccb25ca9f130/go.mod h1:mPBs5jNgx2GuQGvFwUvVKqtn6HsUw9nP64BedgvqEsQ=
289293
google.golang.org/genproto/googleapis/api v0.0.0-20230711160842-782d3b101e98/go.mod h1:rsr7RhLuwsDKL7RmgDDCUc6yaGr1iqceVb5Wv6f6YvQ=

images/kube-webhook-certgen/rootfs/go.mod

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ require (
2626
github.com/google/go-cmp v0.5.9 // indirect
2727
github.com/google/gofuzz v1.1.0 // indirect
2828
github.com/google/pprof v0.0.0-20230926050212-f7f687d19a98 // indirect
29-
github.com/google/uuid v1.3.0 // indirect
29+
github.com/google/uuid v1.3.1 // indirect
3030
github.com/imdario/mergo v0.3.6 // indirect
3131
github.com/inconshreveable/mousetrap v1.1.0 // indirect
3232
github.com/josharian/intern v1.0.0 // indirect

images/kube-webhook-certgen/rootfs/go.sum

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -202,8 +202,8 @@ github.com/google/pprof v0.0.0-20230926050212-f7f687d19a98/go.mod h1:czg5+yv1E0Z
202202
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
203203
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
204204
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
205-
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
206-
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
205+
github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
206+
github.com/google/uuid v1.3.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
207207
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
208208
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
209209
github.com/googleapis/gnostic v0.5.1/go.mod h1:6U4PtQXGIEt/Z3h5MAT7FNofLnw9vXk2cUuW7uA/OeU=

internal/ingress/controller/config/config.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -987,7 +987,7 @@ func NewDefault() Configuration {
987987
BindAddressIpv6: defBindAddress,
988988
OpentracingTrustIncomingSpan: true,
989989
OpentelemetryTrustIncomingSpan: true,
990-
OpentelemetryConfig: "/etc/nginx/opentelemetry.toml",
990+
OpentelemetryConfig: "/etc/ingress-controller/telemetry/opentelemetry.toml",
991991
OtlpCollectorPort: "4317",
992992
OtelServiceName: "nginx",
993993
OtelSampler: "AlwaysOn",

internal/ingress/controller/nginx.go

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import (
2222
"encoding/json"
2323
"errors"
2424
"fmt"
25+
"io/fs"
2526
"net"
2627
"net/http"
2728
"os"
@@ -180,7 +181,11 @@ func NewNGINXController(config *Configuration, mc metric.Collector) *NGINXContro
180181
}
181182

182183
filesToWatch := []string{}
183-
err = filepath.Walk("/etc/nginx/geoip/", func(path string, info os.FileInfo, err error) error {
184+
185+
if err := os.Mkdir("/etc/ingress-controller/geoip/", 0o755); err != nil && !os.IsExist(err) {
186+
klog.Fatalf("Error creating geoip dir: %v", err)
187+
}
188+
err = filepath.WalkDir("/etc/ingress-controller/geoip/", func(path string, info fs.DirEntry, err error) error {
184189
if err != nil {
185190
return err
186191
}
@@ -1107,7 +1112,7 @@ func createOpentracingCfg(cfg *ngx_config.Configuration) error {
11071112
// Expand possible environment variables before writing the configuration to file.
11081113
expanded := os.ExpandEnv(configData)
11091114

1110-
return os.WriteFile("/etc/nginx/opentracing.json", []byte(expanded), file.ReadWriteByUser)
1115+
return os.WriteFile("/etc/ingress-controller/telemetry/opentracing.json", []byte(expanded), file.ReadWriteByUser)
11111116
}
11121117

11131118
func createOpentelemetryCfg(cfg *ngx_config.Configuration) error {

internal/ingress/controller/store/store.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1215,7 +1215,7 @@ func (s *k8sStore) setConfig(cmap *corev1.ConfigMap) {
12151215
s.backendConfig.UseGeoIP2 = false
12161216
}
12171217

1218-
s.writeSSLSessionTicketKey(cmap, "/etc/nginx/tickets.key")
1218+
s.writeSSLSessionTicketKey(cmap, "/etc/ingress-controller/tickets.key")
12191219
}
12201220

12211221
// Run initiates the synchronization of the informers and the initial

internal/ingress/controller/template/template.go

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1230,11 +1230,11 @@ func buildOpentracing(c, s interface{}) string {
12301230

12311231
//nolint:gocritic // rewriting if-else to switch statement is not more readable
12321232
if cfg.DatadogCollectorHost != "" {
1233-
buf.WriteString("opentracing_load_tracer /usr/local/lib/libdd_opentracing.so /etc/nginx/opentracing.json;")
1233+
buf.WriteString("opentracing_load_tracer /usr/local/lib/libdd_opentracing.so /etc/ingress-controller/telemetry/opentracing.json;")
12341234
} else if cfg.ZipkinCollectorHost != "" {
1235-
buf.WriteString("opentracing_load_tracer /usr/local/lib/libzipkin_opentracing_plugin.so /etc/nginx/opentracing.json;")
1235+
buf.WriteString("opentracing_load_tracer /usr/local/lib/libzipkin_opentracing_plugin.so /etc/ingress-controller/telemetry/opentracing.json;")
12361236
} else if cfg.JaegerCollectorHost != "" || cfg.JaegerEndpoint != "" {
1237-
buf.WriteString("opentracing_load_tracer /usr/local/lib/libjaegertracing_plugin.so /etc/nginx/opentracing.json;")
1237+
buf.WriteString("opentracing_load_tracer /usr/local/lib/libjaegertracing_plugin.so /etc/ingress-controller/telemetry/opentracing.json;")
12381238
}
12391239

12401240
buf.WriteString("\r\n")

0 commit comments

Comments
 (0)