Skip to content

Commit

Permalink
Add ingress-dns addon
Browse files Browse the repository at this point in the history
  • Loading branch information
Josh Woodcock committed Oct 1, 2019
1 parent 7d59b0d commit a0d0d02
Show file tree
Hide file tree
Showing 9 changed files with 662 additions and 0 deletions.
6 changes: 6 additions & 0 deletions cmd/minikube/cmd/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,12 @@ var settings = []Setting{
validations: []setFn{IsValidAddon},
callbacks: []setFn{EnableOrDisableAddon},
},
{
name: "ingress-dns",
set: SetBool,
validations: []setFn{IsValidAddon},
callbacks: []setFn{EnableOrDisableAddon},
},
{
name: "hyperv-virtual-switch",
set: SetString,
Expand Down
143 changes: 143 additions & 0 deletions deploy/addons/ingress-dns/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# Minikube Ingress DNS
![Build Status](https://gitlab.com/cryptexlabs/public/development/minikube-ingress-dns/badges/master/pipeline.svg)

DNS service for ingress controllers running on your minikube server

## Overview

### Problem
When running minikube locally you are highly likely to want to run your services on an ingress controller so that you
don't have to use minikube tunnel or NodePorts to access your services. While NodePort might be ok in a lot of
circumstances in order to test some features an ingress is necessary. Ingress controllers are great because you can
define your entire architecture in something like a helm chart and all your services will be available. There is only
1 problem. That is that your ingress controller works basically off of dns and while running minikube that means that
your local dns names like `myservice.test` will have to resolve to `$(minikube ip)` not really a big deal except the
only real way to do this is to add an entry for every service in your `/etc/hosts` file. This gets messy for obvious
reasons. If you have a lot of services running that each have their own dns entry then you have to set those up
manually. Even if you automate it you then need to rely on the host operating system storing configurations instead of
storing them in your cluster. To make it worse it has to be constantly maintained and updated as services are added,
remove, and renamed. I call it the `/ets/hosts` pollution problem.

### Solution
What if you could just access your local services magically without having to edit your `/etc/hosts` file? Well now you
can. This project acts as a DNS service that runs inside your kubernetes cluster. All you have to do is install the
service and add the `$(minikube ip)` as a DNS server on your host machine. Each time the dns service is queried an
API call is made to the kubernetes master service for a list of all the ingresses. If a match is found for the name a
response is given with an IP address as the `$(minikube ip)` which was provided when the helm chart was installed. So
for example lets say my minikube ip address is `192.168.99.106` and I have an ingress controller with the name of
`myservice.test` then I would get a result like so:

```text
#bash:~$ nslookup myservice.test $(minikube ip)
Server: 192.168.99.169
Address: 192.168.99.169#53
Non-authoritative answer:
Name: myservice.test $(minikube ip)
Address: 192.168.99.169
```

## Installation

### Start minikube
```
minikube start
```

### Install the kubernetes resources
```bash
minikube addons enable ingress-dns
```

### Add the minikube ip as a dns server

#### Mac OS
Create a file in /etc/resolver/minikube-profilename-test
```
domain test
nameserver 192.168.99.169
search_order 1
timeout 5
```
Replace `192.168.99.169` with your minikube ip and `profilename` is the name of the minikube profile for the
corresponding ip address

#### Linux

TODO

## Testing

### Add the test ingress
```bash
kubectl apply -f https://github.com/kubernetes/minikube/blob/master/deploy/addons/ingress-dns/example/example.yaml
```
Note: Minimum Kubernetes version for example ingress is 1.14.7

### Validate DNS queries are returning A records
```bash
nslookup hello-john.test $(minikube ip)
nslookup hello-jane.test $(minikube ip)
```

### Validate domain names are resolving on host OS
```bash
ping hello-john.test
```
Expected results:
```text
PING hello-john.test (192.168.99.169): 56 data bytes
64 bytes from 192.168.99.169: icmp_seq=0 ttl=64 time=0.361 ms
```
```bash
ping hello-jane.test
```
```text
PING hello-jane.test (192.168.99.169): 56 data bytes
64 bytes from 192.168.99.169: icmp_seq=0 ttl=64 time=0.262 ms
```

### Curl the example server
```bash
curl http://hello-john.test
```
Expected result:
```text
Hello, world!
Version: 1.0.0
Hostname: hello-world-app-557ff7dbd8-64mtv
```
```bash
curl http://hello-jane.test
```
Expected result:
```text
Hello, world!
Version: 1.0.0
Hostname: hello-world-app-557ff7dbd8-64mtv
```

## Known issues

### .localhost domains will not resolve on chromium
.localhost domains will not correctly resolve on chromium since it is used as a loopback address. Instead use .test, .example, or .invalid

### .local is a reserved TLD
Do not use .local as this is a reserved TLD for mDNS and bind9 DNS servers

### Mac OS

#### mDNS reloading
Each time a file is created or a change is made to a file in `/etc/resolver` you may need to run the following to reload Mac OS mDNS resolver.
```bash
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist
```

## TODO
- Add a service that runs on the host OS which will update the files in `/etc/resolver` automatically
- Start this service when running `minikube addons enable ingress-dns` and stop the service when running
`minikube addons disable ingress-dns`

## Contributors
- [Josh Woodcock](https://github.com/woodcockjosh)
67 changes: 67 additions & 0 deletions deploy/addons/ingress-dns/example/example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world-app
namespace: default
spec:
selector:
matchLabels:
app: hello-world-app
template:
metadata:
labels:
app: hello-world-app
spec:
containers:
- name: hello-world-app
image: gcr.io/google-samples/hello-app:1.0
ports:
- containerPort: 8080
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: example-ingress
namespace: kube-system
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-john.test
http:
paths:
- path: /|/(.+)
backend:
serviceName: hello-world-app
servicePort: 80
- host: hello-jane.test
http:
paths:
- path: /|/(.+)
backend:
serviceName: hello-world-app
servicePort: 80
---
apiVersion: v1
kind: Service
metadata:
name: hello-world-app
namespace: kube-system
spec:
type: ExternalName
externalName: hello-world-app.default.svc.cluster.local
---
apiVersion: v1
kind: Service
metadata:
name: hello-world-app
namespace: default
spec:
ports:
- name: http
port: 80
targetPort: 8080
protocol: TCP
type: NodePort
selector:
app: hello-world-app
51 changes: 51 additions & 0 deletions deploy/addons/ingress-dns/ingress-dns-configmap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Copyright 2016 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

---
apiVersion: v1
data:
map-hash-bucket-size: "128"
hsts: "false"
kind: ConfigMap
metadata:
name: minikube-ingress-dns-nginx-load-balancer-conf
namespace: kube-system
labels:
app: minikube-ingress-dns
app.kubernetes.io/part-of: kube-system
addonmanager.kubernetes.io/mode: EnsureExists
---
apiVersion: v1
kind: ConfigMap
metadata:
name: minikube-ingress-dns-tcp-services
namespace: kube-system
labels:
app: minikube-ingress-dns
app.kubernetes.io/part-of: kube-system
addonmanager.kubernetes.io/mode: EnsureExists
data:
53: "kube-system/kube-ingress-dns-minikube:5353"
---
apiVersion: v1
kind: ConfigMap
metadata:
name: minikube-ingress-dns-udp-services
namespace: kube-system
labels:
app: minikube-ingress-dns
app.kubernetes.io/part-of: kube-system
addonmanager.kubernetes.io/mode: EnsureExists
data:
53: "kube-system/kube-ingress-dns-minikube:5353"
102 changes: 102 additions & 0 deletions deploy/addons/ingress-dns/ingress-dns-dns-server-pod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# Copyright 2016 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

---
apiVersion: v1
kind: ServiceAccount
metadata:
name: minikube-ingress-dns
namespace: kube-system
labels:
app: minikube-ingress-dns
kubernetes.io/bootstrapping: rbac-defaults
app.kubernetes.io/part-of: kube-system
addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: minikube-ingress-dns
namespace: kube-system
labels:
app: minikube-ingress-dns
kubernetes.io/bootstrapping: rbac-defaults
app.kubernetes.io/part-of: kube-system
addonmanager.kubernetes.io/mode: Reconcile
rules:
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- patch
resourceNames:
- tcp-services
- udp-services
- apiGroups:
- ""
- "extensions"
- "networking.k8s.io"
resources:
- ingresses
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: minikube-ingress-dns
namespace: kube-system
labels:
app: minikube-ingress-dns
kubernetes.io/bootstrapping: rbac-defaults
app.kubernetes.io/part-of: kube-system
addonmanager.kubernetes.io/mode: EnsureExists
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: minikube-ingress-dns
namespace: kube-system
---
apiVersion: v1
kind: Pod
metadata:
name: kube-ingress-dns-minikube
namespace: kube-system
labels:
app: minikube-ingress-dns
app.kubernetes.io/part-of: kube-system
addonmanager.kubernetes.io/mode: EnsureExists
spec:
serviceAccountName: minikube-ingress-dns
containers:
- name: minikube-ingress-dns
image: "cryptexlabs/minikube-ingress-dns:0.1.1"
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5353
hostPort: 5353
protocol: TCP
- containerPort: 5353
hostPort: 5353
protocol: UDP
env:
- name: DNS_PORT
value: "5353"
Loading

0 comments on commit a0d0d02

Please sign in to comment.