registry addon broken for consumption within cluster

[castlemilk]

Due to the change enabling the docker registry addon to be consumed externally via the PRs #4341 and #4529, it will result on the registry Service to load balance across both the registry-proxy DaemonSet and the registry ReplicaController, consequently resulting in 50% of requests failing when originating within this cluster.

This gives rise to the issue described in #4604. A common pattern used when using technology like Tekton, running within your minikube cluster, where Kaniko needs to cache and/or push images to the internal registry.

The exact command to reproduce the issue:

kubectl run test-curl --image=castlemilk/curl --restart=Never --rm curl registry.kube-system.svc.cluster.local -vvv

The full output of the command that failed:

SUCCESS REQUEST

*   Trying 10.107.6.165:80...
* TCP_NODELAY set
* Connected to registry.kube-system.svc.cluster.local (10.107.6.165) port 80 (#0)
> POST / HTTP/1.1
> Host: registry.kube-system.svc.cluster.local
> User-Agent: curl/7.65.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Cache-Control: no-cache
< Date: Tue, 25 Jun 2019 11:44:10 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8
< 
* Connection #0 to host registry.kube-system.svc.cluster.local left intact

FAILED REQUEST

/ # curl -XPOST registry.kube-system.svc.cluster.local -vvv
*   Trying 10.107.6.165:80...
* TCP_NODELAY set
* connect to 10.107.6.165 port 80 failed: Connection refused
* Failed to connect to registry.kube-system.svc.cluster.local port 80: Connection refused
* Closing connection 0

The output of the minikube logs command:
NA

The operating system version:
minikube 1.2.0

Resolution

Potential options for resolution:

1. remove the label selector from the DaemonSet, which is being used by the Service, as it isn't needed anyway, because the DaemonSet is used for proxying external access.
2. add a flag that only enables the DaemonSet registry-proxy when explicitly specified