Cannoot access services via NodePort on MacOS with Docker driver #11193
Comments
|
This is a known issue, Docker Desktop networking doesn't support ports. You will have to use minikube tunnel. https://docs.docker.com/docker-for-mac/networking/#known-limitations-use-cases-and-workarounds https://minikube.sigs.k8s.io/docs/handbook/accessing/ You could also run minikube in a VM, to get a proper IP. Such a driver can be selected with |
afbjorklund
added
co/docker-driver
minikube publishes the ssh port automatically
There are other solutions to set up tunneling and networking, but nothing that is provided out-of-the-box. |
|
This issue is similar to #9404
test.yml
@afbjorklund |
|
@zhan9san But #11070 (and #8008) is about exporting the API server outside of localhost ? Using docker publishing for apps is not recommended, because it requires restarting the container. I'm personally looking into NoRouter... It would offer a HTTP proxy, for the cluster. |
Minikube provides us a very convenient way to verify the application in localhost/development, |
|
I agree with you, Regarding NoRouter, it's a great tool providing mutual interconnectivity across multiple remote hosts. Besides, using a one-time pod to access k8s cluster running in container is another choice. |
That's just how the docker publish works, it only takes affect when the container is started. So when you add a new app, the new NodePort would have to be added to the publish list. |
|
@afbjorklund My opinion is that a k8s cluster created by minikube is ephemeral, that's to say, once quick verification is done, destroy cluster as well. The scenario you mentioned will not exist. Here is a bold idea, how about adding |
That just means to publish all ports that are exposed in the container image |
|
You are right. Let's focus on the root cause, different from the Linux system, there is no bridge0 on Macos, and it makes container IP unreachable from host. Fortunately, we can define the port range in both apiserver and docker publishing at the same time. Please see sample below. The small range is for performance. |
|
I don't want to open more ports, and I want to close down 2376 and 5000 whenever possible...
This is not only an issue for the Docker Desktop drivers (Mac and Win), but also the SSH driver. For now, it's a known shortcoming and it is being addressed by It could definitely be documented better:
Also interesting why tunneling doesn't work:
|
|
Note that you can set up your own ssh tunnels for
There should probably be a command for it. Or at least some options for |
|
Is this feature in the plan? |
|
Hi
Please refer to minikube makes it easy to open this exposed endpoint in your browser: |
|
Disregard that mention, wrong issue number... |
|
Hello, For me, what worked was this solution by zhan9san: minikube start --driver=docker --extra-config=apiserver.service-node-port-range=32760-32767 --ports=127.0.0.1:32760-32767:32760-32767 This brings the services to 127.0.0.1 on Mac OS. Although not ideal, it is the closest we can get, using then the 127.0.0.1 host ip instead of the minikube ip. Can I close the issue? I think someone else looking for help and reaching here will probably have a good idea of wat can be Done. Thanks guys. |
|
I'm glad it works for you. Feel free to close it. |
|
Hello, Can you check #11577 please? It is kind of the same thing but now itt's happening on Linux, after some updates. I will close this one. |
|
What worked for me was running |
|
Are the solutions in this post still valid? I've tried every single one of them and as of this writing, I get only errors. Would the fact that I'm using an Apple M1 chip be a factor? In any case, here is what I'm trying with errors: There is no build of virtualbox for M1 chips, so using that driver is not an option. The hyperkit driver shows all the same errors as the docker driver. To this day I have never been able to get a web app to display on an Apple M1 no matter what directions or tutorials I follow. If you have successfully done so, can someone please provide step by step instructions on how to do so? Interestingly enough, I can see the dashboard at 127.0.0.1:51301, however. Tried: Error: Tried: Errors:
trying the NodePort CLUSTER-IP at both ports; tryng 127.0.0.1 at both ports; double checking that the NodePort, Port, and TargetPort in the yaml file are correct, and trying the 192.168.49.2 address with all referenced ports also does not work. Tried: Finally, trying the alternative to use kubectl port fowarding per the docs also gives an error: What am I doing wrong? |
|
Hi @blue928
Does the
ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType.
What are the
Besides, the docker container network is not directly reachable on MacOS, so a tunnel is created by Hope I express myself clearly. |
|
In order to get a minimum reproducible scenario, would you mind running some basic test and sending me the result? Terminal 1 Note: Terminal 2 Add the following line to the bottom of the /etc/hosts file. |
|
Terminal 1: Terminal 2: Terminal 1: (automatic response) Terminal 2: A curl response!! Yes! Why does this work? I also notice the |
You can find detailed info from Support Ingress on MacOS, driver docker and the comments in this issue.
No, it couldn't. The above sample means both As you mentioned If |
|
@zhan9san Thank you so much! Turning on the Ingress Controller and adding an Ingress Resource have been the missing elements. Once I started playing with this in the wild I had one more gotcha that may help others referencing this.
Without the Selector field, the Service will not automatically create the endpoints necessary for traffic to make it to its final destination. The The takeaway is if you're getting 502, 503 errors, double check that your Deployment has a
References: |
|
for m1 users, when starting minikube run this command minikube start --ports=port-target:node-port for eg; minikube start --ports=30000:30000 and for exposing multiple ports, separate them by comma, eg; minikube start --ports=30000:30000,32000:32000 and now you access the pod via: http://localhost:30000 |
|
Is it no longer possible to expose a range of ports using the --ports flag? (CC: @toonvanstrijp) Using the example provided by @marlon-sousa above:
on minikube v1.25.2, yields the following error: `😄 minikube v1.25.2 on Darwin 12.3.1 (arm64) ❌ Exiting due to MK_USAGE: Sorry, one of the ports provided with --ports flag is not valid [127.0.0.1:32760-32767:32760-32767]` |
|
@mattbator minikube considers 127.0.0.1:32760-32767:32760-32767 as one port mapping as you can see from the error message it displayed an array with one element, a workaround I did was separate them by a comma. However, I don't know what to do with bulk ports |
|
None of the fixes here worked for me :/ |
|
@babreu-ncsa can you share with us some screenshots or the error messages? |
|
@nour-karoui |
|
@nour-karoui thanks for asking! I felt like I should provide details, but since the issue is closed and there are so many examples, I just wanted to bring it back to life somehow. But yes, makes sense to post the details! Now here's my apiVersion: apps/v1
kind: Deployment
metadata:
name: helloworld
spec:
selector:
matchLabels:
app: helloworld
replicas: 1 # tells deployment to run 1 pods matching the template
template: # create pods using pod definition in this template
metadata:
labels:
app: helloworld
spec:
containers:
- name: helloworld
image: karthequian/helloworld:latest
ports:
- containerPort: 80Create it verify pods Now expose the deployment: check then hit it: and it will stay forever like this. |
|
@leizhangcnnz can you try --ports=31923:31923 (remove the 127.0.0.1) |
|
@nour-karoui still failing. |
It is expected. I don't think the Instead, if the service type is The #13806 is merged recently.
|
|
@zhan9san thanks for your comment! Unfortunately, none of these is working for me. Here are the details. NodePortDifferent terminal http://127.0.0.1:8080/61983 (and 8080) give "connection reset". http://127.0.0.1:8080/30818 also gives connection reset. LoadBalancerTerminal 2 Terminal 1 http://127.0.0.1:8080 again gives connection reset
I think I tried both options according to https://minikube.sigs.k8s.io/docs/handbook/accessing/#nodeport-access , but in case I'm missing something... please let me know. |
|
The image in doc, But for your own image, the service exposed port is For NodePort
For LoadBalancer
|
|
Thanks again, @zhan9san ! I didn't notice that and it's probably due to my lack of Kubernetes knowledge. Which is the reason why I'm doing helloworld...
Do you have any insights? I appreciate your time! |
|
I am not sure you have to use Could you tried |
|
@zhan9san fantastic!! It worked!! Thank you very much!! |
This comma separated list worked after rebuilding the |
how to do it with multinode cluster |
|
@nour-karoui's Reply helped me figure this out on my M1 MBP.
For me, the gotcha was that you have to do this the first time you start a minikube cluster with the docker driver, otherwise the ports aren't actually exposed. Once I stopped, deleted, and created the minikube cluster specifying the |
|
In Mac with minikube you can use this command: minikube service apache1 --url where apache1 is the name of your service. And It show ip and port which you could use in a browser |
|
After starting the minikube by using as below. port is not open. Can any one suggest how to solve this. minikube start --driver=docker --ports=30100:30100 docker port minikube |
open http://127.0.0.1:53414 in your browser |
|
Hello @tennessine Tried as suggested but find the details below. minikube service client-node-port --url ❌ Exiting due to SVC_NOT_FOUND: Service 'client-node-port' was not found in 'default' namespace. viswa@:/opt/k8s-demo$ minikube service list ❌ Exiting due to SVC_NOT_FOUND: Service 'client-node-port' was not found in 'default' namespace. unfortunately this url is also not working (http://192.168.49.2:32760) |
|
you can use docker desktop NodePort service. Once you deploy your service .yaml file in your cluser follow these steps:
whoever using minikube, plz run this command minikube service your-service-name --url Note: I have pasted here my service file for your reference with minimal self explanatory apiVersion: v1 |
Hello,
I am facing issues accessing services via node port on Mac OS.
System information:
Ninikube version:
kubectl version:
docker version:
Minikube installed with homebrew version:
minikub init command
minikube start --driver=docker --mount --mount-string $(PWD)/dev/volumes:/volumes
Steps to reproduce the issue:
issue
But I cannot access this service using port 30080, and here comes the issue:
If I try
I get 192.168.49.2 as response. This is what happens also on Linux. However, this ip is kind of "useless", in the sense that I can not ping it from terminal nor can I telnet or CURL it at port 30080 (or at any other port I have tried).
This is different from Linux, where I can either ping or telnet / CURL / PSQL / whatever I need at the appropriate ports and services work. on Mac OS, every telnet / CURL / psql I try on this IP hangs indefinitely.
I suppose this has to do somehow with the docker driver, but I couldn't find documentation on that nor could I find anything useful in logs.
Strangely, minikube ssh works flowlessly, as also works minikube service wiremock. More strange is that minikube tunnel also hangs indefinitely.
I cannot use minikube service tunelling for several reazons, and I am thinking what can be done to fix that. NodePort is an important feature of Minikube, so if the docker driver doesn't support that I would like to know.
As I have no errors outputs, I will post no logs here now, but I am whiling to do everything I can to help to track this, so feel free to ask me for extra logs.
Full output of failed command:
There is not failure of output commands, at least that I have seen.
Full output of
minikube startcommand used, if not already included:The text was updated successfully, but these errors were encountered: