add documentation for WSL2

[cheslijones]

1. minikube start --driver=docker
2. minikube ip = 172.17.0.2

How are you supposed to access the web app from a browser when the IP is 172.17.0.2? Browser just spins for a while before saying it can't connect. Only occurs when --driver=docker. Just about every other driver gives you an ip that is accessible from browser.

[tstromberg]

If you are on Docker for macOS or Docker for Windows, networking works a bit differently, as there is no direct IP connectivity to the container. On these platforms, Docker requires individual ports to be proxied through.

See the Deploy applications section of https://minikube.sigs.k8s.io/docs/start/ for instructions on how to get access to your service. Please let me know if this helps!

[cheslijones]

Sorry, I should have specified I'm experimenting with WSL2 and installing docker, kubectl, minikube, and skaffold directly into it and not using Docker for Windows.

I followed the steps in that I basically copied and pasted all of them and was still unable to get any kind of contact between the cluster and browser.

$ kubectl get all
NAME                                  READY   STATUS    RESTARTS   AGE
pod/balanced-77b47f955f-z8z44         1/1     Running   0          2m26s
pod/hello-minikube-5655c9d946-t6fh6   1/1     Running   0          5m31s

NAME                     TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)          AGE
service/balanced         LoadBalancer   10.107.211.77   10.107.211.77   8000:31796/TCP   2m20s
service/hello-minikube   NodePort       10.107.31.112   <none>          8080:32498/TCP   5m26s
service/kubernetes       ClusterIP      10.96.0.1       <none>          443/TCP          22m

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/balanced         1/1     1            1           2m26s
deployment.apps/hello-minikube   1/1     1            1           5m31s

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/balanced-77b47f955f         1         1         1       2m26s
replicaset.apps/hello-minikube-5655c9d946   1         1         1       5m31s

When I did the hello-minikube examples it gave:

$ minikube service hello-minikube
|-----------|----------------|-------------|-------------------------|
| NAMESPACE |      NAME      | TARGET PORT |           URL           |
|-----------|----------------|-------------|-------------------------|
| default   | hello-minikube |        8080 | http://172.17.0.2:32498 |
|-----------|----------------|-------------|-------------------------|
🎉  Opening service default/hello-minikube in default browser...
👉  http://172.17.0.2:32498

The balanced examples gave me 10.107.211.77. In both cases, the browser eventually just said "Hmmmm.... can't reach this page".

If I could get the examples to work, I could more than likely figure out how to apply this to my ingress controller for my actual application.

[winhtaikaung]

I am facing this too

[cheslijones]

@winhtaikaung Any luck on your end? If not, would love to hear from a dev or someone who knows how to get around this issue.

[afbjorklund]

Like stated above, this is different when running on Mac or Win - the IP is accessible on Linux...

This is a Docker VM limitation, unfortunately. You might also want to try other minikube drivers ?

https://docs.docker.com/docker-for-mac/networking/#known-limitations-use-cases-and-workarounds

https://docs.docker.com/docker-for-windows/networking/#known-limitations-use-cases-and-workarounds

[cheslijones]

There are no other minikube driver choices for WSL2. You can't have virtualization in WSL2 so scratch kvm2, vmware, hyperv, virtualbox, etc. none doesn't work either. Pretty much just leaves docker.

To expand on "none doesn't work either"...

$ sudo minikube start --driver=none --kubernetes-version=1.15.10
😄  minikube v1.9.2 on Ubuntu 20.04
✨  Using the none driver based on existing profile
👍  Kubernetes 1.18.0 is now available. If you would like to upgrade, specify: --kubernetes-version=1.18.0
👍  Starting control plane node  in cluster minikube
🔄  Restarting existing none bare metal machine for "minikube" ...
ℹ️  OS release is Ubuntu 20.04 LTS

❌  [NONE_DOCKER_EXIT_1] Failed to enable container runtime enable docker.: sudo systemctl start docker: exit status 1
stdout:

stderr:
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

💡  Suggestion: Either systemctl is not installed, or Docker is broken. Run 'sudo systemctl start docker' and 'journalctl -u docker'
📘  Documentation: https://minikube.sigs.k8s.io/docs/reference/drivers/none
⁉️   Related issue: https://github.com/kubernetes/minikube/issues/4498

$ sudo systemctl start docker
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

Docker is running, by the way... so the error makes no sense.

[afbjorklund]

There are no other minikube driver choices for WSL2. You can't have virtualization in WSL2 so scratch kvm2, vmware, hyperv, virtualbox, etc. none doesn't work either. Pretty much just leaves docker.

WSL (1 is container, 2 is vm) is a little like LXD, it will take more effort to get minikube working there.

See #5392 for details.

The error seems to be due to WSL2 not using systemd, can probably be worked around ? #6954

Sorry for missing "WSL"

[sharifelgamal]

@eox-dev any luck with the above advice?

[cheslijones]

@sharifelgamal Well, basically using docker as the driver doesn't work. Fortunately, podman does although I haven't worked with it extensively so far.

EDIT: Got to play with this for a few hours today. Same issue as with --driver=docker. Get an IP address that I can't access from the browser. Pretty sure not an ingress-nginx as I am using the same version in Pop!_OS and everything works perfectly over there with --driver=kvm2.

Would be great if Microsoft would implement nested virtualization in WSL2 just so you can install kvm or something.

[medyagh]

@eox-dev we had a PR that added support for WSL
do you mind trying the binary in this PR to see if that fixes your problem for docker or podman driver?

#7420 (comment)

[cheslijones]

@medyagh I gave it a shot, but still running into the same issue where the running application cannot be accessed from the browser.

work@RBA15-WINDOWS:~$ ./minikube-linux-amd64 ip
127.0.0.1

It is usually 172.17.0.2 or 172.17.0.3 with latest release of minikube.

When I deploy the ingress-nginx to route traffic to my app's endpoints though, ingress controller uses 172.17.0.3 and I can't access it.

Maybe, it is something I'm just not overlooking, but I have zero issues in macOS with hyperkit and Linux with kvm2. Maybe it is just an issue with WSL2 or the Docker driver.

At some point today I plan on going through this: https://kubernetes.io/blog/2020/05/21/wsl-docker-kubernetes-on-the-windows-desktop/. Maybe that will get me up and running with WSL.

[medyagh]

@medyagh I gave it a shot, but still running into the same issue where the running application cannot be accessed from the browser.

work@RBA15-WINDOWS:~$ ./minikube-linux-amd64 ip
127.0.0.1

It is usually 172.17.0.2 or 172.17.0.3 with latest release of minikube.

When I deploy the ingress-nginx to route traffic to my app's endpoints though, ingress controller uses 172.17.0.3 and I can't access it.

Maybe, it is something I'm just not overlooking, but I have zero issues in macOS with hyperkit and Linux with kvm2. Maybe it is just an issue with WSL2 or the Docker driver.

At some point today I plan on going through this: https://kubernetes.io/blog/2020/05/21/wsl-docker-kubernetes-on-the-windows-desktop/. Maybe that will get me up and running with WSL.

thank you for providing more context, seems like you found a bug !
I belive u can fix that by using 127.0.0.1 IP and we will need to open a port for the ingress port,

is the port for ingress 80 ?
when you used to access 172.17.0.2, did u curl it on http: or https?

[cheslijones]

Yes, using port 80 for local dev. curl on http.

[medyagh]

for docker driver we will need to create a port on 80 during container creation time, this is do-able.

this would be a good fit for v1.12.0 to make docker driver the default driver

@eox-dev do you mind porviding me extact steps, steps by step so I make a PR to fix this?

[Aut0R3V]

@medyagh Hi, sorry for the delay on getting this to you. I'm so jaded with trying to get this working after so many hours, it takes a lot of motivation to get me looking at it again.
I made this repo with a lengthy README.md with the steps. Let me know if you'd like me to post those steps here as well:
https://github.com/eox-dev/minikube-wsl2

@eox-dev that is cool ! I would be happy to review the PR if you like to add this as a tutorial to our docs
https://minikube.sigs.k8s.io/docs/tutorials/
Hey, I just wanted to know if this documentation was updated. If not, then I'd like to help with this.

[cheslijones]

@Aut0R3V Hi, I never updated and sent a PR because I could never get it working.

[Aut0R3V]

@cheslijones thanks for the reply. I also tried
solving it using that documentation but I'm getting some issues.

[igortas]

Is there any news on this topic? How to use localhost for nodeport, loadbalancer or ingress, when minikube, kubectl and docker are installed in WSL2, but there is no docker-desktop at all, or minikube as windows executable. Address is constantly 192.168.49.2, and can't be accessed from browser or postman(insomnia).

[cheslijones]

@igortas This is what has been working for me. I've actually been using WSL2 for a few months or so for development, which I thought I'd never see the day that would happen.

This is what I've been doing and tested on several installs:

1. Install WSL2 (obviously)... I've just been using Ubuntu 20.04 and make sure you are always working in ~/ and not the Windows file system /mnt/c/.
2. Install docker directly into WSL2 without Docker Desktop using the Linux instructions. (Also, make sure to create docker group and add user, restart WSL2, and make sure it is running with sudo service docker start).
3. Install kubectl using the Linux instructions.
4. Install minikube using the Linux instructions.
5. Install skaffold (not necessary but that is what I use to build the docker images and deploy to minikube)... install using the Linux instructions.
6. Create your minikube profile with: minikube start --driver=docker.
7. For ingress-nginx I run the following command. It is actually labeled as being for Docker for Mac, but it works with Docker Desktop for Windows, Docker Desktop for macOS, and apparently WSL2 without Docker Desktop for Windows: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.45.0/deploy/static/provider/cloud/deploy.yaml.
8. Start-up your cluster. In my case it is just: skaffold dev.
9. When it is starting up, or after, run: minikube tunnel in a separate terminal.
10. You will be prompted for your sudo and it will say something about needing to access privileged port, or something to that effect.
11. You won't see any output in that terminal window after sudo (that tripped me up the first time), but when you go to 127.0.0.1 or localhost in the browser, you'll see the application running as expected. (minikube ip and minikube profile list will always say 192.168.49.2, just ignore it).

That is my use case and that has worked for me on any computer I've tried it on. Kind of lame having minikube tunnel using up a terminal window, but it is what it is. Don't know about NodePort, or LoadBalancer, I just use ingress-nginx.

The thing I like about this is the directions are basically the same whether using macOS, WSL2, or Windows, so it is easier to communicate. Doing this on Linux doesn't quite work the same due to Docker networking differently in Linux, but it sounds like it is being addressed: #10812. On Linux using the --driver=docker actually gives it a unique IP address (I believe).

I'm just using the Beta Channel Windows, but it worked for me on the Relase Channel as well. No modifications whatsoever to WSL2 or the kernel, just a clean install of WSL2.

I'm even running mkcert for TLS certificates in local development due to working with external APIs. If you need that as well I can provide instructions on that as well.

[igortas]

@cheslijones
I will update your answer about my working solution with services exposed as nodePort and ingress.

1. I have custom ingress file, the key host is host: localhost
2. In separate terminal, i'm running minikube tunnel
3. curl localhost

For now minikube don't like to work if i set custom host in ingress, even I've tried to update /etc/localhosts/ as:
192.168.49.2 local.dev, for example

Update:
I like to add another point which is connected to kube-apiserver and installation with snap.
Snap was throwing error: cannot communicate with server: Post http://localhost/v2/snaps/kube-apiserver: dial unix /run/snapd.socket: connect: no such file or directory
First there is currently one script on github:
https://github.com/DamionGans/ubuntu-wsl2-systemd-script
which enables systemd on wsl2 and helps to run with snap, but after the restart my VS Code terminal was mess and stop to sync everything from git, started mounting /mnt/c, instead linux etc..
So this is the reference to the correct answer.
microsoft/WSL#2972 (comment)

[medyagh]

@cheslijones
thank you for sharing your interesting story of using minikube, I am glad you got it working in WSL2,

I agree it is kind of not ideal to have another terminal to use the loadbalancer feature, to avoid that you could use the minikube server command

if you have a nodeport or hostport service you could list them with minikube service list
and access then using minikube service svc_name

@igortas do you feel like we should a documentation page just for WSL2 on our website ? if yes would you be interested to add a section in docker driver for WSL2 ?

https://minikube.sigs.k8s.io/docs/drivers/docker/

[shankar-forgerock]

Hi Medya, Thank you for the pointer. https://minikube.sigs.k8s.io/docs/drivers/docker/  looks very promising. For WSL2 environments, it will be worth clarifying if Docker should be installed on WSL2 guest or in Windows host. Regards Shankar

On 5/12/21 11:55 AM, Medya Ghazizadeh wrote: @cheslijones <https://github.com/cheslijones> thank you for sharing your interesting story of using minikube, I am glad you got it working in WSL2, || I agree it is kind of not ideal to have another terminal to use the loadbalancer feature, to avoid that you could use the minikube server command if you have a nodeport or hostport service you could list them with |minikube service list| and access then using minikube service svc_name @igortas <https://github.com/igortas> do you feel like we should a documentation page just for WSL2 on our website ? if yes would you be interested to add a section in docker driver for WSL2 ? https://minikube.sigs.k8s.io/docs/drivers/docker/ <https://minikube.sigs.k8s.io/docs/drivers/docker/> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7879 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AISGH6PRJQK5VIHL6OBEYGLTNLFINANCNFSM4MPUXWPA>.

-- ForgeRock <https://www.forgerock.com/> *Shankar Raman* Senior Technical Writer  |  ForgeRock ***@***.*** ***@***.***> *web* www.forgerock.com <https://www.forgerock.com/>

…

-- ForgeRock values your Privacy <https://www.forgerock.com/your-privacy>

[igortas]

@shankar-forgerock Install and configure everything inside WSL2. U don't need docker-desktop.

@medyagh Looks like a good idea. Where we can continue to chat on this topic?

[dgoldssfo]

Any news yet on whether this might be fixed with Windows 11?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[sayantani11]

Is the issue solved? I also use minikube on WSL, but I haven't faced this problem as of now. But I am also unable to go through the entire conversation, can someone summaries may be?

[sayantani11]

There are no other minikube driver choices for WSL2. You can't have virtualization in WSL2 so scratch kvm2, vmware, hyperv, virtualbox, etc. none doesn't work either. Pretty much just leaves docker.

To expand on "none doesn't work either"...

$ sudo minikube start --driver=none --kubernetes-version=1.15.10
😄  minikube v1.9.2 on Ubuntu 20.04
✨  Using the none driver based on existing profile
👍  Kubernetes 1.18.0 is now available. If you would like to upgrade, specify: --kubernetes-version=1.18.0
👍  Starting control plane node  in cluster minikube
🔄  Restarting existing none bare metal machine for "minikube" ...
ℹ️  OS release is Ubuntu 20.04 LTS

❌  [NONE_DOCKER_EXIT_1] Failed to enable container runtime enable docker.: sudo systemctl start docker: exit status 1
stdout:

stderr:
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

💡  Suggestion: Either systemctl is not installed, or Docker is broken. Run 'sudo systemctl start docker' and 'journalctl -u docker'
📘  Documentation: https://minikube.sigs.k8s.io/docs/reference/drivers/none
⁉️   Related issue: https://github.com/kubernetes/minikube/issues/4498

$ sudo systemctl start docker
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

Docker is running, by the way... so the error makes no sense.

Try:
minikube start --vm-driver=docker
minikube ip = <ip>

[sayantani11]

And we should have docker running in the background or else it won't work I guess

[afbjorklund]

Trying to use systemd where not available is probably a bug, though: #12276 (comment)

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[sh2sg]

To get host applications such as Browser, IDE, Postman to connect to services/applications deployed in Minikube via Ingress,

• on Mac I had to use Hyperkit driver (docker didn't work).
• on Windows WSL2, I have tried docker and kvm2 driver, unfortunately none of them enabled direct IP access from host to minikube, So far I had to fire API calls within WSL2 via cURL, I am seeking a workaround too.

[gsowjanya97]

Hi everyone, I'm new to open source contribution but worked with minikube using docker driver before. Is there still a need to add documention for this issue?

[Claudio-code]

@igortas This is what has been working for me. I've actually been using WSL2 for a few months or so for development, which I thought I'd never see the day that would happen.

This is what I've been doing and tested on several installs:

1. Install WSL2 (obviously)... I've just been using Ubuntu 20.04 and make sure you are always working in ~/ and not the Windows file system /mnt/c/.
2. Install docker directly into WSL2 without Docker Desktop using the Linux instructions. (Also, make sure to create docker group and add user, restart WSL2, and make sure it is running with sudo service docker start).
3. Install kubectl using the Linux instructions.
4. Install minikube using the Linux instructions.
5. Install skaffold (not necessary but that is what I use to build the docker images and deploy to minikube)... install using the Linux instructions.
6. Create your minikube profile with: minikube start --driver=docker.
7. For ingress-nginx I run the following command. It is actually labeled as being for Docker for Mac, but it works with Docker Desktop for Windows, Docker Desktop for macOS, and apparently WSL2 without Docker Desktop for Windows: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.45.0/deploy/static/provider/cloud/deploy.yaml.
8. Start-up your cluster. In my case it is just: skaffold dev.
9. When it is starting up, or after, run: minikube tunnel in a separate terminal.
10. You will be prompted for your sudo and it will say something about needing to access privileged port, or something to that effect.
11. You won't see any output in that terminal window after sudo (that tripped me up the first time), but when you go to 127.0.0.1 or localhost in the browser, you'll see the application running as expected. (minikube ip and minikube profile list will always say 192.168.49.2, just ignore it).

That is my use case and that has worked for me on any computer I've tried it on. Kind of lame having minikube tunnel using up a terminal window, but it is what it is. Don't know about NodePort, or LoadBalancer, I just use ingress-nginx.

The thing I like about this is the directions are basically the same whether using macOS, WSL2, or Windows, so it is easier to communicate. Doing this on Linux doesn't quite work the same due to Docker networking differently in Linux, but it sounds like it is being addressed: #10812. On Linux using the --driver=docker actually gives it a unique IP address (I believe).

I'm just using the Beta Channel Windows, but it worked for me on the Relase Channel as well. No modifications whatsoever to WSL2 or the kernel, just a clean install of WSL2.

I'm even running mkcert for TLS certificates in local development due to working with external APIs. If you need that as well I can provide instructions on that as well.

Now install minikube in WSL it more simple you need install docker in wsl and follow linux guides