-
Notifications
You must be signed in to change notification settings - Fork 304
install: Add minikube instructions #445
install: Add minikube instructions #445
Conversation
/cc @krsna1729 |
041fa7e
to
fa3ed93
Compare
@krsna1729 @GabyCT - feedback rolled in thanks. Updated. |
@jodh-intel @jcvenegas - oh, Travis gave me a:
Hmm, shame it did not give me details - and, what are the rules - do all lines have to be executable in the CI/travis system? That no doubt won't work for this minikube code (no minikube installed for a start) - I'm prone to flip all the |
@grahamwhaley - The rules are in https://github.com/kata-containers/documentation/blob/master/Documentation-Requirements.md#code-blocks. The problem with your file is that you are using bash code blocks and intermingling commands and command output. If you want this file to be executable, you will need to remove all command output from those blocks. If you really want to show commands and output (not generally recommended), use a |
Right, I don't think we can realistically execute these |
Ack. It's a shame though as I was hoping we could offer an automated install alternative to Docker. Maybe that can come later? |
It might. discussed with @egernst yesterday if we can add a minikube ci setup/test for kata-deploy testing at least - we might get the script to auto-install and setup minikube from that. |
I'd almost prefer we wait until we can do that since once it's a valid auto-installation option, we can test (and continue to test) that doc to guarantee it doesn't go stale as these things have a habit of doing on their own ;) |
@grahamwhaley thanks for the changes |
When attempting to follow the instructions I came across this bug too. The pod was failing at sandbox creation, and when I ran a Talked with @grahamwhaley and started doing a little bit of debugging. The first step was getting the last known good version up and running. So I did an |
@gabibeyer Just happened to have a look at your error message, that does come from the network code: @grahamwhaley Is that the same error you are seeing during the setup? |
I just so happened to be trying these instructions (from the gist) today and got the same error deploying the example: |
@amshinde yep, same error. @wilsonianb could be! Minikube is running a v4.15 kernel and a rootfs constructed from a buildroot. It has a config.gz in the fs, which I had a look in - afaict it has all the netlink filtering enabled. @amshinde , any clues on what to look for then? I suspect an
Let us know what to look for (or, of course, you can make a minikube by following the doc in this PR ;-) ). And then... to confirm - if you run up the minikube and kata, but before you launch the example pod/deployment, OK, let's see if we can figure out why tcfilter does not work for Minikube, and then either:
|
fa3ed93
to
639611c
Compare
I'm sticking a DNM on here until we figure out what to do about the |
@amshinde - I had a probe around (not that I'm sure what I'm looking for ;-) ) to see if anything obvious jumped out in the minikube about why I did notice that minikube does not appear to have the I did also spot some error/warnings in the dmesg, but I'm not convinced they are the problem, but will post here just in case:
It might just be co-incidence that we get an error on |
@amshinde OK, I think the MACVTAP and other module missing may be red herring - I think they are not modules, but built into the minikube kernel. |
@amshinde OK, I did a vendor update for the netlink module (if you do that, watch out for vishvananda/netlink#357 btw if you fail to compile...) - and the same fault still happens :-( |
@amshinde in case it helps, I added a few logger lines into network.go and dumped some of the attrs and tapattrs info, just in case anything leapt out. Sorry for the format, this is go's interpretation of the data structures:
If you need me to add more logging, let me know (or throw me a diff/patch)...
|
@grahamwhaley finally taking a look at this, this fell out of sight! |
@amshinde - I will get to look at this, but in case you want to beat me to it, I think the minikube kernel config is over at https://github.com/kubernetes/minikube/blob/master/deploy/iso/minikube-iso/configs/minikube_defconfig |
Heh, oh, that link to the minikube repo is to the buildroot config... here is the one to the kernel config file: From that file:
All those look to be set @amshinde . Any more ideas? or, we can start trying to add more debug (but I have a feeling ultimately the bottom level Oh, if you want to re-create, just follow the minikube doc guide attached to this PR with any Kata > v1.6.2 (so the kata-deploy will pick that up for you and should fail as is today) |
/test |
I've stuck a |
@grahamwhaley - yep, we seemed to have lost those labels in the switch to the new set (amazing how these things pass through review eh? :) I've now re-added those labels to all repos and raised the following to update the master database of labels: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @grahamwhaley. Some backticks changes required I think but also, it does look to me like all these commands could be made non-interactive. If so, we could make this document testable.
|
||
## Prerequisites | ||
|
||
This installation guide has only been verified under a `Minikube` `Linux`* installation, using the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not clear why we need backticks for these two? If they are project or product names, they can just be left capitalised I think, but use backticks for command names. Also, I guess drop the *
as discussed on my runtime PR?
This installation guide may not work for `MacOS` installations of `Minikube`, due to the lack of | ||
nested virtualization support on that platform. | ||
|
||
This installation guide has not been tested on a `Windows` installation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Againk, backticks?
This installation guide has only been verified under a `Minikube` `Linux`* installation, using the | ||
[kvm2](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#kvm2-driver) driver. | ||
|
||
This installation guide may not work for `MacOS` installations of `Minikube`, due to the lack of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Backticks?
|
||
| what | why | | ||
| ---- | --- | | ||
| --vm-driver kvm2 | The host VM driver | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please can you add backticks to all these options?
> | ||
> | what | why | | ||
> | ---- | --- | | ||
> | --feature-gates=RuntimeClass=true | Kata needs to use the RuntimeClass k8s feature | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing backticks.
Check you have virtualization enabled inside your `Minikube`: | ||
|
||
```sh | ||
minikube ssh |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Missing
$
prompt? - Could this become something like?:
$ minikube ssh egrep 'vmx|svm' /proc/cpuinfo
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. We have to add "'s or it goes horribly wrong though:
$ minikube ssh "egrep 'vmx|svm' /proc/cpuinfo"
I would have loved to add the --color
option in there as well to make it pop out, but Minikube is based on busybox, so does not have it :-(
|
||
|
||
```sh | ||
$ kubectl get nodes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this become something like?:
$ kubectl get nodes | grep -E "\<minikube\>" | grep -E "\<Ready\>"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we were automating a spin wait then maybe, but I think minikube
waits until the cluster is notionally up and Ready
before it returns to the prompt anyway. Thus, if it failed to start and configure you'd get something like:
The connection to the server localhost:8080 was refused - did you specify the right host or port?
but when it works you see the fairly obvious:
NAME STATUS ROLES AGE VERSION
minikube Ready master 99m v1.14.3
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, so kubectl
will block on any command until it can talk to the cluster? OOI I wonder if there is either a formal is-the-cluster-up-yet
command or a timeout / async option? If the cluster never comes up, does get nodes
error / timeout or just hang forever?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
errors out immediately if the cluster is not up (as it cannot connect to the socket), or returns an answer (even if that is 'no nodes ready yet') otherwise afaik
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yuck. Oh well...
to check it is running: | ||
|
||
```sh | ||
$ kubectl get pods |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this become something like?:
$ kubectl get pods | grep -- -apache-kata | grep -E "\<Running\>"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sold. It looks OK on my machine as my distro by default aliases grep to use colour - but, I think having the whole output of kubectl get pods
is more normal and useful in a k8s scenario, it's more what the user expects I would think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Surely there must be some way to detect the pod is running without having to manually eyeball the output though? Maybe a --json
option or something to make parsing reliable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is - I'll update to kubectl rollout status deployment php-apache-kata-qemu
|
||
```sh | ||
$ minikube ssh | ||
$ ps -ef | fgrep qemu-system |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this and the uname
command become two minikube ssh ...
command instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure. found a pgrep
string command that does a nice job.
inside the `Kata` VM. | ||
|
||
```sh | ||
$ kubectl get pods |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this become something like?:
$ pod=$(kubectl get pods | grep -E "\<Running\>" | grep kata- | head -1)
$ kernel=$(kubectl exec -ti "$pod" uname -r)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
similar to above, will replace with:
podname=$(kubectl get pods -o=name | fgrep php-apache-kata-qemu | sed 's?pod/??')
kubectl exec ${podname} -- uname -a
639611c
to
1e9f962
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated and pushed.
Updates applied.
I also tested the latest Minikube (v1.1.1) with the latest Kata from kata-deploy (v1.7.1), and it now works fine.
The full command is therefore: | ||
|
||
```sh | ||
$ minikube start --vm-driver kvm2 --memory 6144 --network-plugin=cni --enable-default-cni --container-runtime=cri-o --bootstrapper=kubeadm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Explained in the table above - as Kata has 2G default pod RAM sizes, but the Minikube default is only 2G, so v.likely we cannot launch a single default Kata inside a default Minikube without extending the RAM allocation.
and the `kvm_intel` driver, you can perform the check by: | ||
|
||
```sh | ||
$ cat /sys/module/kvm_intel/parameters/nested |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the big issue with testability is our reliance on the external docker kvm2 driver (where we link out to the installation instructions), and the fact that drive has some issues as a native installation on some distros.
I'll clean up all the other bits and update the PR, and then see if I can work out how do do a 'docs execution' locally and peek at how hard it would be to make it runnable.
We would really like to have minikube in the CI, particuarly for kata-deploy, but I'm not sure if we can realistically use this doc to help with that.
|
||
|
||
```sh | ||
$ kubectl get nodes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we were automating a spin wait then maybe, but I think minikube
waits until the cluster is notionally up and Ready
before it returns to the prompt anyway. Thus, if it failed to start and configure you'd get something like:
The connection to the server localhost:8080 was refused - did you specify the right host or port?
but when it works you see the fairly obvious:
NAME STATUS ROLES AGE VERSION
minikube Ready master 99m v1.14.3
Check you have virtualization enabled inside your `Minikube`: | ||
|
||
```sh | ||
minikube ssh |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. We have to add "'s or it goes horribly wrong though:
$ minikube ssh "egrep 'vmx|svm' /proc/cpuinfo"
I would have loved to add the --color
option in there as well to make it pop out, but Minikube is based on busybox, so does not have it :-(
```sh | ||
$ minikube ssh | ||
$ cd /opt/kata | ||
# If the following directory is not yet created and populated, wait a short while and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@krsna1729 heh, sleep 100
. sigh. hmm, I'm not over the moon with that ;-)
Let me see if there is some way to check kata-deploy has entered its infinity sleep state (which is what it does when it completes). Maybe we need to add some magic to kata-deploy to create a 'I am done' marker at the end of its work loop...
```sh | ||
$ minikube ssh | ||
$ cd /opt/kata | ||
# If the following directory is not yet created and populated, wait a short while and |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@krsna1729 @jodh-intel @amshinde @egernst Well, best I can come up with to check if kata-deploy has finished its work is:
podname=$(kubectl -n kube-system get pods -o=name | fgrep kata-deploy | sed 's?pod/??')
active=$(kubectl -n kube-system exec ${podname} -- ps -ef | fgrep infinity)
Better than guessing I guess - but, not great, and only really works in the single node case - does not scale out well.
I had a quick look at daemonsets - no obvious in-built way to manage 'status' - I think it would have to be built on some other infra method, and have the kata-deploy pods push status out to something else.
For the minikube docs, I'll update to the above ps sleep infinity
hack, and note this only works for single node clusters.
to check it is running: | ||
|
||
```sh | ||
$ kubectl get pods |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sold. It looks OK on my machine as my distro by default aliases grep to use colour - but, I think having the whole output of kubectl get pods
is more normal and useful in a k8s scenario, it's more what the user expects I would think.
|
||
```sh | ||
$ minikube ssh | ||
$ ps -ef | fgrep qemu-system |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure. found a pgrep
string command that does a nice job.
inside the `Kata` VM. | ||
|
||
```sh | ||
$ kubectl get pods |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
similar to above, will replace with:
podname=$(kubectl get pods -o=name | fgrep php-apache-kata-qemu | sed 's?pod/??')
kubectl exec ${podname} -- uname -a
/test |
| what | why | | ||
| ---- | --- | | ||
| `--vm-driver kvm2` | The host VM driver | | ||
| `--memory 6144` | Allocate sufficient memory, as Kata Containers default to 1 or 2Gb | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: You could sort the options alphabetically.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could, but I think as they are they keep related items (particularly the cni ones for instance) together. They also align somewhat with the order in the upstream Minikube docs iirc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If they group logically, you could add a new column for that to make it clear (and then sort by that grouping column and sub-sort by the what
column).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if the only way to get the PR landed is to sort them .... table now sorted.
This installation guide may not work for MacOS installations of Minikube, due to the lack of | ||
nested virtualization support on that platform. | ||
|
||
This installation guide has not been tested on a Windows installation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The MacOS and Windows sentences could be turned into a "Note" or a "Warning" maybe?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
> **Note:** This example deploys Kata Containers using CRI-O. The same procedure should also work | ||
> for a `containerd` based Kubernetes installation, but is not covered in this document. | ||
|
||
> **Note:** Prior to Minikube/Kubernetes v1.14, the beta `RuntimeClass` feature also needed enabling with |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two notes next to each other, so I'd turn them into a list:
Notes:
- item one.
- item two.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. and for the new pair of notes earlier on. let's see if I got the formatting right..
$ minikube start --vm-driver kvm2 --memory 6144 --network-plugin=cni --enable-default-cni --container-runtime=cri-o --bootstrapper=kubeadm | ||
``` | ||
|
||
> **Note:** For Kata Containers >v1.6.2, the now default `tcfilter` networking of Kata Containers |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Can you expand
>
and<
into words for clarity? - s/did not work for/does not work with/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
||
|
||
```sh | ||
$ kubectl get nodes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, so kubectl
will block on any command until it can talk to the cluster? OOI I wonder if there is either a formal is-the-cluster-up-yet
command or a timeout / async option? If the cluster never comes up, does get nodes
error / timeout or just hang forever?
```sh | ||
$ kubectl get nodes | ||
NAME STATUS ROLES AGE VERSION | ||
minikube Ready master 78m v1.13.4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know you are trying to be helpful by providing command output throughout this doc. But I believe the best practice is to avoid showing any command output unless absolutely necessary; commands have a nasty habit of changing their output format so showing output that is different to what the user sees is worse than not showing output. I've tried to allude to that at the bottom of https://github.com/kata-containers/documentation/blob/master/Documentation-Requirements.md#code-blocks.
Maybe @klynnrif can comment on this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jodh-intel I agree. In general, if you can avoid showing output you should avoid it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dropped the output and added a relevant genericised sentence on what to look for.
|
||
```sh | ||
$ podname=$(kubectl -n kube-system get pods -o=name | fgrep kata-deploy | sed 's?pod/??') | ||
# If 'sleep infinity' is running, the pod has completed its work. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would probably be worth referencing https://github.com/kata-containers/packaging/blob/master/kata-deploy/scripts/kata-deploy.sh as that is what (currently) runs the sleep
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done. reworded. moved the comment out of the code block.
There are a couple of ways to verify it is running with Kata Containers. Nominally, it | ||
should be hard to tell - the idea of Kata Containers is that your pod will run inside a VM, | ||
but look and feel just like it would as a normal software pod. In | ||
theory, you should not be able to tell your pod is running as a Kata Containers container. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These two sentences are essentially saying the same thing. I'd consider dropping the "Nominally, " sentence.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
||
## Wrapping up | ||
|
||
This guide should have shown a relatively easy way to get a Minikube up with Kata Containers installed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this be simplified to the following?:
This guide has shown an easy way to setup a Minikube with Kata Containers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
||
This guide should have shown a relatively easy way to get a Minikube up with Kata Containers installed. | ||
Be aware, this is only a small single node Kubernetes cluster running under a nested virtualization setup. | ||
It will have limitations, but, as a first introduction to Kata Containers, and how to install it under Kubernetes, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/It will have/It has/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rephrased a bit
## Introduction | ||
|
||
[Minikube](https://kubernetes.io/docs/setup/minikube/) is an easy way to try out a Kubernetes (k8s) | ||
cluster locally. It utilises running a single node Kubernetes stack in a local VM. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/utilises running/creates/ ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
[Minikube](https://kubernetes.io/docs/setup/minikube/) is an easy way to try out a Kubernetes (k8s) | ||
cluster locally. It utilises running a single node Kubernetes stack in a local VM. | ||
|
||
[Kata Containers](https://github.com/kata-containers) can be installed into a Minikube cluster using |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe turn this around and make it prescriptive:
Install Kata Containers in a Minikube cluster using kata-deploy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think prescriptive works in the Introduction - this is describing what can be done, not what to do or how to do it. Later in the 'installing Kata Containers' section is/can be prescriptive.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed.
The full command is therefore: | ||
|
||
```sh | ||
$ minikube start --vm-driver kvm2 --memory 6144 --network-plugin=cni --enable-default-cni --container-runtime=cri-o --bootstrapper=kubeadm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm still not clear on this magic number - why/how is 6144
"sufficient"?
1e9f962
to
3850767
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for updating @grahamwhaley. A bunch more comments.
@klynnrif - ptal.
The full command is therefore: | ||
|
||
```sh | ||
$ minikube start --vm-driver kvm2 --memory 6144 --network-plugin=cni --enable-default-cni --container-runtime=cri-o --bootstrapper=kubeadm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still don't think this is explained clearly enough in the document but let's see what other think...
|
||
|
||
```sh | ||
$ kubectl get nodes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yuck. Oh well...
|
||
```sh | ||
# No color option, as minikube is running busybox | ||
$ minikube ssh "egrep 'vmx|svm' /proc/cpuinfo" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about a compromise:
$ egrep -o -m 1 'vmx|svm' /proc/cpuinfo
You can then explain that if the command above returns any output, the system is usable. That's better than a number and imho one heck of a lot better than the rest of the noise in that file which users shouldn't care about (and which just adds noise to our docs imho).
Y | ||
``` | ||
|
||
If your system does not report `Y` from the `nested` parameter, then details on how |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is perfect as the paragraph explains the output of the shell block above. But that means you can drop the command output since, as confirmed by the doc team, we should avoid displaying command output unless absolutely necessary (see https://github.com/kata-containers/documentation/blob/master/Documentation-Requirements.md#code-blocks).
There are still quite a few examples of command output in this doc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤷♂️ nuked.
nested virtualization support on that platform. | ||
> - This instal lation guide has not been tested on a Windows installation. | ||
|
||
Before commencing installatin, it is strongly recommended you read the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- typo: "installation".
- nit: double space.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
| what | why | | ||
| ---- | --- | | ||
| `--vm-driver kvm2` | The host VM driver | | ||
| `--memory 6144` | Allocate sufficient memory, as Kata Containers default to 1 or 2Gb | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If they group logically, you could add a new column for that to make it clear (and then sort by that grouping column and sub-sort by the what
column).
3850767
to
2e18172
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
repushed
|
||
```sh | ||
# No color option, as minikube is running busybox | ||
$ minikube ssh "egrep 'vmx|svm' /proc/cpuinfo" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
went with -c
- I'd thought the count was just the retval, but is also on stdout, so is OK to explain.
nested virtualization support on that platform. | ||
> - This instal lation guide has not been tested on a Windows installation. | ||
|
||
Before commencing installatin, it is strongly recommended you read the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
| what | why | | ||
| ---- | --- | | ||
| `--vm-driver kvm2` | The host VM driver | | ||
| `--memory 6144` | Allocate sufficient memory, as Kata Containers default to 1 or 2Gb | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if the only way to get the PR landed is to sort them .... table now sorted.
The full command is therefore: | ||
|
||
```sh | ||
$ minikube start --vm-driver kvm2 --memory 6144 --network-plugin=cni --enable-default-cni --container-runtime=cri-o --bootstrapper=kubeadm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've added an extra Note: to try and help.
Y | ||
``` | ||
|
||
If your system does not report `Y` from the `nested` parameter, then details on how |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤷♂️ nuked.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Scrubbed for grammar, structure, and flow. Sentence rewrites suggested to keep an active voice. Thanks!
|
||
Alternatively, and for other architectures, the Kata Containers built in | ||
[`kata-check`](https://github.com/kata-containers/runtime#hardware-requirements) | ||
command can be used *inside Minikube* once Kata has been installed, to check for compatability. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spelling error "compatability" => compatibility
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
||
## Enabling Kata Containers | ||
|
||
> **Note:** This step is only required for Minikube/Kubernetes versions <= 1.13. Since version |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggested rewrite to stay active:
Minikube/Kubernetes versions <= 1.13 only requires this step.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done, but I moved the 'only' earlier:
Only Minikube/Kubernetes versions <= 1.13 require this step.
$ kubectl apply -f kata-qemu-runtimeClass.yaml | ||
``` | ||
|
||
Kata Containers should now be installed and enabled in the Minikube cluster. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggested rewrite to stay active:
The Kata Containers installation process should be complete and enabled in the Minikube cluster.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
2e18172
to
e091edb
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@klynnrif - changes applied and pushed. thx
|
||
Alternatively, and for other architectures, the Kata Containers built in | ||
[`kata-check`](https://github.com/kata-containers/runtime#hardware-requirements) | ||
command can be used *inside Minikube* once Kata has been installed, to check for compatability. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
## Setting up Minikube | ||
|
||
To enable Kata Containers under Minikube, you need to add a few configuration options to the | ||
default Minikube setup. This is easily accomplished, as Minikube supports them on the setup commandline. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
$ kubectl apply -f kata-qemu-runtimeClass.yaml | ||
``` | ||
|
||
Kata Containers should now be installed and enabled in the Minikube cluster. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
|
||
Launch a container that is defined to run on Kata Containers. The enabling is configured by | ||
the following lines in the YAML file. See the Kubernetes | ||
[Runtime Class Documentaiton](https://kubernetes.io/docs/concepts/containers/runtime-class/#usage) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
$ kubectl apply -f test-deploy-kata-qemu.yaml | ||
``` | ||
|
||
An Apache PHP pod should be deployed with the Kata Containers runtime. This may take a few |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've moved the reference to Apache PHP to the previous sentence to avoid having to try and keep the post-tense reference.
|
||
An Apache PHP pod should be deployed with the Kata Containers runtime. This may take a few | ||
moments if the container image needs to be pulled down into the cluster. Progress can be | ||
checked using: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
[kvm2](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#kvm2-driver) driver. | ||
|
||
> **Notes:** | ||
> - This installation guide may not work for MacOS installations of Minikube, due to the lack of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"MacOS" isn't in the dictionary I'm afraid ;) To correct that, all you need to do is add MacOS/B
(exactly as I've written it) in one of the spell checkers data files, say one of:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so, open another PR in another repo, sigh....
and, looks like it is macOS
(capitalisation).
OK, let me go make another PR...
Over at kata-containers/tests#1826
I'll see if that lands, or if we need to add a depends on this PR for it.
This guide has shown an easy way to setup Minikube with Kata Containers. | ||
Be aware, this is only a small single node Kubernetes cluster running under a nested virtualization setup. | ||
As such, it has limitations, but as a first introduction to Kata Containers, and how to install it under Kubernetes, | ||
it should suffice for initial learning and experimentation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice. Is there an obvious "next step" (or "further information") you could point users at?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, not one that I can immediately think of. Next step would be to pick your route to a 'full k8s install', and you'd still use the above kube-deploy
style install probably. I think there are too many possibilities to choose one to suggest.
[Minikube](https://kubernetes.io/docs/setup/minikube/) is an easy way to try out a Kubernetes (k8s) | ||
cluster locally. It utilises running a single node Kubernetes stack in a local VM. | ||
|
||
[Kata Containers](https://github.com/kata-containers) can be installed into a Minikube cluster using |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed.
Add instructions of how to setup and install Kata under Minikue. Fixes: kata-containers#444 Signed-off-by: Graham Whaley <[email protected]>
Add Minikube to the list of 'cloud' providers installation instructions. Whilst there, order the list alphabetically. Signed-off-by: Graham Whaley <[email protected]>
e091edb
to
8e29464
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated, pushed.
We may want to wait for the 'macOS' spellcheck PR to land before we spin a CI again.
[kvm2](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#kvm2-driver) driver. | ||
|
||
> **Notes:** | ||
> - This installation guide may not work for MacOS installations of Minikube, due to the lack of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so, open another PR in another repo, sigh....
and, looks like it is macOS
(capitalisation).
OK, let me go make another PR...
Over at kata-containers/tests#1826
I'll see if that lands, or if we need to add a depends on this PR for it.
This guide has shown an easy way to setup Minikube with Kata Containers. | ||
Be aware, this is only a small single node Kubernetes cluster running under a nested virtualization setup. | ||
As such, it has limitations, but as a first introduction to Kata Containers, and how to install it under Kubernetes, | ||
it should suffice for initial learning and experimentation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, not one that I can immediately think of. Next step would be to pick your route to a 'full k8s install', and you'd still use the above kube-deploy
style install probably. I think there are too many possibilities to choose one to suggest.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @grahamwhaley.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm - thanks!
Add a doc on how to setup and install Kata into Minikube.