Windows support hostprocess

[fabi200123]

Description

This PR helps moving the Windows Calico hostprocess from kubernetes-sigs/sig-windows-tools to this repository. This PR is based on the old PRs created in projectcalico/node and projectcalico/cni-plugin.

THIS PR IS STILL WORK IN PROGRESS!

Related issues/PRs

• #1201
• #1148

##Used for installing/building:

• containerd v1.7.0
• calico-windows.yaml, kube-calico-rbac.yml

[CLAassistant]

All committers have signed the CLA.

[coutinhop]

Hi @fabi200123!

Apologies for the long delay. Thanks so much for your contribution, but I don't believe we should follow through with this though, seeing as projectcalico/node#1201 and projectcalico/cni-plugin#1148 were both closed due to kubernetes/enhancements#2865 (comment).

We already have hostprocess support (currently in tech-preview), you can find the powershell scripts and Dockerfile in https://github.com/projectcalico/calico/tree/master/node/windows-packaging, and the docs at https://docs.tigera.io/calico/latest/getting-started/kubernetes/windows-calico/quickstart#install-calico-for-windows-using-hostprocess-containers

Granted, I'm still trying to wrap my head around the original PRs you linked, but it doesn't seem like these changes ended up being necessary to support windows hostprocess containers? Please do comment back if there's any issue...

[jsturtevant]

closed due to kubernetes/enhancements#2865 (comment).

This is no longer an issue with containerd 1.7. In cluster configuration with Hostprocess containers is now supported with no changes.

Granted, I'm still trying to wrap my head around the original PRs you linked, but it doesn't seem like these changes ended up being necessary to support windows hostprocess containers? Please do comment back if there's any issue...

Last time I looked it still required some custom set up (maybe this has changed). In general, I am in favor of getting rid of Powershell scripts installed directly on the host and moving towards go binaries for installation and running the calico components.

The current preview installation requires installing scripts on the host in a well-known spot and then referencing them from there. It would be cleaner to not have scripts installed directly on the node. It makes installation and cleanup easier. This also follows the installation process that Linux has (deploy a few templates and get calico!) instead of having to do custom configuration on the node.

[coutinhop]

Thanks @jsturtevant! I see, and I agree with you that doing away with powershell scripts on the host would be best. Having containerd v1.7.0+ as a requirement might make this more complex (if we do want to support earlier versions we'd need to still have the scripts for them, @mgleung @caseydavenport @fasaxc any thoughts?)

[jsturtevant]

Having containerd v1.7.0+ as a requirement might make this more complex

Agreed but I also thought since the current HPC work is preview and by the time something like this stabilizes 1.7 will be pretty far along the adoption curve.

Would need to maintain the "install on host and run as services" until 1.6.x went out of support, which is a definite maintaince concern, since you would have two ways of doing it. I think this can be mitigated to a degree though by tweaking and re-using some of the powershell scripts to support both formats.

[caseydavenport]

Throwing in my 2 cents as a drive-by - we should probably just support containerd v1.7.0+ and go all-in on the hostprocess method of install.

[davhdavh]

A bit of feedback after trying to actually set this up.

1. the docs at https://docs.tigera.io/calico/latest/getting-started/kubernetes/windows-calico/quickstart#install-calico-for-windows-using-hostprocess-containers is missing the step where you setup the calico-windows-vxlan.yaml and windows-kube-proxy.yaml because they are not default included anywhere.
2. The scripts to setup things are all over the place. A couple of examples:

• cni folder is in c:\etc\cni, or c:\k\cni or \cni depending on which script you use, and the defaults are different all over the place
• hns.ps1 file is in c:\k\ or in C:\CalicoWindows\libs\hns
• nssm is installed twice, both as part of containerd and in C:\CalicoWindows\nssm
• Kubernetes gets installed in c:\k and C:\k\kubernetes

3. The various calico help files is all over the place. Half the places you have to install the api server manually, half the places you have to install calicoctl manually.
4. The helm chart for calico does not wait for all crd to be installed, and there is no method to setup the IPAM config etc necessary for a windows setup.
5. the default calico-windows-vxlan.yaml doesnt actually work at all. It assumes the default working path is somewhere 2 levels deep, but the actual default working path is c:. The correct path to the various scripts has to be updated to "$env:CONTAINER_SANDBOX_MOUNT_POINT\host-process-install.ps1" and the working path updated to "c:\CalicoWindows".
6. The requirements to the vSwitch is mentioned nowhere, and error messages is absolutely horrible in this case. E.g. if you dont have a free hardware card to make the vSwitch on, it will just spin forever waiting for the vSwitch to be created and the windows events will just say "bad interface"

[jsturtevant]

2. The scripts to setup things are all over the place. A couple of examples:

I think that is the advantage of using containers with the scripts embedded (the approach proposed here). There will be a few things to be install on the host but otherwise when the container goes away we don't leave tons of stuff on the host.

[coutinhop]

@fabi200123 @jsturtevant I'd like to thank you for your PR and I hope you don't take issue with the fact that I built upon it to push #7857 and tigera/operator#2732, still also works in progress, but which will add GA support for Windows HPC to Calico. I'd like to close this issue and ask you to review the above PRs if you'd like. Please let me know if that's OK with you!

@davhdavh thanks for your feedback. Yes, our current (non-HPC) Calico for Windows install has quite a bit of issues, but as @jsturtevant said, moving to hostprocess containers will clean this up quite a bit. I'd like to invite you to check out the PRs above as well.

[jsturtevant]

I think this can be closed. Thanks @coutinhop and @fabi200123 and everyone else that helped get this support merged into calico 🥳