Introduce kvm-libvirt platform

[pothos]

This documents how Lokomotive can be set up with nodes as VM on KVM through the libvirt Terraform provider. It provides a controller and a worker Terraform module to start with.

It provides a quick and easy way to test Lokomotive with a defined Flatcar image and doesn't require to have access to one of the cloud providers or spare hardware.

Testing

Please test it with the steps mentioned in the documentation.

Depends on kinvolk-archives/terraform-render-bootkube#12

[invidian]

Just a few nits in the comments. I tested it and it works fine. It's actually cool how fast it deploys.

[pothos]

Thanks for the comments!

[chrskhl]

Is this something we could use for the CI? We currently have another VM setup for that.

[chrskhl]

I just noticed that @squeed did something like this for Typhoon in poseidon/typhoon#266

Is this based on that?

[invidian]

Is this something we could use for the CI? We currently have another VM setup for that.

That doesn't use Matchbox and PXE, so I doubt in can be used for CI. We should test it on CI later on though.

[pothos]

Is this based on that?

@invidian Pointed me to this as well but the structure is different from the packet/aws files which I used as a basis. In the linked PR the controllers and workers get names instead of the count as done in packet/aws. I took the same mechanism to assign stable IPs to the controllers (starting with .10) and having a machine domain variable but the rest is different, e.g., this PR here does not change the host DNS server for IP resolution but uses node IPs in the configuration, and also uses a temporary disk image pool instead the default one.

[johananl]

@pothos I started reviewing this but haven't finished yet. I'm submitting what I currently have in the meantime and will do my best to finish testing this on Monday.

[johananl]

Is this something we could use for the CI? We currently have another VM setup for that.

@blixtra our CI process uses the bare-metal target. This one adds a new target (libvirt).

[pothos]

I have to fix a problem with multiple nodes.

[invidian]

BTW I stripped the code a bit and it's actually also a cool way of having a bunch of Flatcar instances running locally :)

[pothos]

I think I've fixed the multi-node configuration count mistake.

[kosyfrances]

While trying to apply terraform, I got hit with

Error: Error applying plan:

2 error(s) occurred:

* module.worker-pool-one.libvirt_domain.worker-machine: 1 error(s) occurred:

* libvirt_domain.worker-machine: Error creating libvirt domain: virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: 2019-06-20T11:05:59.605317Z qemu-system-x86_64: -drive file=/var/tmp/vmszqzs07/vmcluster-one-worker-0.qcow2,format=qcow2,if=none,id=drive-virtio-disk0: Could not open '/var/tmp/vmszqzs07/vmcluster-one-worker-0.qcow2': Permission denied')
* module.controller.libvirt_domain.controller-machine: 1 error(s) occurred:

* libvirt_domain.controller-machine: Error creating libvirt domain: virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: 2019-06-20T11:05:59.605666Z qemu-system-x86_64: -drive file=/var/tmp/vmszqzs07/vmcluster-controller-0.qcow2,format=qcow2,if=none,id=drive-virtio-disk0: Could not open '/var/tmp/vmszqzs07/vmcluster-controller-0.qcow2': Permission denied')

Left a couple of notes from my review so far. I will try setting up and continuing with review again some other time.

[pothos]

@kosyfrances Thanks for the review, can we have a look how to solve the error together?

[invidian]

Just a few nits, but otherwise LGTM!

[pothos]

I've removed the ?ref=thisbranch part of the terraform URLs to make it ready for merging.