Add integration tests to test helm+flux #1101
Conversation
… flux. Use build tag integration_test to enable.
…ep after a minikube start to give the cluster time to stabilize.
|
Thanks @ncabatoff for putting in this work. @stefanprodan has contributed the flux helm chart in #1107, which (once merged) you should be able to rebase on to remove some of the LoC here. I will try to take a closer look in the next day or two. Cheers! |
Fix a race: if helm hasn't yet deployed a release, we can't get its history yet. Don't treat a minikube delete error as a failure since if there's nothing to delete it will fail. Provide --context to kubectl for the version command, since it's contacting the cluster.
|
Hey @squaremo, sounds good. Yes the flux helm chart I use is based on https://github.com/stefanprodan/k8s-podinfo/tree/master/charts/weave-flux. I'm not sure how easy it will be to use it unchanged since I rely on the ssh-known-hosts configmap being mounted, but I'm sure we can work something out. |
Ahh yes I see, that does make it a bit tricky. Maybe @stefanprodan will be kind enough to comment here on how it could be adapted to serve both purposes (I know he is a fan of mounting Aside from converging on the Helm chart, it would be really nice if we can get this to run in CI -- that would help avoid the situation that prompted it (the prior tests rotting). Can it be done with the |
The helm chart now has a parameter for supplying a host key, which gets mounted into the fluxd container. Is that enough to rework this PR around the chart herein? (I will attempt so, eventually, but feel free to beat me to it :-) |
|
The |
|
I'm finally coming back to this patch. Thanks for pointing out the fix in 0.28.1. I will look into working with the upstream charts soon. For now my focus has been on getting this working with TravisCI, as I agree with you that should be done to prevent rot. Yes, I'm almost certain it can be done with minikube --vm-driver=none. Main change required is that since we don't have a host node to use as our git repo host any longer, I'm deploying a pod to be the git server instead. I've got it working that way locally. In travis I've got it to the point where kubernetes comes up and we can interact with it, but the tests are still timing out while waiting to see the flux-sync tag get updated. I expect I should have that fixed and an updated PR issued within the week. |
|
@ncabatoff Great! For testing against self-hosted repos, I mashed together https://github.com/squaremo/git-serv at one point -- I can adapt it to what's needed here, if that would be helpful. |
|
Update: I've got it "working" on Travis now. The scare quotes are because it works at best half the time even now that I've ironed out most of the kinks. But Travis seems especially flaky lately: often the test hangs when trying to download dependencies, rather than during the tests themselves. I guess maybe I'll try circleci, since that's what flux is using anyway. I'll give your git-serv a try as well. So far I've been using jkarlos/git-server-docker, but we may as well use one we control. |
|
I've switched build-from-forks on in CircleCI. I hope that means you can push changes to the CI to this PR, and see if they work! |
|
I've been continuing my experiments in my flux-tester repo: I feel a bit more free to do whatever since it's more isolated from your repo and I still don't really know what I'm doing git-wise. Sadly I fear circleci is not going to be viable for running k8s integration tests, at least with minikube. I really wanted it to work, because the config format seems nicer and better documented and the system is much more responsive and reliable, based on the little I've used it so far. However:
An alternative might be https://github.com/kubernetes-sigs/kubeadm-dind-cluster, but it doesn't look anywhere near as stable or well supported as Minikube. My experience today with travis, incidentally (it ran builds every time I tried a tweak to the circleci config.yml) was pretty positive today: out of 9 builds, 7 passed. Unfortunately I think it's a bit hit or miss, but it may still be our best option for now. |
|
Via @hiddeco: looks like it may still be possible to run |
|
Rebased and modestly worked-on branch at |
|
We have the Flux dev meeting coming up next week. One of the action item from last time was to take a look at this effort and see if the remaining work could be broken up into smaller pieces. In broad strokes: what's still missing and where could people help? |
|
We now have an end-to-end smoke test, which does not quite exercise as much of flux as here, but is likely to be the base on which more end-to-end tests are added. |
See the README for more details. This fixes #919, and adds tests for flux+helm. Besides the tests copied from test-flux (test that the sync tag gets updated, test that automation works) it adds tests to ensure that charts in the git repo get deployed, that changes made to charts in the git repo result in a helm release upgrade, and that helm changes made outside of git get reverted.