Implement testEnv for e2e tests

[chanwit]

This PR implements multiple test modes for e2e.
In addition to fakeKubeManager, we add 2 modes of testEnv:

• testEnv's controlplane only, and
• the existing cluster mode

This PR also implements 2 test cases for flux install and flux install --component-extras using the existing cluster mode.

Signed-off-by: Chanwit Kaewkasi chanwit@gmail.com

[allenporter]

Would it make sense to always run under testEnv and remove usage of the fake? Generally I saw that it seems to be preferred and better supported over the fake client, but wasn't sure if there were large down sides (e.g. super slow)

[chanwit]

I'm supporting all possible modes at the moment because fake is still useful in many cases, and I still don't want to break flow of unit tests here and there. We could gradually remove fake tests over time as we see fit, wdyt?

[allenporter]

I'm supporting all possible modes at the moment because fake is still useful in many cases, and I still don't want to break flow of unit tests here and there. We could gradually remove fake tests over time as we see fit, wdyt?
…
On Sun, Aug 8, 2021, 1:54 AM Allen Porter @.***> wrote: Would it make sense to always run under testEnv and remove usage of the fake? Generally I saw that it seems to be preferred and better supported over the fake client, but wasn't sure if there were large down sides (e.g. super slow) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1697 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACTKW4KC36KDBDOBOSDFLT3V6PHANCNFSM5BXUHWYQ .

I only added these fake unit tests a few days ago, so, if the env test works just as good happy to switch. I'm happy to try it out and then update and remove, just figured you might have a better idea of it's it's worth it just to switch all at once.

[chanwit]

@stefanprodan We don't have testEnv v1.21.1.

unable to fetch checksum for requested version: unable fetch metadata for kubebuilder-tools-1.21.1-linux-amd64.tar.gz -- got status "404 Not Found" from GCS

[stefanprodan]

@chanwit my mistake, the version should be set from https://github.com/kubernetes-sigs/controller-runtime/releases

[stefanprodan]

Looks like we need to add the Kubernetes version to out action, as described here https://github.com/kubernetes-sigs/controller-runtime/tree/master/tools/setup-envtest

[stefanprodan]

@chanwit I figure it out, please set the version to 1.21.x

[chanwit]

It seems to be working at a certain level.
https://github.com/fluxcd/flux2/runs/3319357290?check_suite_focus=true#step:10:1

@stefanprodan could you please take a look

[stefanprodan]

LGTM

Thanks @chanwit 🥇

[allenporter]

I'm updating #1696 now that this is merged and was curious since it was not immediately obvious when reading the test: What are the sleeps for? I was curious if we could explicitly wait for the condition it is expecting, but wasn't sure what the context was.

[stefanprodan]

@allenporter there are many issues with the current install testing, as the namespace finalizer can take anywhere from 1 to 5 minutes :( I will get rid of the sleep.