charts: Mark all test resources to be only created while running tests

[akshaymankar]

Checklist

• Add a new entry in an appropriate subdirectory of changelog.d
• Read and follow the PR guidelines

[jschaul]

As long as CI is happy, this looks good.

I'm not sure configmaps need this annotation, but it probably also doesn't hurt.

[akshaymankar]

I'm not sure configmaps need this annotation, but it probably also doesn't hurt.

I thought why create these resources at all in the non-testing scenarios.

[akshaymankar]

helm test --logs assumes that every test resource is a pod 🤦
helm/helm#11236

[jschaul]

helm test --logs assumes that every test resource is a pod facepalm helm/helm#11236

I'm reworking how helm test works in #3040

then this issue goes away. So review and merge that PR first, then this one might go green?

[jschaul]

@akshaymankar your latest commit here with patched helm might be in vain: The command "helm test" runs inside the cailleachImage, not the wire-server image, for some reason. Check the wire-server-pr pipeline - I also got bitten by this. Feel free to change this (though maybe then other dependencies are missing)

[akshaymankar]

Ah thanks! I was very confused to see the test fail in exactly the same way. I will change the pipeline I guess.

[jschaul]

Ah thanks! I was very confused to see the test fail in exactly the same way. I will change the pipeline I guess.

Or bump the version of helm in use inside cailleach with the same patch, that would also work I suppose.

[akshaymankar]

I vaguely remember wanting to get rid of cailleach-image as much as possible for running integration tests and since we don't rely on any sops machinery anymore we shouldn't need it. I have changed the pipeline already. Hopefully things go green now 🤞