Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional flag to use v1beta1 CronJob (pre 1.21) #6030

Merged
merged 2 commits into from
Mar 1, 2023

Conversation

absoludity
Copy link
Contributor

Description of the change

This PR adds an optional boolean flag that for the apprepository controller, --v1-beta1-cron-jobs, which can be set to ensure that the app repository controller creates and watches the older v1beta1 cronjobs.

Note that, although enabling this functionality to help with #6021 , we won't be removing the restriction in the Chart.yaml for > k8s 1.21, so to install the chart on a cluster prior to 1.21 you'll need to manually comment out that line in the Chart.yaml (this just ensures we don't go back and start supporting older k8s versions officially).

To use the flag, you can add the following to your values.yaml:

apprepository:
  extraFlags:
    - "--v1-beta1-cron-jobs"

Benefits

People who need to can choose to install Kubeapps explicitly on 1.20 .

Possible drawbacks

Applicable issues

Additional information

@mecampbellsoup Can you please test the resulting image on your 1.20 dev cluster and works as you expect (I tested locally with a 1.20 kind cluster and see app repos being synced etc.)

@netlify
Copy link

netlify bot commented Feb 27, 2023

Deploy Preview for kubeapps-dev canceled.

Name Link
🔨 Latest commit 2ab51a7
🔍 Latest deploy log https://app.netlify.com/sites/kubeapps-dev/deploys/63fc231d52442c0008864e7e

Signed-off-by: Michael Nelson <[email protected]>
@mecampbellsoup
Copy link
Contributor

@absoludity does this also include the other fix #6016? i.e. is its history linear with main?

@absoludity
Copy link
Contributor Author

@absoludity does this also include the other fix #6016? i.e. is its history linear with main?

Yep, it's the previous commit shown in the commit history of the branch at: https://github.com/vmware-tanzu/kubeapps/commits/6021-cronjob-api

@mecampbellsoup
Copy link
Contributor

@absoludity does this also include the other fix #6016? i.e. is its history linear with main?

Yep, it's the previous commit shown in the commit history of the branch at: 6021-cronjob-api (commits)

OK cheers, sorry I missed that 😓 will test ASAP and report back!

@absoludity
Copy link
Contributor Author

@absoludity does this also include the other fix #6016? i.e. is its history linear with main?

Yep, it's the previous commit shown in the commit history of the branch at: 6021-cronjob-api (commits)

OK cheers, sorry I missed that sweat will test ASAP and report back!

Cool thanks. Just remember, although the code is all in this one branch, it's two separate images that you'll need, one for the apprepository-controller and the other for the kubeapps-apis service.

@mecampbellsoup
Copy link
Contributor

@absoludity working great!!

image

image

image

@absoludity
Copy link
Contributor Author

Great? Not sure if the blank info in the first screenshot or the error in the third are something you expected?

@mecampbellsoup
Copy link
Contributor

Great? Not sure if the blank info in the first screenshot or the error in the third are something you expected?

Ran out of time to do proper thorough testing but initial tests are working as expected. I'll do more in the morning and confirm back here. Thanks again.

@mecampbellsoup
Copy link
Contributor

@absoludity regarding the secrets-related error above, I am seeing 500s from HAProxy to k8s api for both configmaps and secrets:

(⎈ default:cloud)mcampbell-1 :: ~ » k -n cloud logs deployments/cloud-app-kubernetes-ingress | grep " 500 "
10.240.77.49:35992 [28/Feb/2023:15:10:51.473] https~ default_kubernetes_https/<NOSRV> -1/-1/-1/-1/0 500 208 - - PR-- 5/4/0/0/0 0/0 {|cloud-app-kubernetes-ingress.cloud|Bearer vag9nt7hjiuh6ig1ty26xocs9j1wlvbu|kubeapps-apis/v0.0.0 (linux/amd64) kubernetes/$Format} "POST cloud-app-kubernetes-ingress.cloud/k8s/api/v1/namespaces/tenant-dev-24fdb0-one/secrets HTTP/2.0"
10.241.126.222:58620 [28/Feb/2023:15:20:27.291] https~ default_kubernetes_https/<NOSRV> -1/-1/-1/-1/0 500 208 - - PR-- 4/3/0/0/0 0/0 {|cloud-app-kubernetes-ingress.cloud|Bearer vag9nt7hjiuh6ig1ty26xocs9j1wlvbu|kubeapps-apis/v0.0.0 (linux/amd64) kubernetes/$Format} "POST cloud-app-kubernetes-ingress.cloud/k8s/api/v1/namespaces/tenant-dev-24fdb0-one/secrets HTTP/2.0"
10.240.77.49:41020 [28/Feb/2023:15:23:54.763] https~ default_kubernetes_https/SRV_1 0/0/0/4/4 404 500 - - ---- 5/4/0/0/0 0/0 {|cloud-app-kubernetes-ingress.cloud|Bearer vag9nt7hjiuh6ig1ty26xocs9j1wlvbu|Go-http-client/2.0} "GET cloud-app-kubernetes-ingress.cloud/k8s/api/v1/namespaces/tenant-dev-24fdb0-one/configmaps/test01-filebrowser HTTP/2.0"
10.240.77.49:41020 [28/Feb/2023:15:23:54.863] https~ default_kubernetes_https/<NOSRV> -1/-1/-1/-1/0 500 208 - - PR-- 5/4/0/0/0 0/0 {|cloud-app-kubernetes-ingress.cloud|Bearer vag9nt7hjiuh6ig1ty26xocs9j1wlvbu|kubeapps-apis/v0.0.0 (linux/amd64) kubernetes/$Format} "POST cloud-app-kubernetes-ingress.cloud/k8s/api/v1/namespaces/tenant-dev-24fdb0-one/secrets HTTP/2.0"

Still looking into it.

@absoludity
Copy link
Contributor Author

OK. I'm off today but will look deeper tomorrow. I think I only had time to test locally that the repos were being synced, so it could be that I can reproduce the issue if I deploy an app on a 1.20 cluster.

@mecampbellsoup
Copy link
Contributor

@mecampbellsoup Can you please test the resulting image on your 1.20 dev cluster and works as you expect (I tested locally with a 1.20 kind cluster and see app repos being synced etc.)

Everything looking good on my end.

Our errors came about from a bug w/ HAProxy similar to haproxy/haproxy#1766.

// a v1 batch listener or a v1beta1 batch listener.
//
// This allows users on 1.20 to continue to use the latest release.
func (c *Controller) setBatchLister(useV1Beta1 bool, kubeInformerFactory kubeinformers.SharedInformerFactory) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind of makes me wonder if this flag is necessary at all. Could the code just iterate over both v1 and v1beta1 and query the k8s API to see which resource is defined in the cluster, and use that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then you probably wouldn't need as many if checks/logic branches in the rest of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could just iterate implicitly but that has drawbacks as well: we'd need to do that iteration at each call-site (where the if logic branches are, which could be extracted, yep), or just iterate once here and keep the conditionals, but more importantly, it'd become an implicit decision: I'm happy for this to be explicit with a branch in each part because we want to remove it again soon. It's not a supported feature (why I left the chart requirement unchanged).

@absoludity absoludity merged commit 07d413d into main Mar 1, 2023
@absoludity absoludity deleted the 6021-cronjob-api branch March 1, 2023 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable CronJob api group to be configurable
3 participants