Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG-5656] Annotate Jobs with parent ScaledJob generation #5876

Merged
merged 8 commits into from
Jul 30, 2024

Conversation

josefkarasek
Copy link
Member

@josefkarasek josefkarasek commented Jun 10, 2024

As reported in #5656, keda-operator Pod restart causes the deletion of existing Jobs (child resources of ScaledJob). The Jobs are re-created immediately, but the state/progress of the running Jobs is lost.

My approach to fixing this is to maintain relation between the ScaledJob and child Jobs using an Annotation:
"scaledjob.keda.sh/generation": $(ScaledJob.Generation)
This way the operator, during startup, will only delete Jobs with "scaledjob.keda.sh/generation" that doesn't match the generation of the parent ScaledJob.

Checklist

Fixes #5656

@josefkarasek josefkarasek marked this pull request as ready for review June 17, 2024 09:15
@josefkarasek josefkarasek requested a review from a team as a code owner June 17, 2024 09:15
@josefkarasek josefkarasek changed the title Annotate Jobs with parent ScaledJob generation [BUG-5656] Annotate Jobs with parent ScaledJob generation Jun 17, 2024
Copy link
Member

@zroubalik zroubalik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, thanks! Could you please also update changelog?

pkg/scaling/executor/scale_jobs.go Outdated Show resolved Hide resolved
@zroubalik
Copy link
Member

zroubalik commented Jun 25, 2024

/run-e2e internal
Update: You can check the progress here

zroubalik

This comment was marked as outdated.

Copy link
Member

@zroubalik zroubalik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josefkarasek the static check si failing on the Changelog entry, it needs to be alphabetically sorted, also please move this into Improvements section - it is an user-facing issue.

The rest looks good, thanks

@zroubalik zroubalik self-requested a review June 25, 2024 19:51
Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Josef Karasek <[email protected]>
@zroubalik
Copy link
Member

zroubalik commented Jul 30, 2024

/run-e2e internal
Update: You can check the progress here

Signed-off-by: Zbynek Roubalik <[email protected]>
@zroubalik zroubalik merged commit 7ca6708 into kedacore:main Jul 30, 2024
16 checks passed
@JorTurFer JorTurFer mentioned this pull request Jul 30, 2024
23 tasks
JorTurFer pushed a commit to JorTurFer/keda that referenced this pull request Jul 30, 2024
)

* Annotate Jobs with parent ScaledJob generation

Signed-off-by: Josef Karasek <[email protected]>

* fix tests

Signed-off-by: Josef Karasek <[email protected]>

* fix lint

Signed-off-by: Josef Karasek <[email protected]>

* fix log message

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

---------

Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Zbynek Roubalik <[email protected]>
Co-authored-by: Zbynek Roubalik <[email protected]>
JorTurFer pushed a commit to JorTurFer/keda that referenced this pull request Jul 30, 2024
)

* Annotate Jobs with parent ScaledJob generation

Signed-off-by: Josef Karasek <[email protected]>

* fix tests

Signed-off-by: Josef Karasek <[email protected]>

* fix lint

Signed-off-by: Josef Karasek <[email protected]>

* fix log message

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

---------

Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Zbynek Roubalik <[email protected]>
Co-authored-by: Zbynek Roubalik <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>
JorTurFer added a commit that referenced this pull request Jul 31, 2024
* bump golang

Signed-off-by: Jorge Turrado <[email protected]>

* chore: build with keda-tools:1.22.5 (#5971)

* chore: build with keda-tools:1.22.5
to resolve CVE-2024-24790, CVE-2024-24789, and CVE-2024-24791
bump github.com/Azure/azure-sdk-for-go/sdk/azidentity to resolve CVE-2024-35255

Signed-off-by: Paul Yu <[email protected]>

* chore: use go install instead of go get and replacing deprecated tools

Signed-off-by: Paul Yu <[email protected]>

* chore: vendor dependency cleanup

Signed-off-by: Paul Yu <[email protected]>

* Update missing references to 1.21

Signed-off-by: Jorge Turrado <[email protected]>

---------

Signed-off-by: Paul Yu <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>
Co-authored-by: Jorge Turrado Ferrero <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* [BUG-5922] Report failing ScaledJob triggers in status (#5916)

Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* [BUG-5656] Annotate Jobs with parent ScaledJob generation (#5876)

* Annotate Jobs with parent ScaledJob generation

Signed-off-by: Josef Karasek <[email protected]>

* fix tests

Signed-off-by: Josef Karasek <[email protected]>

* fix lint

Signed-off-by: Josef Karasek <[email protected]>

* fix log message

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

---------

Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Zbynek Roubalik <[email protected]>
Co-authored-by: Zbynek Roubalik <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* fix: `+srv` mongodb url scheme parsing bug (#5773)

This commit fixs issue #5760. where OP was facing problem with +srv schema

Signed-off-by: Rishikesh Betigeri <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* fix: issue when GitHub organization contains more than 30 repos (#5746)

Signed-off-by: Simon Kobler <[email protected]>
Signed-off-by: Simon Kobler <[email protected]>
Co-authored-by: Jorge Turrado Ferrero <[email protected]>
Co-authored-by: Simon Kobler <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* Fix scaler leak during cache refresh (#5807)

Signed-off-by: Guillaume Jacquet <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* Prepare release v2.14.1

Signed-off-by: Jorge Turrado <[email protected]>

* add missing change

Signed-off-by: Jorge Turrado <[email protected]>

* update changelog

Signed-off-by: Jorge Turrado <[email protected]>

* fix: e2e test regex check tag (#5831)

Signed-off-by: Jan Wozniak <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* Validate regex before building image for e2e test (#5783)

* added regex pre check before building image

Signed-off-by: Yaxhveer <[email protected]>

* updated changelog

Signed-off-by: Yaxhveer <[email protected]>

* refactored

Signed-off-by: Yaxhveer <[email protected]>

* corrected

Signed-off-by: Yaxhveer <[email protected]>

* corrected changelog

Signed-off-by: Yaxhveer <[email protected]>

* updated the workflow

Signed-off-by: Yaxhveer <[email protected]>

* updated the workflow

Signed-off-by: Yaxhveer <[email protected]>

---------

Signed-off-by: Yaxhveer <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* fix some pending tasks

Signed-off-by: Jorge Turrado <[email protected]>

* use AAD-Pod-Identity always

Signed-off-by: Jorge Turrado <[email protected]>

* use AAD-Pod-Identity always

Signed-off-by: Jorge Turrado <[email protected]>

---------

Signed-off-by: Jorge Turrado <[email protected]>
Signed-off-by: Paul Yu <[email protected]>
Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Zbynek Roubalik <[email protected]>
Signed-off-by: Rishikesh Betigeri <[email protected]>
Signed-off-by: Simon Kobler <[email protected]>
Signed-off-by: Simon Kobler <[email protected]>
Signed-off-by: Guillaume Jacquet <[email protected]>
Signed-off-by: Jan Wozniak <[email protected]>
Signed-off-by: Yaxhveer <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>
Co-authored-by: Paul Yu <[email protected]>
Co-authored-by: Josef Karasek <[email protected]>
Co-authored-by: Zbynek Roubalik <[email protected]>
Co-authored-by: Rishikesh <[email protected]>
Co-authored-by: Simon Kobler <[email protected]>
Co-authored-by: Simon Kobler <[email protected]>
Co-authored-by: Guillaume Jacquet <[email protected]>
Co-authored-by: Jan Wozniak <[email protected]>
Co-authored-by: Yashveer <[email protected]>
jkyros pushed a commit to jkyros/keda that referenced this pull request Aug 2, 2024
* bump golang

Signed-off-by: Jorge Turrado <[email protected]>

* chore: build with keda-tools:1.22.5 (kedacore#5971)

* chore: build with keda-tools:1.22.5
to resolve CVE-2024-24790, CVE-2024-24789, and CVE-2024-24791
bump github.com/Azure/azure-sdk-for-go/sdk/azidentity to resolve CVE-2024-35255

Signed-off-by: Paul Yu <[email protected]>

* chore: use go install instead of go get and replacing deprecated tools

Signed-off-by: Paul Yu <[email protected]>

* chore: vendor dependency cleanup

Signed-off-by: Paul Yu <[email protected]>

* Update missing references to 1.21

Signed-off-by: Jorge Turrado <[email protected]>

---------

Signed-off-by: Paul Yu <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>
Co-authored-by: Jorge Turrado Ferrero <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* [BUG-5922] Report failing ScaledJob triggers in status (kedacore#5916)

Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* [BUG-5656] Annotate Jobs with parent ScaledJob generation (kedacore#5876)

* Annotate Jobs with parent ScaledJob generation

Signed-off-by: Josef Karasek <[email protected]>

* fix tests

Signed-off-by: Josef Karasek <[email protected]>

* fix lint

Signed-off-by: Josef Karasek <[email protected]>

* fix log message

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

* update changelog

Signed-off-by: Josef Karasek <[email protected]>

---------

Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Zbynek Roubalik <[email protected]>
Co-authored-by: Zbynek Roubalik <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* fix: `+srv` mongodb url scheme parsing bug (kedacore#5773)

This commit fixs issue kedacore#5760. where OP was facing problem with +srv schema

Signed-off-by: Rishikesh Betigeri <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* fix: issue when GitHub organization contains more than 30 repos (kedacore#5746)

Signed-off-by: Simon Kobler <[email protected]>
Signed-off-by: Simon Kobler <[email protected]>
Co-authored-by: Jorge Turrado Ferrero <[email protected]>
Co-authored-by: Simon Kobler <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* Fix scaler leak during cache refresh (kedacore#5807)

Signed-off-by: Guillaume Jacquet <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* Prepare release v2.14.1

Signed-off-by: Jorge Turrado <[email protected]>

* add missing change

Signed-off-by: Jorge Turrado <[email protected]>

* update changelog

Signed-off-by: Jorge Turrado <[email protected]>

* fix: e2e test regex check tag (kedacore#5831)

Signed-off-by: Jan Wozniak <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* Validate regex before building image for e2e test (kedacore#5783)

* added regex pre check before building image

Signed-off-by: Yaxhveer <[email protected]>

* updated changelog

Signed-off-by: Yaxhveer <[email protected]>

* refactored

Signed-off-by: Yaxhveer <[email protected]>

* corrected

Signed-off-by: Yaxhveer <[email protected]>

* corrected changelog

Signed-off-by: Yaxhveer <[email protected]>

* updated the workflow

Signed-off-by: Yaxhveer <[email protected]>

* updated the workflow

Signed-off-by: Yaxhveer <[email protected]>

---------

Signed-off-by: Yaxhveer <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>

* fix some pending tasks

Signed-off-by: Jorge Turrado <[email protected]>

* use AAD-Pod-Identity always

Signed-off-by: Jorge Turrado <[email protected]>

* use AAD-Pod-Identity always

Signed-off-by: Jorge Turrado <[email protected]>

---------

Signed-off-by: Jorge Turrado <[email protected]>
Signed-off-by: Paul Yu <[email protected]>
Signed-off-by: Josef Karasek <[email protected]>
Signed-off-by: Zbynek Roubalik <[email protected]>
Signed-off-by: Rishikesh Betigeri <[email protected]>
Signed-off-by: Simon Kobler <[email protected]>
Signed-off-by: Simon Kobler <[email protected]>
Signed-off-by: Guillaume Jacquet <[email protected]>
Signed-off-by: Jan Wozniak <[email protected]>
Signed-off-by: Yaxhveer <[email protected]>
Signed-off-by: Jorge Turrado <[email protected]>
Co-authored-by: Paul Yu <[email protected]>
Co-authored-by: Josef Karasek <[email protected]>
Co-authored-by: Zbynek Roubalik <[email protected]>
Co-authored-by: Rishikesh <[email protected]>
Co-authored-by: Simon Kobler <[email protected]>
Co-authored-by: Simon Kobler <[email protected]>
Co-authored-by: Guillaume Jacquet <[email protected]>
Co-authored-by: Jan Wozniak <[email protected]>
Co-authored-by: Yashveer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Restart of keda-operator causes ScaledJob object to be updated
2 participants