Skip to content

Conversation

@mkuratczyk
Copy link
Contributor

There were 3 issues:

  • the STS could get deleted before deletion label propagated, leading to some pods terminating for a long time, because they didn't skip quorum checks
  • there was an unnecessary explicit STS deletion
  • addRabbitmqDeletionLabel was not filtering by namespace, potentially performing operations on wrong/too many pods (if there were multiple clusters with the same name in different namespaces)

There were 3 issues:
* the STS could get delete before deletion label propagated,
  leading to some pods terminating for a long time, because they
  didn't skip quorum checks
* there was an unnecessary explicit STS deletion
* addRabbitmqDeletionLabel was not filtering by namespace,
  potentially performing operations on wrong/too many pods
  (if there were multiple clusters with the same name in different
  namespaces)
Copy link
Member

@Zerpet Zerpet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing to object, the solution looks good to me 👍 Glad you figured out the problem!

Out of curiosity, did you consider using CreateOrUpdate() function in removeFinalizer() ?
CreateOrUpdate does essentially what you are doing of Get prior to Update()

@mkuratczyk
Copy link
Contributor Author

I had not tried before but I've just pushed a commit which uses CreateOrUpdate and seems to work just as well.
Happy to keep this version if you prefer

@Zerpet Zerpet added this to the 2.12.2 milestone May 6, 2025
@Zerpet
Copy link
Member

Zerpet commented May 6, 2025

Ship it! :shipit:

@mkuratczyk mkuratczyk merged commit fe56e19 into main May 6, 2025
13 checks passed
@mkuratczyk mkuratczyk deleted the wait-for-label-propagation-before-deleting branch May 6, 2025 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants