Skip to content

🐛 Fix deregistering of deleted CAPI Machines #4701

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Dec 19, 2023
Merged

🐛 Fix deregistering of deleted CAPI Machines #4701

merged 1 commit into from
Dec 19, 2023

Conversation

tobiasgiese
Copy link
Member

@tobiasgiese tobiasgiese commented Dec 18, 2023 •
edited
Loading

What type of PR is this?

/kind bug

What this PR does / why we need it:

We have an issue that there is a gap between the CAPI Machine deletion and the AWSMachine deletion. In this timeframe the kube-apiserver to the to be deleted control plane is no longer possible, but the deregistering of the LB member will be performed only for AWSMachine deletions. This PR fixes the gap and also deregisteres the instance if the CAPI Machine is deleted.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #4697

Special notes for your reviewer:

Checklist:

  • squashed commits
  • includes documentation
  • includes emojis
  • adds unit tests
  • adds or updates e2e tests

Release note:

Resolved issue where deleted control planes were still registered in the kube-apiserver load balancer, causing potential disruptions.

Tobias Giese [email protected], Mercedes-Benz Tech Innovation GmbH, legal info/Impressum

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-priority size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Dec 18, 2023
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Dec 18, 2023
@tobiasgiese tobiasgiese mentioned this pull request Dec 18, 2023
Closed
@tobiasgiese tobiasgiese force-pushed the tobiasgiese/fix-deregister-scaledown branch from 7f832cb to 2cac3e7 Compare December 18, 2023 19:46
@tobiasgiese
Fix deregistering of deleted CAPI Machines
38ef1d3 
We have an issue that there is a gap between the CAPI Machine deletion and the AWSMachine deletion.
In this timeframe the kube-apiserver to the to be deleted control plane is no longer possible, but the deregistering of the LB member will be performed only for AWSMachine deletions.
This PR fixes the gap and also deregisteres the instance if the CAPI Machine is deleted.

Signed-off-by: Tobias Giese <[email protected]>
@tobiasgiese tobiasgiese force-pushed the tobiasgiese/fix-deregister-scaledown branch from 2cac3e7 to 38ef1d3 Compare December 18, 2023 19:56
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Dec 18, 2023
@richardcase
Copy link
Member

/test ?

@k8s-ci-robot
Copy link
Contributor

@richardcase: The following commands are available to trigger required jobs:

  • /test pull-cluster-api-provider-aws-build
  • /test pull-cluster-api-provider-aws-test
  • /test pull-cluster-api-provider-aws-verify

The following commands are available to trigger optional jobs:

  • /test pull-cluster-api-provider-aws-apidiff-main
  • /test pull-cluster-api-provider-aws-e2e
  • /test pull-cluster-api-provider-aws-e2e-blocking
  • /test pull-cluster-api-provider-aws-e2e-clusterclass
  • /test pull-cluster-api-provider-aws-e2e-conformance
  • /test pull-cluster-api-provider-aws-e2e-conformance-with-ci-artifacts
  • /test pull-cluster-api-provider-aws-e2e-eks
  • /test pull-cluster-api-provider-aws-e2e-eks-gc
  • /test pull-cluster-api-provider-aws-e2e-eks-testing

Use /test all to run the following jobs that were automatically triggered:

  • pull-cluster-api-provider-aws-apidiff-main
  • pull-cluster-api-provider-aws-build
  • pull-cluster-api-provider-aws-test
  • pull-cluster-api-provider-aws-verify

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@richardcase
Copy link
Member

Until the e2e passes:

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 19, 2023
@richardcase
Copy link
Member

/test pull-cluster-api-provider-aws-e2e

@richardcase
Copy link
Member

/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 19, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: richardcase

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 19, 2023
@richardcase
Copy link
Member

@tobiasgiese (or anyone else) - when the e2e run finished feel free to do an unhold.

@richardcase
Copy link
Member

/cherrypick release-2.3

@k8s-infra-cherrypick-robot
Copy link

@richardcase: once the present PR merges, I will cherry-pick it on top of release-2.3 in a new PR and assign it to you.

In response to this:

/cherrypick release-2.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

AndiDog
AndiDog reviewed Dec 19, 2023
View reviewed changes
Copy link
Contributor

@AndiDog AndiDog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

but as usual, changes will haunt us later if there's no test. I'll have maintainers decide here, since I'm not an expert of this part of the code.

@tobiasgiese
Copy link
Member Author

but as usual, changes will haunt us later if there's no test. I'll have maintainers decide here, since I'm not an expert of this part of the code.

Indeed this should be done. But I think it's fair to say that this will be addressed in a follow up PR/issue.
WDYT @richardcase

@tobiasgiese
Copy link
Member Author

e2e tests passed
/unhold

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 19, 2023
@tobiasgiese
Copy link
Member Author

@AndiDog issue #3820 will cover this issue IMO

@k8s-ci-robot k8s-ci-robot merged commit bba04c4 into kubernetes-sigs:main Dec 19, 2023
@k8s-infra-cherrypick-robot
Copy link

@richardcase: new pull request created: #4704

In response to this:

/cherrypick release-2.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Dec 19, 2023
Merged
@tobiasgiese tobiasgiese deleted the tobiasgiese/fix-deregister-scaledown branch December 19, 2023 11:10
@tobiasgiese
Copy link
Member Author

/cherrypick release-2.2

I think we should backport it @richardcase

@k8s-infra-cherrypick-robot
Copy link

@tobiasgiese: new pull request created: #4706

In response to this:

/cherrypick release-2.2

I think we should backport it @richardcase

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Dec 20, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AndiDog AndiDog AndiDog left review comments

@fiunchinho fiunchinho Awaiting requested review from fiunchinho

@nrb nrb Awaiting requested review from nrb

Assignees

@richardcase richardcase

@AndiDog AndiDog

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority release-note Denotes a PR that will be considered when it comes time to generate release notes. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Control plane node replacement causes kube-apiserver access issues due to delayed AWSMachine deletion
5 participants
@tobiasgiese @richardcase @k8s-ci-robot @k8s-infra-cherrypick-robot @AndiDog