Skip to content

Backport of fix(control-plane): acl tokens deleted while pods in graceful shutdown into release/1.1.x#3742

Merged
DanStough merged 2 commits intorelease/1.1.xfrom
backport/dans/NET-8345/graceful-shutdown-acl-race/generally-internal-piglet
Mar 20, 2024
Merged

Backport of fix(control-plane): acl tokens deleted while pods in graceful shutdown into release/1.1.x#3742
DanStough merged 2 commits intorelease/1.1.xfrom
backport/dans/NET-8345/graceful-shutdown-acl-race/generally-internal-piglet

Conversation

@hc-github-team-consul-core
Copy link
Copy Markdown
Collaborator

Backport

This PR is auto-generated from #3736 to be assessed for backporting due to the inclusion of the label backport/1.1.x.

🚨

Warning automatic cherry-pick of commits failed. If the first commit failed,
you will see a blank no-op commit below. If at least one commit succeeded, you
will see the cherry-picked commits up to, not including, the commit where
the merge conflict occurred.

The person who merged in the original PR is:
@DanStough
This person should manually cherry-pick the original PR into a new backport PR,
and close this one when the manual backport PR is merged in.

merge conflict error: POST https://api.github.com/repos/hashicorp/consul-k8s/merges: 409 Merge conflict []

The below text is copied from the body of the original PR.


Changes proposed in this PR

  • Users have reported that when GracefulShutdown is enabled, ACL tokens are deleted immediately when a pod goes in to TERMINATING, which can disrupt consul DNS and an app's shutdown procedure if that is required.
  • The new behavior is that if a pod still exists and has GracefulShutdown enabled, we will skip ACL and node cleanup, instead re-queue the event for the longest shutdown period, plus some buffer to cut down on noise.

How I've tested this PR

  • Manual testing in a local kube cluster, deploying an app w/ ACLs and graceful shutdown. I delete a pod and watch the control-plane and consul-dataplane logs to verify when the ACL token is cleaned up.
  • Additional unit tests added.

How I expect reviewers to test this PR

👀 ☕

Checklist


Overview of commits

@hashicorp-cla
Copy link
Copy Markdown

hashicorp-cla commented Mar 14, 2024

CLA assistant check
All committers have signed the CLA.

@DanStough DanStough force-pushed the backport/dans/NET-8345/graceful-shutdown-acl-race/generally-internal-piglet branch from 9f1fec4 to d90547f Compare March 14, 2024 16:19
@DanStough DanStough marked this pull request as ready for review March 14, 2024 16:19
@DanStough DanStough merged commit 6e2f282 into release/1.1.x Mar 20, 2024
@DanStough DanStough deleted the backport/dans/NET-8345/graceful-shutdown-acl-race/generally-internal-piglet branch March 20, 2024 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants