feat: invalidate SSM cache upon AMI deprecation

[jmdeal]

Fixes #N/A

Description

When an EKS-optimized AMI is deprecated, and a user is using a family@latest alias, Karpenter will continue to launch with that AMI until the 24 hour SSM cache entry has expired. This cache ensures Karpenter doesn't cause a thundering herd by upgrading all users within a region simeltaneously upon a SSM parameter rollout. However, Karpenter should respond faster in the event of an EKS-optimized AMI deprecation.

This PR introduces a mechanism to invalidate the SSM cache upon detection of a deprecated EKS-optimized AMI. This cache invalidation is still staggered over 30 minutes to reduce the risk of a thundering herd, but this is still up to 48x the reaction time Karpenter previously had. Additionally, thanks to the 24 hour cache, only a subset of users using family@latest will have been upgraded to the deprecated AMI in the first place, further reducing the chances of a thundering herd upon rollback.

Some assumptions made:

• The latest SSM parameter for EKS optimized AMIs should be rolled back before AMI deprecation. If this is not the case, Karpenter will continuously invalidate the SSM cache until the parameter points to a non-deprecated AMI.
• Karpenter will only need to rollback a subset of users if the issue is detected and the AMI is deprecated within 24 hours of the AMI's release. Otherwise, we expect all users with a family@latest alias (and non-blocking drift budgets / PDBs) to have been upgraded to the now-deprecated AMI.

How was this change tested?
make test (additional tests in-progress)

Does this change impact docs?

• Yes, PR includes docs updates
• Yes, issue opened: #
• No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[netlify]

✅ Deploy Preview for karpenter-docs-prod canceled.

Name	Link
🔨 Latest commit	106b0e7
🔍 Latest deploy log	https://app.netlify.com/sites/karpenter-docs-prod/deploys/672415096affee00083a328f

[jmdeal]

/karpenter snapshot

[github-actions]

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-93bd2cd2c2124923e5686e057a4af017b25b360a.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-93bd2cd2c2124923e5686e057a4af017b25b360a" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

[coveralls]

Pull Request Test Coverage Report for Build 11621534000

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 90 of 117 (76.92%) changed or added relevant lines in 12 files are covered.
• 9 unchanged lines in 2 files lost coverage.
• Overall coverage decreased (-0.2%) to 82.679%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controllers/controllers.go	0	2	0.0%
pkg/apis/v1/ec2nodeclass.go	15	18	83.33%
pkg/operator/operator.go	0	4	0.0%
pkg/controllers/providers/ssm/invalidation/controller.go	35	43	81.4%
pkg/apis/v1/zz_generated.deepcopy.go	0	10	0.0%

Files with Coverage Reduction	New Missed Lines	%
pkg/providers/instance/instance.go	2	89.12%
pkg/providers/instancetype/types.go	7	96.63%

Totals
Change from base Build 11567721194:	-0.2%
Covered Lines:	5704
Relevant Lines:	6899

---

💛 - Coveralls

[jonathan-innis]

Muy bonita 🎉

[jonathan-innis]

LGTM 🚀

[jonathan-innis]

LGTM 🚀