[iam-role-for-service-accounts-eks] Karpenter IAM Policy expects a tag

[dewjam]

Description

When using the iam-role-for-service-accounts-eks module to create the Karpenter Controller IAM policy, a condition is set by default which assumes all resources created by Karpenter have a specific Tag (karpenter.sh/discovery). As a result, you must also supply this Tag in the Karpenter Provisioner spec otherwise Karpenter will not have permissions to manage all the necessary AWS resources. (related to aws/karpenter-provider-aws#1488)

It seems it would be ideal to make this condition configurable and have it default to no condition. This isn't really a bug, so looking for feedback about how this should be handled.

---

• Terraform: v1.1.6

• Provider(s):

on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.4.0
+ provider registry.terraform.io/hashicorp/cloudinit v2.2.0
+ provider registry.terraform.io/hashicorp/helm v2.4.1
+ provider registry.terraform.io/hashicorp/tls v3.1.0

• Module: iam-role-for-service-accounts-eks, Karpenter IRSA

Reproduction

Steps to reproduce the behavior:

1. Deploy the Terraform plan from the gist below. This will create an EKS cluster and deploy Karpenter to the cluster.
2. Deploy the Karpenter provisioner.yaml spec from the gist below.
3. Create a deployment and scale it up so Karpenter attempts to create new nodes.
4. Karpenter will fail to launch instances due to permissions problems with the RunInstances action.

https://gist.github.com/dewjam/a17f428dab130a5252b355e5c2c1851b

Expected behavior

We expect Karpenter to be able to execute the RunInstances, TerminateInstances, or DeleteLaunchTempate actions on resources it creates.

Actual behavior

By default, Karpenter is unable to execute the RunInstances, TerminateInstances, or DeleteLaunchTempate actions.

Additional Info

You can work around this by including the necessary tags in the provisioner spec.

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:
  tags:
     karpenter.sh/discovery: <cluster_name>

[bryantbiggs]

thanks for the issue @dewjam - I've re-opened my pull request aws/karpenter-provider-aws#1332 related to this. Once I have a resolution with the karpenter project, we can make the necessary changes to suit (if needed)

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.