-
-
Notifications
You must be signed in to change notification settings - Fork 4.3k
feat: Add EKS Fargate support #866
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 12 commits
00f216b
8cd11c3
c3e5c43
ed47451
23dd5bf
1843863
026e422
064aca0
799c9f9
29509af
fe188c9
3ef8991
b770ef3
3cba018
1cbbaf4
ad96814
6822b43
0eef01f
40051b9
c054b7a
9cc9cec
9553f5c
e4c0d49
0a8e0a6
0885fdf
b6c8522
7199e01
47816b4
ba4aa63
85f1319
cdbc078
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| module "fargate" { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This needs a dependency on the aws-auth configmap. Otherwise spinning up fresh clusters with fargate enabled may fail due to the race condition of who creates the configmap first
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you provide some guidance on how to best achieve that?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Adding this dependency creates a cycle. Are you sure the fargate module would cause the
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, AWS automatically creates the aws-auth configmap when you create a fargate profile on a cluster that does not have the configmap. This is a race condition that caused some pain for early adopters of the managed node groups in this module. The problem with the cycle is using the cluster_name from the null_resource to create the IAM role. This could be avoided by using a dependency variable instead of the null_resource. Then only the |
||
| source = "./modules/fargate" | ||
| create = var.create_eks && var.create_eks_fargate | ||
| cluster_name = aws_eks_cluster.this[0].name | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| profiles = var.eks_fargate_profiles | ||
| subnets = var.subnets | ||
| tags = var.tags | ||
| cluster_primary_security_group_id = element(concat(aws_eks_cluster.this[*].vpc_config[0].cluster_security_group_id, list("")), 0) | ||
| worker_security_group_id = local.worker_security_group_id | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| # Allow Fargate pods and EC2 workers to communicate | ||
|
|
||
| resource "aws_security_group_rule" "eks_fargate1" { | ||
| count = var.create ? 1 : 0 | ||
| type = "ingress" | ||
| from_port = 0 | ||
| to_port = 65535 | ||
| protocol = "all" | ||
| security_group_id = var.worker_security_group_id | ||
| source_security_group_id = var.cluster_primary_security_group_id | ||
| } | ||
|
|
||
| resource "aws_security_group_rule" "eks_fargate2" { | ||
| count = var.create ? 1 : 0 | ||
| type = "ingress" | ||
| from_port = 0 | ||
| to_port = 65535 | ||
| protocol = "all" | ||
| security_group_id = var.cluster_primary_security_group_id | ||
| source_security_group_id = var.worker_security_group_id | ||
| } | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
|
|
||
| # EKS Fargate Pod Execution Role | ||
|
|
||
| data "aws_iam_policy_document" "eks_fargate_pod_assume_role" { | ||
itssimon marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| count = var.create ? 1 : 0 | ||
| statement { | ||
| effect = "Allow" | ||
| actions = ["sts:AssumeRole"] | ||
|
|
||
| principals { | ||
| type = "Service" | ||
| identifiers = ["eks-fargate-pods.amazonaws.com"] | ||
| } | ||
| } | ||
| } | ||
|
|
||
| resource "aws_iam_role" "eks_fargate_pod" { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you add in the |
||
| count = var.create ? 1 : 0 | ||
| name = format("%s-fargate", var.cluster_name) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The cluster and worker roles are created using
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the only open topic to resolve now. How would you like to handle the naming here @barryib ? |
||
| assume_role_policy = join("", data.aws_iam_policy_document.eks_fargate_pod_assume_role.*.json) | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| tags = merge(var.tags, { "kubernetes.io/cluster/${var.cluster_name}" = "owned" }) | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| } | ||
|
|
||
| resource "aws_iam_role_policy_attachment" "eks_fargate_pod" { | ||
| count = var.create ? 1 : 0 | ||
| policy_arn = "arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy" | ||
| role = join("", aws_iam_role.eks_fargate_pod.*.name) | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| } | ||
|
|
||
|
|
||
| # EKS Fargate profiles | ||
|
|
||
| resource "aws_eks_fargate_profile" "this" { | ||
| count = var.create ? local.profile_count : 0 | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| cluster_name = var.cluster_name | ||
| fargate_profile_name = format("%s-fargate-%s", var.cluster_name, var.profiles[count.index].namespace) | ||
| pod_execution_role_arn = join("", aws_iam_role.eks_fargate_pod.*.arn) | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| subnet_ids = var.subnets | ||
| tags = merge(var.tags, { "kubernetes.io/cluster/${var.cluster_name}" = "owned" }) | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| selector { | ||
| namespace = var.profiles[count.index].namespace | ||
| labels = var.profiles[count.index].labels | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| locals { | ||
| profile_count = length(var.profiles) | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| output "iam_role_name" { | ||
| description = "IAM role name for EKS Fargate pods" | ||
| value = element(concat(aws_iam_role.eks_fargate_pod.*.name, list("")), 0) | ||
| } | ||
|
|
||
| output "iam_role_arn" { | ||
| description = "IAM role ARN for EKS Fargate pods" | ||
| value = element(concat(aws_iam_role.eks_fargate_pod.*.arn, list("")), 0) | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| variable "create" { | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| description = "Controls if EKS Fargate resources should be created." | ||
| type = bool | ||
| default = false | ||
| } | ||
|
|
||
| variable "cluster_name" { | ||
| description = "Name of parent cluster." | ||
| type = string | ||
| } | ||
|
|
||
| variable "profiles" { | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| description = "List of EKS Fargate profiles to create." | ||
| type = list(object({ | ||
| namespace = string | ||
| labels = map(string) | ||
| })) | ||
| default = [] | ||
| } | ||
|
|
||
| variable "subnets" { | ||
| description = "A list of subnets for the EKS Fargate profiles." | ||
| type = list(string) | ||
| } | ||
|
|
||
| variable "tags" { | ||
| description = "A map of tags to add to all resources." | ||
| type = map(string) | ||
| } | ||
|
|
||
| variable "cluster_primary_security_group_id" { | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| description = "Cluster primary security group ID created by the EKS cluster on 1.14 or later. Referred to as 'Cluster security group' in the EKS console." | ||
| type = string | ||
| } | ||
|
|
||
| variable "worker_security_group_id" { | ||
itssimon marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| description = "Security group ID attached to the EKS workers." | ||
| type = string | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.