[Bug]: aws_security_group_rule: Changes to cidr_blocks causes Error: [WARN] A duplicate Security Group rule was found

[YakDriver]

This is a specific scenario related to a family of longstanding challenges with aws_security_group and aws_security_group_rule causing A duplicate Security Group rule was found. There are two purposes to this issue:

1. Shine light on a specific aspect of the bug
2. Act as an umbrella for the family of issues (see References below 👇 )

IMPORTANT NOTE

We highly recommend using aws_vpc_security_group_egress_rule and aws_vpc_security_group_ingress_rule instead of aws_security_group_rule. It may be useful to think of aws_security_group_rule as semi deprecated.

Terraform Core Version

1.9.2

AWS Provider Version

5.59.0

Affected Resource(s)

• aws_security_group_rule

Expected Behavior

Applying configuration should succeed recreating missing cidr_blocks.

Actual Behavior

Applying the configuration, after an out-of-band change to the CIDR blocks, causes an error.

Relevant Error/Panic Output Snippet

Error: [WARN] A duplicate Security Group rule was found on (sg-0738e88c121d67831). This may be
a side effect of a now-fixed Terraform issue causing two security groups with
identical attributes but different source_security_group_ids to overwrite each
other in the state. See https://github.com/hashicorp/terraform/pull/2376 for more
information and instructions for recovery. Error: InvalidPermission.Duplicate: the specified rule "peer: 19.0.0.0/24, TCP, from port: 636, to port: 636, ALLOW" already exists
       status code: 400, request id: 7b3a6f74-2027-4370-b44f-148f9028faf6

   with aws_security_group_rule.ec2_sg_rule_ecp_636[0],
   on main.tf line 25, in resource "aws_security_group_rule" "ec2_sg_rule_ecp_636":
   25: resource "aws_security_group_rule" "ec2_sg_rule_ecp_636" {

Terraform Configuration Files

resource "aws_security_group_rule" "ec2_sg_rule_ecp_636" {
  count             = 1
  description       = "Allow LDAP from on-premise"
  from_port         = 636
  protocol          = "tcp"
  security_group_id = aws_security_group.example.id
  to_port           = 636
  cidr_blocks       = ["19.0.0.0/24", "17.0.0.0/24", "20.0.0.0/24"]
  type              = "ingress"
}

Steps to Reproduce

1. Apply configuration
2. Remove one of the CIDR blocks with CLI or Console
3. Apply configuration

References

There seem to be many error reports in the same neighborhood:

• aws_security_group_rule requires replacement when ordering of cidr_blocks argument changes #19082
• Modifying cidr_blocks of resource aws_security_group_rule should not force a new resource #14966
• resource/aws_security_group_rule: Terraform state leaks on interrupt #31729
• [Bug]: TF often tries to create duplicate security group rules #29797
• [Bug]: Creation of aws_security_group_rule succeeds but is not recorded in state file  #27024
• A duplicate Security Group rule was found #26125
• A duplicate Security Group rule was found after second apply #25965
• r/aws_security_group_rule create_before_destroy fails when changing CIDRs #25173
• aws_security_group_rule requires replacement when ordering of cidr_blocks argument changes #19082
• aws_security_group_rule InvalidPermission.Duplicate - Should be additive #12450
• A security group with a combination of a CIDR, a SG, and self=true, applies but still shows changes #7425

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[YakDriver]

Potential workarounds:

GOOD: One rule per aws_security_group_rule resource

resource "aws_security_group_rule" "example" {
  description       = "Allow LDAP from on-premise"
  from_port         = 636
  protocol          = "tcp"
  security_group_id = aws_security_group.example.id
  to_port           = 636
  cidr_blocks       = ["18.0.0.0/24"]
  type              = "ingress"
}

resource "aws_security_group_rule" "example2" {
  description       = "Allow LDAP from on-premise"
  from_port         = 636
  protocol          = "tcp"
  security_group_id = aws_security_group.example.id
  to_port           = 636
  cidr_blocks       = ["19.0.0.0/24"]
  type              = "ingress"
}

BETTER: One rule per aws_security_group_rule resource (for_each)

variable "my_list" {
  type = list(string)
  default = ["18.0.0.0/24", "19.0.0.0/24"]
}

resource "aws_security_group_rule" "example" {
  for_each          = toset(var.my_list)
  description       = "Allow LDAP from on-premise"
  from_port         = 636
  protocol          = "tcp"
  security_group_id = aws_security_group.example.id
  to_port           = 636
  cidr_blocks       = [each.value]
  type              = "ingress"
}

BEST: One rule per aws_vpc_security_group_ingress_rule resource

variable "my_list" {
  type = list(string)
  default = ["18.0.0.0/24", "19.0.0.0/24"]
}

resource "aws_vpc_security_group_ingress_rule" "example" {
  for_each          = toset(var.my_list)
  description       = "Allow LDAP from on-premise"
  from_port         = 636
  ip_protocol       = "tcp"
  security_group_id = aws_security_group.example.id
  to_port           = 636
  cidr_ipv4         = each.value
}

[YakDriver]

For fans of history, the error message below was penned by Mitchell Hashimoto 9 years ago. Most error messages have moved to a more modern style. This one has been left as a tribute.

[WARN] A duplicate Security Group rule was found on (<id>). This may be
a side effect of a now-fixed Terraform issue causing two security groups with
identical attributes but different source_security_group_ids to overwrite each
other in the state. See https://github.com/hashicorp/terraform/pull/2376 for more
information and instructions for recovery. Error: <error>

[thefirstofthe300]

Question here: how do we enforce security group rule state, i.e. if someone creates a security group rule manually, how can we get Terraform to remove it? Using the aws_security_group ingress and egress blocks seemed to be the method of doing that and then was superseded by the aws_security_group_rule and aws_vpc_security_group_{ingress,egress}_rule resources. AFAIK, the individual resources are not fully-authoritative of what rules are in the security group.