ingress_with_cidr_blocks do not use prefix_list_ids from map entry

[Remy-Mollandin-SK5]

Description

Complete example shows a map entry named "prefix_list_ids". However, the code shows that only var.ingress_prefix_list_ids is used..

• ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]: 4.17
• Terraform version: 1.3.9
• Provider version(s): 4.60

Reproduction Code [Required]

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_ec2_managed_prefix_list" "one" {
  address_family = "IPv4"
  max_entries    = 1
  name           = "pl-one"
}

resource "aws_ec2_managed_prefix_list" "two" {
  address_family = "IPv4"
  max_entries    = 1
  name           = "pl-two"
}

module "prefix_list" {
  source  = "terraform-aws-modules/security-group/aws"
  version = "~> 4.17"

  name        = "pl-sg"
  description = "Security group with prefix list"
  vpc_id      = aws_vpc.main.id

  ingress_with_cidr_blocks = [
    {
      from_port       = 9100
      to_port         = 9100
      protocol        = 6 # "tcp"
      description     = "Arbitrary TCP port"
      prefix_list_ids = join(",", [aws_ec2_managed_prefix_list.one.id, aws_ec2_managed_prefix_list.two.id])
    },
  ]
}

Steps to reproduce the behavior:

terraform init && terraform apply

Expected behavior

Security group rule using prefix_list_ids mentioned in the map

Actual behavior

Security group rule with empty prefix_list_ids. Apply running for 5 minutes before crashing:

╷
│ Error: waiting for Security Group (sg-01bb4ea80ed15aede) Rule (sgrule-3779539315) create: couldn't find resource
│ 
│   with module.prefix_list.aws_security_group_rule.ingress_with_cidr_blocks[0],
│   on .terraform/modules/prefix_list/main.tf line 191, in resource "aws_security_group_rule" "ingress_with_cidr_blocks":
│  191: resource "aws_security_group_rule" "ingress_with_cidr_blocks" {
│ 

Rule during plan:

  # module.prefix_list.aws_security_group_rule.ingress_with_cidr_blocks[0] will be created
  + resource "aws_security_group_rule" "ingress_with_cidr_blocks" {
      + cidr_blocks              = []
      + description              = "Arbitrary TCP port"
      + from_port                = 9100
      + id                       = (known after apply)
      + prefix_list_ids          = []
      + protocol                 = "tcp"
      + security_group_id        = (known after apply)
      + security_group_rule_id   = (known after apply)
      + self                     = false
      + source_security_group_id = (known after apply)
      + to_port                  = 9100
      + type                     = "ingress"
    }

Terminal Output Screenshot(s)

Additional context

This is troublesome if we have multiple ingress_with_cidr_blocks or ingress_with_source_security_group_id for the same port as it will try to add the prefix list rule to each.

Example:

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_ec2_managed_prefix_list" "one" {
  address_family = "IPv4"
  max_entries    = 1
  name           = "pl-one"
}

resource "aws_ec2_managed_prefix_list" "two" {
  address_family = "IPv4"
  max_entries    = 1
  name           = "pl-two"
}

resource "aws_security_group" "one" {
  vpc_id = aws_vpc.main.id
}

resource "aws_security_group" "two" {
  vpc_id = aws_vpc.main.id
}

module "prefix_list" {
  source  = "terraform-aws-modules/security-group/aws"
  version = "~> 4.17"

  name        = "pl-sg"
  description = "Security group with prefix list"
  vpc_id      = aws_vpc.main.id

  ingress_prefix_list_ids = [
    aws_ec2_managed_prefix_list.one.id,
    aws_ec2_managed_prefix_list.two.id
  ]
  ingress_with_source_security_group_id = [
    {
      from_port       = 9100
      to_port         = 9100
      protocol        = 6 # "tcp"
      description     = "Arbitrary TCP port"
      source_security_group_id = aws_security_group.one.id
    },
    {
      from_port       = 9100
      to_port         = 9100
      protocol        = 6 # "tcp"
      description     = "Arbitrary TCP port"
      source_security_group_id = aws_security_group.two.id
    }
  ]
}

Results:

│ Error: [WARN] A duplicate Security Group rule was found on (sg-01bb4ea80ed15aede). This may be
│ a side effect of a now-fixed Terraform issue causing two security groups with
│ identical attributes but different source_security_group_ids to overwrite each
│ other in the state. See https://github.com/hashicorp/terraform/pull/2376 for more
│ information and instructions for recovery. Error: InvalidPermission.Duplicate: the specified rule "peer: pl-09b782041201388e9, TCP, from port: 9100, to port: 9100, ALLOW" already exists
│       status code: 400, request id: 02bc7f1d-35d9-483b-945b-f4ec90164cb1
│ 
│   with module.prefix_list.aws_security_group_rule.ingress_with_source_security_group_id[1],
│   on .terraform/modules/prefix_list/main.tf line 103, in resource "aws_security_group_rule" "ingress_with_source_security_group_id":
│  103: resource "aws_security_group_rule" "ingress_with_source_security_group_id" {

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions]

This issue was automatically closed because of stale in 10 days

[busuusteph]

fwiw I have just seen the same

[cnorthfield]

Can this be looked at?

[Remy-Mollandin-SK5]

A workaround is to instantiate another module with create_sg set to False and configure the prefix list in this module. This issue has been closed so it might be worth to open a new one ?

I've opened a PR few months ago that was fixing this (among a lot of other changes): #282.

[cnorthfield]

Thanks, @Remy-Mollandin-SK5. It's a shame #282 got closed without review.

The lighter-weight workaround I'm using is to create a separate rule (in my case a aws_vpc_security_group_egress_rule resource) and attach it to the security group:

module "example" {
  source = "terraform-aws-modules/security-group/aws"
  # ...
}

resource "aws_vpc_security_group_egress_rule" "contact_importer_lambda_vpc_s3_tls" {
  security_group_id = module.example.security_group_id
  # ...
  prefix_list_id = "prefix"
}

[antonbabenko]

I believe this has been improved in #258 . Please use version 5.1.0 of this module and check out the examples:

terraform-aws-security-group/examples/complete/main.tf

Lines 455 to 477 in 43974e9

	module "prefix_list_sg" {
	source = "../../"
	name = "prefix-list-sg"
	description = "Security group using prefix list and custom ingress rules"
	vpc_id = data.aws_vpc.default.id
	use_name_prefix = false
	ingress_prefix_list_ids = [aws_ec2_managed_prefix_list.prefix_list_sg_example.id]
	ingress_with_prefix_list_ids = [
	{
	from_port = 80
	to_port = 80
	protocol = "tcp"
	},
	{
	from_port = 443
	to_port = 443
	protocol = "tcp"
	},
	]
	}

[cnorthfield]

The issue still appears to be present for egress.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.