feat!: Upgrade AWS provider and min required Terraform version to 6.0 and 1.5.7 respectively

[bryantbiggs]

Description

List of backwards incompatible changes

• Terraform v1.5.7 is now minimum supported version
• AWS provider v6.0.0 is now minimum supported version
• The attributes used to construct the container definition(s) have been changed from HCL's norm of snake_case to camelCase to match the AWS API. There currently isn't a resource nor data source for the container definition, so one is constructed entirely from HCL in the container-definition sub-module. This definition is then rendered as JSON when presented to the task definition (or task set) APIs. Previously, the variable names used were snake_case and then internally converted to camelCase. However, this does not allow for using the container-definition sub-module on its own due to the mismatch between casing. Its probably going to trip a few folks up, but hopefully we'll remove this for a data source in the future.
• security_group_rules has been split into security_group_ingress_rules and security_group_egress_rules to better match the AWS API and allow for more flexibility in defining security group rules.
• Default permissive permissions for SSM parameter ARNs and Secrets Manager secret ARNs have been removed throughout. While this made it easier for users since it "just worked", it was not secure and could lead to unexpected access to resources. Users should now explicitly define the permissions they need in their IAM policies.
• The "hack" put in place to track the task definition version when updating outside of the module has been removed. Instead, users should rely on the track_latest variable to ensure that the latest task definition is used when updating the service. Any issues with tracking the task definition version should be reported to the ECS service team as it is a limitation of the AWS ECS service/API and not the module itself.
• The inline policy for the Tasks role of the service sub-module has been replaced with a standalone IAM policy. In some organizations, inline policies are not allowed.
• The default for the container-definition user has been changed from 0 to null.

Additional changes

Added

• Support for region parameter to specify the AWS region for the resources created if different from the provider region.
• Support for ECS infrastructure IAM role creation in the service sub-module. This role is used to manage ECS infrastructure resources https://docs.aws.amazon.com/AmazonECS/latest/developerguide/infrastructure_IAM_role.html

Modified

• Variable definitions now contain detailed object types in place of the previously used any type.

Variable and output changes

1. Removed variables:

   • default_capacity_provider_use_fargate

   • fargate_capacity_providers

   • cluster sub-module

     • fargate_capacity_providers; part of default_capacity_provider_strategy now
     • default_capacity_provider_use_fargate

   • container-definition sub-module

     • None

   • service sub-module

     • inference_accelerator

2. Renamed variables:

   • cluster_settings -> cluster_setting

   • cluster sub-module

     • cluster_configuration - configuration
     • cluster_settings - setting
     • cluster_service_connect_defaults - service_connect_defaults

   • container-definition sub-module

     • dependencies - dependsOn
     • disable_networking - disableNetworking
     • dns_search_domains - dnsSearchDomains
     • dns_servers - dnsServers
     • docker_labels - dockerLabels
     • docker_security_options - dockerSecurityOptions
     • environment_files - environmentFiles
     • extra_hosts - extraHosts
     • firelens_configuration - firelensConfiguration
     • health_check - healthCheck
     • linux_parameters - linuxParameters
     • log_configuration - logConfiguration
     • memory_reservation - memoryReservation
     • mount_points - mountPoints
     • port_mappings - portMappings
     • psuedo_terminal - pseudoTerminal
     • readonly_root_filesystem - readonlyRootFilesystem
     • repository_credentials - repositoryCredentials
     • start_timeout - startTimeout
     • system_controls - systemControls
     • volumes_from - volumesFrom
     • working_directory - workingDirectory

   • service sub-module

     • None

3. Added variables:

   • cloudwatch_log_group_class

   • default_capacity_provider_strategy

   • cluster sub-module

     • cloudwatch_log_group_class
     • default_capacity_provider_strategy - replaces fargate_capacity_providers and default_capacity_provider_use_fargate functionality

   • container-definition sub-module

     • log_group_class
     • restartPolicy - defaults to enabled = true
     • versionConsistency - defaults to "disabled" Issues with "software version consistency" feature aws/containers-roadmap#2394

   • service sub-module

     • availability_zone_rebalancing
     • volume_configuration
     • vpc_lattice_configurations
     • enable_fault_injection
     • track_latest
     • create_infrastructure_iam_role
     • infrastructure_iam_role_arn
     • infrastructure_iam_role_name
     • infrastructure_iam_role_use_name_prefix
     • infrastructure_iam_role_path
     • infrastructure_iam_role_description
     • infrastructure_iam_role_permissions_boundary
     • infrastructure_iam_role_tags

4. Removed outputs:

   • cluster sub-module
     • None
   • container-definition sub-module
     • None
   • service sub-module
     • task_definition_family_revision

5. Renamed outputs:

   • cluster sub-module
     • None
   • container-definition sub-module
     • None
   • service sub-module
     • None

6. Added outputs:

   • cluster sub-module
     • None
   • container-definition sub-module
     • None
   • service sub-module
     • infrastructure_iam_role_arn
     • infrastructure_iam_role_name

Motivation and Context

• Resolves container-definition module not usable on its own #147
• Resolves port_mappings are always empty #151
• Resolves Replace hack with track_latest attribute #164
• Resolves The recommended workaround for ignoring task definition changes causes the service's container_definitions to be overwritten on every Terraform apply, even ones that don't touch the service. #165
• Resolves Service module defaulting user to root #221
• Resolves customized_metric_specification metrics is missing #225
• Resolves Can we support RestartPolicy #230
• Resolves Add support for managed_storage_configuration block #233
• Resolves Add containerPortRange definition support for tasks #234
• Resolves Add support for managed_draining for capacity provider #235
• Resolves Dynamically add container(s) to container_definitions #240
• Resolves Add support for created cloudwatch logs with user defined log group class #241
• Resolves Explicit task_exec_secret_arns #244
• Resolves Encryption for Fargate Ephemoral Storage #258
• Resolves Support of EBS volumes #254
• Resolves Enable Fault Injection in ECS task #259
• Resolves can't provide secrets to service #260
• Resolves Feature Request: Availability Zone Rebalancing Input #262
• Resolves Add support for IAM task role policy path in ECS service module #265
• Resolves Feature Request: Attach EBS volumes on fargate #281
• Resolves service_registries dynamic block incorrectly wraps map in list, breaking input processing #287
• Resolves service_connect_configuration should allow multiple services #286
• Resolves service_registries dynamic block incorrectly wraps map in list, breaking input processing #285
• Resolves Feature: service connect multiple service #289
• Resolves module fails with aws provider v6.0 #291
• Resolves service_connect_configuration does not support multiple port mappings #292
• Resolves Unsupported block type #297
• Resolves terraform-aws-ecs Warning: Deprecated attribute #298
• Resolves terraform-aws-ecs Error: Unsupported block type. Line: dynamic "inference_accelerator" #299
• Closes feat: Added managed storage KMS configuration #274
• Closes feat: Add version consistency support in container definition #280
• Closes fix: Limit aws provider version to < 6 #294

Breaking Changes

• Yes, see docs/UPGRADE-6.0.md

How Has This Been Tested?

• I have updated at least one of the examples/* to demonstrate and validate my change(s)
• I have tested and validated these changes using one or more of the provided examples/* projects

• I have executed pre-commit run -a on my pull request

[kkrastev-cloudoffice]

any ETA on this?

[BramRoets]

@bryantbiggs Could you please provide an update on this PR? It adds multiple important features which are standard to AWS ECS.

It's been open for a very long time. Is this still being maintained or should we move away from this project?

[bryantbiggs]

yes, part of it is finding time since these large, breaking changes do take a considerable amount of time to test and document, and part of it is balancing the number of times we take a breaking change (major version bump). with v6 of the provider coming, I'm half inclined to wait and set the minimum provider version to 6.0 hashicorp/terraform-provider-aws#41101 to have a stable path forward for quite some time after

[nZac]

@bryantbiggs if I can be of use in testing I'd be glad to help run this through the paces. Do you have a pattern or practice you use and documentation you would need to validate that testing is sufficient?

[webyneter]

FYI, had to set these to work around validation errors, in a scenario when none of these were meant to be set :

tasks_iam_role_statements = []
security_group_ingress_rules = {}
security_group_egress_rules = {}

[webyneter]

Also, FYI, var.track_latest description is off: It says, the default is false whereas the actual default = true.

In addition to that, there's no way to set var.track_latest from the root module, there's neither a track_latest parameter on the module.service, nor the corresponding root module variable.

[nikita-toffee-ai]

The enabled field in the var.service_connect_configuration is now enabled = true by default, so even if I might not need service discovery, it now requires me to specify the namespace regardless.

[webyneter]

tag_specifications = optional(list(object({
        propagate_tags = optional(string, "TASK_DEFINITION")
        resource_type  = string
        tags           = optional(map(string))
      })))

I think, we should make resource_type = optional(string, "volume"), besides, "volume" is the only value that's currently allowed.

[bryantbiggs]

note to self: default_capacity_provider_strategy was properly borked the way it was previously configured 😅

[bryantbiggs]

I win 😅

[antonbabenko]

Looks great!

[antonbabenko]

Can tflint recognize inline comments to mark just unsupported places? Other tools like tfsec and checkov can, so maybe tflint can also?

[bryantbiggs]

oh that was much easier than I thought it would be 😅 - fixed in 6d0ff2b

[willfish]

I'm integrating this in our repo, here, and all seems to work as before trade-tariff/trade-tariff-platform-aws-terraform#416

[antonbabenko]

This PR is included in version 6.0.0 🎉

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.