Feature: ECS Fargate Resources

[oharaandrew314]

SAM is currently designed to be used with Lambda for creating serverless applications.

But what about Fargate for ECS? This service allows you to run docker applications serverless-ly, but It requires an obscene amount of complicated boilerplate CFN to get even the simplest web application running.

I think it would be beneficial to have a virtual resource, e.g. AWS::Serverless::FargateApp that implicitly generates a service, task definition, and all associated public-facing load balancer resources if necessary. To keep things simple, a FargateApp will have only one container definition.

I can help contribute a spec if this idea is accepted.

[sanathkr]

This is an interesting idea, but I don't know if it will fit our immediate focus of making serverless application development experience super awesome. I will tag it as a feature request and keep the issue open to gather +1s and revisit later.

[joeyvmason]

Any update on this? Would be really useful IMO. Configuring even a basic Fargate task in CloudFormation is incredibly daunting.

[brettstack]

Use cases and suggestions on how SAM can simplify/abstract would be very helpful. Thanks.

[clareliguori]

Hi all, I'm from Fargate/ECS, and want to reiterate what Brett said: If you have some examples for how you'd like to see Fargate fit into SAM, I would definitely like to hear them!

So far on the Fargate team, we've focused on building AWS CDK applets to abstract Fargate applications in a declarative format (very similar to the FargateApp suggestion by @oharaandrew314 with service, task def, load balancer, etc all generated). See example here, and docs here. If you have feedback on the applets, we're eager to hear it on the CDK repo. But again, if you have suggestions specifically for Fargate-in-SAM, fire away!

[hoegertn]

I have written a CFN macro for fagate (https://github.com/taimos/cfn-macro-fargate) that could be a base for a discussion on how to add Fargate to SAM. I talked to @cmmeyer about this at re:Invent

[jlhood]

@hoegertn Took a look through your macro. Seems like a very useful abstraction! Not sure we would support this as a native SAM resource type, but have you considered sharing the lambda function backing this macro in SAR? SAR doesn't allow the actual AWS::CloudFormation::Macro resource to be published, but you could share the macro function and then consumers could use nested apps to pull it into their template and then create the macro themselves.

[deleugpn]

@clareliguori The problem with CDK is that it brings a high overhead of re-learning and redesigning the DevOps mechanics, specially from a company that uses CodePipeline with CloudFormation as the deploy step. Switching from Containers (Source, CodeBuild, CloudFormation) to (Source, CodeBuild, CloudFormation) where CodeBuild used to build docker containers and now just aws cloudformation package instead is far easier than switching from CloudFormation to CDK.

The Serverless framework provides a simple, yet powerful abstraction on top of the extreme complexity of CloudFormation. Simplifying Fargate deployments could be an amazing improvement. Here's some of my proposals as requested by @brettstack (Note: I think this works well in combination of #721 (comment))

FargateContainer:
  Type: AWS::Serverless::Fargate
  Properties:
    Role: !GetAtt ContainerRole.Arn
    Image: !Sub '${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/my-repository:{TagParameter}'
    Command: my-docker-command
    DesireCount: 1
    MemorySize: 1024
    CpuSize: 512
    Environment:
      Variables:
        MyVariable: 'MyValue'

This would provide the bare-minimum for a Fargate Container (as it works with Lambda). This part is the simplest one and would mostly abstract just the Task Definition.

  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      ContainerDefinitions:
        - Environment:
            - Name: MyVariable
              Value: MyValue

          Name: Optionally available at FargateContainer.Name
          Essential: true
          Image: Available at FargateContainer.Image
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: !Ref LogGroup
              awslogs-region: !Ref 'AWS::Region'
              awslogs-stream-prefix: task

          Privileged: 'false'
      Cpu: Available at FargateContainer.CpuSize
      Memory: Available at FargateContainer.MemorySize
      Family: Optionally available at FargateContainer.Name or FargateContainer.Name?
      NetworkMode: awsvpc
      ExecutionRoleArn: Optionally available at FargateContainer.Role but fallback to !Sub 'arn:aws:iam::${AWS::AccountId}:role/ecsTaskExecutionRole'
      TaskRoleArn: Optionally available at FargateContainer.Role but fallback to !Sub 'arn:aws:iam::${AWS::AccountId}:role/ecsTaskExecutionRole'
      RequiresCompatibilities: [FARGATE]

  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      RetentionInDays: 30

I'd expect Global Environment variables to be applied to the Task Definition.

---

Things get more interesting when we bring the Events tag to Fargate. The first and most useful for me would be ALB.

    VpcConfig:
      SecurityGroupIds: [!ImportValue ContainerSecurityGroup]
      SubnetIds: !Split [',', !ImportValue PrivateSubnets]
    Events:
      WebContainer:
        Type: ALB
        Properties:
          LoadBalancerArn: !ImportValue LoadBalancer
          ListenerArn: !ImportValue Listener
          CertificateArn: !ImportValue Certificate
          Condition: [...]
          HealthCheckPath: /healthy.php

This would generate the following boilerplate:

  HttpsListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      Actions:
        - Type: forward
          TargetGroupArn:
            Ref: TargetGroup
      Conditions: {Available at Event Property}
      ListenerArn: {Available At Event Property}
      Priority: // This is a tricky one, see https://github.com/awslabs/serverless-application-model/issues/721#issuecomment-470358435

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      HealthCheckIntervalSeconds: 30
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 15
      HealthyThresholdCount: 2
      UnhealthyThresholdCount: 3
      HealthCheckPath: {Available at Event Property}
      Matcher:
        HttpCode: '200'
      Port: 80
      Protocol: HTTP
      TargetGroupAttributes:
        - Key: deregistration_delay.timeout_seconds
          Value: '30'
      TargetType: ip
      VpcId: {I guess this can be figured out by the Private Subnet?}

  Service:
    Type: AWS::ECS::Service
    Properties:
      Cluster: {Either make a new one or optionally get it from FargateContainer.Cluster}
      LaunchType: FARGATE
      DesiredCount: {FargateContainer.DesiredCount}
      LoadBalancers:
        - ContainerName: TaskDefinition.ContainerDefinitions.0.Name
          ContainerPort: 80
          TargetGroupArn: !Ref TargetGroup
      NetworkConfiguration:
        AwsvpcConfiguration:
          SecurityGroups: {FargateContainer.VpcConfig.SecurityGroupIds}
          Subnets: {FargateContainer.VpcConfig.SubnetIds}
      TaskDefinition: !Ref TaskDefinition

Obviously I'm biased and don't know what would be a sensible default for the health check configuration. But I believe AWS has the resources to analyze what would be a sensible default. I'd even be willing to just accept whatever AWS defines as default so I don't have to configure that amount of variables.

From all this, I feel there's one tricky metric: TaskDefinition.Properties.ContainerDefinitions.0.PortMappings. However, I think we might agree that worst-case scenario we'd have 1 more attribute at the AWS::Serverless::Fargate resource to define the port and it would cascade to Task Definition, Target Group and Service.

---

For Scheduled Task (provided someday Fargate finally provides Scheduled Tasks, per aws/containers-roadmap#392), I'd expect the syntax to be somewhat like the following:

      Events:
        HourlyScheduling:
          Type: Schedule
          Properties:
            Schedule: cron(0 * * * ? *)

Which would create the following resource

  HourlySchedulingRule:
    Type: AWS::Events::Rule
    Properties:
      Description: {Optionally Available?}
      ScheduleExpression: cron(0 * * * ? *)
      State: ENABLED
      Targets:
        - Id: {The name defined for the generated resource TaskDefinition}
          EcsParameters:
            LaunchType: FARGATE
            PlatformVersion: LATEST
            TaskCount: 1
            TaskDefinitionArn: {Task Definition Resource}
            NetworkConfiguration:
              AwsvpcConfiguration:
                SecurityGroups:
                  - {Available Globally or in the AWS::Serverless::Fargate Resource}
                Subnets: {Available Globally or in the AWS::Serverless::Fargate Resource}
          Arn: [Create one or optionally take it from FargateContainer.Cluster]
          RoleArn: [Make a role with the permission to ecs:RunTask and iam:PassRole

This is my guess to what scheduling Fargate on CloudFormation would be based on AWS CLI documentation.

I'd also propose a syntax for running containers triggered by SQS, but I'm going to hold off on that for now to see what AWS / the community has to say about what I'm proposing so far first.

[deleugpn]

@mgrandis needs feedback from whom? What kind of feedback?

[ghost]

ping

[mstone-modulus]

Hi,

I agree with the rest of the thread that it would be helpful to specify Fargate/ecs resources through SAM.

Additionally, the previously linked example and docs for working with fargate/ecs through cdk are no longer available. Are there other suggested practices for building serverless applications that use fargate/ecs?

Thanks!

[clareliguori]

Here are updated links for CDK with ECS/Fargate: docs, example

[Barhuumi]

I support this. SAM support for Fargate would add massive value, there is a lot to configure currently and adds a lot of friction to development

[phishy]

I just want to run an ffmpeg task thats longer than the Lambda timeout and orchestrate it with StepFunctions like I do with the rest of my SAM stuff. I guess now I need to make a little CDK app, and find out how to link them together. ¯_(ツ)_/¯

[RavenX86]

Same as above. I need to be able to run the same docker image for which an ECR was auto-generated inside an EC2, or ECS and be able to run the task longer than 15 minutes and connect to some ports in some cases. Lambda is perfect for 95-99% of the cases but not for all. While there is the option to create a CDK app in the same repo it would most likely end up with a second ECR and a second tempalte.yaml with no way to share the VPC with the lambda since we do not use statically named elements but let the system auto-generate the name and then pass them along with attributes.

[austinrbrown01]

I just want to run an ffmpeg task thats longer than the Lambda timeout and orchestrate it with StepFunctions like I do with the rest of my SAM stuff. I guess now I need to make a little CDK app, and find out how to link them together. ¯_(ツ)_/¯

Yep, same as above. I kinda realized after deploying my app that serverless compute simply didn't afford me the granularity with regard to compute and memory that I needed. 10gb ram is way too much for my needs, but since compute and memory are linked, you have to pay for needlessly large memory overhead.

Given the fact that SAM already supports building and testing serverless apps in AL2 Docker containers, I was surprised to learn that SAM doesn't support the same with fargate. seems like a missing link

[aksel]

I'm building a data pipeline with a number of processing steps. Some of these are best suited for Lambdas, while others are better suited for Fargate tasks.

I was hoping to tie all of this together with SAM.

It would be really nice to be able to quickly define Fargate tasks alongside state machines, something like AWS::Serverless::Task.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Serverless data pipeline, using Fargate tasks and Lambdas.

Resources:
  DataProcessingPipeline:
    Type: AWS::Serverless::StateMachine
    Properties:
      Definition:
        StartAt: Preprocessing
        States:
          # Initial step is a docker-based lambda.
          Preprocessing:
            Type: Task
            Resource: !GetAtt Preprocessing.Arn
            Next: Processing

          # Second step is a long-running Fargate task.
          Processing:
            Type: Task
            Resource: arn:aws:states:::ecs:runTask
            Parameters:
              TaskDefinition: !GetAtt Processing.Arn
            Next: CollectDiagnostics

          # A simple inlined lambda.
          CollectDiagnostics:
            Type: Task
            Resource: !GetAtt CollectDiagnostics.Arn
            Next: PostProcessing

          # Finally, another lambda.
          PostProcessing:
            Type: Task
            Resource: !GetAtt CollectDiagnostics.Arn
            End: true

  # Preprocessing lambda
  Preprocessing:
    Type: AWS::Serverless::Function
    Metadata:
      Dockerfile: docker/preprocessingLambda.Dockerfile
      DockerContext: .
    Properties:
      PackageType: Image

  # Processing Fargate task
  Processing:
    Type: AWS::Serverless::Task
    Properties:
      ContainerDefinitions:
        - Name: default
          Image: # Local docker image
            Dockerfile: docker/processingTask.Dockerfile
            DockerContext: .
          HealthCheck: # Takes a while to start up!
            Command: ["CMD-SHELL", "curl http://localhost:8000/health"]
            Interval: 300
            Timeout: 60
            Retries: 10
            StartPeriod: 300
        - Name: somethingElse
          Image: # Some other image, built and pushed elsewhere
            Uri: 123456789000.dkr.ecr.eu-central-1.amazonaws.com/my-cool-image:latest
      CPU: 4096
      Memory: 16384
      EphemeralStorage:
        SizeInGib: 80

  # Diagnostics lambda
  CollectDiagnostics:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: python3.9
      InlineCode: |
        def handler(event, context):
          print("Wow, diagnostics!")

  # Post-processing lambda
  PostProcessing:
    Type: AWS::Serverless::Function
    Metadata:
      Dockerfile: docker/postProcessingLambda.Dockerfile
      DockerContext: .
    Properties:
      PackageType: Image

Or maybe, rather than ECS, then doing it via Batch?

Anyway, for now, I handle this by building ECR images for the tasks separately, and then providing them as parameters.

[costincaraivan]

SAM is fairly mature compared to 2018, maybe it's time to tackle this? Lambda isn't the be-all-end-all of serverless and Fargate brings a lot of much needed flexibility with very low operational overhead.

[Thomas-McKanna]

It would be very useful if SAM at the very least offered a way to build and push Docker images to ECR, since it would unify all deployment steps into SAM. This is particularly useful for serverless applications which benefit from mixing Lambda functions and ECS tasks.

Currently, if you have a custom Docker container and want to define an ECS service with SAM, you have to build the Docker containers separately from your sam build command. Any guide that you look at which attempts to deploy an ECS service with SAM has this awkward requirement (for example, https://containersonaws.com/pattern/sam-fargate).

Since SAM already has functionality for Docker-based Lambda images via metadata, perhaps it isn't so far off from providing a more generic container resource like:

MyDockerImage:
    type: AWS::Serverless::Image
    DockerTag: foobar
    DockerContext: ./hello-world
    Dockerfile: Dockerfile

# Then we can refer to build image's ECR URI below:

MyECSTaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: long-running-service
      Cpu: '256'
      Memory: '512'
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      ExecutionRoleArn: !Ref ExecutionRole
      TaskRoleArn: !Ref TaskRole
      ContainerDefinitions:
        - Name: my-image
          Image: !Ref MyDockerImage
          PortMappings:
            - ContainerPort: 80