Using Copilot with EFS to publish a public Docker image

[bvtujo]

Continued from discussion on #2137 with @thstart. I'm capturing those comments in this discussion for better searchability.

Original Comment: Exposes a counterintuitive behavior where copying folders into the efs mount directory at container build time does not result in those files appearing in EFS.

Response to original: Solution is to perform the copy at container runtime, i.e. in the ENTRYPOINT or CMD directives, by wrapping the service initialization code in a script which also performs the copy.

This was unacceptable because the final image is being published in public and the customer does not want their test data mounted to EFS to be part of that image.

The current question:

I'm using Copilot because it creates all permissions and setting and is working very reliably.

I would like to publish my container using the EFS at AWS Marketplace.
How to use Copilot to make that task easier? For example mounting the EFS to the container?

Copilot currently doesn't support publishing to AWS Marketplace. You can definitely use Copilot to build the container, but all of the EFS infrastructure is dependent on your customers using a Copilot application, environment, and service to run it. Certain assumptions we make about the architecture may not be true in all customer environments that Copilot did not create.

Am I understanding properly?

[bvtujo]

@thstart if you need EFS for testing purposes, you can hydrate your filesystem in a couple of ways. First, and the recommended option, would be to use the command field in the manifest to override the default command. You could invoke a script which downloads data from S3 and puts it in your EFS volume, then launches your service.

The other option is to hydrate with a sidecar or a Scheduled Job. There are details below if you click the little arrow.

The manifest configuration would be much the same, and you'd have to copy your EFS files into either the sidecar image or the scheduled job image. This should fit your use case since you wouldn't necessarily publish either of those.

Sidecar details

To set this up with a sidecar, you could do something like this in your service manifest:

storage:
  volumes:
    myEFSVolume:
      read_only: false
      path: /app/efs
      efs:
        id: fs-12345

sidecars:
  efsCopier:
    image: ${ECR_IMAGE_URI}
    mount_points:
      - source_volume: myEFSVolume
        path: /efs
        read_only: false

Then, in your Dockerfile for the sidecar, you could do:

FROM alpine:latest
COPY . /data
CMD /bin/sh copy-script.sh

#!/bin/sh
cp -r /data/* /efs
tail -f /dev/null # Keep alive indefinitely so that the sidecar's exit doesn't cause the task to be killed.

This sidecar option will get a little easier in our next release; we've got a change going out which will let you mark sidecars as nonessential.

Hope this helps, or gives you some options on how to proceed!