[ECS] [secrets]: support parameter name from AWS Secrets Manager

[ramilexe]

Tell us about your request
I want to be able to inject Secret Key item from Secrets Manager to ECS container_definitions.secrets

Which service(s) is this request for?
ECS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
In task definition I have the following configuration:

"secrets": [
  {
    "valueFrom": "arn:aws:secretsmanager:us-east-1:account:secret:SECRET_NAME",
    "name": "RABBITMQ_DEFAULT_USER"
  }
]

but it gets injected as whole json object with other parameters:

Is it possible to get only one value from the secret?
Thank you

[nsbialek]

This came up in a recent conversation as if the feature already exists. If someone has documentation for how to implement this please drop a note here. It would be much appreciated.

[kadrach]

Ran into this issue earlier this week with RDS credentials generated by SecretsManager. The ECS ContainerDefinition.Secrets only support an Arn for ValueFrom.

CloudFormation offers support for "dynamic references" for SecretsManager of the form {{resolve:secretsmanager:secret-id:secret-string:json-key:version-stage:version-id}} (note json-key).

The downside of this approach is that it bakes the secret into the stack on create/update operations, meaning a rotation of the secret would not be picked up by new tasks until the stack is updated.

[hspak]

I would love this functionality so we can have 1 secret in secrets manager per ECS task instead of 1 per every secret a task requires.

[ramilexe]

Any updates on that @coultn ? Is it on roadmap?

[galindro]

This is something really nice because it would avoid unnecessary costs.

For example:

Imagine that, to access an API, we should inform username and password. We could simply store them together in one secret:

Secret name: prod/api/trackify
Secret Value:

{
  "username": "myusername",
  "password": "mypassword"
}

With the current limitation, we can store each json property on its own secret, incurring in more costs:

Secret name: prod/api/trackify/username
Secret Value: myusername

Secret name: prod/api/trackify/password
Secret Value: mypassword

[coultn]

Any updates on that @coultn ? Is it on roadmap?

Apologies for the delay in responding - we are aware of the request and are evaluating the priority. Additional thumbs-up on the issue are always welcome and we do pay attention to that when prioritizing.

[stephen-164]

Also note that this requirement (one plaintext value per secret) is incompatible with the key rotation mechanisms and secret target attachments (eg RDS attachment, which requires a JSON structure to add the host/engine/port values). This makes the implementation particularly confusing.

[jritsema]

I agree that this would be a great option to have in ECS/Fargate. Maybe a new setting like injectSecretProperties that would inject each json property key/value as an envvar.

In the mean time, we created a docker shim that can do this.
https://github.com/turnerlabs/sm_entrypoint

We also use parameter store to achieve this but would prefer to standardize on Secrets Manager.

[cervantek]

Heads up that CodeBuild team just released exactly this with the following format. The ECS team should be talking to the CodeBuild team about these type of things, you guys are solving a lot of the same problems.

secret-id:json-key:version-stage:version-id

https://aws.amazon.com/about-aws/whats-new/2019/11/aws-codebuild-adds-support-for-aws-secrets-manager/