Fix #3973: Document how to use KMS encrypted variables for AWS Lambda

[alex1x]

Fixes #3973

Document how to use KMS encrypted variables for AWS Lambda.

There is nothing particularly novel here, just a detailed explanation of how to use KMS encrypted variables in your Lambdas, as this has been a topic of confusion, particularly in the context of its relation to the kms_key_arn parameter (which actually is unrelated): hashicorp/terraform#12225

This guide proposes a simple workflow for creating Lambdas with KMS encrypted environment variables via Terraform.

[rberlind]

I found this extra documentation very helpful to understanding how to use KMS with AWS Lambda functions. Is there some reason it has not yet been merged into the master branch so that it could be visible under the AWS Provider on the Terraform docs website?

Thanks.

[dkheyman]

Wouldn't it be possible to create an interpolation that will call aws kms encrypt on the environment variable plaintext value, in the Terraform declaration for the aws_lambda_function under environment {variables {}}. Something like:

        variables = {
            my-variable = "${encrypt("variable_value", <kms_key_arn>)}"
            }
       }

or perhaps even use the output of a null_resource local_exec that runs aws kms encrypt behind the scenes.

It seems like currently there's no way to automate the 'encryption in transit' bit for Lambda using native Terraform.

[austinorth]

Any movement on this? Found this super helpful, and the current documentation is pretty lackluster regarding this functionality.

[ispulkit]

Any movement here? This will be really useful for us.

[teamterraform]

Notification of Recent and Upcoming Changes to Contributions

Thank you for this contribution! There have been a few recent development changes that affect this pull request. We apologize for the inconvenience, especially if there have been long review delays up until now. Please note that this is automated message from an unmonitored account. See the FAQ for additional information on the maintainer team and review prioritization.

If you are unable to complete these updates, please leave a comment for the community and maintainers so someone can potentially continue the work. The maintainers will encourage other contributors to use the existing contribution as the base for additional changes as appropriate. Otherwise, contributions that do not receive updated code or comments from the original contributor may be closed in the future so the maintainers can focus on active items.

For the most up to date information about Terraform AWS Provider development, see the Contributing Guide. Additional technical debt changes can be tracked with the technical-debt label on issues.

As part of updating a pull request with these changes, the most current unit testing and linting will run. These may report issues that were not previously reported.

Action Required: Terraform 0.12 Syntax

Reference: #8950
Reference: #14417

Version 3 and later of the Terraform AWS Provider, which all existing contributions would potentially be added, only supports Terraform 0.12 and later. Certain syntax elements of Terraform 0.11 and earlier show deprecation warnings during runs with Terraform 0.12. Documentation and test configurations, such as those including deprecated string interpolations (some_attribute = "${aws_service_thing.example.id}") should be updated to the newer syntax (some_attribute = aws_service_thing.example.id). Contribution testing will automatically fail on older syntax in the near future. Please see the referenced issues for additional information.

Action Required: Terraform Plugin SDK Version 2

Reference: #14551

The Terraform AWS Provider has been upgraded to the latest version of the Terraform Plugin SDK. Generally, most changes to contributions should only involve updating Go import paths in source code files. Please see the referenced issue for additional information.

Removal of website/aws.erb File

Reference: #14712

Any changes to the website/aws.erb file are no longer necessary and should be removed from this contribution to prevent merge issues in the near future when the file is removed from the repository. Please see the referenced issue for additional information.

Upcoming Change of Git Branch Naming

Reference: #14292

Development environments will need their upstream Git branch updated from master to main in the near future. Please see the referenced issue for additional information and scheduling.

Upcoming Change of GitHub Organization

Reference: #14715

This repository will be migrating from https://github.com/terraform-providers/terraform-provider-aws to https://github.com/hashicorp/terraform-provider-aws. No practitioner or developer action is anticipated and most GitHub functionality will automatically redirect to the new location. Go import paths including terraform-providers can remain for now. Please see the referenced issue for additional information and scheduling.

[austinorth]

https://www.youtube.com/watch?v=U7CZcd-UYmU

[ghost]

Any news on 'encryption in transit' ?

[austinorth]

I see that @ispulkit and @liamraemclauchlan approved these changes. Any chance one of you could fix these merge conflicts and merge the PR?

[amihura]

Would be good to have

[zhelding]

Pull request #21306 has significantly refactored the AWS Provider codebase. As a result, most PRs opened prior to the refactor now have merge conflicts that must be resolved before proceeding.

Specifically, PR #21306 relocated the code for all AWS resources and data sources from a single aws directory to a large number of separate directories in internal/service, each corresponding to a particular AWS service. This separation of code has also allowed for us to simplify the names of underlying functions -- while still avoiding namespace collisions.

We recognize that many pull requests have been open for some time without yet being addressed by our maintainers. Therefore, we want to make it clear that resolving these conflicts in no way affects the prioritization of a particular pull request. Once a pull request has been prioritized for review, the necessary changes will be made by a maintainer -- either directly or in collaboration with the pull request author.

For a more complete description of this refactor, including examples of how old filepaths and function names correspond to their new counterparts: please refer to issue #20000.

For a quick guide on how to amend your pull request to resolve the merge conflicts resulting from this refactor and bring it in line with our new code patterns: please refer to our Service Package Refactor Pull Request Guide.

[abhaynayak24]

while the feature for encryption in transit is in the pipeline, are there any quick workarounds that we can use?

[thekev]

the kms helper stuff just calls kms:Encrypt for you. so, you can do that yourself as a "workaround", presuming your provider role has permission:

data "vault_generic_secret" "secret" {
  path = "secrets/lambda_thingy"
}
data "aws_kms_ciphertext" "secret" {
  key_id = aws_kms_key.some_key.key_id
  plaintext = data.vault_generic_secret.secret.data_json
}
resource "aws_lambda_function" "magic" {
  function_name = "knows-how-to-decrypt"
  role          = aws_iam_role.magic.arn
  timeout       = 30
  runtime       = "python3.9"
  handler       = "magic.lambda_handler"
  filename      = data.archive_file.lambda_zip.output_path
  source_code_hash = data.archive_file.lambda_zip.output_base64sha256
  environment {
    variables = {
      FOOBAR = "notsecret"
      CONFIG = data.aws_kms_ciphertext.secret.ciphertext_blob
    }
  }
  kms_key_arn   = aws_kms_key.some_key.arn
}

The problem is the ciphertext is non-deterministic. It will be a change delta on every plan.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.