Experiment: Ephemeral input variables, and the terraform.applying symbol

[apparentlymart]

This PR includes a subset of the work from #35078, extended with what I consider to be a pragmatic amount of tests for a language experiment so that we can see that it's fundamentally working while avoiding making it too hard to remove later if this experiment is not stabilized.

My intention with this is to allow us to get some early experience and more concrete feedback about ephemeral values using alpha releases in the v1.10 series, now that v1.9 has entered its prerelease period and so the main branch represents v1.10.

This focuses on only the subset of the ephemeral values ideas that we can implement relatively-non-disruptively only in this codebase:

• Ephemeral input variables, which is intended to allow such things as using a time-limited JSON web token in the plan phase and then using a new, separately-issued JSON web token in the apply phase.
• The terraform.applying symbol, which is an ephemeral boolean flag that is true only during the apply phase.
• Ephemeral output values, although this is not quite a sufficient implementation of them for general use yet because we've not designed how the CLI layer ought to deal with them.

Here's an example of a mostly-realistic configuration that could be written with this subset of the functionality:

terraform {
  # This currently requires an opt-in, and so will only be available
  # in alpha releases.
  experiments = [ephemeral_values]

  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
}

variable "aws_jwt" {
  type      = string
  ephemeral = true
  sensitive = true
}

locals {
  aws_read_role_arn  = "arn:aws:iam::XXXXX:role/terraform-read"
  aws_write_role_arn = "arn:aws:iam::XXXXX:role/terraform-full"
}

provider "aws" {
  region = "us-west-2"

  assume_role_with_web_identity {
    role_arn           = terraform.applying ? local.aws_write_role_arn : local.aws_read_role_arn
    web_identity_token = var.aws_jwt
  }
}

---

The remaining ideas for ephemeral values (e.g. ephemeral resources and write-only attributes for managed resources) are not really ready for even experimental implementation yet, because they require provider plugin protocol design, plugin framework design, and devising a strategy for how to avoid the ephemeral. reference prefix being a breaking change for a hypothetical existing provider named "ephemeral".

Therefore they are intentionally excluded here, but you can find a partial implementation of ephemeral resources in the original prototype PR #35078.

We might later decide to stabilize this subset of the functionality separately from the rest, since it's useful in its own right for some real-world use-cases, but that remains to be seen as we continue with technical design and project planning for the overall project. For now my goal is only to make it easier to play with the concepts, get practical experience, and encourage some early feedback to help make sure we're on the right track before we invest too deeply.

To keep this more tightly scoped to start I didn't yet teach the stacks runtime about ephemeral values. However, I'm hoping to do that in a followup PR because patterns like the one above are particularly useful in stacks where JSON web tokens are the way to handle provider authentication. (Stacks currently uses a workaround of passing the ephemeral JSON web token indirectly via a file on disk, whereas supporting ephemeral input values would allow us to pass the tokens only in RAM.)

[liamcervante]

This looks fine to me, seems like everything is well protected behind the flag.

I'm just curious why terraform.applying should be ephemeral. Is it just because it's value can change between plan and apply so we want to make sure it doesn't get written anywhere?

[apparentlymart]

Indeed, in fact it must change between plan and apply (by definition) and so it wouldn't be valid to use it in any context that must remain constant between plan and apply. For example, if it were used to populate something in a resource block then the final plan created during the apply phase could differ from the initial plan, which would invalidate the plan and cause apply to fail. It being ephemeral means we can catch that mistake during the plan phase instead.

The primary place to use it would be in a provider block, where ephemeral values are allowed.

[github-actions]

Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.