Experiment: Ephemeral input variables, and the terraform.applying symbol #35273

This isn't actually used anywhere yet. We'll introduce uses of it in subsequent commits.

This doesn't actually do anything yet, but in future commits it will enable some new language features for marking input variables and output values as "ephemeral", meaning that they don't get saved as part of state snapshots or saved plan files.

When the ephemeral_values experiment is active, a module author can designate individual input variables and output values as being "ephemeral", which doesn't currently do anything but in future commits will represent that the values can be used only in locations that don't require Terraform to persist the value as part of state snapshots or saved plan files.

This is a checkpoint commit on the path to supporting ephemeral values as a cross-cutting concern in the Terraform language. An ephemeral value is one that lives only in RAM during a single phase and so cannot persist from the plan phase to the apply phase, or between consecutive plan/apply rounds. Terraform tracks whether each value is ephemeral using the cty "marks" concept, thus achieving the same dynamic analysis as we already employ for the concept of "sensitive values" that prevents displaying a value in the user interface. This commit is just a starting point which gets some of the basics into place: - input variables and output values can both be statically declared as being ephemeral. Only ephemeral inputs/outputs can have ephemeral values assigned to them, and the recipient of the value sees it as ephemeral even if the assigned value was not already ephemeral. This creates a dynamic analysis cutoff point at module boundaries so that it's possible to determine in isolation whether a single module is using ephemeral values correctly, without having to test it in every possible calling context. - Managed and data resources cannot have ephemeral values assigned into their configurations because Terraform and providers both expect the resource attributes to persist between phases. - Ephemeral values _can_ be used in provider and provisioner configurations, because both of those effectively meet the definition of the new "ephemeral" concept despite it not previously having a name. - Ephemeral markings propagate through all of the built-in language features for dynamic analysis purposes, largely relying on cty's efforts to do that in similar vein as for sensitive marks. In particular, it's possible to define a local value whose expression produces an ephemeral result, and passing ephemeral values to functions should propagate the ephemeral mark to the results when appropriate. (I've not yet thoroughly reviewed all of the built-in functions for correct marks handling though, so there may be some gaps to address in later commits.) The next step for this work will be to change the modules runtime to have support for a workflow involving ephemeral _root_ input variables, where their values must be re-supplied during the apply phase. That will then achieve (in experimental capacity) the first goal of ephemeral values: to be able to provide non-persistent settings such as time-limited API tokens to use in provider configuration blocks.

We'll use this to track what subset of the ephemeral input variables were set in the planning options, and which ones must therefore be re-supplied in the apply phase options. This new plan field is not yet populated anywhere. The uses of this will follow in subsequent commits.

The new concept of "ephemeral input variables" creates the possibility of needing to provide an input variable value during the apply phase, rather than just retaining it in the plan. Now we'll remember in the plan the names of the variables that need to be re-supplied during apply -- that is, any ephemeral values whose plan-time values were non-null -- and then check at the start of the apply phase whether those variables (and _only_ those variables) were provided in the planning options. This doesn't yet update Terraform CLI to actually populate this stuff, so as of this commit any plan with apply-time variables is effectively unapplyable. We'll deal with that in future commits.

When the ephemeral variables experiment is active we can potentially have input variables whose values need to be provided separately in both the plan and apply phases, as a compromise to avoid writing those values as part of a saved plan file and to allow the given value to vary between the two phases if necessary. The CLI layer must therefore re-process the given input variable values during the apply phase whenever this experiment is active for the root module and the plan recorded at least one apply-time variable name. To reduce the risk of this new logic accidentally impacting non-experimental usage, the whole call is guarded by whether the root module is participating in the experiment. Checking just the root module is sufficient here because only the root input variables are directly handled by the CLI layer; input variables for descendent modules are handled entirely within the modules runtime.

To support ephemeral values we need a more complicated set of rules about what input variables can and must be set when applying a saved plan. The command itself does not have enough information to implement those rules itself, so we'll let them pass through and check this in the local backend's apply phase instead. The local backend's apply phase already had basic support for dealing with apply-time variable values, but it'll now also be responsible for rejecting attempts to set variables when the experiment isn't enabled, to keep all of this logic in roughly the same place.

This new symbol returns an ephemeral boolean flag that is true only during the apply phase and false otherwise. Since this involves an ephemeral value and those are still experimental, this symbol is also available only when opted in to the ephemeral_values experiment. The intended use for this is to configure a provider with a different (and probably more privileged) role or username during the apply phase when Terraform is actually trying to change infrastructure, so that planning can be done with a more limited level of access that might, for example, only allow _reading_ from the remote system. role_arn = ( terraform.applying ? local.write_role_arn : local.read_role_arn )

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experiment: Ephemeral input variables, and the terraform.applying symbol #35273

Experiment: Ephemeral input variables, and the terraform.applying symbol #35273

Commits on May 31, 2024