command: terraform show -format=json

[apparentlymart]

This set of changes adds a -format=json option to the terraform show command to allow users to obtain machine-readable plan and state output.

There are (and have been) several other variants of adding some JSON output support to Terraform:

• In [WIP] JSON plan format #4610 I tried to make JSON the primary format for plans, for consistency with states. However, this would've likely tempted people to try to hand-edit plan files, which is likely to cause Terraform to misbehave or crash.
• Initial stab at JSON plan output #3170 proposes to add JSON output to the terraform plan command itself. That would work, but is a little problematic because the plan command produces more output than just the plan, including the log of refreshing any existing resources and warnings about various things, and so it feels better (to me) to leave terraform plan as human-readable and provide a separate command for JSONifying plans.
• Allow JSON output where it makes sense #2460 is a bit of a "meta-issue" aiming to get JSON support in a bunch of different places. JSON-readable logs of Terraform's ongoing behavior would be interesting, but is a lot of work to bite off.

So this change focuses on one primary use-case: I already made a plan using terraform plan -out=... and now I want to use some separate tool to analyze the plan.

$ terraform plan -out=tfplan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but
will not be persisted to local or remote state storage.


The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed. Cyan entries are data sources to be read.

Your plan was also saved to the path below. Call the "apply" subcommand
with this plan file and Terraform will exactly execute this execution
plan.

Path: tfplan

(... etc, etc ...)

$ terraform show -format=json tfplan
{
   ...a big blob of JSON...
}

terraform show is used because it is focused on obtaining information without any other side-effects. This keeps the multi-format complexity from spreading all over the command module and should make a reasonable interface for producing a plan once and then safely rendering it in a number of different formats. (A maximum of two to start, but who knows...)

Since it is the terraform show command that is being extended here, it also supports JSON output of states. This is pretty redundant, since the state is already JSON-readable in exactly the same format in the local state file. It is thus included just for consistency and completeness for now, but could become more useful in future if, for example, we were to implement encryption of the local state file at rest (#9556), or if the local cache were to go away altogether as part of some forthcoming redesign of the remote state management workflow (#1964 or something like it).

This includes some of the work I did for #4610, though it required some tweaking to catch up with changes to Terraform's data structures in the mean time.

---

A big reason to be cautious about supporting something like this is that it creates a pretty big API surface area to maintain compatibility with. Terraform is still evolving very fast, so it's basically impossible to retain 100% compatibility with any machine-readable format. As a pragmatic compromise this change adds a JSON format for adventurous people to use, but specifically disclaims any defined behavior of it across Terraform versions and declines to document it in any way.

My thought was that for now this would remain a "buyer beware" sort of feature for adventurous early adopters, and then we'll get some experience with what people tend to do with it and then, at some later point, possibly commit to some level of compatibility for this format and produce some documentation on it.

[mitchellh]

There is probably meaningful work in here, but we're going to be completely revamping terraform show since it is not usable very quickly as state size increases. JSON output may help with that but there are other things I don't like about this:

The JSON format now creates some sort of API promise implicitly (though we definitely explicitly don't) of the structure, but the diff/state structure is going to continue to change heavily in upcoming versions, so anyone relying on this format will have to update their tools for each new version.

This has been a primary de-motivator for me for JSON output for state/diffs and why I push folks to just use UNIX tools with terraform state commands if possible for example. I understand there is a large class of problems that can't be solved that way, though.

And the people super motivated can easily (Go knowledge required, but past that it is easy) use the Terraform Go API (not promised stable but you can vendor) to parse states/plans and do clever things. That is a VERY high touch solution but at least its definitely "buyer beware" as you might say.

So given that... I'm gonna say "no" to this and close it but I don't think your work will be a waste: we probably can and should adopt various parts of your diff in the future and knowing this exists will be useful.

[apparentlymart]

Cool... that all makes sense. Thanks for the explanation, @mitchellh. A chunk of the code here was already from another rejected PR so maybe it will get a third chance later. 😀

As an aside: I previously tried to make use of Terraform's "API" from another Go program in the past; I guess it was probably my prototype of wrapping Terraform in a Packer-like UI. Ever since Terraform started using govendor this sort of usage has been generally broken, because any time a vendored library appears in Terraform's public interface (which happens with e.g. HIL's AST and HCL's concepts) you end up with types whose names cannot be written down by any code outside of the Terraform package.

Of course I don't expect anything to be done to fix that, since Terraform is primarily an application rather than a library, but just something to keep in mind when thinking about ways people might want to integrate with Terraform.

[jalil]

Hey Michell,
Can you give use the link to the Terraform Go API with some samples usages.

Thank you

[hraban]

Hi @jalil I think he's talking about this very package we're commenting on right now. See #415 (comment)

[gazoakley]

It's a shame we can't have this built into Terraform - it'd be particularly helpful to include an option to strip away any attributes marked as sensitive in the schema. You could then build tools which take in plans/states for analysis. Maybe the exported JSON could include a schema semver?

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.