Internalize all of the Go packages in this repository

[apparentlymart]

Now that the SDK has moved out into a separate repository, this main Terraform repository is a main Go module rather than a library Go module, and so none of the Go package APIs in here are intended for external consumption and they are not subject to semver-style backward compatibility promises.

Terraform is an older codebase that predates the introduction of the internal mechanism for to the Go toolchain that packages are not for external use, but since that is now the expected way to communicate that we're going to retroactively move all of Terraform's Go packages (aside from package main in the root) down under the internal directory.

We're intending to do this prior to tagging a Terraform 1.0 in order to avoid making it appear to the Go toolchain that our v1.0.0 tag is a stable release of a Go Module containing library packages. However, we will have some delay before doing it in order to give a chance to land some already-opened pull requests that would likely end up with merge conflicts once the top-level packages have new import paths.

[wata727]

I'm so interested in this. As far as I know, some third-party utilities (including TFLint) are importing packages from this repository. And some of the packages are not included in terraform-plugin-sdk. For example, TFLint uses an internal API to load configuration files and evaluate expressions.

Are you planning to offer such functionality as a separate repository? If don't, I'm considering writing a Go module like "libterraform", just like when I opened #17904.

I agree with this change. I would like to discuss the policy of future third-party products. Thanks to the Terraform team.

[apparentlymart]

Hi @wata727,

The supported public interfaces for Terraform are protocols and file formats rather than in-process Go function calls, so I'd recommend integrating via external implementations of the documented protocols and formats. Because Terraform is open source, it's fine to use copies of code in this repository as the basis of those implementations, but it's important to distinguish between the documented interfaces and the current implementation details if you are building something that should work with multiple versions of Terraform.

Part of the 1.0 release will be some stronger promises about which parts of Terraform's behavior are integration points you can rely on.

[apparentlymart]

This is now done, in #28723.

If you end up here because you had an external codebase importing packages from Terraform, you can continue to import from older releases as a stop-gap measure for the moment, but I'd strongly recommend migrating to other approaches using the CLI executable via its machine-readable interfaces. Most of the supported surface area is available via the terraform-exec library, which wraps execution of Terraform CLI and includes parsers for the machine-readable interfaces. If you need something that CLI offers but terraform-exec doesn't support yet, please open a feature request for that library.

Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.