Speed up by caching plugins and providers

[lkysow]

With the latest terraform >= 0.9 providers come as plugins that need to be downloaded. Right now, we're using terraform init which downloads the plugins into the current directory under .terraform/plugins/{os}_{runtime} for every project.

If we move the downloaded plugin to ~/.terraform.d/plugins/ (or ~/.terraform.d/plugins/{os}_{runtime} then terraform won't download the plugin again for any project.

So my proposed solution is:

• on every terraform init, check if there are any files in {pwd}/.terraform/plugins/os_runtime other than lock.json. If so, move all of them to ~/.terraform.d/plugins/.

Now tf will check ~/.terraform.d/plugins/ before downloading any plugins so it will be back up to speed. Since tf will download any plugins that aren't already existing, we can rely on it to do the downloading.

[anubhavmishra]

Some providers are versioned and people might request two different versions of the same provider. Do we account for that? Just a question.

[lkysow]

Yeah terraform names the plugin files depending on the version so we can have multiple files cached and tf will just use whatever version it needs

[automaticgiant]

can we not just use https://www.terraform.io/docs/configuration/providers.html#provider-plugin-cache ?
well, in theory.
i was trying to set TF_PLUGIN_CACHE_DIR and terraform init to cache deps in the dockerfile, but the env doesn't persist to child tf. sad. we have a corporate proxy, so this is awkward. so now i'm loading them into ~/.terraform.d/plugins.

[lkysow]

Nice! I didn't know that was possible. I think it probably makes sense to always execute terraform with the TF_PLUGIN_CACHE_DIR variable set to "$HOME/.terraform.d/plugin-cache"

[atlantisbot]

This issue was migrated to runatlantis/atlantis#34. Read about why here.