External data source can't depend on resource

[Sebmaster]

Terraform Version

Terraform v0.11.0
+ provider.external v1.0.0
+ provider.ignition v1.0.0
+ provider.tls v1.0.1

Terraform Configuration Files

provider "external" {
  version = "~> 1.0"
}

provider "tls" {
  version = "~> 1.0"
}

provider "ignition" {
  version = "~> 1.0"
}

resource "tls_private_key" "ca" {
  algorithm   = "ECDSA"
}

data "external" "some_program" {
  program = ["never_executed"]

  query = {
    cert = "${tls_private_key.ca.private_key_pem}"
  }
}

data "ignition_config" "etcd_init" {
  replace {
    source = "${data.external.some_program.result.url}"
  }
}

(kinda convoluted to reduce it as much as possible.)
Changing any of the dependencies makes plan run successfully.

Debug Output

https://gist.github.com/Sebmaster/71dc67a935e1d58cd10ac74f2bf45cfc

Expected Behavior

According to the docs, data sources should wait until their dependencies are evaluated to run themselves.

Actual Behavior

Produces:

* data.ignition_config.etcd_init: 1 error(s) occurred:

* data.ignition_config.etcd_init: Resource 'data.external.some_program' does not have attribute 'result.url' for variable 'data.external.some_program.result.url'

It seems like the ignition config is evaluating even though it shouldn't (because it needs to wait for the external script, which depends on the resource, so should only be run in the apply phase). The external source is not evaluated (as expected).

Steps to Reproduce

1. terraform init
2. terraform apply

Important Factoids

None

References

[jbardin]

Hi @Sebmaster,

Thanks for filing this with an excellent example.

The behavior here is the confluence of a couple issues. The evaluation of "external.some_program is delayed, and consequently ignition_config.etcd_init should probably be transitively handled in the same manner. Indexing into an unknown map should also return an unknown value, rather than a "does not have attribute" error.

These are known issues that we hope to be tackling soon, however I'll keep this open for now as I don't think we have an example posted of how these dependent data sources interact.

[stijndehaes]

Hi all, I bumped into this issue yesterday. There still seems to be an issue with this. Anyone currently working on this? Otherwise I might have try, but I have never looked at terraform code yet. So some pointers on where to look would be appreciated.

[jbardin]

Hi @stijndehaes,

I've confirmed that this is working in the current master branch.

This should work in most cases with the latest 0.11 release as well, as long as you access the result map attributes with via an index:

source = "${data.external.some_program.result["url"]}"

[piersf]

Thanks! This saved my day on another workaround I was working on( #6884 ).

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.