Level limitation to provider inheritance in modules? #4443

wholroyd · 2015-12-26T03:54:23Z

Is there currently a limitation to how many modules you can chain together before a provider definition is ignored?

One Module

I have a custom module that builds a network in AWS (VPC, subnets, routing tables, etc), which is called by code to define a service. The custom module only contains resources, no providers. The service defines the aws_provider with its keys and region (hardcoded) and all variables have a default value. I shouldn't get prompted for jack. If I do a terraform get && terraform plan on the service code, everything works fine, it spits out a plan, I can build a graph, etc.

Hierarchy...

service
- module

Graph...

Notice how provider.aws on root is marked as disabled, but it still works?

digraph {
    compound = "true"
    newrank = "true"
    subgraph "root" {
        "[root] module.custom.aws_internet_gateway.ig" [label = "aws_internet_gateway.ig", shape = "box"]
        "[root] module.custom.aws_route_table_association.subnet-a" [label = "aws_route_table_association.subnet-a", shape = "box"]
        "[root] module.custom.aws_route_table_association.subnet-b" [label = "aws_route_table_association.subnet-b", shape = "box"]
        "[root] module.custom.aws_subnet.vpc-a" [label = "aws_subnet.vpc-a", shape = "box"]
        "[root] module.custom.aws_subnet.vpc-b" [label = "aws_subnet.vpc-b", shape = "box"]
        "[root] module.custom.aws_vpc.vpc" [label = "aws_vpc.vpc", shape = "box"]
        "[root] module.custom.provider.aws" [label = "provider.aws", shape = "diamond"]
        "[root] provider.aws (disabled)" [label = "provider.aws (disabled)", shape = "diamond"]
        "[root] module.custom.aws_internet_gateway.ig" -> "[root] module.custom.aws_vpc.vpc"
        "[root] module.custom.aws_route_table_association.subnet-a" -> "[root] module.custom.aws_subnet.vpc-a"
        "[root] module.custom.aws_route_table_association.subnet-b" -> "[root] module.custom.aws_subnet.vpc-b"
        "[root] module.custom.aws_subnet.vpc-a" -> "[root] module.custom.aws_vpc.vpc"
        "[root] module.custom.aws_subnet.vpc-b" -> "[root] module.custom.aws_vpc.vpc"
        "[root] module.custom.aws_vpc.vpc" -> "[root] module.custom.provider.aws"
        "[root] module.custom.provider.aws" -> "[root] provider.aws (disabled)"
    }
}

Two Modules

If make the service code a module and call it from code that defines an environment, then all of a sudden, I get prompted for the AWS provider details. Even though it's hard coded in the files. The environment code contains nothing of it's own (no providers whatsoever, that is defined in service), it literally just calls the service module and passes it's vars to the module.

Hierarchy...

environment
- service
  - module

Graph...

Notice how module.service.provider.aws on root is disabled, which is the provider.aws from root in the example above, but now it doesn't work?

digraph {
    compound = "true"
    newrank = "true"
    subgraph "root" {
        "[root] module.service.module.custom.aws_internet_gateway.ig" [label = "aws_internet_gateway.ig", shape = "box"]
        "[root] module.service.module.custom.aws_route_table_association.subnet-a" [label = "aws_route_table_association.subnet-a", shape = "box"]
        "[root] module.service.module.custom.aws_route_table_association.subnet-b" [label = "aws_route_table_association.subnet-b", shape = "box"]
        "[root] module.service.module.custom.aws_subnet.vpc-a" [label = "aws_subnet.vpc-a", shape = "box"]
        "[root] module.service.module.custom.aws_subnet.vpc-b" [label = "aws_subnet.vpc-b", shape = "box"]
        "[root] module.service.module.custom.aws_vpc.vpc" [label = "aws_vpc.vpc", shape = "box"]
        "[root] module.service.module.custom.provider.aws" [label = "provider.aws", shape = "diamond"]
        "[root] module.service.provider.aws (disabled)" [label = "provider.aws (disabled)", shape = "diamond"]
        "[root] provider.aws (disabled)" [label = "provider.aws (disabled)", shape = "diamond"]
        "[root] module.service.module.custom.aws_internet_gateway.ig" -> "[root] module.service.module.custom.aws_vpc.vpc"
        "[root] module.service.module.custom.aws_route_table_association.subnet-a" -> "[root] module.service.module.custom.aws_subnet.vpc-a"
        "[root] module.service.module.custom.aws_route_table_association.subnet-b" -> "[root] module.service.module.custom.aws_subnet.vpc-b"
        "[root] module.service.module.custom.aws_subnet.vpc-a" -> "[root] module.service.module.custom.aws_vpc.vpc"
        "[root] module.service.module.custom.aws_subnet.vpc-b" -> "[root] module.service.module.custom.aws_vpc.vpc"
        "[root] module.service.module.custom.aws_vpc.vpc" -> "[root] module.service.module.custom.provider.aws"
    }
}

According to the debug output on plan, this keeps asynchronously repeating every 5 seconds to the log (same pieces, never in same order)...

2015/12/25 22:13:49 [DEBUG] vertex provider.module.service.module.custom.aws (close), waiting for: module.service.module.custom.aws_internet_gateway.ig
2015/12/25 22:13:49 [DEBUG] vertex module.service.module.custom.aws_route_table_association.subnet-a, waiting for: module.service.module.custom.aws_subnet.vpc-a
2015/12/25 22:13:49 [DEBUG] vertex module.service.module.custom.aws_subnet.vpc-b, waiting for: module.service.module.custom.aws_vpc.vpc
2015/12/25 22:13:49 [DEBUG] vertex module.service.module.custom.aws_internet_gateway.ig, waiting for: module.service.module.custom.aws_vpc.vpc
2015/12/25 22:13:49 [DEBUG] vertex module.service.module.custom.aws_route_table_association.subnet-b, waiting for: module.service.module.custom.aws_subnet.vpc-b
2015/12/25 22:13:49 [DEBUG] vertex module.service.module.custom.aws_vpc.vpc, waiting for: module.service.module.custom.provider.aws
2015/12/25 22:13:49 [DEBUG] vertex module.service.module.custom.aws_subnet.vpc-a, waiting for: module.service.module.custom.aws_vpc.vpc
2015/12/25 22:13:49 [DEBUG] vertex root, waiting for: provider.module.service.module.custom.aws (close)

The name of that last item looks odd, cause it doesn't exist by that name. Not sure if that's part of the issue or just a mistake in the output.

Repro

Using 0.6.8, clone https://github.com/wholroyd/terraform-bug, and edit c_service/main.tf and add in your AWS keys.
Navigate to c_service/ and run terraform get && terraform plan. This should work, you should get a plan generated.
Navigate to b_environment/ and run terraform get && terraform plan. This should fail as you will get AWS key and region prompts instead.

The only difference between c_service and b_environment is that it's an additional level in the graph and service contains the provider. It's literally the same content in both main.tf files with different module name and source values otherwise.

From what I understand, as long as you have a provider defined at the same level or in an ancestral level to a resource that needs to use it, this should work.

The text was updated successfully, but these errors were encountered:

wholroyd · 2015-12-27T02:32:28Z

It seems this was reported before and closed in #3931, which referenced #2832 instead.

While this does seem related to #2832, in that issue the provider in the root is unknown to grandchild modules with resources attempting to use it. My particular in this issue is that the provider doesn't exist until the great-grandchild, which calls modules that contains resources it calls from there. It seems when there are too many module levels in either direction of where the provider is defined, it will get lost.

Using my sample project above, look at how the graph looks from the root of the project...

digraph {
        compound = "true"
        newrank = "true"
        subgraph "root" {
                "[root] module.region.module.environment.module.service.module.custom.aws_internet_gateway.ig" [label = "aws_internet_gateway.ig", shape = "box"]
                "[root] module.region.module.environment.module.service.module.custom.aws_route_table_association.subnet-a" [label = "aws_route_table_association.subnet-a", shape = "box"]
                "[root] module.region.module.environment.module.service.module.custom.aws_route_table_association.subnet-b" [label = "aws_route_table_association.subnet-b", shape = "box"]
                "[root] module.region.module.environment.module.service.module.custom.aws_subnet.vpc-a" [label = "aws_subnet.vpc-a", shape = "box"]
                "[root] module.region.module.environment.module.service.module.custom.aws_subnet.vpc-b" [label = "aws_subnet.vpc-b", shape = "box"]
                "[root] module.region.module.environment.module.service.module.custom.aws_vpc.vpc" [label = "aws_vpc.vpc", shape = "box"]
                "[root] module.region.module.environment.module.service.module.custom.provider.aws" [label = "provider.aws", shape = "diamond"]
                "[root] module.region.module.environment.module.service.provider.aws (disabled)" [label = "provider.aws (disabled)", shape = "diamond"]
                "[root] module.region.module.environment.provider.aws (disabled)" [label = "provider.aws (disabled)", shape = "diamond"]
                "[root] module.region.module.environment.module.service.module.custom.aws_internet_gateway.ig" -> "[root] module.region.module.environment.module.service.module.custom.aws_vpc.vpc"
                "[root] module.region.module.environment.module.service.module.custom.aws_route_table_association.subnet-a" -> "[root] module.region.module.environment.module.service.module.custom.aws_subnet.vpc-a"
                "[root] module.region.module.environment.module.service.module.custom.aws_route_table_association.subnet-b" -> "[root] module.region.module.environment.module.service.module.custom.aws_subnet.vpc-b"
                "[root] module.region.module.environment.module.service.module.custom.aws_subnet.vpc-a" -> "[root] module.region.module.environment.module.service.module.custom.aws_vpc.vpc"
                "[root] module.region.module.environment.module.service.module.custom.aws_subnet.vpc-b" -> "[root] module.region.module.environment.module.service.module.custom.aws_vpc.vpc"
                "[root] module.region.module.environment.module.service.module.custom.aws_vpc.vpc" -> "[root] module.region.module.environment.module.service.module.custom.provider.aws"
                "[root] module.region.module.environment.module.service.provider.aws (disabled)" -> "[root] module.region.module.environment.provider.aws (disabled)"
        }
}

Notice there is no provider at root, or on the region module. There is one on the environment module according to the graph (but disabled), yet it doesn't exist in the code. I'd imagine the graph in #3931 would look similar where the provider would exist in the graph close to the root and eventually disappearing towards the grandchild module.

phinze · 2016-01-13T20:00:12Z

Thanks for the report, @wholroyd. I'm suspicious that it's the same root issue as #2832, but I see what you're saying about this one having a slightly different shape.

Either way, this "provider details lost through module inheritance" but is on my shortlist of priority core bugs, so you can expect to see movement here soon!

catsby · 2016-01-13T20:04:09Z

#4607 has been closed as a duplicate of this, but may have additional information

Tanbouz · 2016-01-26T14:44:04Z

I posted recently in the #2659 (deleted comment) thinking it was related to terraform.tfvars not being read. After messing around, it seems I'm also affected by this.
main.tf -> cluster.tf -> node.tf

main.tf defines general configs, provider and clusters.
webservers/cluster.tf module defines several nodes
webservers/nginx/node.tf defines a single aws resource to create one node in the cluster

node.tf was asking for provider region interactively when running terraform plan when it was defined in main.tf with values provided by terraform.tfvars

Weirdly, sometimes it worked by printing out the plan but most of the time it didn't.

v0.6.9

jonapich · 2016-02-19T15:09:46Z

Is there any known safe workaround or recovery steps to this issue?

I also use nested modules, and I was able to apply everything correctly. I am trying to remove a part of it by commenting out the module from the root, but then terraform complains about the lack of provider.

I cannot use "destroy" because it won't let me target a full module. There are tons of resources involved, I can't possibly handle them one at a time manually, especially given that it's a new module I'm testing and I have to provision/delete it several times a day.

jonapich · 2016-02-19T15:11:14Z

I was able to skip through the issue using environment variables, but it really only works because I'm using a single region/creds set. Anyone encountering this issue over a multi-region architecture will have a very, very bad day...

The flattening process was not properly drawing dependencies between provider nodes in modules and their parent provider nodes. Fixes #2832 Fixes #4443 Fixes #4865

ghost · 2020-04-26T02:22:49Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

radeksimko added question core labels Dec 28, 2015

wholroyd mentioned this issue Jan 12, 2016

aws provider info isn't available to grandchild module in 0.6.9 #4607

Closed

phinze added bug and removed question labels Jan 13, 2016

phinze mentioned this issue Apr 14, 2016

core: fix provider config inheritence for deeply nested modules #6186

Merged

jen20 closed this as completed in #6186 Apr 18, 2016

martin-cossy-atwork mentioned this issue Sep 6, 2016

terraform prompts several times to enter a provider.aws.region when a certain number of nested modules is reached. #8680

Closed

revprez mentioned this issue Jul 11, 2017

Terraform 0.9.11 prompting user for value for provider.aws.region #15524

Closed

ghost locked and limited conversation to collaborators Apr 26, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Level limitation to provider inheritance in modules? #4443

Level limitation to provider inheritance in modules? #4443

wholroyd commented Dec 26, 2015

wholroyd commented Dec 27, 2015

phinze commented Jan 13, 2016

catsby commented Jan 13, 2016

Tanbouz commented Jan 26, 2016

jonapich commented Feb 19, 2016

jonapich commented Feb 19, 2016

ghost commented Apr 26, 2020

Level limitation to provider inheritance in modules? #4443

Level limitation to provider inheritance in modules? #4443

Comments

wholroyd commented Dec 26, 2015

One Module

Hierarchy...

Graph...

Two Modules

Hierarchy...

Graph...

Repro

wholroyd commented Dec 27, 2015

phinze commented Jan 13, 2016

catsby commented Jan 13, 2016

Tanbouz commented Jan 26, 2016

jonapich commented Feb 19, 2016

jonapich commented Feb 19, 2016

ghost commented Apr 26, 2020