Feature Request: Support full templating functionality of templatefile() in module

[Elgeario]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

** template_file_variables - Update or alternative option to template_file_variables in enterprise scale**

In order to make the management of policy easier i want to be able to define lists of valid values in one place and then refer to those lists via variables in the policy code. I can use template_file_variables to specify a single value and the code is happy, but as soon as i attempt to make a list of values it spits out

│ The given value is not suitable for child module variable "template_file_variables" defined at
│ .terraform\modules\enterprise_scale\modules\archetypes\variables.tf:56,1-35: all map elements must have the same type.

So for example i want to provide 1 master list of valid cost centres and then use that variable to enforce policy without the need of repeating the list across multiple files requiring multiple updates when changes to these values are required.

If there is a way to achieve this in the current code please advise.

Outside of the CAF code i have successfully used Json to perform this.

[krowlandson]

@Elgeario... thank you for raising this issue.

This appears to be down to the variable type being declared as map(any) which requires all map elements to be the same. This is due to the way Go works when creating strongly typed objects.

Changing this to just any appears to resolve the problem, but I need to do some more testing.

If you would like to implement and test this before we get a PR merged and released, please locate and update the following values from map(any) to any in the local copy of the module once you're run terraform init (found in the .terraform/modules folder of your root module):

• terraform-azurerm-caf-enterprise-scale/variables.tf

  Line 485 in 4c64874

  type = map(any)

• terraform-azurerm-caf-enterprise-scale/modules/archetypes/variables.tf

  Line 57 in 4c64874

  type = map(any)

I believe this should allow you to use all options of templatefile() function.

Please let me know how you get on.

[Elgeario]

Excellent work, I will get on this today

…

On Wed, 19 Jan 2022, 10:08 Kevin Rowlandson, ***@***.***> wrote: @Elgeario <https://github.com/Elgeario>... thank you for raising this issue. This appears to be down to the variable type being declared as map(any) which requires all map elements to be the same. This is due to the way Go works when creating strongly typed objects. Changing this to just any appears to resolve the problem, but I need to do some more testing. If you would like to implement and test this before we get a PR merged and released, please locate and update the following values from map(any) to any in the local copy of the module once you're run terraform init (found in the .terraform/modules folder of your root module): - https://github.com/Azure/terraform-azurerm-caf-enterprise-scale/blob/4c64874539456b3596fd3c6030d114693258bf04/variables.tf#L485 - https://github.com/Azure/terraform-azurerm-caf-enterprise-scale/blob/4c64874539456b3596fd3c6030d114693258bf04/modules/archetypes/variables.tf#L57 I believe this should allow you to use all options of templatefile() <https://www.terraform.io/language/functions/templatefile> function. Please let me know how you get on. — Reply to this email directly, view it on GitHub <#253 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATFKRTC3INNSIK4BZ7IBIR3UW2ETRANCNFSM5MHG7MMQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.*** .com>

[Elgeario]

I have updated the two variable.tf files above to "any" and updated main.tf with below and am still getting an error

template_file_variables = {
listOfAllowedpmo = ["Override","AlwaysOn","AlwaysOff","AlwaysOffSchedule"]
}

ERROR =====

│ Error: Error in function call
│
│ on .terraform\modules\enterprise_scale\modules\archetypes\locals.policy_set_definitions.tf line 30, in locals:
│ 30: filepath => jsondecode(templatefile("${local.custom_library_path}/${filepath}", local.template_file_vars))
│ ├────────────────
│ │ local.custom_library_path is "./lib"
│ │ local.template_file_vars is object with 12 attributes
│
│ Call to function "templatefile" failed: ./lib/policy_set_definitions/policy_set_definition_es_enforce_mandatory_tagging_initiative_coop_subscription.json:83,35-51: Invalid template interpolation value;
│ Cannot include the given value in a string template: string required., and 1 other diagnostic(s).

[krowlandson]

@Elgeario

Thank you for testing and coming back so quickly. Are you able to confirm what is your intent with this? Are you wanting Terraform to convert the list input into a valid JSON string value within the template, copy the list in as a JSON list, or are you trying to create a copy loop within the template?

I believe you may have to follow the guidance for Generating JSON or YAML from a template
but if you are able to share a copy of the template file ./lib/policy_set_definitions/policy_set_definition_es_enforce_mandatory_tagging_initiative_coop_subscription.json and what you are trying to achieve, I would be happy to help.

[Elgeario]

After looking again, I am probably making things more complicated than i need to. Let me try to explain.

What I wanted to do was have a list of environments (for example) that will restrict people to provide only those values when creating an environment TAG. That 1 list however would then be referenced by different policies meaning that should we want to update our environment list, it would be in 1 place and then multiple policies would update reflecting the change.

In tagging I want to ensure that subscriptions, resource groups and resources are all restricted to the same list but this requires multiple policies to accomplish, or maybe one really well written one, so I wanted a way to define a list and reference it within multiple policies.

        "listOfAllowedEnvironments": {
        "type": "Array",
        "metadata": {
            "displayName": "list Of Allowed environment values",
            "description": "The list of environments that can be specified when deploying resources."
          },
          "allowedValues": ["${listOfAllowedEnvironments}"],
          "defaultValue": ["${listOfAllowedEnvironments}"]
        }

[krowlandson]

Thank you for the additional information @Elgeario

In this particular case I believe you are incorrectly writing the template. The initial diagnosis of updating the variable is still correct for the module to support inputs other than strings, but the processing of a non-string value (to generate valid JSON output) is slightly different.

I believe you have two options here:

1. Use the templating language to correctly generate the JSON value
2. Input the values as a JSON encoded string

NOTE: In both cases, you need to be aware that the resultant template file will not pass JSON validation. Unfortunately this is just one of the side-effects of using templating languages.

For option 1, I believe your code would look a bit like the following:

        "listOfAllowedEnvironments": {
        "type": "Array",
        "metadata": {
            "displayName": "list Of Allowed environment values",
            "description": "The list of environments that can be specified when deploying resources."
          },
          "allowedValues": ${jsonencode([for env in listOfAllowedEnvironments : "${env}"])},
          "defaultValue": ${jsonencode([for env in listOfAllowedEnvironments : "${env}"])}
        }

For option 2, you would need to update your input and the template as follows:

template_file_variables = {
  listOfAllowedpmo = jsonencode(["Override","AlwaysOn","AlwaysOff","AlwaysOffSchedule"])
}

        "listOfAllowedEnvironments": {
        "type": "Array",
        "metadata": {
            "displayName": "list Of Allowed environment values",
            "description": "The list of environments that can be specified when deploying resources."
          },
          "allowedValues": ${listOfAllowedEnvironments},
          "defaultValue": ${listOfAllowedEnvironments}
        }

Please note that I've been unable to test these, so please let me know how you get on.

[krowlandson]

I note that the example you gave earlier within template_file_variables doesn't match the example in the JSON template, but the same principles apply. I've left them as provided but you should be able to see what is required.

Also, you probably note that I prefer to use jsonencode() on an HCL object rather than trying to manually generate a valid JSON string for obvious reasons.

[krowlandson]

Added to release:
https://github.com/Azure/terraform-azurerm-caf-enterprise-scale/releases/tag/v1.1.2