Plugins fail when evaluating an attribute with a heredoc value

[velkovb-nuk]

I have an issue with a ecr_lifecycle_policy defined with a heredoc.

resource "aws_ecr_lifecycle_policy" "test" {
  depends_on = [aws_ecr_repository.test]
  repository = "test-${var.services[count.index]}"
  count      = length(var.services)
  policy     = <<EOF
{
    "rules": [
        {
            "rulePriority": 1,
            "description": "Expire untagged images older than 7 days",
            "selection": {
                "tagStatus": "untagged",
                "countType": "sinceImagePushed",
                "countUnit": "days",
                "countNumber": 1
            },
            "action": {
                "type": "expire"
            }
        },
        {
            "rulePriority": 2,
            "description": "Keep last 10 PR images",
            "selection": {
                "tagStatus": "tagged",
                "tagPrefixList": ["PR-"],
                "countType": "imageCountMoreThan",
                "countNumber": 100
            },
            "action": {
                "type": "expire"
            }
        }
    ]
}
EOF
}

Version

tflint -v
TFLint version 0.23.0
+ ruleset.aws (0.1.1-bundled)Terraform v0.14.3
terraform -v
Terraform v0.14.3

It was working fine on v0.22.0

[bendrucker]

Folks, refrain from "+1" comments that don't add new information to the issue. You can keep up to date by subscribing to the thread and register your interest in the issue by adding a 👍🏻 to the original post.

[bendrucker]

Not gonna have time to fix this right this second, but leaving behind some bread crumbs.

This minimal heredoc case passes:

locals {
    heredoc = <<EOF
Here!
EOF
}

Failure happens when the AWS plugin is introduced:

resource "aws_ecr_lifecycle_policy" "test" {
  policy     = <<EOF
{}
EOF
}

With output:

Failed to check ruleset. An error occurred:

Error: Failed to check `aws_ecr_lifecycle_policy_invalid_policy` rule: main.tf:4,4-4: Unterminated template string; No closing marker was found for the string.

The originally submitted configuration is not valid on its own and therefore triggers a different error if evaluated:

10:14:49 runner_eval.go:108: [ERROR] Failed to eval an expression in main.tf:4; Reference to undeclared input variable: An input variable with the name "services" has not been declared. This variable can be declared with a variable "services" {} block.
10:14:49 server.go:418: rpc: gob error encoding body: gob: type not registered for interface: tflint.Error
Failed to check ruleset. An error occurred:

Error: Failed to check `aws_ecr_lifecycle_policy_invalid_policy` rule: reading body EOF

2021-01-04T10:14:49.690-0800 [DEBUG] plugin.tflint: 10:14:49 stream.go:16: [ERR] plugin: stream copy 'stderr' error: stream closed
2021-01-04T10:14:49.691-0800 [DEBUG] plugin.tflint: 10:14:49 rpc_server.go:48: [ERR] plugin: plugin server: accept unix /var/folders/7p/z_kw6fc9693fly126tljpml40000gn/T/plugin733321968: use of closed network connection
2021-01-04T10:14:49.692-0800 [DEBUG] plugin: plugin process exited: path=/usr/local/bin/tflint pid=88813
2021-01-04T10:14:49.692-0800 [DEBUG] plugin: plugin exited

The issue reporting the "Reference to undeclared input variable" is a separate one.

[ovcharenko]

@bendrucker You may try to use jsonencode() instead of HEREDOC

[bendrucker]

Are you suggesting that as a workaround? That's certainly well and good, I don't write literal JSON in heredocs either. But users should generally refrain from updating rather than rewriting their code. Or are you saying there's another issue?

[velkovb-nuk]

Moving the heredoc to locals and then referencing the value works.

I had to remove the variables from the configuration before submitting it. With proper values the errors is the same.

[bendrucker]

Moving the heredoc to locals and then referencing the value works.

TFLint doesn't evaluate locals at the moment (#571) so that will prevent TFLint from evaluating the attribute at all. If this is acceptable, disabling the aws_ecr_lifecycle_policy_invalid_policy rule would be a more direct way to get the same effect.

[velkovb-nuk]

I see, was not aware of that.

I tried disabling the rule but my problem is I call tflint in a CI environment for multiple terraform configurations some of which are not for AWS and don't have this rule.

I think for now I will stay with locals setup until there is a fix.

Thanks for the info!

[ovcharenko]

Are you suggesting that as a workaround? That's certainly well and good, I don't write literal JSON in heredocs either. But users should generally refrain from updating rather than rewriting their code. Or are you saying there's another issue?

Nope, just a quick workaround.

[Stromweld]

Moving the heredoc to locals and then referencing the value works.

TFLint doesn't evaluate locals at the moment (#571) so that will prevent TFLint from evaluating the attribute at all. If this is acceptable, disabling the aws_ecr_lifecycle_policy_invalid_policy rule would be a more direct way to get the same effect.

In your above breadcrumbs comment you mentioned the minimal heredoc case passes, but that's set in a locals block and your stating a local's block isn't evaluated by tflint. So there may be an issue with the minimal case test.

I'm assuming this error is going to pop up anywhere someone tries to use heredoc outside of locals block and referencing it.

Just for reference I'm getting the same error with a different resource.

TFLint version 0.23.0
+ ruleset.aws (0.1.1-bundled)
$ tflint --module
Failed to check ruleset. An error occurred:
Error: Failed to check `aws_iam_role_invalid_assume_role_policy` rule: .terraform/modules/vmware-cloudhealth/cloud-health-main.tf:25,4-4: Unterminated template string; No closing marker was found for the string.

[bendrucker]

In your above breadcrumbs comment you mentioned the minimal heredoc case passes, but that's set in a locals block and your stating a local's block isn't evaluated by tflint. So there may be an issue with the minimal case test.

The notable bit about that repro is that it shows that the issue comes from evaluating the attribute value rather than when parsing the doc. That can be inferred from the error too but isn't necessarily obvious.

And yes this will apply equally to any rule that tries to evaluate a heredoc expression.

[wata727]

This seems to be an HCL issue when parsing heredoc as an expression. If you are interested in the implementation, please see here as well. hashicorp/hcl#441

As a workaround, I think there's a way to always add a newline at the end when the plugin parses the expression, and I would like to hear opinions from the HCL dev team about this.

[idavydiak]

Hey @bendrucker I have the same problem while upgrading to terraform 0.13.
The problem is that we use a lot here-doc to paste multi-lines, like

resource "aws_cloudwatch_metric_alarm" "error_5xx_alarm" {
  alarm_name        = "High CloudFront 5XX Error Rate: ${var.readable_name}"
  alarm_description = <<-EOF
  More than ${local.threshold_5xx_alarm}% of CloudFront the requests served by
  the CloudFront distribution '${var.distribution_id}' (${var.readable_name})
  are errors!
  https://console.aws.amazon.com/cloudfront/v2/home?#/monitoring/${var.distribution_id}
  Priority: P1
  EOF

  ...

  tags = local.tags
}

And it is very convenient for us. But it fails with tflint 0.23. I moved EOF block to locals

locals {
  error_5xx_alarm_description = <<EOF
More than ${local.threshold_5xx_alarm}% of CloudFront the requests served by
the CloudFront distribution '${var.distribution_id}' (${var.readable_name})
are errors!

Priority: P1
EOF
}

resource "aws_cloudwatch_metric_alarm" "error_5xx_alarm" {

  alarm_name          = "High CloudFront 5XX Error Rate: ${var.readable_name}"
  alarm_description   = local.error_5xx_alarm_description

  ...
  tags = local.tags
}

It seems to work.
My question is, if it is a good approach to use locals for here-doc, using AWS provider, or maybe I should use something else?
Thanks in advance!

Best regards,
Iaroslav