[Bug]: Provider Crash when applying masking policies

[AaronCoquet-Easypark]

Terraform CLI Version

1.9.8

Terraform Provider Version

1.0.1

Company Name

Easypark

Terraform Configuration

locals {
  masking_file_contents = flatten([
    for filename in fileset(local.masking_file_prefix, "**/*.json") : [
      for qual_table, obj in jsondecode(file("${local.masking_file_prefix}/${filename}")) : [
        for idx, c_obj in obj.columns : {
          database       = split(".", qual_table)[0]
          schema         = split(".", qual_table)[1]
          table          = split(".", qual_table)[2]
          column         = c_obj.column
          type           = c_obj.type
          masking_policy = c_obj.masking_policy
          clearance      = c_obj.clearance
        }
      ]
    ]
  ])

  mapped_masking_file_contents = {
    for idx, obj in local.masking_file_contents :
    format(
      "%s|%s|%s|%s",
      obj.database,
      obj.schema,
      obj.table,
      obj.column
      ) => merge(
      obj,
      {
        bypass_roles = concat(
          # Ternary split across lines!
          # if the clearance is PII or BOTH, then we need to add the PII clearance role
          contains(["PII", "BOTH"], obj.clearance) ? ([
            format("%s__%s__CLEARANCE__PII", obj.database, obj.schema)
          ]) : [],
          # if the clearance is SENSITIVE or BOTH, then we need to add the SENSITIVE clearance role
          contains(["SENSITIVE", "BOTH"], obj.clearance) ? ([
            format("%s__%s__CLEARANCE__SENSITIVE", obj.database, obj.schema)
          ]) : [],
          # if the clearance is neither PII nor SENSITIVE, then we need to add an empty string
          !contains(["PII", "SENSITIVE", "BOTH"], obj.clearance) ? [""] : []
        )
        "shortish_name" = format(
          "%s__%s__%s__%s__%s__%s__%s",
          obj.database,
          obj.schema,
          obj.table,
          obj.column,
          obj.type,
          obj.masking_policy,
          obj.clearance
        )
        "qualified_name" = format(
          # "\"database\".\"schema\".\"shortish_name\""
          # where shortish_name is:
          # "database__schema__table__column__type__masking_policy__clearance"
          "%s.%s.\"%s__%s__%s__%s__%s__%s__%s\"",
          obj.database == upper(obj.database) ? obj.database : "\"${obj.database}\"",
          obj.schema == upper(obj.schema) ? obj.schema : "\"${obj.schema}\"",
          obj.database,
          obj.schema,
          obj.table,
          obj.column,
          obj.type,
          obj.masking_policy,
          obj.clearance
        )
      }
    )
  }

  mapped_masking_file_contents_filtered = {
    for key, obj in local.mapped_masking_file_contents :
    key => obj
    if(true
      && can(local.structure[obj.database])
      && can(local.structure[obj.database]["schemas"][obj.schema])
      && can(local.structure[obj.database]["schemas"][obj.schema]["tables"][obj.table])
    )
  }
  policies_sensitive_varchar_full = {
    for key, obj in local.mapped_masking_file_contents_filtered :
    key => obj
    if obj.clearance == "SENSITIVE"
    && obj.type == "VARCHAR"
    && obj.masking_policy == "FULL"
  }
}

resource "snowflake_masking_policy" "sensitive_varchar_full" {
  provider = snowflake.masking_admin
  for_each = local.policies_sensitive_varchar_full

  name     = each.key
  database = each.value.database
  schema   = each.value.schema

  body    = <<-SQL
    CASE WHEN
      IS_ROLE_IN_SESSION('${each.value.bypass_roles[0]}')
      THEN val
      ELSE REPEAT('*', LEAST(LEN(val), 10))
    END
  SQL
  comment = <<-EOT
    Generic masking policy for columns in ${each.value.database}.${each.value.schema}.
    Replaces original VARCHAR value with *-s, capped at length 10.
    NULLs will remain as NULL.
    Bypassed with SENSITIVE clearance, via ${each.value.bypass_roles[0]} role.
  EOT

  argument {
    name = "VAL"
    type = "VARCHAR"
  }

  return_data_type = "VARCHAR"
}

resource "snowflake_table_column_masking_policy_application" "sensitive_varchar_full" {
  provider = snowflake.masking_admin
  for_each = local.policies_sensitive_varchar_full

  table          = format("\"%s\".\"%s\".\"%s\"", each.value.database, each.value.schema, each.value.table)
  column         = each.value.column
  masking_policy = snowflake_masking_policy.sensitive_varchar_full[each.key].name

}

Category

category:resource

Object type(s)

resource:masking_policy

Expected Behavior

1. The resource "snowflake_masking_policy.sensitive_varchar_full" is created in each schema that needs that type of masking
2. After all of those masking policies are created, they are applied to specified table columns within the related schema

Actual Behavior

1. The resource "snowflake_masking_policy.sensitive_varchar_full" is created in each schema that needs that type of masking
2. The provider crashes

Steps to Reproduce

example.json

1. Create a JSON file like the attached, adjusted to reference a database, schema, table and column that exists.
2. Adjust the first local to look at the JSON file
3. Terraform Plan
4. Terraform Apply

How much impact is this issue causing?

Medium

Logs

https://gist.github.com/AaronCoquet-Easypark/d9367f5ce67ad0b246b3b103f0088a51

Additional Information

Some, but not all, of the masking policies were created. None of the masking policies were applied.

Would you like to implement a fix?

• Yeah, I'll take it 😎

[sfc-gh-asawicki]

Hey @AaronCoquet-Easypark. Thanks for reaching out to us.

Can you please share the logs from the crash? The provided logs above show only the plan output and not the result of the apply. Please run apply with TF_LOG=DEBUG environment variable set. We don't need the whole logs, just the part with the crash and the stacktrace.

[AaronCoquet-Easypark]

Turns out the whole logs were large enough that GitHub rejected them!
I've updated the logs to only include the two resources listed (snowflake_masking_policy.sensitive_varchar_full and snowflake_table_column_masking_policy_application.sensitive_varchar_full). New Gist is linked in original message.

[sfc-gh-jmichalak]

Logs look good now, thanks. The issue is that in snowflake_table_column_masking_policy_application, masking_policy field expects a fully qualified name of the referenced object. You can simply use masking_policy = snowflake_masking_policy.sensitive_varchar_full[each.key].fully_qualified_name instead.

The panic is not ideal here, and an error should be returned instead. We plan to rework tables and the associations soon, so this will be fixed.

[AaronCoquet-Easypark]

That seems to have mostly fixed it!

However, it raises a different issue: when I use fully_qualified_name, I get this drift on every plan/apply (my schema and table are both lowercase, so need to be quoted):

~ masking_policy = "database.\"schema\".\"database|schema|table|column\"" -> "\"database\".\"schema\".\"database|schema|table|column\""

[sfc-gh-jmichalak]

I'm sorry, I did not verify this properly. The diff suppressor in the preview resources is sometimes not correct. Still, you need to use fully qualified names here, but with a matching format, so please use the TF function format for now (similarly to what you do in the table) field.

This permadiff shouldn't occur, and during the rework, we will fix this.

[AaronCoquet-Easypark]

As a fix, I set my DB, Schema, Table, and Column names to be quoted if they aren't all caps:

format("%s.%s.%s",
database == upper(database) ? database : format("\"%s\"", database),
... schema ...,
... table ...
)