azurerm_synapse_workspace creation with CMK failing and requires activation

[jamescross91]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

• Terraform v1.0.3
• AzureRM 2.82.0

Affected Resource(s)

• azurerm_synapse_workspace
• azurerm_synapse_workspace_key

Terraform Configuration Files

data "azurerm_client_config" "current" {}

resource "random_password" "password" {
  length            = 32
  special           = true
  override_special  = "_%@"
  min_upper         = 5
  min_lower         = 5
  min_numeric       = 5
  min_special       = 5
}

resource "azurerm_synapse_workspace" "synapse_workspace" {
  name                                 = "${var.synapse_workspace_name}-${var.resource_name_suffix}"
  managed_resource_group_name          = "${var.synapse_managed_resource_group_name}-${var.resource_name_suffix}"
  resource_group_name                  = var.rg_name
  location                             = var.rg_location
  storage_data_lake_gen2_filesystem_id = var.storage_account_synapse_file_system_id
  sql_administrator_login              = var.sql_server_admin_login
  sql_administrator_login_password     = random_password.password.result
  data_exfiltration_protection_enabled = var.synapse_data_exfiltration_protection_enabled
  managed_virtual_network_enabled      = var.synapse_managed_virtual_network_enabled
  sql_identity_control_enabled         = var.synapse_sql_identity_control_enabled

  customer_managed_key {
    key_versionless_id = var.synapse_encryption_key_versionless_id
    key_name           = var.synapse_encryption_key_name
  }

  aad_admin {
    login     = var.analytics_ad_group_name
    object_id = var.analytics_ad_group_id
    tenant_id = data.azurerm_client_config.current.tenant_id
  }
}
# Depends on both the creation of the keyvault and the workspace, so created here to avoid a circular dependency between the two modules 
resource "azurerm_key_vault_access_policy" "workspace_policy" {
  key_vault_id = var.keyvault_id
  tenant_id    = azurerm_synapse_workspace.synapse_workspace.identity[0].tenant_id
  object_id    = azurerm_synapse_workspace.synapse_workspace.identity[0].principal_id

  key_permissions = [
    "Get", "WrapKey", "UnwrapKey"
  ]
}

resource "azurerm_synapse_workspace_key" "synapse_workspace_key" {
  customer_managed_key_versionless_id = var.synapse_encryption_key_versionless_id
  synapse_workspace_id                = azurerm_synapse_workspace.synapse_workspace.id
  active                              = true
  cusomter_managed_key_name           = var.synapse_encryption_key_name
  depends_on                          = [azurerm_key_vault_access_policy.workspace_policy]
}

Debug Output

# module.synapse.azurerm_key_vault_access_policy.workspace_policy will be created
+ resource "azurerm_key_vault_access_policy" "workspace_policy" {
    + id              = (known after apply)
    + key_permissions = [
        + "Get",
        + "WrapKey",
        + "UnwrapKey",
      ]
    + key_vault_id    = "/subscriptions/d0fc0e0c-1888-47b0-8b01-1425bbce0aad/resourceGroups/aza-dev-james-jmcdevuk4/providers/Microsoft.KeyVault/vaults/keyvault-jmcdevuk4"
    + object_id       = (known after apply)
    + tenant_id       = (known after apply)
  }
# module.synapse.azurerm_synapse_workspace.synapse_workspace will be created
+ resource "azurerm_synapse_workspace" "synapse_workspace" {
    + aad_admin                            = [
        + {
            + login     = "aza-dev-james-jmcdevuk4-analytics"
            + object_id = "24bab146-a747-4e22-a6f1-1d2361d1018e"
            + tenant_id = "5f6b9ea3-fb44-458a-b837-98b92fc69a4d"
          },
      ]
    + connectivity_endpoints               = (known after apply)
    + data_exfiltration_protection_enabled = true
    + id                                   = (known after apply)
    + identity                             = (known after apply)
    + location                             = "westeurope"
    + managed_resource_group_name          = "synapse-rg-jmcdevuk4"
    + managed_virtual_network_enabled      = true
    + name                                 = "synapse-jmcdevuk4"
    + resource_group_name                  = "aza-dev-james-jmcdevuk4"
    + sql_administrator_login              = "sqladminuser"
    + sql_administrator_login_password     = (sensitive value)
    + sql_identity_control_enabled         = true
    + storage_data_lake_gen2_filesystem_id = "https://datastorejmcdevuk4.dfs.core.windows.net/synapse"
    + customer_managed_key {
        + key_name           = "encryption-key"
        + key_versionless_id = "https://keyvault-jmcdevuk4.vault.azure.net/keys/synapse-encryption-key"
      }
  }
# module.synapse.azurerm_synapse_workspace_key.synapse_workspace_key will be created
+ resource "azurerm_synapse_workspace_key" "synapse_workspace_key" {
    + active                              = true
    + cusomter_managed_key_name           = "encryption-key"
    + customer_managed_key_versionless_id = "https://keyvault-jmcdevuk4.vault.azure.net/keys/synapse-encryption-key"
    + id                                  = (known after apply)
    + synapse_workspace_id                = (known after apply)
  }
plan: 3 to add, 8 to change, 0 to destroy.
 Running terraform apply tfplan

Error: A resource with the ID "/subscriptions/blah/resourceGroups/aza/providers/Microsoft.Synapse/workspaces/synapse-jmcdevuk4" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_synapse_workspace" for more information.

with module.synapse.azurerm_synapse_workspace.synapse_workspace,
  on synapse/main.tf line 13, in resource "azurerm_synapse_workspace" "synapse_workspace":
 13: resource "azurerm_synapse_workspace" "synapse_workspace" {

Expected Behaviour

Workspace is created with CMK

Actual Behaviour

Workspace is created, but terraform complains that it's inactive. Re-running terraform results in a resource already exists error

Steps to Reproduce

Run the above

[trotaylor]

My configuration is the same as above and getting the same error about activation of workspace.

│ Error: updating Synapse Workspace "ab0001-workspace-test" Sql Admin (Resource Group "ab0001-workspace-test"): synapse.WorkspaceAadAdminsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="WorkspaceActivationRequired" Message="Workspace needs to be activated, by adding the managed identity in the KeyVault containing the customer managed key and activating the workspace through the keys subresource."
│
│ with module.workspace.azurerm_synapse_workspace.workspace,
│ on modules/main.tf line 21, in resource "azurerm_synapse_workspace" "workspace":
│ 21: resource "azurerm_synapse_workspace" "workspace" {
│
╵
Error: Process completed with exit code 1.

[jamescross91]

@katbyte - can you help us get some traction on this issue? We've raised this with Microsoft through our premium support plan too

[ms-henglu]

Hi all,

I checked this issue, the cause is that aad admin can't be added to a workspace whose cmk is not activated.
This issue happens to sql_aad_admin block as well. In v2.85.0, restriction that doesn't allow user to use both aad_admin and customer_managed_key is introduced.

Would you please try the following configuration? It removes aad_admin block and use azurerm_synapse_workspace_aad_admin resource( released in v2.85.0) instead.

resource "azurerm_synapse_workspace" "test" {
  name                                 = "henglusw"
  resource_group_name                  = azurerm_resource_group.test.name
  location                             = azurerm_resource_group.test.location
  storage_data_lake_gen2_filesystem_id = azurerm_storage_data_lake_gen2_filesystem.test.id
  sql_administrator_login              = "sqladminuser"
  sql_administrator_login_password     = "H@Sh1CoR3!"
  customer_managed_key {
    key_versionless_id = azurerm_key_vault_key.test.versionless_id
    key_name           = "enckey"
  }
}

resource "azurerm_key_vault_access_policy" "workspace_policy" {
  key_vault_id = azurerm_key_vault.test.id
  tenant_id    = azurerm_synapse_workspace.test.identity[0].tenant_id
  object_id    = azurerm_synapse_workspace.test.identity[0].principal_id

  key_permissions = [
    "Get", "WrapKey", "UnwrapKey"
  ]
}

resource "azurerm_synapse_workspace_key" "test" {
  customer_managed_key_versionless_id = azurerm_key_vault_key.test.versionless_id
  synapse_workspace_id                = azurerm_synapse_workspace.test.id
  active                              = true
  customer_managed_key_name           = "enckey"
  depends_on                          = [azurerm_key_vault_access_policy.workspace_policy]
}

// aad_admin block
resource "azurerm_synapse_workspace_aad_admin" "test" {
   login     = "AzureAD Admin"
   object_id = data.azurerm_client_config.current.object_id
   tenant_id = data.azurerm_client_config.current.tenant_id

   depends_on = [azurerm_synapse_workspace_key.test]
}

[mybayern1974]

@jamescross91 , do you think the solution provided by ms-henglu could unblock you? If so we may intend to close this issue.

[jamescross91]

@mybayern1974 - that works but there's a null pointer exception configuring sql_aad_admin - I see a fix for that has been merged this week pending release is 2.87.0 - hopefully that will fix things

[jamescross91]

still failing on 2.87.0 when trying to set sql_aad_admin

[ms-henglu]

Hi @jamescross91,

still failing on 2.87.0 when trying to set sql_aad_admin

You mean still get a null pointer exception or the problem mentioned in this issue?

[jamescross91]

The problem mentioned in the issue - NPE is resolved but with the SQL AAD Admin block - it creates the workspace and fails claiming it's inactive

[ms-henglu]

Got it. Please wait for the same fix as aad_admin block. The PR is #14341
And there's workaround for it, you can remove the sql_aad_admin block, then apply it, after the workspace with CMK created, add the sql_aad_admin block back.

[ms-henglu]

Hi @jamescross91 , the fix has been merged and released in https://github.com/hashicorp/terraform-provider-azurerm/releases/tag/v2.90.0, would you please try the latest version?

[mybayern1974]

@jamescross91 , with assuming the issue reported by you have been fixed and the release post v2.90 could work, I would like to close this issue within the next couple of days if there are no more updates from you.

[jamescross91]

@mybayern1974 we can close this - looks to be working - thanks all!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.