azurerm_databricks_workspace - changing the sku no longer forces a new resource unless it is required

[neil-yechenwei]

fixes #9124

[tombuildsstuff]

LGTM 👍

[tombuildsstuff]

@neil-yechenwei running the tests for this both of these, unfortunately this fails due to the Network Intent Policy on the Subnet:

------- Stdout: -------
=== RUN   TestAccDatabricksWorkspace_complete
=== PAUSE TestAccDatabricksWorkspace_complete
=== CONT  TestAccDatabricksWorkspace_complete
TestAccDatabricksWorkspace_complete: testing.go:745: Error destroying resource! WARNING: Dangling resources
may exist. The full state and error is shown below.
Error: errors during apply: 2 problems:
- removing Network Security Group Association from Subnet "acctest-sn-public-201210111515241321" (Virtual Network "acctest-vnet-201210111515241321" / Resource Group "acctestRG-db-201210111515241321"): network.SubnetsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="NetworkSecurityGroupCannotBeRemovedDueToNipOnSubnet" Message="Network security group cannot be removed from subnet /subscriptions/*******/resourceGroups/acctestRG-db-201210111515241321/providers/Microsoft.Network/virtualNetworks/acctest-vnet-201210111515241321/subnets/acctest-sn-public-201210111515241321 because it has network intent policy /subscriptions/*******/resourceGroups/acctestRG-db-201210111515241321/providers/Microsoft.Network/networkIntentPolicies/adb-westeurope-npip-1b0e118dea45e229146ca580 applied." Details=[]
- removing Network Security Group Association from Subnet "acctest-sn-private-201210111515241321" (Virtual Network "acctest-vnet-201210111515241321" / Resource Group "acctestRG-db-201210111515241321"): network.SubnetsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="NetworkSecurityGroupCannotBeRemovedDueToNipOnSubnet" Message="Network security group cannot be removed from subnet /subscriptions/*******/resourceGroups/acctestRG-db-201210111515241321/providers/Microsoft.Network/virtualNetworks/acctest-vnet-201210111515241321/subnets/acctest-sn-private-201210111515241321 because it has network intent policy /subscriptions/*******/resourceGroups/acctestRG-db-201210111515241321/providers/Microsoft.Network/networkIntentPolicies/adb-westeurope-npip-91c62c7255ba715a396e8d20 applied." Details=[]

------- Stdout: -------
=== RUN   TestAccDatabricksWorkspace_update
=== PAUSE TestAccDatabricksWorkspace_update
=== CONT  TestAccDatabricksWorkspace_update
TestAccDatabricksWorkspace_update: testing.go:684: Step 2 error: errors during apply:
Error: removing Network Security Group Association from Subnet "acctest-sn-public-201210111515266957" (Virtual Network "acctest-vnet-201210111515266957" / Resource Group "acctestRG-db-201210111515266957"): network.SubnetsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="NetworkSecurityGroupCannotBeRemovedDueToNipOnSubnet" Message="Network security group cannot be removed from subnet /subscriptions/*******/resourceGroups/acctestRG-db-201210111515266957/providers/Microsoft.Network/virtualNetworks/acctest-vnet-201210111515266957/subnets/acctest-sn-public-201210111515266957 because it has network intent policy /subscriptions/*******/resourceGroups/acctestRG-db-201210111515266957/providers/Microsoft.Network/networkIntentPolicies/adb-westeurope-npip-58d4a17cdfa748ada099fb39 applied." Details=[]
Error: removing Network Security Group Association from Subnet "acctest-sn-private-201210111515266957" (Virtual Network "acctest-vnet-201210111515266957" / Resource Group "acctestRG-db-201210111515266957"): network.SubnetsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="NetworkSecurityGroupCannotBeRemovedDueToNipOnSubnet" Message="Network security group cannot be removed from subnet /subscriptions/*******/resourceGroups/acctestRG-db-201210111515266957/providers/Microsoft.Network/virtualNetworks/acctest-vnet-201210111515266957/subnets/acctest-sn-private-201210111515266957 because it has network intent policy /subscriptions/*******/resourceGroups/acctestRG-db-201210111515266957/providers/Microsoft.Network/networkIntentPolicies/adb-westeurope-npip-28f84f89b11ce8f2be6e1c3f applied." Details=[]
TestAccDatabricksWorkspace_update: testing.go:745: Error destroying resource! WARNING: Dangling resources
may exist. The full state and error is shown below.
Error: errors during apply: Error deleting Network Security Group "acctest-nsg-private-201210111515266957" (Resource Group "acctestRG-db-201210111515266957"): network.SecurityGroupsClient#Delete: Failure sending request: StatusCode=400 -- Original Error: Code="InUseNetworkSecurityGroupCannotBeDeleted" Message="Network security group /subscriptions/*******/resourceGroups/acctestRG-db-201210111515266957/providers/Microsoft.Network/networkSecurityGroups/acctest-nsg-private-201210111515266957 cannot be deleted because it is in use by the following resources: /subscriptions/*******/resourceGroups/acctestRG-db-201210111515266957/providers/Microsoft.Network/virtualNetworks/acctest-vnet-201210111515266957/subnets/acctest-sn-private-201210111515266957, /subscriptions/*******/resourceGroups/acctestRG-db-201210111515266957/providers/Microsoft.Network/virtualNetworks/acctest-vnet-201210111515266957/subnets/acctest-sn-public-201210111515266957. In order to delete the Network security group, remove the association with the resource(s). To learn how to do this, see aka.ms/deletensg." Details=[]

As such it appears that we may need to manage the Network Intent Policy within Terraform, so that we can guarantee the tear-down order here - would you be able to take a look?

Thanks!

[neil-yechenwei]

@tombuildsstuff , looks like that the test cases you mentioned failed for long time and are not related with this PR.
Per the error message,seems the root cause is that "networkIntentPolicy" is automatically applied to Virtual Network by default. Seems the API behavior is different than before. And "networkIntentPolicy" RP hasn't been implemented by terraform. For the fix, I assume we have to implement the RP "networkIntentPolicy" first and then explicitly remove the association from the terraform config if my understanding is correct. So maybe fixing those TC requires non-trivial efforts since it requires new resource "networkIntentPolicy".
So what do you think can we merge this PR first to solve the reported issue? Or do we have to make a perfect fix by adding a new resource?

[katbyte]

Thanks @neil-yechenwei - LGTM aside from one comment

[ghost]

This has been released in version 2.43.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 2.43.0"
}
# ... other configuration ...

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!