azurerm_resource_group_template_deployment - fix delete for nested template deployment

[teowa]

Community Note

• Please vote on this PR by adding a 👍 reaction to the original PR to help the community and maintainers prioritize for review
• Please do not leave "+1" or "me too" comments, they generate extra noise for PR followers and do not help prioritize for review

Description

The resource provider and types is not returned if nested template is used, Azure/azure-rest-api-specs#28517
If that happens, existing logic to retrieve API version will fail, this PR tries to send a fake ID and get the usable API version from error message.

PR Checklist

• I have followed the guidelines in our Contributing Documentation.
• I have checked to ensure there aren't other open Pull Requests for the same update/change.
• I have checked if my changes close any open issues. If so please include appropriate closing keywords below.
• I have updated/added Documentation as required written in a helpful and kind way to assist users that may be unfamiliar with the resource / data source.
• I have used a meaningful PR title to help maintainers and other users understand this change and help prevent duplicate work.
  For example: “resource_name_here - description of change e.g. adding property new_property_name_here”

Changes to existing Resource / Data Source

• I have added an explanation of what my changes do and why I'd like you to include them (This may be covered by linking to an issue above, but may benefit from additional explanation).
• I have written new tests for my resource or datasource changes & updated any relevent documentation.
• I have successfully run tests with my changes locally. If not, please provide details on testing challenges that prevented you running the tests.
• (For changes that include a state migration only). I have manually tested the migration path between relevant versions of the provider.

Testing

TF_ACC=1 go test -v ./internal/services/resource -parallel 5 -run TestAccResourceGroupTemplateDeployment_nestedTemplate -timeout 2h -ldflags="-X=github.com/hashicorp/terraform-provider-azurerm/version.ProviderVersion=acc"
=== RUN   TestAccResourceGroupTemplateDeployment_nestedTemplate
=== PAUSE TestAccResourceGroupTemplateDeployment_nestedTemplate
=== CONT  TestAccResourceGroupTemplateDeployment_nestedTemplate
--- PASS: TestAccResourceGroupTemplateDeployment_nestedTemplate (214.03s)
PASS
ok      github.com/hashicorp/terraform-provider-azurerm/internal/services/resource      214.054s

• My submission includes Test coverage as described in the Contribution Guide and the tests pass. (if this is not possible for any reason, please include details of why you did or could not add test coverage)

Change Log

Below please provide what should go into the changelog (if anything) conforming to the Changelog Format documented here.

• azurerm_resource_group_template_deployment - fix delete for nested template deployment [GH-00000]

This is a (please select all that apply):

• Bug Fix
• New Feature (ie adding a service, resource, or data source)
• Enhancement
• Breaking Change

Related Issue(s)

Fixes #25458

Note

If this PR changes meaningfully during the course of review please update the title and description as required.

[tombuildsstuff]

@teowa I've taken a look through and left some comments inline, but unfortunately we can't just use a fake API version here - that'd end up causing issues for Service Teams which would invariably come back to us as a bug report.

Digging into the logic for this one, I suspect the approach used won't work here since the caching key currently only supports top-level resources, which I believe is the root cause of this?

As such can we update the test to ensure it fails in main (since I suspect this one would pass?) - and then update the caching logic to fix this issue?

Thanks!

[tombuildsstuff]

This would lead to some interesting logs for the Service Teams to deal with..

Instead - hitting this endpoint:

az rest --method get --url "https://management.azure.com/subscriptions/XXXX/providers/Microsoft.Insights?api-version=2024-03-01"

Returns the (truncated) JSON below, which contains the nested Resource Type defined above (activityLogAlerts from Microsoft.Insights/activityLogAlerts):

{
  "authorizations": [],
  "id": "/subscriptions/XXXX/providers/microsoft.insights",
  "namespace": "microsoft.insights",
  "registrationPolicy": "RegistrationRequired",
  "registrationState": "Registered",
  "resourceTypes": [
    {
      "apiProfiles": [
        {
          "apiVersion": "2017-04-01",
          "profileVersion": "2018-06-01-profile"
        }
      ],
      "apiVersions": [
        "2020-10-01",
        "2017-04-01",
        "2017-03-01-preview"
      ],
      "capabilities": "SupportsTags, SupportsLocation",
      "locations": [
        "Global",
        "West Europe",
        "North Europe"
      ],
      "resourceType": "activityLogAlerts"
    }
  ]
}

(I've checked, the same information is returned from API Version 2022-09-01, which is currently being called).

---

Digging into the code, it appears the issue is coming from here:

terraform-provider-azurerm/internal/services/resource/template_deployment_common.go

Lines 285 to 288 in 18af7d0

	// NOTE: there's an enhancement in that not all RP's necessarily offer everything in every version
	// but the majority do, so this is likely sufficient for now
	resourceProviderApiVersions[strings.ToLower(providerId.ProviderName)] = *apiVersion
	break

.. and as such I don't believe this'll actually fix the issue? and I don't believe the acceptance test covers using a nested resource?

Instead I suspect that we'd need to fix this by changing the caching key to be something along the lines of lower(ResourceProvider)-lower(ResourceType), and then ensuring the API Version we're storing is the latest API version returned from the API (since some resources can't be modified using an earlier API Version than the one they're created using).

[teowa]

Hi @tombuildsstuff , thanks for the comment. Actually there are two cases:

1. The nested template issue in the azurerm_resource_group_template_deployment - API version information for RP "Microsoft.Insights" ("") was not found - activityLogAlerts #25458, the template used is to deploy a template(a Microsoft.Resources/deployments resource), the nested template deploy a Microsoft.Insight/activityLogAlerts resource, we call it nested template temporarily. The nested template occurs if directly using bicep build template in azurerm template deployment resource as in azurerm_resource_group_template_deployment - API version information for RP "Microsoft.Insights" ("") was not found - activityLogAlerts #25458. And when the azurerm provider tries to determine which to delete by retrieving the the template it only gets Microsoft.Resource RP and the Microsoft.Insight/activityLogAlerts resource is only displayed as ID in below outputResources field, full response can be found at [BUG] Microsoft.Resources/stable/2020-06-01 Deployments_Get method doesn't return resource providers used in deployment template Azure/azure-rest-api-specs#28517

"providers": [
      {
        "namespace": "Microsoft.Resources",
        "resourceTypes": [
          {
            "resourceType": "deployments",
            "locations": [
              null
            ]
          }
        ]
      }
    ],
    "dependencies": [],
    "outputResources": [
      {
        "id": "/subscriptions/xxx/resourceGroups/wt-test-rgtd/providers/Microsoft.Insights/activityLogAlerts/test"
      }
    ]

2. another is nested resource in the template, which has more than one resources in single template, the PR azurerm_resource_group_template_deployment - fix nested resources cannot be deleted #18319 introduce the logic to find the correct API version in the response error message if API received the wrong API version.

I totally agree with you to cache the resource type in key and use the correct API versions at the very begining to better solve the nested resource issue. But to address the nested template issue(above point 1) , where the REST API doesn't return the API version for these resources, we need to use other method, like in this PR, to guess a API version first. WDYT?

Thanks.