Support for Auto-Approval when adding Synapse Managed Private Endpoints

[djpirra]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Currently we can use the Synapse provider to create managed private endpoints.
Unfortunately at the moment these private endpoints need to be approved manually and it prevents a full deployment using Terraform.

New or Affected Resource(s)

azurerm_synapse_managed_private_endpoint

Potential Terraform Configuration

is_manual_connection = false

[owenfarrell]

@djpirra - I did a bit of digging on this one, as I was interested in the same thing. Ultimately, I've concluded that the API used by the Azure portal to list private endpoints doesn't have an equivalent function in the Go SDK.

At the end of the day, we need to get the ID of the private endpoint associated with the managed private endpoint (which is different from the managed private endpoint itself).

• Attempt 1: When you create a new managed private endpoint, Azure returns the ID of the managed private endpoint and the ID of the private link resource

  • but Azure doesn't get the ID of the private endpoint that is Pending that resides in Microsoft's managed network

• Attempt 2: We could calculate the ID of the private endpoint using:

  • the Synapse workspace name and Synapse managed private endpoint name
    These are already known as part of the Managed Private Endpoint creation process

  • the subscription ID and resource group name of the managed network
    These are dynamic based on the result of the Managed Private Endpoint creation process, and I don't see any way to retrieve them 😔

• Attempt 3: We could query the list of private endpoints and then search the list for the name of the private endpoint that we care about.

  • you need subscription ID and resource group name of the managed network as part of the request 😔

[djpirra]

For now I have used Az CLI on a null provider to perform the approval on all the managed endpoints that are pending. While this resolves my issue, I believe a solution for having a better terraform integration should be found. How can we help?

On Sat, 25 Sep 2021 at 12:40 AM Owen Farrell ***@***.***> wrote: @djpirra <https://github.com/djpirra> - I did a bit of digging on this one, as I was interested in the same thing. Ultimately, I've concluded that the API used by the Azure portal to list private endpoints doesn't have an equivalent function in the Go SDK. At the end of the day, we need to get the ID of the private endpoint associated with the managed private endpoint (which is different from the *managed* private endpoint itself). - Attempt 1: When you create a new managed private endpoint, Azure returns the ID of the managed private endpoint and the ID of the private link resource - but Azure *doesn't* get the ID of the private endpoint that is Pending that resides in Microsoft's managed network - Attempt 2: We could calculate the ID of the private endpoint using: - the Synapse workspace name and Synapse managed private endpoint name *These are already known as part of the Managed Private Endpoint creation process* - the subscription ID and resource group name *of the managed network* the resource group name *of the managed network* *These are dynamic based on the result of the Managed Private Endpoint creation process, and I don't see any way to retrieve them* 😔 - Attempt 3: We could query the list of private endpoints and then search the list for the name of the private endpoint that we care about. - you need subscription ID and resource group name *of the managed network* as part of the request 😔 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13107 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZ5Q5RANCTQ6CZNSP36EXDUDTO4LANCNFSM5CVKY4TQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Melhores Cumprimentos, Luis Simões

[owenfarrell]

@djpirra - so I did some more digging and found #11975 is describing your configuration setup. In looking at that issue report, I now realize that I was looking at the wrong set of APIs.

Unfortunately, I don't think there is a one-size-fits-all approach to resolving this issue. Since private endpoint connections are managed on the target resources themselves,

I'm happy to jump in to this, but I need some guidance from @tombuildsstuff on how to approach as there are implications to both. Below are the two options that I've considered, but I'm not married to either.

Option 1: Add a is_manual_connection

The advantage to this approach is that it's super simple on end-users.

The disadvantage to this approach is that it pushes logic/complexity in to the synapse_managed_private_endpoint_resource implementation. That single resource is going to have to:

1. parse the target_resource_id
2. identify the type of resource
3. dispatch the creation function based on the type of resource
4. poll for state changes based on the type of resource

And that's no trivial effort - the synapse_managed_private_endpoint_resource would need to access 13ish clients in other packages.

Example HCL

resource "azurerm_synapse_managed_private_endpoint" "example" {
  name                 = "example-endpoint"
  synapse_workspace_id = azurerm_synapse_workspace.example.id
  target_resource_id   = azurerm_storage_account.example_connect.id
  subresource_name     = "blob"
  is_manual_connection = false
}

Option 2: Create xxxxx_private_endpoint_connection resources

The advantage to this approach is that it simplifies the implementation as each Terraform resource is pinned to a single Azure resource. So there's no identifying or dispatching required.

The down side is that it effectively pushes the complexity on to end users.

Example HCL

resource "azurerm_synapse_managed_private_endpoint" "example" {
  name                 = "example-endpoint"
  synapse_workspace_id = azurerm_synapse_workspace.example.id
  target_resource_id   = azurerm_storage_account.example_connect.id
  subresource_name     = "blob"
}

resource "azurerm_storage_blob_private_endpoint_connection" "example" {
  storage_account_id    = azurerm_storage_account.example_connect.id
  private_endpoint_name = azurerm_synapse_managed_private_endpoint.example_connect.name

  status      = "Approved"
  description = "An (optional) custom message to include with the approval"
}

As much as I like the simplicity of option 1, I think option 2 is the "right" approach since it (1) reinforces clarity on what resources are supported and (2) can be used to auto-approve endpoints beyond Synapse. But I do like the simplicity of the first option.

Thoughts/feedback welcome!

[djpirra]

I do believe that Option 1 is the best way to go, no doubt.
It follows a more homogenous integration in my opinion and it simplifies the usage for the developer.

Identifying the resource type can be done through the subresource name or maybe from the attributes assigned to the particular target resource ID, no?