Feature Request: Support for Azure Stack

[hashibot]

This issue was originally opened by @Tasquith as hashicorp/terraform#7299. It was migrated here as part of the provider split. The original body of the issue is below.

---

Hi,

I'm looking into Azure Stack and terraform/packer at the moment. I've not found a way of configuring the API endpoint to access anything other than the default public cloud via these tools.

I can see that the azure-sdk-for-go supports this, I'm not sure about riviera though.

Azure/azure-sdk-for-go#330

Are there any timescales for adding configurable endpoint support? I understand Azure Stack is not generally available until november, but it would be nice to prove against the technical preview.

Regards,

Tom

[tombuildsstuff]

Just noticed this blog post explaining that it's possible to run Azure Stack on Azure now - which would allow us a test environment to develop + test this feature.

[hodyroff]

Indeed the demand for Azure Stack seems to pick up right now, is this on the Terraform roadmap? Any ETA?

[martinbjorgan]

+1

[tombuildsstuff]

👋 hey folks

We've been looking into how we'd support Azure Stack for a while now - unfortunately it appears that it's going to be more complex than just adding support for specifying a custom endpoint. We've previously had access to an Azure Stack instance which allowed us to identify the following issues:

1. Not all services are supported at the same API versions as Azure Public (e.g. managed disks aren't supported and validation requirements are different)

2. Some services are completely different in Azure Stack (e.g. SQL Servers/DB's use an external SQL Server rather than SQL Azure)

Both of these issues means that we need to do some investigation on how's best to proceed here, I think the next step would be to prototype supporting a few Azure Stack resources (e.g. VM's and VMSS's) to try and give us an indication of how drastically some of the existing resources would need to be changed.

Given this requires some more thought/investigation - I've tagged this thinking and we'll post an update when we've had a chance to look into this :)

Thanks!

[martinbjorgan]

Hi @tombuildsstuff and thanks for the update, though a bit sad.
Looking forward to hear more about the investigation. Hopefully it will result in stack support :)

Will inform my org about this latest update.

Cheers!

[charliejllewellyn]

Hey @tombuildsstuff, I'd be really interested to see how this is coming along. Have you seen this thread on the Ansible repo which seems like the same problems you are trying to solve. A common approach here could be really valuable? ansible/ansible#26195

Also we have a lab we'd be happy to get you access to if this is an impediment?

[ClaudiaBaur]

hi @tombuildsstuff,
for a first POC, we did some private changes with the September version of the azurerm provider and got Terraform working with Azure Stack for basic infrastructure like VM, network and storage. However, we changed the sources in a way that the newly built AzureRm provider now works with Azure Stack (but no longer with Azure…) and we did just overwrote stuff which cannot be taken to merge it back.
However, it might give you an idea about minimum changes to make the AzureRM provider work with Azure Stack for the basic infrastructure.

Add the environment for Azure Stack and define its endpoints

• Source: vendor\github.com\Azure\go-autorest\autorest\azure\environments.go

• Add the environment AZURELOCALCLOUD and define the following endpoints in the section LocalCloud.
  var (
  LocalCloud = Environment{
  Name: "AzureLocalCloud",
  ManagementPortalURL: "https://management.local.azurestack.external",
  PublishSettingsURL: "",
  ServiceManagementEndpoint: "",
  ResourceManagerEndpoint: "https://management.local.azurestack.external",
  ActiveDirectoryEndpoint: "https://login.windows.net",
  GalleryEndpoint: "https://adminportal.local.azurestack.external:30015/",
  KeyVaultEndpoint: "https://vault.local.azurestack.external",
  GraphEndpoint: "https://graph.windows.net/",
  StorageEndpointSuffix: "local.azurestack.external",
  SQLDatabaseDNSSuffix: "",
  TrafficManagerDNSSuffix: "",
  KeyVaultDNSSuffix: "vault.local.azurestack.external",
  ServiceBusEndpointSuffix: "",
  ServiceManagementVMDNSSuffix: "",
  ResourceManagerVMDNSSuffix: "",
  ContainerRegistryDNSSuffix: "",
  ActiveDirectoryResourceId: "Your ADResourceID...",
  }

• To use this environment later during your Terraform deployments, you have to set the
  environment variable ARM_ENVIRONMENT = "Local".
  Not setting this variable, the environment points to "Public" as a default.

• A proper solution depends on the way you want to go. If you plan to offer one common provider which works with Azure and Azure Stack, the environment variable could be the way to go. This is also the way Terraform currently uses to differentiate between several Azure clouds like German cloud, government cloud, and so on…

Change the API versions used to older versions supported by Azure Stack

• Sources: *.go
• Within the Terraform provider, you find tons of API versions hard coded for any infrastructure component. You can easily find them when searching for "const APIVersion" in the *.go files…
• We changed the versions for all infrastructure components used.
• To set API versions which are valid for Azure and Azure Stack, our hard-coded version changes are not sufficient as you cannot differentiate between API versions for Azure and for Azure Stack. To get his done, little more logic needs to be added to use API versions which depend from the underlying ARM environment…
• Using kind of a profile for API versions as proposed by @charliejllewellyn makes sense, because API versions will change with nearly every version of Azure Stack and always stay behind Azure...

Change on Azure Stack Authentication using ADAL Library

• Source: vendor\github.com\Azure\go-autorest\autorest\adal\token.go and azurerm\config.go
  The Terraform authentication uses the ADAL library. This does not work when not using the Azure Active Directory Resource ID for authentication in vendor\github.com\Azure\go-autorest\autorest\adal\token.go, fct. refreshInternal().
  As a workaround, we added an endpoint to set the ActiveDirectoryResourceId to the AzureRM environment and we used this endpoint in fct refreshInternal().
  But, with our implementation, you need to rebuilt the AzureRm provider after changing the
  ActiveDirectoryResourceId. For customers, the ActiveDirectoryResourceId needs to be changeable without rebuild of the provider…

[paultyng]

@ClaudiaBaur this is very helpful, thanks!

@charliejllewellyn thanks for the pointer to the thread, it is a very similar discussion to what we are having, I'll have to keep an eye on it, unfortunately its not on the Go SDK so the work won't be directly applicable. But we have definitely been weighing the big switch vs version specific resources approaches and haven't come to any conclusions.

We are still working on this per Tom's post above, its on our roadmap, but I don't have any hard information beyond that.

[bespinfw]

Hi we are also interested to use Terraform for deployments to Azure and Azure Stack.

In a conclusion this means that Azure Stack needs a seperate Terraform provider right? Until then VMs can be created also via the AzureRM provider.

Is an Azure Stack provider in the dev pipeline or on any roadmap?

Thanks!

[paultyng]

It's definitely on our roadmap, but as above, I don't have any firm timelines or anything to give out now, but there is definitely a lot of interest and top of mind for us.

[dsboulder]

I'm also very interested in either extending the azurerm provider to support stack, or perhaps a new provider. Sounds like the azure golang SDK does already support stack, but there are a huge number of properties you have to pass through in order to use it.

[louisleon90]

As an Azure Stack Operator to provide the AppService and SQL/MySQL resource providers on AzureStack it would be very helpfull to have Terrafrom in place. Also it would be a killer argument to use Terraform if we can support/consult our customers with it on both Azure public and the Azure Stack.

[sasukeh]

I wish I could use azurestackrm as terraform provider. Absolutely, we gonna be able to use hybrid on-demand cloud infrastructure. That would be able to use DevTest usage as well. How's progress for this?

[tombuildsstuff]

👋 hey @sasukeh @louisleon90 @dsboulder @bespinfw @ClaudiaBaur @charliejllewellyn @martinbjorgan @hodyroff @Tasquith

Just to let you know that we've released a new Azure Stack Provider for Terraform (which is available on Github here) - which can be used like so:

provider "azurestack" {
  version = "=0.1.0"
}

# Create a resource group
resource "azurestack_resource_group" "network" {
  name     = "production"
  location = "West US"
}

# Create a virtual network within the resource group
resource "azurestack_virtual_network" "network" {
  name                = "production-network"
  address_space       = ["10.0.0.0/16"]
  location            = "${azurestack_resource_group.network.location}"
  resource_group_name = "${azurestack_resource_group.network.name}"

  subnet {
    name           = "subnet1"
    address_prefix = "10.0.1.0/24"
  }

  subnet {
    name           = "subnet2"
    address_prefix = "10.0.2.0/24"
  }

  subnet {
    name           = "subnet3"
    address_prefix = "10.0.3.0/24"
  }
}

Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!