Creating a API Management policy for an API gives an ValidationError

[tverhoeven]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

$ terraform -v
Terraform v0.12.5
+ provider.azurerm v1.31.0

Affected Resource(s)

• azurerm_api_management_api_policy

Terraform Configuration Files

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://keybase.io/hashicorp

# Create default policy for example API
resource "azurerm_api_management_api_policy" "apim-api-example-policy" {

  resource_group_name = "${var.customer}-rg"
  api_management_name = azurerm_api_management.apim.name
  api_name            = azurerm_api_management_api.apim-api-example.name

# Set the policy here
  xml_content = file("${path.module}/default-api-policy.xml")

}

The XML file referenced is available in this gist: https://gist.github.com/tverhoeven/abf1054ff868ee9f602b4308fe606449

Debug Output

https://gist.github.com/tverhoeven/abf1054ff868ee9f602b4308fe606449

Panic Output

Expected Behavior

The policy is created without throwing an error.

Actual Behavior

Terraform report a error, status 400, ValidationError coming from the AzureRM:

Error: Error creating or updating API Policy (Resource Group "poc-test-rg" / API Management Service "poc-test-apim" / API "poc-test-api-example"): apimanagement.APIPolicyClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="ValidationError" Message="One or more fields contain incorrect values:" Details=[{"code":"ValidationError","message":"'\u003c', hexadecimal value 0x3C, is an invalid attribute character. Line 10, position 63.","target":"representation"}]

Steps to Reproduce

1. terraform apply

Important Factoids

I've tried this with both inline XML in the terraform, or with external files containing the XML. I've also tried with the external file formatted as ASCII or UTF-8 (checked with the file command). I always get the same error.

To me it looks like there is an UTF-8 encoding to many happening. See this snippet from the debug output: "\u003cinbound\u003e\n".

I also get this both with Terraform running on macOS and Ubuntu Linux. So it is not OS specific.

References

• #0000

[tverhoeven]

Any update on this? Anything that I can already do to further debug this?

[tombuildsstuff]

hey @tverhoeven

Thanks for opening this issue - sorry for the delayed response here.

Taking a look into this the Azure API requires that this XML is submitted within the HTTP Request, which is a JSON API - which I believe is why this is being encoded in this fashion.

From the HTTP Response being returned:

2019-07-24T11:45:40.346+0200 [DEBUG] plugin.terraform-provider-azurerm_v1.31.0_x4: {"error":{"code":"ValidationError","message":"One or more fields contain incorrect values:","details":[{"code":"ValidationError","target":"representation","message":"'<', hexadecimal value 0x3C, is an invalid attribute character. Line 10, position 63."}]}}

it appears that this doesn't like this line:

<when condition="@(context.Variables.GetValueOrDefault("urlId").Equals("1"))">

as such I'm wondering if you can confirm if this Policy works when set in the Portal?

Thanks!

[tverhoeven]

I can confirm that the policy itself works fine. For testing I first setup everything manually, including the policy. Then I made my Terraform code. The policy is copied from the policy editor in the portal into the XML file referenced in the Terraform code.

[riordanp]

Hi, I have encountered a similar issue where policies that validate correctly when applied in the portal gives an error when applied through terraform. Here's the minimal policy I tested with:

<policies>
    <inbound>
        <base />
        <set-variable name="abc" value="@(context.Request.Headers.GetValueOrDefault("X-Header-Name", ""))" />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
    </on-error>
</policies>

And here's the error returned by the API to terraform:

apimanagement.APIPolicyClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="ValidationError" Message="One or more fields contain incorrect values:" Details=[{"code":"ValidationError","message":"'X-Header-Name' is an unexpected token. Expecting white space. Line 4, position 86.","target":"representation"}]

This is what terraform read from the file:

xml_content         = "<policies>\r\n    <inbound>\r\n        <base />\r\n        <set-variable name=\"abc\" value=\"@(context.Request.Headers.GetValueOrDefault(\"X-Header-Name\", \"\"))\" />\r\n    </inbound>\r\n    <backend>\r\n        <base />\r\n    </backend>\r\n    <outbound>\r\n        <base />\r\n    </outbound>\r\n    <on-error>\r\n        <base />\r\n    </on-error>\r\n</policies>"

As with @tverhoeven I've tried inline the policy and setting the encoding to UTF-8 explicitly. This is a slightly different error to the original issue but it still seems like it could be related to an incorrectly encoded '"' character.

[riordanp]

I have looked into this a bit more and may have found a fix, when I make a request like this to the management API directly (this is what the provider is trying to do at the moment):

{
  "properties": {
    "format": "xml",
    "value": "<policies>\r\n    <inbound>\r\n        <base />\r\n        <set-variable name=\"abc\" value=\"@(context.Request.Headers.GetValueOrDefault(\"X-Header-Name\", \"\"))\" />\r\n    </inbound>\r\n    <backend>\r\n        <base />\r\n    </backend>\r\n    <outbound>\r\n        <base />\r\n    </outbound>\r\n    <on-error>\r\n        <base />\r\n    </on-error>\r\n</policies>"
  }
}

I get the same error as in terraform: 'X-Header-Name' is an unexpected token. Expecting white space. Line 4, position 86.

However when changing the properties.format field to rawxml the policy passes validation and is updated. See the docs for this resource at https://docs.microsoft.com/en-us/rest/api/apimanagement/2019-01-01/apipolicy/createorupdate#apimanagementcreateapipolicynonxmlencoded

Hope this helps.

[opticyclic]

You can test this with the az cli.

Assuming you have a resource group called test and an apim called testing-1234 with the Echo API in it.

This doesn't work:

az rest -m put -u "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/test/providers/Microsoft.ApiManagement/service/testing-1234/apis/echo-api/policies/policy?api-version=2019-01-01" -b '{"properties":{"format":"xml","value":"<policies><inbound><base /><set-variable name=\"abc\" value=\"@(context.Request.Headers.GetValueOrDefault(\"X-Header-Name\", \"\"))\" /> </inbound> <backend> <forward-request />  </backend><outbound /></policies>"}}'

Changing to the rawxml as mentioned does work:

az rest -m put -u "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/test/providers/Microsoft.ApiManagement/service/testing-1234/apis/echo-api/policies/policy?api-version=2019-01-01" -b '{"properties":{"format":"xml","value":"<policies><inbound><base /><set-variable name=\"abc\" value=\"@(context.Request.Headers.GetValueOrDefault(\"X-Header-Name\", \"\"))\" /> </inbound> <backend> <forward-request />  </backend><outbound /></policies>"}}'

It seems to be something with the quoting and escaping because a more simple example works fine with the xml format:

az rest -m put -u "https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/test/providers/Microsoft.ApiManagement/service/testing-1234/apis/echo-api/policies/policy?api-version=2019-01-01" -b '{"properties":{"format":"xml","value":"<policies> <inbound /> <backend>    <forward-request />  <\/backend>  <outbound /><\/policies>"}}'

Trying to embed the command in a local-exec causes more problems as there is an extra level of escaping required.

However, you can put the command in a shell script and call that as a workaround. e.g.

resource "null_resource" "apim-policy" {
  provisioner "local-exec" {
    command     = "./policy.sh"
    interpreter = ["/bin/bash", "-c"]
    working_dir = "apim"
  }
  depends_on = ["azurerm_api_management.apim"]
}

[timmyreilly]

Found another workaround, you can use 'character entities' for quotes, slashes, and angle brackets. The xml file I now pass to xml_content in azurerm_api_management_api_operation_policy looks something like this for inbound policies:

<inbound>
        <!-- Authenticate APIM with Blob Storage -->
        <set-header name="x-ms-version" exists-action="override">
            <value>2019-02-02</value>
        </set-header>
        <authentication-managed-identity resource="https://storage.azure.com/" />
        
        <!-- Set Block Blob as blob type -->
        <set-header name="x-ms-blob-type" exists-action="append">
            <value>BlockBlob</value>
        </set-header>
        <set-variable name="Base64EncodedSnapshot" value="@{
        JObject requestBody = context.Request.Body.As&lt;JObject&gt;(preserveContent: true); 
            JToken imageNameJtoken = requestBody.GetValue(&#34;Base64EncodedSnapshot&#34;); 
            return imageNameJtoken.ToString(); 
        }" />
        <rewrite-uri template="@{ 
            JObject requestBody = context.Request.Body.As&lt;JObject&gt;(preserveContent: true); 

            string SourceIp = requestBody.GetValue(&#34;SourceIp&#34;).ToString();
            string Id = requestBody.GetValue(&#34;Id&#34;).ToString(); 
            string ImageName = requestBody.GetValue(&#34;ImageName&#34;).ToString(); 

            string fullyQualifiedPath = SourceIp + &#34;&#47;&#34; + Id + &#34;&#46;&#34; + &#34;jpeg&#34;;
            
            return &#34;images&#47;&#34; + fullyQualifiedPath;
        }" 
        />
        <set-body>@{
            return Convert.FromBase64String((string)context.Variables["Base64EncodedSnapshot"]); 
        }</set-body>
        <base />
        
    </inbound>

[jeanpaulsmit]

Holy cow, you saved me with this!
Encoded:

" to "
< to &#60;
> to &#62;

and the following expression started to work with "azurerm_api_management_api_policy"

<value>@(context.Variables.GetValueOrDefault&#60;string&#62;(&#34;myvalue&#34;))</value>

By the way, alternatively you can use a small Powershell script to do this.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!