Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: API Management support #1177

Closed
pietro opened this issue Apr 27, 2018 · 28 comments · Fixed by #3151
Closed

Feature Request: API Management support #1177

pietro opened this issue Apr 27, 2018 · 28 comments · Fixed by #3151

Comments

@pietro
Copy link
Contributor

pietro commented Apr 27, 2018

I couldn't find anything on the documentation about API Gateways, policies, etc. for Azure API Management[0][1].

[0] https://docs.microsoft.com/en-us/azure/api-management/
[1] https://docs.microsoft.com/en-us/rest/api/apimanagement/

@woosley
Copy link

woosley commented Apr 29, 2018

I am looking for the same thing, is this in any plan?

@torresdal
Copy link
Contributor

I've created a pull request for the API Management service #1516. Once this is working correctly we can add other API Management resources.

@torresdal
Copy link
Contributor

torresdal commented Jul 8, 2018

I've started to look at the other relevant api-management resources to implement.

Edit: After using and learning API Management in Azure, it looks like all resources mentioned below are relevant for TF, and that deployment really don't come into play

I would like your input on where to draw the line between IaC (Infrastructure as Code) and deployment. The api-management-service is clearly IaC in my opinion and I didn't hesitate to implement it. Other relevant resources of api-management is not necessarily as clear.

I've listed possible api-management Terraform resources extracted from the Azure API Management REST API. After this exercise I was left with the following questions:

1. Which of these are clearly IaC and should be resources in Terraform? Or which ones are clearly not IaC and does not belong in Terraform?
2. In which priority? "Without resource x, y, z, we can't configure API Management in Terraform"
2. Are some of these resources better handled elsewhere (other tooling?)
3. Should all be separate resources, and does some belong together/merged?

Note: The ones with strikethrough are resources I personally think is irrelevant for Terraform.

  • api-management-api
  • api-management-api-diagnostic
  • api-management-api-issue
  • api-management-api-logger
  • api-management-api-operation
  • api-management-api-operation-policy
  • api-management-api-policy
  • api-management-api-release
  • api-management-api-schema
  • api-management-api-version-set
  • api-management-authorization-server
  • api-management-backend
  • api-management-certificate
  • api-management-delegation
  • api-management-diagnostic
  • api-management-email-template
  • api-management-group
  • api-management-identity-provider
  • api-management-issue
  • api-management-logger
  • api-management-notification
  • api-management-oidc-provider
  • api-management-policy
  • api-management-product
  • api-management-product-group
  • api-management-product-policy
  • api-management-product-subscription
  • api-management-property
  • api-management-tag
  • api-management-user

@lfshr
Copy link
Contributor

lfshr commented Jul 10, 2018

@torresdal this is something I'd be interested in picking up as well if you're looking for a partner in crime.

@scottzilla
Copy link
Contributor

I love this so much.

I cannot wait to migrate my brittle azurerm_template_deployment resources to native.

@anoff
Copy link

anoff commented Jul 23, 2018

For my use case I would require the base resources needed to manage products and APIs (rate limits, backends, rewrites, auth, ..). I assume that maps to the following resources from your list.

Mandatory to implement functionality:

  • api-management-api
  • api-management-api-policy
  • api-management-api-schema
  • api-management-backend
  • api-management-product
  • (api-management-product-policy)
  • (api-management-policy)

Lower Priority (ordered):

  • api-management-api-diagnostic
  • api-management-tag
  • api-management-email-template
  • api-management-certificate
  • api-management-identity-provider

Edit: Imho there is currently no tooling available to handle any of those resources in another way; so for me it is hard to draw a objective line saying some things should be TF and others should not. One question that popped into my head is how TF should handle self-service subscriptions that will not be generated via TF if you have a api-management-product-subscription resource available. But maybe just putting that resource in the backlog to avoid this scenario will be enough to get this feature off the ground :)

@torresdal
Copy link
Contributor

torresdal commented Aug 29, 2018

Hi all! Quick status. Picking this up again after a rather long summer vacation :-)

Currently the pull request (#1516) for the api management service is awaiting approval (expect this to go out very soon after great reviews and help by @tombuildsstuff @jeffreyCline), and I've started the work on api-management-api.

@lfshr You mentioned you might be interested in contributing - that would be great! It will probably be easier to do so when both api-management and api-management-api is merged into master, but I'm open to suggestions. Let me know what you think?

@torresdal
Copy link
Contributor

With regards to feature prioritization I totally agree with your assessment @anoff, and I think this is what we should aim for. Btw if anyone wants to contribute to any of these features, that would be very welcome. I've just started with the basic stuff, but too much work for me to do all these features on my own 😄

@torresdal
Copy link
Contributor

While working on the api-management-api implementation, I'm curious to what people commonly use/need here. Import from Swagger/Wsdl/... or specify each API operation manually (eg GET /users/{user})? What are common usecases?

Allowing users to explicitly define all operations of an API is by all means doable, but will require some work. A very important factor will be to get the abstraction level right, to make it usable.

Any input you have on this would be great. Thanks.

@alastairtree
Copy link
Contributor

Import from swagger would be lovely - so that I could just check in a swagger file with some .tf files and apply those rather than a giant pile of ARM template JSON

@torresdal
Copy link
Contributor

@alastairtree Import is what I've implemented so far and will probably be what we support in the first release.

The more I think about it, the more I think being able to add individual operations is not a good idea. If someone have good arguments for the opposite, please let me know.

@alastairtree
Copy link
Contributor

Yeah until terraform import can actually generates the HCL in .tf files for you you are probably right. But the ability to define your api mangagement config in HCL with references to swagger files for the api definition would be an nice feature I think.

Any idea on an ETA on API management support in the azurerm terraform provider?

@svscorp
Copy link

svscorp commented Oct 8, 2018

@alastairtree as I see it was merged 7 days ago: MR #1516

Hopefully updates will follow :)

@acarroll
Copy link

I've implemented the CRUD for product on the side (separate provider), have decided to merge it back into the main provider and am looking to complete that and raise a pull request hopefully this week, if I can get enough time.

For the project on which I'm currently working, the things that're candidates for TF provisioning are the APIMS, products, product policies, name value pairs and API version sets. These are the things we don't consider to be very volatile. We're often redeploying Swagger specs and versions during development so from this perspective I haven't considered API as a TF candidate.

@tombuildsstuff tombuildsstuff modified the milestones: Future, Being Sorted Oct 25, 2018
@torresdal
Copy link
Contributor

Just letting you know that I wont be contributing any more to this repository and the API Management resources, until I see major improvements in lead times. After months of coding resources for Azure API Management, only the first resource is merged (api_management). My second (api_management_api) has been in review for 2 months and nothing is happening and nobody is answering my questions - so I’ve now closed it (#2073). Hope other contributors gets a better experience contributing here, because I’m done wasting my time.

@mcwienczek
Copy link

mcwienczek commented Jan 14, 2019

Can someone from Hashicorp address why this PR couldn't be merged? And most importantly when those features might land in terraform?

@torresdal
Copy link
Contributor

Also interested In that.

@mcwienczek I guess that question should go to Hashicorp and not Microsoft, since this is a Hashicorp open source project.

@michaelmeelis
Copy link

I would like to see api-management-settings-virtual-network. I'm using the apim in combination with AKS and I need to put the APIM into an existing vnet->subnet. Is this a different issue or does this belong to this feature request.

@AndyMoore
Copy link
Contributor

so in the absence of this, we're looking at either embedding arm into terraform configuration, or just moving to aws?

@tombuildsstuff
Copy link
Contributor

tombuildsstuff commented Feb 26, 2019

hey @torresdal

So firstly I apologies that it's taken so long to reply to this.

Spending some time thinking about this issue, based on the comments above I think this ultimately comes down to 3 questions:

  1. Why did this PR sit for so long?
  2. When will these features be landing?
  3. What are we doing to improve this?

So:

Why did this PR sit for so long?

This is mostly unintentional, from our side we were busy with other things (personally that's some holiday, HashiConf and then the Requiring Imports changes) - and due to taking a quick skim over this PR I believed this was still waiting on changes to be made at your end. In either case unfortunately this is my fault for missing that you'd pushed changes here - so apologies about that.

Whilst I appreciate that probably isn't helpful at this point - we're aware there are pain points in the current flow (and I'll cover more about and our plan to improve that below).

When will these features be landing?

So I have good news on this front, we're going to be working on adding support for the following API Management Resources in the next few releases:

Whilst I don't have an exact timeline for each Resource, after spending some time trying to work out the dependencies between these resources - this is the (more or less) ordered list we're planning on supporting over the next few releases.

In addition we also plan to enhance the azurerm_api_management resource to add support for some of the fields which aren't currently supported.

What are we doing to improve this?

Whilst I appreciate this experience has been frustrating - we're aware there's issues that require fixing here and we're working to improve this going forward.

One of the benefits of investing heavily in the Azure Provider over the last couple of years has been the marked increase in adoption, both in terms of folks using the provider but also in terms of issues and PR's.

Whilst on the one hand we've done a fairly good job in terms of adding support for new resources - on the other hand we've not done so well in terms of bug fixes or community PR's - and we're intending on fixing that. Unfortunately for historical reasons our roadmaps pretty fixed until the end of April - but from that point we're intending on switching the ratios such that we can spend more time on Community PR's, Bug Fixes and Issues (as well as new resource development) - rather than being solely focused on new resource development.

Taking a look at where we are today, we're trying to review PR's and get them merged within a day or two of them coming in, which tends to work for the smaller PR's. On the flip side some of the larger PR's can take longer and sometimes fall through the cracks - in particular where they require more thought (such as #1805) or have an upstream dependency blocking them (such as #2582).

From our side @katbyte has spent some time adding linters to try and ensure the low-hanging comments are covered - and ensuring the codebase is a little more consistent insofar as when users look to existing resources, they're not using a pattern we're deprecating. In addition, we've been trying to ensure the PR reviews that we do are comprehensive - which whilst this may introduce a larger number of comments does reduce the amount of back and forth so that we can get things merged faster. Finally when reviewing PR's that are otherwise good to merge (for example we've not heard back from a user) we've been pushing the odd commit to the branch so that we can get these merged.

Over the past few months we've been working to try and make what's being worked on more visible through the project's GitHub Milestones, both in terms of the work we're planning on doing and in terms of PR's we plan to review.

This includes the use of the 'blocked' milestone where appropriate (e.g. where a PR is blocked on a broken API such as #2582) - where we've recently switched tact to temporarily closing PR's which are blocked on an upstream issue but leaving them assigned to the blocked milestone, which allows us to focus on other tasks whilst the upstream issues are resolved, rather than having them left open indefinitely.

Thinking ahead slightly further, we're also looking to onboard additional maintainers for the Azure Provider - but I don't have a timeframe for this unfortunately.

In the case of this particular Pull Request - apologies that this has slipped through the cracks, this was an error on my part. If you're still interested in contributing support for API Management API's we'd love to take another look and get this over the line - in either case I'd like to thank you for this contribution even though I appreciate it's been a frustrating experience.

Thanks!

@tombuildsstuff tombuildsstuff added this to the 1.23.0 milestone Feb 26, 2019
@michaelmeelis
Copy link

Hi @tombuildsstuff,

Great to hear that this part is getting some well deserved love. I'm still curious if the vnet part will be picked up as well. I've currently deployed the APIM with an ARM template to circumvent this problem but would love to see this back into this resource.

Kind regards,
Michael

@tombuildsstuff
Copy link
Contributor

@michaelmeelis support for API Management Virtual Networks is being implemented in #2582 - however it's blocked on an upstream issue, which is why that PR is temporarily closed

@michaelmeelis
Copy link

@tombuildsstuff thanks for the quick response! I will look at that issue.

@NathanielRose
Copy link

Hey @tombuildsstuff,

While testing the Azure API Management Service Provider I noticed a few things.

  • The terraform output from a provisioned API-mgmt service returns an scm_url with no credentials. This requires us to run a separate script to generate the tokens for us to push to the repo.
  • Once keys are generated you then have to deploy your commit to the API-mgmt service through the rest api adding another step in an automated script.
  • Is there a way to point the scm_url to a pre-existing git repo? (More of an API-mgmt service request)

@tombuildsstuff
Copy link
Contributor

tombuildsstuff commented Mar 5, 2019

@NathanielRose would you mind opening a separate issue for those? Thanks!

@torresdal
Copy link
Contributor

For the record, thanks for coming back with an explanation @tombuildsstuff! Looks like the activity on APIM has really picked up and things are moving. Good to see!

@ghost
Copy link

ghost commented Apr 3, 2019

This has been released in version 1.24.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
	version = "~> 1.24.0"
}
# ... other configuration ...

@ghost
Copy link

ghost commented May 3, 2019

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!

@ghost ghost locked and limited conversation to collaborators May 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.