APICast Authentication in seperate policy

[pimg]

currently the Authentication is part of the APICast core policy (apicast/gateway/src/apicast/policy/apicast/apicast.lua) which invokes the post_action in the proxy (apicast/gateway/src/apicast/proxy.lua)
that makes the authrep call to the 3Scale rest service. Essentially hard coupling the APICast gateway to the 3Scale authentication service (either key based or oauth based). Since the execution of the apicast policy is required.

In my oppinion it would be benificial to create a more loosely coupled and flexible architecture by removing the hard link between APICast and the 3Scale authentication service enabling other authentication services to act as a
authentication service for the APICast gateway. Making the authrep functionality available in a seperate policy enables users to provide for an alternative authentication/reporting policy.
We have a concrete example of an organization which would like to use their custom authentication service which is available via a SOAP service as the authentication service for APICast.
currently this can only be done by adding the custom authentication in a seperate policy ontop of the 3Scale one.

To seperate the authentication functionality in a seperate policy is also in line with the "trend" to have more and more core functionality of APICAst/3Scale available in seperate policies.

I am aware of a flag in the request context which can skip te post action:
https://github.com/3scale/apicast/blob/master/gateway/src/apicast/policy/apicast/apicast.lua#L60
if context.skip_apicast_post_action then return end

But somehow this doen not feel right, and I don't know about any unforseen side effects by skipping the post_action altogether without providing an alternative.

I am also aware of issue 795 (#795) although I am not exactly sure if this issue is part of 795. If it should be part of this issue feel free to close this one.

Kind regards,

Pim

[mikz]

@pimg We are very much on the same note and plan to offer different authentication/authorization methods. But the 3scale authentication provides more than just "authentication". It is coupled to analytics, rate limiting, usage rules, billing, ...

What functionality of 3scale you'd like to use without the APIcast policy? If you remove 3scale authentication there is not much left over. Rate limiting, analytics, mapping rules - none of that is going to work.
APIcast can already work without the APIcast policy already. We are using it in tests and development mode. (BTW really bad name choice, we are going to rename it to 3scale soon). It is not possible to do this from 3scale UI because it would essentially disable the UI functionality like usage rules, analytics, rate limits, ...

[pimg]

I think it would make sense to have those seperate functionalities you mention like mapping rule matching, rate limiting and analytics as seperate policies. To allow for finer grained "mixing and matching".

The organization I mentioned in the issue for example would like to use the metrics and rate limiting part but not the mapping rule matching and use their SOAP service which returns if a request with a certain token is allowed access to an API. A workaround is to simply declare a "/" mapping rule. But having those functionality in seperate policies would first eliminate the need for a workaround and allow for other use cases where somebody would like to have the mapping rule matcher but not the analytics (just an example). Or have them all custom, although that might be ambitious.

I noticed the rate limiting functionality is already available in a seperate policy, so it seems like a natural path.

I also see the need of a seperate analytics/metric policy since some organisations use version numbers in HTTP headers and can't differentiate calls to different versions in different 3Scale methods/metrics. But maybe that should be a seperate issue?

[mikz]

The rate limiting policy does not use 3scale and just uses Redis. It is not compatible with 3scale metrics / usage rules / analytics. It is totally standalone.

How you want to use metrics and rate limiting part without mapping rules and authentication ?
3scale rate limiting is authentication. If you want to define rate limits on 3scale, then you have to use the 3scale authentication.
Incoming request is transformed into metrics by mapping rules. Then these metrics plus authentication are sent to 3scale backend for verification. If the identified application is active and did not reach a rate limit then it is allowed. Later this data is shown as analytics and used for billing.

If you skip 3scale authentication, how and when you want to use rate limits? At some point (before/after your custom auth) it would have to call 3scale anyway to verify this application did not reach the rate limit.
3scale backend can do authorize, authorize+report, report. We are using authorize+report for performance reasons, but it could be split in two. To verify the app exists, is allowed to use the API a, and did not reach the rate limit we need to call authorize. Then we can call report to actually make the rate limiting and analytics work. Just doing report would keep Analytics working (just total hits if no metrics would be reported). But authorize is required to verify the applications' rate limits.

Extracting data from HTTP headers to use as metrics is doable by custom policies already.

[pimg]

Ok, I did not realize the rate limiting, mapping rules and metrics where that integrated and, also from a functional perspective and basically are the authentication. Although I still do not fully understand the mapping rule part, since the authrep call takes the API key(s) and a usage object. The mapping rules are matched by APICast (configured by the config loader if I understand correctly) and result in a usage object (or an error) which is passed in the authrep. Or am I missing something completely?

Anyway, I think we can basically take this into 2 directions.

1 - don't use any of the 3scale API manager functionality and you are on your own when it comes to authentication, rate limiting and analytics. Batteries are not included....
2 - a custom authentication policy that would basically a adapter or wrapper around the authrep call. That way you still leverage all 3scale functionality, but provide a more natural place to implement your own authentication implementation on top of the 3scale functionality. For example implement something custom to extract the API key(s) from, perform additional calls to do your own checks and transform the authentication response of your custom logic to the 3Scale authrep call.

The first one, kind of depends on what the future of apicast is going to be, what I mean is if it always will be part of the 3scale offering or will someday be a standalone product.

The second one I view similar to the 3scale code plugins. But instead of providing a Java/Ruby/PHP/etc. Library a lua base policy is available which can be extended with custom implementation. Something which is now difficult since it is in the required apicast policy.

If I am totally missing something completely, I am glad to hear it :)

[mikz]

re mapping rules:
Yes, mapping rules are basically patters that are matched against the URL to produce usage/transaction. That is a list of tuples of "metric" and "delta". How much to increment each of the matched metrics. So you can say mapping rules produce metrics from the incoming requests. This usage/transaction is used to do the authrep to 3scale backend and verify this request does not go over any usage limits on those reported metrics.

1. Yes. Without the APIcast policy (hopefully called 3scale in the future) it would make sense only to be used without 3scale API Manager. But it is something we will push harder in the future with APIcast standalone configuration #795 by making it easier to configure standalone. We plan to support both use cases.

2. I think this is already doable. We have some "extra" checks - OAuth 2.0 Token Introspection policy - makes an extra HTTP call to verify the JWT with the Identity Provider.
   We also allow extracting credentials by another policy ([THREESCALE-586] Default credentials policy #741):
   https://github.com/3scale/apicast/blob/63d0c23918e67b817147682520cd35955453f1ea/gateway/src/apicast/proxy.lua#L228-L229
   We have a policy that handles the 3scale backend response Cache policy #546 and adds extra params to the 3scale backend call: Add 3scale referrer policy #728.
   I'm happy to hear about the exact use case and suggest ways to implement it or make necessary changes to make that possible.

[pimg]

ah great, I completely missed the extracted credentials line. I definetely need to spend some more time in the code. I also noticed the example of setting the credentials in the default_credentials policy. So that's a great example.

Like I mentioned in the opening ticket and you mentioned in the last reply the standalone version is already covered in #795 So that part is tracked in that issue, no need to keep this one open.

For the initial use case with the Soap authentication service, I think it will have a similar setup as the token_introspection policy. In general it kind of depends if the want to use the 3Scale API Manager, or in the future provide the batteries themselve.

[mikz]

OK, closing as there is nothing to do. It is possible to extend handing credentials with a policy as well as implement a policy to do some extra authentication (like token introspection does).