Remove Superuser permissions

[jersiovic]

I'm trying without success to suppress SuperUserHandler service because I don't want superuser admin has permissions for certain things.
In order to do so, in my custom module Startup.cs file I did the following:

public override void ConfigureServices(IServiceCollection services)
        {
            var handlerToRemove = services.FirstOrDefault(s => s.ServiceType == typeof(IAuthorizationHandler) 
                                    && s.ImplementationType == typeof(SuperUserHandler)).ToList();
            if (handlerToRemove != null )
            {
                services.Remove(handlerToRemove);
            }
...

To be sure SuperUserHandler is registered previous to that code I added "OrchardCore.Users" as a dependency of my custom module.
In fact, debugging I see the handler is registered and this code actually removes the handler.
However when I get an access token with admin user and call to a an api method calling to await _authorizationService.AuthorizeAsync(User, Permissions.APermissionAssignedToUsersThatAreNotAdmin), it returns true when it should return false.

Any idea of what could be causing this issue?