OpenID Connect Support (research)

[dervoeti]

In general

Authentication via OIDC works, but there does not seem to be native support for authorization. We could ship an existing provider for example for Keycloak, but we will most likely implement our own UserGroupProvider for this that uses our UserInfoFetcher.
This means that, for now, authorization has to be defined somewhere else and does not come from the OIDC provider. As an easy example, I specified an initial admin user for NiFi ("Initial User Identity 1" in authorizers.xml in the examples that follow later). If the OIDC authentication succeeds and the identity (by default email, but can be configured to any other claim by specifying nifi.security.user.oidc.claim.identifying.user) of the authenticated user matches the identity of the initial admin user, the user is now logged in to NiFi and has admin permissions. If another user with a different email address authenticates, authentication itself still works, but the user can't do anything because it is not known by NiFi. More users can for example be added by the admin user via the NiFi UI, these users are then also associated with identities of the OIDC provider upon login.

There are at least three possibilites how to configure NiFi to use OIDC for authentication:

• Every OIDC user is able to login to NiFi and has the same permissions (not sure how exactly authorization works in this case, but it works). A pre-defined admin user is also able to login via API.
• Only users that are known by NiFi are able to login via OIDC (e.g. an initial admin user). This user is also able to login via API. Other users can for example be added to NiFi by the initial admin user.
• Every OIDC user is able to login to NiFi, but authorization is provided by a custom UserGroupProvider. We need to check how API authentication works in this case (API authentication is needed to create the reporting task that enables JVM and NiFi metrics).

The third solution is probably the best regarding the flexibility, but it needs some more research and we will most likely need to build our own UserGroupProvider.
Examples for the first two cases follow below.

Basic setup

stackablectl operator install commons=0.0.0-dev secret=0.0.0-dev nifi=0.0.0-dev zookeeper=0.0.0-dev
(I ran this on 2023-10-30, will most likely match SDP version 23.11)

Zookeeper cluster and Znode:

apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperCluster
metadata:
  name: simple-zk
spec:
  image:
    productVersion: 3.8.0
    stackableVersion: "23.7.0"
  servers:
    roleGroups:
      default:
        replicas: 3

apiVersion: zookeeper.stackable.tech/v1alpha1
kind: ZookeeperZnode
metadata:
  name: simple-nifi-znode
spec:
  clusterRef:
    name: simple-zk

Admin credentials for NiFi:

apiVersion: v1
kind: Secret
metadata:
  name: simple-admin-credentials
stringData:
  admin: admin
---
apiVersion: authentication.stackable.tech/v1alpha1
kind: AuthenticationClass
metadata:
  name: simple-nifi-users
spec:
  provider:
    static:
      userCredentialsSecret:
        name: simple-admin-credentials

Example NiFi Cluster for the first configuration variant ("Every OIDC user is able to login to NiFi"):

apiVersion: nifi.stackable.tech/v1alpha1
kind: NifiCluster
metadata:
  name: simple-nifi
spec:
  clusterConfig:
    authentication:
      - authenticationClass: simple-nifi-users
    listenerClass: external-unstable
    sensitiveProperties:
      autoGenerate: true
      keySecret: nifi-sensitive-property-key
    zookeeperConfigMapName: simple-nifi-znode
  image:
    productVersion: 1.21.0
    stackableVersion: 23.7.0
  nodes:
    configOverrides:
      nifi.properties:
        nifi.security.user.oidc.client.id: CLIENT_ID
        nifi.security.user.oidc.client.secret: CLIENT_SECRET
        nifi.security.user.oidc.discovery.url: https://KEYCLOAK_HOST/realms/KEYCLOAK_REALM/.well-known/openid-configuration
    roleGroups:
      default:
        replicas: 1

Example NiFi Cluster for the second configuration variant ("Only users that are known by NiFi are able to login via OIDC")

Define authorizers:

apiVersion: v1
data:
  authorizers.xml: |
    <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
    <authorizers>

      <userGroupProvider>
        <identifier>file-user-group-provider</identifier>
        <class>org.apache.nifi.authorization.FileUserGroupProvider</class>
        <property name="Users File">./conf/users.xml</property>
        <property name="Legacy Authorized Users File"></property>
        <property name="Initial User Identity 1">admin</property>
        <property name="Initial User Identity other-nifis">CN=generated certificate for pod</property>
      </userGroupProvider>

      <accessPolicyProvider>
        <identifier>file-access-policy-provider</identifier>
        <class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
        <property name="User Group Provider">file-user-group-provider</property>
        <property name="Authorizations File">./conf/authorizations.xml</property>
        <property name="Initial Admin Identity">admin</property>
        <property name="Legacy Authorized Users File"></property>
        <property name="Node Identity other-nifis">CN=generated certificate for pod</property>
        <property name="Node Group"></property>
      </accessPolicyProvider>

      <authorizer>
        <identifier>managed-authorizer</identifier>
        <class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
        <property name="Access Policy Provider">file-access-policy-provider</property>
      </authorizer>

    </authorizers>
kind: ConfigMap
metadata:
  name: nifi-oidc-authorizers

Mount this file in our NiFi cluster:

apiVersion: nifi.stackable.tech/v1alpha1
kind: NifiCluster
metadata:
  name: simple-nifi
spec:
  image:
    productVersion: 1.21.0
    stackableVersion: 23.7.0
  clusterConfig:
    listenerClass: external-unstable
    authentication:
      - authenticationClass: simple-nifi-users
    extraVolumes:
      - name: nifi-oidc-authorizers
        configMap:
          name: nifi-oidc-authorizers
    sensitiveProperties:
      keySecret: nifi-sensitive-property-key
      autoGenerate: true
    zookeeperConfigMapName: simple-nifi-znode
  nodes:
    configOverrides:
      nifi.properties:
        nifi.security.user.authorizer: managed-authorizer
        nifi.security.user.oidc.client.id: CLIENT_ID
        nifi.security.user.oidc.client.secret: CLIENT_SECRET
        nifi.security.user.oidc.discovery.url: https://KEYCLOAK_HOST/realms/KEYCLOAK_REALM/.well-known/openid-configuration
        nifi.security.user.oidc.claim.identifying.user: preferred_username
        nifi.authorizer.configuration.file: /stackable/userdata/nifi-oidc-authorizers/authorizers.xml
    roleGroups:
      default:
        replicas: 1

If a user with the preferred_username "admin" logs in via the OIDC provider, the user is associated with the initial NiFi admin user. Since the username of the initial admin user provided in authorizers.xml is the same as in the simple-admin-credentials Secret, the "admin" user is also able to authenticate to the API via the credentials provided in that Secret.

[fhennig]

Hey! Thanks for the write-up, I've linked this ticket in our epic, and I'm closing #507 in favor of this one.

For context, two links from #507 that might be useful as well (but probably not, since you already wrote a lot!):

• https://bryanbende.com/development/2017/10/03/apache-nifi-openid-connect
• https://github.com/orgs/stackabletech/discussions/29#discussioncomment-6657116