Skip to content
This repository has been archived by the owner on Mar 6, 2023. It is now read-only.

zlabjp/k8s-athenz-syncer

 
 

Repository files navigation

k8s-athenz-syncer

K8s-athenz-syncer is a controller that synchronizes the Athenz domain data including the roles, services and policies into corresponding Kubernetes AthenzDomain custom resources.

Architecture

Table of Contents

  1. Background
  2. Install
  3. Configuration
  4. Usage
  5. Contribute
  6. Maintenance/Contacts
  7. License

Background

Athenz is a generic RBAC provider for Kubernetes resource access management and Istio service-service authentication and authorization. An Athenz domain contains a set of roles and policies defined by service admins. The policies can grant or deny an Athenz role with permissions to perform specific actions on services or resources. An Athenz role can comprise of a set of principals which could represent end users or other services.

Every Kubernetes namespace is mapped to an Athenz domain and the Athenz roles and policies defined within each domain are used to express access control rules to Kubernetes resources such as deployments, services, ingresses, etc. associated with the namespace.


Kubernetes namespace to Athenz domain name mapping

A kubernetes namespace is mapped to an Athenz domain with the following pattern:

Kubernetes Athenz
Namespace Domain
{user-namespace} {user-managed-domain}
{system-namespace} {special-cluster-admin-managed-domain}
- dashes '-' in namespace replaced with '.' in domain
- double dashes '--' in namespace replaced with '-' in domain
e.g. sports-frontend sports.frontend
Resources Resources
- {namespace-scoped} {user-managed-domain-prefixed}
e.g. sports-frontend/fantasy-prod-deployment sports.frontend:fantasy-dashboard
- {cluster-scoped} {special-admin-domain-prefixed}
Service Service
e.g. sports-frontend/fantasy-dashboard sports.frontend::fantasy-dashboard
RBAC - Role and RoleBindings Roles and Policies
- {Role.rule} {Policy.assertion}
- {RoleBinding.Subjects} {Role}

Note: Kubernetes system namespaces such as "kube-system", "istio-system" are mapped to an equivalent Athenz domain with the format: <cluster-admin-domain>.<cluster-namespace>


While Athenz ZMS provides APIs to perform resource access checks against user/client credentials, caching the relevant policies as Kubernetes resources allows any in-cluster auth provider service to perform in-memory validation of user or client credentials without worrying about the Athenz ZMS/ZTS service availability.

K8s-athenz-syncer calls Athenz ZMS API GetSignedDomains() API to fetch the entire contents of an Athenz domain signed by the ZMS including roles, principals and policies with signatures and creates Kubernetes Custom Resources that store the domain data in the cluster so that applications can do the security checks based on local cached data.

The controller also runs a cron that periodically fetches the list of Athenz domains that were modified during the cron interval and then fetches the signed contents for each domain and stores them as the AthenzDomain Custom Resource in the cluster in order to keep all policies in local cache updated. There is also a full resync cron that adds all the watched namespaces to the controller work queue so that all of Kubernetes AthenzDomains Custom Resources are resynced after a full resync interval.

Example AthenzDomain CR

apiVersion: v1
items:
- apiVersion: athenz.io/v1
  kind: AthenzDomain
  metadata:
    creationTimestamp: 2019-01-01T00:00:00Z
    generation:
    name: home.test
    namespace: home-test
    resourceVersion:
    selfLink:
    uid:
  spec:
    domains:
      domain:
        name: home.test
        ypmId: 0
        roles:
          name: home.test:role.admin
        modified: '2019-01-01T00:00:00.000Z'
        members:
          user.tester
        roleMembers:
          memberName: user.tester
        policies:
        contents:
            domain: home.test
            policies:
              name: home.test:policy.admin
            modified: '2019-01-01T00:00:00.000Z'
            assertions:
              role: home.test:role.admin
                resource: home.test:*
                action: "*"
                effect: ALLOW
        signature:
        keyId: xyz
        services: []
        entities:
        modified: '2019-01-01T00:00:00.000Z'
    signature:
    keyId: xyz

Install

Prerequisite

There are a variety of prerequisites required in order to run this controller, they are specified below.

  • Kubernetes cluster - A running Kubernetes cluster is required to run the controller. More information on how to setup a cluster can be found in the official documentation here. This controller was developed and tested with the 1.13 release.
  • Athenz - Athenz is required for the controller to fetch policy and domain data. More information and setup steps can be found here. The authorization management service (ZMS) and its apis are primarily used for this controller.
  • SIA Provider - A service identity agent (SIA) must be running in the Kubernetes cluster in order to provision X.509 certificates to instances in order to authenticate with Athenz. The approach we currently use in production can be found here.

Setup

Configuration files which must be applied to run k8s-athenz-syncer which can be found in the k8s directory.

Athenz Domain Custom Resource Definition

The Athenz Domain custom resource definition must be first created in order for the controller to sync the custom resource. Run the following command:

kubectl apply -f k8s/athenzdomain.yaml

Service Account

In order to tell SIA which service to provide an X.509 certificate to, a service account must be present. This is required for the controller to authenticate with ZMS for api calls. Run the following command:

kubectl apply -f k8s/serviceaccount.yaml

or

kubectl create serviceaccount k8s-athenz-syncer

ClusterRole and ClusterRoleBinding

This controller requires RBAC to create, update, delete, watch and list all Athenzdomains Custom Resources in the cluster. It also has a watch and list on namespaces in order to know which domains to look up from Athenz.

NOTE: If you are deploying to a non-default namespace, make sure to update the k8s/clusterrolebinding.yaml subject namespace accordingly.

kubectl apply -f k8s/clusterrole.yaml
kubectl apply -f k8s/clusterrolebinding.yaml

Deployment

The deployment for the controller contains three containers: sia init, sia refresh, and the controller itself. Build a docker image using the Dockerfile and publish to a docker registry. Make sure to replace the docker images inside of this spec to the ones which are published in your organization. Also, replace the zms url with your instance. Run the following command in order to deploy:

kubectl apply -f k8s/deployment.yaml

Configuration

K8s-athenz-syncer has a variety of parameters that can be configured, they are given below.

Parameters Description Default
admin-domain Admin domain that can be specified in order to fetch admin domains from Athenz
athenz-contact-time-cm-key Key of ConfigMap to record the latest time that the Update Cron contacted Athenz latest_contact
athenz-contact-time-cm-name Name of ConfigMap to record the latest time that the Update Cron contacted Athenz athenzcall-config
athenz-contact-time-cm-ns Namespace of ConfigMap to record the latest time that the Update Cron contacted Athenz kube-yahoo
auth-header Authentication header field
cacert Path to X.509 ca certificate file to use for zms authentication
cert Path to X.509 certificate file to use for zms authentication /var/run/athenz/service.cert.pem
disable-keep-alives Disable keep alive for zms client true
identity-key Directory containing private keys for service identity /var/run/keys/identity
inClusterConfig Set to true to use in cluster config true
key Path to private key file for zms authentication /var/run/athenz/service.key.pem
kubeconfig Absolute path to the kubeconfig file /root/.kube/config
log-location Log location /var/log/k8s-athenz-syncer/k8s-athenz-syncer.log
log-mode Logger mode INFO
ntoken-expiry Custom nToken expiration duration 1h0m0s
queue-delay-interval Delay interval time for workqueue 250ms
resync-cron Sleep interval for controller full resync cron 1h0m0s
secret-name Secret name that contains private key k8s-athenz-syncer
service-domain Athenz domain that contains k8s-athenz-syncer
service-name Service name k8s-athenz-syncer
system-namespaces A list of cluster system namespaces that you hope the controller to fetch from Athenz
update-cron Sleep interval for controller update cron 1m0s
zms-url Athenz full zms url including api path

Usage

Once the controller is up and running, the controller will create Kubernetes AthenzDomains Custom Resources in the cluster accordingly. Users and Applications can consume those AthenzDomains CR to get security policy information for access control checks.

  1. To see all the AthenzDomains CR created, run kubectl get athenzdomains
  2. In order to use AthenzDomains CR in applications, create AthenzDomains clientset and informers to retrieve the resources.

Contribute

Please refer to the contributing file for information about how to get involved. We welcome issues, questions, and pull requests.

Maintainers

Core Team : [email protected]

License

Copyright 2019 Oath Inc. Licensed under the Apache License, Version 2.0 (the "License")

About

No description, website, or topics provided.

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 99.7%
  • Dockerfile 0.3%