diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/lint.yml similarity index 69% rename from .github/workflows/golangci-lint.yml rename to .github/workflows/lint.yml index 51f8604f2..1208037fc 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/lint.yml @@ -11,16 +11,24 @@ permissions: contents: read # Optional: allow read access to pull request. Use with `only-new-issues` option. # pull-requests: read + jobs: golangci: - name: lint + name: golangci runs-on: ubuntu-latest steps: + - name: Checkout repository + uses: actions/checkout@v4 + # Upgraded Go version? Make sure to upgrade it in the GitHub Actions setup, the Dockerfile and the go.mod as well, so the linter and tests run the same version. - uses: actions/setup-go@v3 with: go-version: 1.21 - - uses: actions/checkout@v3 + + - name: go vet + run: go vet ./... + working-directory: src + - name: golangci-lint uses: golangci/golangci-lint-action@v3 with: @@ -44,4 +52,18 @@ jobs: # skip-pkg-cache: true # Optional: if set to true then the action don't cache or restore ~/.cache/go-build. - # skip-build-cache: true \ No newline at end of file + # skip-build-cache: true + + markdownlint: + name: Check for Markdown errors + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + # Tip: run the markdown lint action locally with '--fix' to automatically fix some of the issues: + # docker run -v $PWD:/workdir ghcr.io/igorshubovych/markdownlint-cli:latest "**/*.md" --fix + - uses: articulate/actions-markdownlint@v1 + with: + config: .markdownlint.json + files: '**/*.md' \ No newline at end of file diff --git a/.markdownlint.json b/.markdownlint.json new file mode 100644 index 000000000..632ae1f5d --- /dev/null +++ b/.markdownlint.json @@ -0,0 +1,5 @@ +{ + "default": true, + "MD013": false, + "MD033": false +} \ No newline at end of file diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 3eb754072..631a8a351 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -55,7 +55,7 @@ further defined and clarified by project maintainers. ## Enforcement Instances of abusive, harassing, or otherwise unacceptable behavior may be -reported by contacting the project team at help@otterize.com. All +reported by contacting the project team at . All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. @@ -68,9 +68,9 @@ members of the project's leadership. ## Attribution This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, -available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html +available at [homepage]: https://www.contributor-covenant.org For answers to common questions about this code of conduct, see -https://www.contributor-covenant.org/faq + diff --git a/README.md b/README.md index 8a309e2bc..765bfae33 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,12 @@ -# Otterize intents operator - - +# Otterize intents operator +logo ![build](https://github.com/otterize/intents-operator/actions/workflows/build.yaml/badge.svg) [![Go Report Card](https://goreportcard.com/badge/github.com/otterize/intents-operator/src)](https://goreportcard.com/report/github.com/otterize/intents-operator/src) [![community](https://img.shields.io/badge/slack-Otterize_Slack-purple.svg?logo=slack)](https://joinslack.otterize.com) -* [About](#about) +* [About](#about) * [Quickstart & Docs](https://docs.otterize.com/) * [How does the intents operator work?](#how-does-the-intents-operator-work) * [Network policies](#network-policies) @@ -22,16 +21,17 @@ * [Contributing](#contributing) * [Slack](#slack) - ## About + The Otterize intents operator is a tool used to easily automate the creation of network policies and Kafka ACLs in a Kubernetes cluster using a human-readable format, via a custom resource. -Users declare each client's intents to access specific servers (represented as the kind `ClientIntents`); -the operator automatically labels the relevant pods accordingly, +Users declare each client's intents to access specific servers (represented as the kind `ClientIntents`); +the operator automatically labels the relevant pods accordingly, and creates the corresponding network policies and Kafka ACLs. Here is an example of a `ClientIntents` resource enabling traffic from `my-client` to `web-server` and `kafka-server`: + ```yaml apiVersion: k8s.otterize.com/v1alpha3 kind: ClientIntents @@ -49,12 +49,14 @@ spec: ## How does the intents operator work? ### Network policies -The intents operator automatically creates, updates and deletes network policies, and automatically labels client and server pods, + +The intents operator automatically creates, updates and deletes network policies, and automatically labels client and server pods, to match declarations in client intents files. -The policies created are `Ingress`-based, so source pods are labeled with a `can-access-=true` +The policies created are `Ingress`-based, so source pods are labeled with a `can-access-=true` while destination pods are labeled with `has-identity=`. -The example above results in the following network policy being created: +The example above results in the following network policy being created: + ```yaml Name: access-to-web-server Spec: @@ -71,175 +73,193 @@ Spec: For more usage example see the [network policy tutorial](https://docs.otterize.com/quick-tutorials/k8s-network-policies). ### AWS IAM + The intents operator, together with the [credentials operator](https://github.com/otterize/credentials-operator), enables the intent-based declarative management of AWS IAM roles and policies. To enable AWS access for a pod, follow these steps: + 1. Label a pod to have an AWS role created for it: -``` -metadata: - labels: - "credentials-operator.otterize.com/create-aws-role": "true" -``` + + ```yaml + metadata: + labels: + "credentials-operator.otterize.com/create-aws-role": "true" + ``` 2. Declare ClientIntents to specify which AWS resources it needs access to: -```yaml -apiVersion: k8s.otterize.com/v1alpha3 -kind: ClientIntents -metadata: - name: server -spec: - service: - name: server - calls: - - name: arn:aws:s3:::otterize-tutorial-bucket-*/* - type: aws - awsActions: - - "s3:PutObject" -``` + + ```yaml + apiVersion: k8s.otterize.com/v1alpha3 + kind: ClientIntents + metadata: + name: server + spec: + service: + name: server + calls: + - name: arn:aws:s3:::otterize-tutorial-bucket-*/* + type: aws + awsActions: + - "s3:PutObject" + ``` Try the [AWS IAM tutorial](https://docs.otterize.com/quickstart/access-control/aws-iam-eks) to learn more. ### Azure IAM + The intents operator, together with the [credentials operator](https://github.com/otterize/credentials-operator), enables the intent-based declarative management of Azure IAM identities and role assignments. To enable Azure access for a pod, follow these steps: + 1. Label a pod to have an Azure managed identity created for it: -``` -metadata: - labels: - "credentials-operator.otterize.com/create-azure-workload-identity": "true" -``` + + ```yaml + metadata: + labels: + "credentials-operator.otterize.com/create-azure-workload-identity": "true" + ``` 2. Declare ClientIntents to specify which Azure scopes it needs access to: -```yaml -apiVersion: k8s.otterize.com/v1alpha3 -kind: ClientIntents -metadata: - name: server -spec: - service: - name: server - calls: - # The Azure resource ID that references the resource(s) for the authorization. Subscription & resource group are automatically appended. - - name: "/providers/Microsoft.Storage/storageAccounts/otterizeazureiamtutorial/blobServices/default/containers/otterizeazureiamtutorialcontainer" - type: azure - # one or more Azure roles that will be provided to the specified resources - azureRoles: - - "Storage Blob Data Contributor" - - name: "/providers/Microsoft.KeyVault/vaults/otterizetutorialazureiamkeyvault" - type: azure - # Optional - Grant Azure Key Vault data plane access by using Key Vault access policy - azureKeyVaultPolicy: - certificatePermissions: - - "all" - keyPermissions: - - "all" - secretPermissions: - - "all" - storagePermissions: - - "get" - - "list" -``` + + ```yaml + apiVersion: k8s.otterize.com/v1alpha3 + kind: ClientIntents + metadata: + name: server + spec: + service: + name: server + calls: + # The Azure resource ID that references the resource(s) for the authorization. Subscription & resource group are automatically appended. + - name: "/providers/Microsoft.Storage/storageAccounts/otterizeazureiamtutorial/blobServices/default/containers/otterizeazureiamtutorialcontainer" + type: azure + # one or more Azure roles that will be provided to the specified resources + azureRoles: + - "Storage Blob Data Contributor" + - name: "/providers/Microsoft.KeyVault/vaults/otterizetutorialazureiamkeyvault" + type: azure + # Optional - Grant Azure Key Vault data plane access by using Key Vault access policy + azureKeyVaultPolicy: + certificatePermissions: + - "all" + keyPermissions: + - "all" + secretPermissions: + - "all" + storagePermissions: + - "get" + - "list" + ``` Try the [Azure IAM tutorial](https://docs.otterize.com/features/azure-iam/tutorials/azure-iam-aks) to learn more. ### Google Cloud IAM + The intents operator, together with the [credentials operator](https://github.com/otterize/credentials-operator), enables the intent-based declarative management of Google Cloud service accounts and policies. To enable Google Cloud access for a pod, follow these steps: + 1. Label a pod to have an GCP service account created for it: -``` -metadata: - labels: - "credentials-operator.otterize.com/create-gcp-sa": "true" -``` + + ```yaml + metadata: + labels: + "credentials-operator.otterize.com/create-gcp-sa": "true" + ``` 2. Declare ClientIntents to specify which GCP resources it needs access to: -```yaml -apiVersion: k8s.otterize.com/v1alpha3 -kind: ClientIntents -metadata: - name: server -spec: - service: - name: server - calls: - # The GCP resource name - # Wildcards can be used in the end of the resource name to match multiple and nested resources - - name: projects/_/buckets/otterize-demo-bucket* - type: gcp - # one or more GCP Roles that will be provided to the specified resources - gcpPermissions: - - "storage.admin" - # Multiple call definitions can be defined for a single service. - - name: projects/_/buckets/otterize-read-only-bucket* - type: gcp - gcpPermissions: - - "storage.objectViewer" -``` + + ```yaml + apiVersion: k8s.otterize.com/v1alpha3 + kind: ClientIntents + metadata: + name: server + spec: + service: + name: server + calls: + # The GCP resource name + # Wildcards can be used in the end of the resource name to match multiple and nested resources + - name: projects/_/buckets/otterize-demo-bucket* + type: gcp + # one or more GCP Roles that will be provided to the specified resources + gcpPermissions: + - "storage.admin" + # Multiple call definitions can be defined for a single service. + - name: projects/_/buckets/otterize-read-only-bucket* + type: gcp + gcpPermissions: + - "storage.objectViewer" + ``` Try the [GCP IAM tutorial](https://docs.otterize.com/features/gcp-iam/tutorials/gcp-iam-gke) to learn more. ### Otterize for PostgreSQL & MySQL + Otterize automates PostgreSQL & MySQL access management and secrets for your workloads, all in Kubernetes. Here is how: + 1. Annotate a pod, requesting a user and a password to be provisioned and bound to the pod, using the following annotation: -``` -credentials-operator.otterize.com/user-password-secret-name: booking-service-secret` -``` + + ```yaml + credentials-operator.otterize.com/user-password-secret-name: booking-service-secret` + ``` 2. Apply a PostgreSQLServerConfig or a MySQLServerConfig to the cluster, specifying the database name, user, and password. -```yaml -apiVersion: k8s.otterize.com/v1alpha3 -kind: PostgreSQLServerConfig -metadata: - name: bookings -spec: - address: db.bookings-database.svc.cluster.local:5432 - credentials: - username: admin - password: password -``` + ```yaml + apiVersion: k8s.otterize.com/v1alpha3 + kind: PostgreSQLServerConfig + metadata: + name: bookings + spec: + address: db.bookings-database.svc.cluster.local:5432 + credentials: + username: admin + password: password + ``` 3. Declare your workload’s ClientIntents, specifying desired permissions. -```yaml -apiVersion: k8s.otterize.com/v1alpha3 -kind: ClientIntents -metadata: - name: booking-service - namespace: flight-search -spec: - service: - name: booking-service - calls: - - name: bookings - type: database - databaseResources: - - databaseName: bookings-db - table: users - operations: - - SELECT - - databaseName: bookings-db - table: products - operations: - - ALL -``` + ```yaml + apiVersion: k8s.otterize.com/v1alpha3 + kind: ClientIntents + metadata: + name: booking-service + namespace: flight-search + spec: + service: + name: booking-service + calls: + - name: bookings + type: database + databaseResources: + - databaseName: bookings-db + table: users + operations: + - SELECT + - databaseName: bookings-db + table: products + operations: + - ALL + ``` Otterize then creates a user and matching grants on the target database. Try the [Just-in-time PostgreSQL users & access](https://docs.otterize.com/features/postgresql/tutorials/postgres) or [Just-in-time MySQL users & access](https://docs.otterize.com/features/mysql/tutorials/mysql) tutorials to learn more. ### Kafka mTLS & ACLs -The intents operator automatically creates, updates, and deletes ACLs in Kafka clusters running within your Kubernetes cluster. + +The intents operator automatically creates, updates, and deletes ACLs in Kafka clusters running within your Kubernetes cluster. It works with the [credentials operator](https://github.com/otterize/credentials-operator) to automatically: -- Establish pod service identities. -- Generate trusted credentials for each client service. -- Deliver the credentials to the pod's containers within a locally-mounted volume. + +* Establish pod service identities. +* Generate trusted credentials for each client service. +* Deliver the credentials to the pod's containers within a locally-mounted volume. With Kafka, you can also control access to individual topics, like so: + ```yaml apiVersion: k8s.otterize.com/v1alpha3 kind: ClientIntents @@ -259,6 +279,7 @@ spec: Read more about it in the [secure Kafka access tutorial](https://docs.otterize.com/quick-tutorials/k8s-kafka-mtls). ### Istio AuthorizationPolicy + The intents operator automatically creates, updates and deletes Istio authorization policies, automatically looks up service accounts for client pods and labels server pods, to reflect precisely the client-to-server calls declared in client intents files. The intents operator can also be configured to process client intents *without* creating and managing network policies, to provide visibility on what would happen once enforcement via Istio authorization policy is activated. More information can be found in the [shadow vs active enforcement documentation](https://docs.otterize.com/shadow-vs-active-enforcement). @@ -273,6 +294,7 @@ Finally, an Istio authorization policy is created that allows communication betw Read more about it in the [Istio AuthorizationPolicy tutorial](https://docs.otterize.com/quick-tutorials/k8s-istio-authorization-policies). ### Identities + Pods in the cluster are dynamically labeled with their owner's identity. If a `ReplicaSet` named `my-client` owns 5 pods and a `Deployment` named `my-server` owns 3 pods, and we enable `my-client` → `my-server` access via `ClientIntents`, all 5 source pods would be able to access all 3 target pods. @@ -280,17 +302,20 @@ source pods would be able to access all 3 target pods. Pod identities can be overridden by setting the value of the custom annotation `intents.otterize.com/service-name` to the desired service name. This is useful, for example, for pods without any owner. - ## Bootstrapping -To bootstrap client intents files for the services running in your cluster, you can use the [Otterize network + +To bootstrap client intents files for the services running in your cluster, you can use the [Otterize network mapper](https://github.com/otterize/network-mapper), which automatically detects pod-to-pod calls. ## Read more -The Otterize intents operator is a part of [Otterize OSS](https://otterize.com/open-source) + +The Otterize intents operator is a part of [Otterize OSS](https://otterize.com/open-source) and is an implementation of [intent-based access control](https://otterize.com/ibac). ## Development + Run the `make` command inside `src/operator` directory. Some useful commands are: + * `make build` to compile the go code. * `make deploy` to generate Kubernetes Deployment object which deploys the project to your local cluster. @@ -298,24 +323,28 @@ To create a local Docker image, execute `make docker-build-local`, and deploy it For utilizing the locally built Docker image on Minikube running on a VM like macOS arm64, use `make minikube-push`. ## Contributing + 1. Feel free to fork and open a pull request! Include tests and document your code in [Godoc style](https://go.dev/blog/godoc) 2. In your pull request, please refer to an existing issue or open a new one. -3. Changes to Kubernetes objects will make changes to the Helm chart in the [helm-charts repo](https://github.com/otterize/helm-charts), +3. Changes to Kubernetes objects will make changes to the Helm chart in the [helm-charts repo](https://github.com/otterize/helm-charts), which is a submodule in this repository, so you'll need to open a PR there as well. 4. See our [Contributor License Agreement](https://github.com/otterize/cla/). ## Slack + To join the conversation, ask questions, and engage with other users, join the Otterize Slack! [![button](https://i.ibb.co/vwRP6xK/Group-3090-2.png)](https://joinslack.otterize.com) ## Usage telemetry + The operator reports anonymous usage information back to the Otterize team, to help the team understand how the software is used in the community and what aspects users find useful. No personal or organizational identifying information is transmitted in these metrics: they only reflect patterns of usage. You may opt out at any time through a single configuration flag. To **disable** sending usage information: -- Via the Otterize OSS Helm chart: `--set global.telemetry.enabled=false`. -- Via an environment variable: `OTTERIZE_TELEMETRY_ENABLED=false`. -- If running an operator directly: `--telemetry-enabled=false`. + +* Via the Otterize OSS Helm chart: `--set global.telemetry.enabled=false`. +* Via an environment variable: `OTTERIZE_TELEMETRY_ENABLED=false`. +* If running an operator directly: `--telemetry-enabled=false`. If the `telemetry` flag is omitted or set to `true`, telemetry will be enabled: usage information will be reported. diff --git a/src/operator/README.md b/src/operator/README.md index 835eaade1..8c535d0bc 100644 --- a/src/operator/README.md +++ b/src/operator/README.md @@ -1,8 +1,12 @@ +# Otterize Intents Operator + ## Getting Started + You’ll need a Kubernetes cluster to run against. You can use [KIND](https://sigs.k8s.io/kind) to get a local cluster for testing, or run against a remote cluster. **Note:** Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster `kubectl cluster-info` shows). ### Running the operator in debug mode + 1. Apply the CRDS to your cluster: `kubectl apply -k config/crd/` 2. Set up your desired environment variables using src/shared/local.env file as template 3. Run main.go with the configured environment variables @@ -10,25 +14,27 @@ You’ll need a Kubernetes cluster to run against. You can use [KIND](https://si ** Note that the above setup does not support webhooks ### Running on the cluster + 1. Install Instances of Custom Resources: -```sh -kubectl apply -f config/samples/ -``` + ```sh + kubectl apply -f config/samples/ + ``` 2. Build and push your image to the location specified by `IMG`: - -```sh -make docker-build docker-push IMG=/operator:tag -``` - + + ```sh + make docker-build docker-push IMG=/operator:tag + ``` + 3. Deploy the controller to the cluster with the image specified by `IMG`: -```sh -make deploy IMG=/operator:tag -``` + ```sh + make deploy IMG=/operator:tag + ``` ### Uninstall CRDs + To delete the CRDs from the cluster: ```sh @@ -36,6 +42,7 @@ make uninstall ``` ### Undeploy controller + UnDeploy the controller to the cluster: ```sh @@ -43,27 +50,30 @@ make undeploy ``` ### How it works + This project aims to follow the Kubernetes [Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) -It uses [Controllers](https://kubernetes.io/docs/concepts/architecture/controller/) -which provides a reconcile function responsible for synchronizing resources untile the desired state is reached on the cluster +It uses [Controllers](https://kubernetes.io/docs/concepts/architecture/controller/) +which provides a reconcile function responsible for synchronizing resources untile the desired state is reached on the cluster ### Test It Out + 1. Install the CRDs into the cluster: -```sh -make install -``` + ```sh + make install + ``` 2. Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running): -```sh -make run -``` + ```sh + make run + ``` **NOTE:** You can also run this in one step by running: `make install run` ### Modifying the API definitions + If you are editing the API definitions, generate the manifests such as CRs or CRDs using: ```sh @@ -80,13 +90,10 @@ Copyright 2022. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 +You may obtain a copy of the License at Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. - diff --git a/src/operator/config/README.md b/src/operator/config/README.md index ab1a46ad0..1439c296b 100644 --- a/src/operator/config/README.md +++ b/src/operator/config/README.md @@ -1 +1,3 @@ -Files in this folder are auto generated. When deploying the intents operator, the manifests from the Helm chart, which is a submodule, are used. When testing, these manifests are used. \ No newline at end of file +# Operator deployment config files + +Files in this folder are auto generated. When deploying the intents operator, the manifests from the Helm chart, which is a submodule, are used. When testing, these manifests are used.