diff --git a/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/_index.md b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/_index.md new file mode 100644 index 0000000000..51de59988b --- /dev/null +++ b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/_index.md @@ -0,0 +1,7 @@ +--- +title: "Managing Control Plane" +linkTitle: "Managing Control Plane" +weight: 4 +description: > + This guide is for administrators and operators wanting to install and configure PipeCD for other developers. +--- diff --git a/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/adding-a-project.md b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/adding-a-project.md new file mode 100644 index 0000000000..e162c6adf5 --- /dev/null +++ b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/adding-a-project.md @@ -0,0 +1,24 @@ +--- +title: "Adding a project" +linkTitle: "Adding a project" +weight: 2 +description: > + This page describes how to set up a new project. +--- + +The control plane ops can add a new project for a team. +Project adding can be simply done from an internal web page prepared for the ops. +Because that web service is running in an `ops` pod, so in order to access it, using `kubectl port-forward` command to forward a local port to a port on the `ops` pod as following: + +``` console +kubectl port-forward service/pipecd-ops 9082 --namespace={NAMESPACE} +``` + +Then, access to [http://localhost:9082](http://localhost:9082). + +On that page, you will see the list of registered projects and a link to register new projects. +Registering a new project requires only a unique ID string and an optional description text. + +Once a new project has been registered, a static admin (username, password) will be automatically generated for the project admin. You can send that information to the project admin. The project admin first uses the provided static admin information to log in to PipeCD. After that, they can change the static admin information, configure the SSO, RBAC or disable static admin user. + +__Caution:__ The Role-Based Access Control (RBAC) setting is required to enable your team login using SSO, please make sure you have that setup before disable static admin user. \ No newline at end of file diff --git a/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/architecture-overview.md b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/architecture-overview.md new file mode 100644 index 0000000000..4166700b69 --- /dev/null +++ b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/architecture-overview.md @@ -0,0 +1,40 @@ +--- +title: "Architecture overview" +linkTitle: "Architecture overview" +weight: 1 +description: > + This page describes the architecture of control plane. +--- + +![](/images/control-plane-components.png) +

+Component Architecture +

+ +The control plane is a centralized part of PipeCD. It contains several services as below to manage the application, deployment data and handle all requests from `piped`s and web clients: + +##### Server + +`server` handles all incoming gRPC requests from `piped`s, web clients, incoming HTTP requests such as auth callback from third party services. +It also serves all web assets including HTML, JS, CSS... +This service can be easily scaled by updating the pod number. + +##### Cache + +`cache` is a single pod service for caching internal data used by `server` service. Currently, this `cache` service is powered by `redis`. +You can configure the control plane to use a fully-managed redis cache service instead of launching a cache pod in your cluster. + +##### Ops + +`ops` is a single pod service for operating PipeCD owner's tasks. +For example, it provides an internal web page for adding and managing projects; it periodically removes the old data; it collects and saves the deployment insights. + +##### Data Store + +`Data store` is a storage for storing model data such as applications and deployments. This can be a fully-managed service such as GCP [Firestore](https://cloud.google.com/firestore), GCP [Cloud SQL](https://cloud.google.com/sql) or AWS [RDS](https://aws.amazon.com/rds/) (currently we choose [MySQL v8](https://www.mysql.com/) as supported relational data store). You can also configure the control plane to use a self-managed MySQL server. +When installing the control plane, you have to choose one of the provided data store services. + +##### File Store + +`File store` is a storage for storing stage logs, application live states. This can be a fully-managed service such as GCP [GCS](https://cloud.google.com/storage), AWS [S3](https://aws.amazon.com/s3/), or a self-managed service such as [Minio](https://github.com/minio/minio). +When installing the control plane, you have to choose one of the provided file store services. diff --git a/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/auth.md b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/auth.md new file mode 100644 index 0000000000..d3d5a8a93b --- /dev/null +++ b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/auth.md @@ -0,0 +1,261 @@ +--- +title: "Authentication and authorization" +linkTitle: "Authentication and authorization" +weight: 3 +description: > + This page describes about PipeCD Authentication and Authorization. +--- + +![Project Settings](/images/settings-project-v0.38.x.png) + +### Static Admin + +When the PipeCD owner [adds a new project](../adding-a-project/), an admin account will be automatically generated for the project. After that, PipeCD owner sends that static admin information including username, password strings to the project admin, who can use that information to log in to PipeCD web with the admin role. + +After logging, the project admin should change the provided username and password. Or disable the static admin account after configuring the single sign-on for the project. + +### Single Sign-On (SSO) + +Single sign-on (SSO) allows users to log in to PipeCD by relying on a trusted third-party service. + +The project can be configured to use a shared SSO configuration (shared OAuth application) instead of needing a new one. In that case, while creating the project, the PipeCD owner specifies the name of the shared SSO configuration should be used, and then the project admin can skip configuring SSO at the settings page. + +**Supported service** + +- GitHub +- Generic OIDC + +> Note: In the future, we want to support such as Google Gmail, Bitbucket... + +#### Github + +Before configuring the SSO, you need an OAuth application of the using service. For example, GitHub SSO requires creating a GitHub OAuth application as described in this page: + +https://docs.github.com/en/developers/apps/creating-an-oauth-app + +The authorization callback URL should be `https://YOUR_PIPECD_ADDRESS/auth/callback`. + +![](/images/settings-update-sso.png) + +#### Generic OIDC + +PipeCD supports any OIDC provider, with tested providers including Keycloak, Auth0, and AWS Cognito. The only supported authentication flow currently is the Authorization Code Grant. + +Requirements and Troubleshooting: + +- The OIDC provider must provide claims for user's roles and username. +- Roles claim value must use same values as pre-configured project RBAC Roles. +- - Claims can be retreived from the IdToken or UserInfo endpoint. The UserInfo endpoint will be used if issuer supports it. +- You can use set a custom claim key name for roles and username in the OIDC provider. Using `usernameClaimKey` and `rolesClaimKey` in the configuration. If not set, the default value will be chosen in the following order: + + - Supported Claims Key for Username (in order of priority): `username`, `preferred_username`,`name`, `cognito:username` + - Supported Claims Key for Role (in order of priority): `groups`, `roles`, `cognito:groups`, `custom:roles`, `custom:groups` + +- If no usable claims are found, `Unable to find user` error will be shown. +- If no roles are found, user can not access any resources. (If `allowStrayAsViewer` is set to `true`, user can access as a viewer) + +Provider Configuration Examples: + +##### Keycloak + +- **Client authentication**: On +- **Valid redirect URIs**: `https://YOUR_PIPECD_ADDRESS/auth/callback` +- **Client scopes**: Add a new mapper to the `-dedicated` scope. For instance, map Group Membership information to the groups claim (Full group path should be off). + +- **Control Plane configuration**: + + ```yaml + apiVersion: "pipecd.dev/v1beta1" + kind: ControlPlane + spec: + sharedSSOConfigs: + - name: oidc + provider: OIDC + oidc: + clientId: + clientSecret: + issuer: https:///realms/ + redirectUri: https:///auth/callback + scopes: + - openid + - profile + ``` + +##### Okta + +- **Allowed Callback URLs**: `https://YOUR_PIPECD_ADDRESS/auth/callback` +- **Control Plane configuration**: + + ```yaml + apiVersion: "pipecd.dev/v1beta1" + kind: ControlPlane + spec: + sharedSSOConfigs: + - name: oidc + provider: OIDC + oidc: + clientId: + clientSecret: + issuer: https://.okta.com + redirectUri: https:///auth/callback + scopes: + - openid + - profile + - groups + ``` + +##### Auth0 + +- **Allowed Callback URLs**: `https://YOUR_PIPECD_ADDRESS/auth/callback` +- **Control Plane configuration**: + + ```yaml + apiVersion: "pipecd.dev/v1beta1" + kind: ControlPlane + spec: + sharedSSOConfigs: + - name: oidc + provider: OIDC + oidc: + clientId: + clientSecret: + issuer: https:// + redirectUri: https:///auth/callback + scopes: + - openid + - profile + ``` + +- **Roles/Groups Claims** + For Role or Groups information mapping using Auth0 Actions, here is an example for setting `custom:roles`: + + ```javascript + exports.onExecutePostLogin = async (event, api) => { + let namespace = "custom"; + if (namespace && !namespace.endsWith("/")) { + namespace += ":"; + } + api.idToken.setCustomClaim(namespace + "roles", event.authorization.roles); + }; + ``` + +##### AWS Cognito + +- **Allowed Callback URLs**: `https://YOUR_PIPECD_ADDRESS/auth/callback` + +- **Control Plane configuration**: + + ```yaml + apiVersion: "pipecd.dev/v1beta1" + kind: ControlPlane + spec: + sharedSSOConfigs: + - name: oidc + provider: OIDC + oidc: + clientId: + clientSecret: + issuer: https://cognito-idp..amazonaws.com/ + redirectUri: https:///auth/callback + scopes: + - openid + - profile + ``` + +##### Custom Claims Key + +In some cases, the OIDC provider may not provide the claims with the default key names like `groups`. You can set the custom claim key name for roles and username in the control plane configuration to map the claims from the OIDC provider. **To be cautious, OIDC providers can not be used if issuer discovery is not supported.** + +```yaml +apiVersion: "pipecd.dev/v1beta1" +kind: ControlPlane +spec: + sharedSSOConfigs: + - name: oidc + provider: OIDC + oidc: + clientId: + clientSecret: + issuer: https:// + redirectUri: https:///auth/callback + scopes: + - openid + - profile + usernameClaimKey: username # change to your custom claim key + rolesClaimKey: roles # change to your custom claim key + avatarUrlClaimKey: picture # change to your custom claim key +``` + +(Optional) You can choose to use the avatar URL from the OIDC provider. Using `avatarUrlClaimKey` in the configuration. If not set, the default value will be chosen in the following order: `picture`, `avatar_url` + +##### Custom OIDC Configuration + +If you want to set your custom endpoint without using the endpoint from the issuer, you can set the `authorization_endpoint`, `token_endpoint`, `userinfo_endpoint` in the control plane configuration. + +```yaml +apiVersion: "pipecd.dev/v1beta1" +kind: ControlPlane +spec: + sharedSSOConfigs: + - name: oidc + provider: OIDC + oidc: + clientId: + clientSecret: + issuer: https:// + redirectUri: https:///auth/callback + scopes: + - openid + - profile + authorization_endpoint: https:///authorize # change to your custom endpoint + token_endpoint: https:///token # change to your custom endpoint + userinfo_endpoint: https:///userinfo # change to your custom endpoint +``` + +### Role-Based Access Control (RBAC) + +Role-based access control (RBAC) allows restricting access on the PipeCD web-based on the roles of user groups within the project. Before using this feature, the SSO must be configured. + +PipeCD provides three built-in roles: + +- `Viewer`: has only permissions to view existing resources or data. +- `Editor`: has all viewer permissions, plus permissions for actions that modify state, such as manually syncing application, canceling deployment... +- `Admin`: has all editor permissions, plus permissions for updating project configurations. + +#### Configuring the PipeCD's roles + +The below table represents PipeCD's resources with actions on those resources. + +| resource | get | list | create | update | delete | +| :---------- | :-: | :--: | :----: | :----: | :----: | +| application | ○ | ○ | ○ | ○ | ○ | +| deployment | ○ | ○ | | ○ | | +| event | | ○ | | | | +| piped | ○ | ○ | ○ | ○ | | +| project | ○ | | | ○ | | +| apiKey | | ○ | ○ | ○ | | +| insight | ○ | | | | | + +Each role is defined as a combination of multiple policies under this format. + +``` +resources=RESOURCE_NAMES;actions=ACTION_NAMES +``` + +The `*` represents all resources and all actions for a resource. + +``` +resources=*;actions=ACTION_NAMES +resources=RESOURCE_NAMES;actions=* +resources=*;actions=* +``` + +#### Configuring the PipeCD's user groups + +User Group represents a relation with a specific team (GitHub)/group (Google) and an arbitrary role. All users belong to a team/group will have all permissions of that team/group. + +In case of using the GitHub team as a PipeCD user group, the PipeCD user group must be set in lowercase. For example, if your GitHub team is named `ORG/ABC-TEAM`, the PipeCD user group would be set as `ORG/abc-team`. (It's follow the GitHub team URL as github.com/orgs/{organization-name}/teams/{TEAM-NAME}) + +Note: You CANNOT assign multiple roles to a team/group, should create a new role with suitable permissions instead. + +![](/images/settings-add-user-group.png) diff --git a/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/configuration-reference.md b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/configuration-reference.md new file mode 100644 index 0000000000..87c1c76a28 --- /dev/null +++ b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/configuration-reference.md @@ -0,0 +1,179 @@ +--- +title: "Configuration reference" +linkTitle: "Configuration reference" +weight: 6 +description: > + This page describes all configurable fields in the Control Plane configuration. +--- + +``` yaml +apiVersion: pipecd.dev/v1beta1 +kind: ControlPlane +spec: + address: https://your-pipecd-address + ... +``` + +## Control Plane Configuration + +| Field | Type | Description | Required | +|-|-|-|-| +| stateKey | string | A randomly generated string used to sign oauth state. | Yes | +| datastore | [DataStore](#datastore) | Storage for storing application, deployment data. | Yes | +| filestore | [FileStore](#filestore) | File storage for storing deployment logs and application states. | Yes | +| cache | [Cache](#cache) | Internal cache configuration. | No | +| address | string | The address to the control plane. This is required if SSO is enabled. | No | +| insightCollector | [InsightCollector](#insightcollector) | Option to run collector of Insights feature. | No | +| sharedSSOConfigs | [][SharedSSOConfig](#sharedssoconfig) | List of shared SSO configurations that can be used by any projects. | No | +| projects | [][Project](#project) | List of debugging/quickstart projects. Please note that do not use this to configure the projects running in the production. | No | + +## DataStore + +| Field | Type | Description | Required | +|-|-|-|-| +| type | string | Which type of data store should be used. Can be one of the following values
`FIRESTORE`, `MYSQL`. | Yes | +| config | [DataStoreConfig](#datastoreconfig) | Specific configuration for the datastore type. This must be one of these DataStoreConfig. | Yes | + +## DataStoreConfig + +Must be one of the following objects: + +### DataStoreFireStoreConfig + +| Field | Type | Description | Required | +|-|-|-|-| +| namespace | string | The root path element considered as a logical namespace, e.g. `pipecd`. | Yes | +| environment | string | The second path element considered as a logical environment, e.g. `dev`. All pipecd collections will have path formatted according to `{namespace}/{environment}/{collection-name}`. | Yes | +| collectionNamePrefix | string | The prefix for collection name. This can be used to avoid conflicts with existing collections in your Firestore database. | No | +| project | string | The name of GCP project hosting the Firestore. | Yes | +| credentialsFile | string | The path to the service account file for accessing Firestores. | No | + + +### DataStoreMySQLConfig + +| Field | Type | Description | Required | +|-|-|-|-| +| url | string | The address to MySQL server. Should attach with the database port info as `127.0.0.1:3307` in case you want to use another port than the default value. | Yes | +| database | string | The name of database. | No (If you set it via URL) | +| usernameFile | string | Path to the file containing the username. | No | +| passwordFile | string | Path to the file containing the password. | No | + + +## FileStore + +| Field | Type | Description | Required | +|-|-|-|-| +| type | string | Which type of file store should be used. Can be one of the following values
`GCS`, `S3`, `MINIO` | Yes | +| config | [FileStoreConfig](#filestoreconfig) | Specific configuration for the filestore type. This must be one of these FileStoreConfig. | Yes | + +## FileStoreConfig + +Must be one of the following objects: + +### FileStoreGCSConfig + +| Field | Type | Description | Required | +|-|-|-|-| +| bucket | string | The bucket name. | Yes | +| credentialsFile | string | The path to the service account file for accessing GCS. | No | + +### FileStoreS3Config + +| Field | Type | Description | Required | +|-|-|-|-| +| bucket | string | The AWS S3 bucket name. | Yes | +| region | string | The AWS region name. | Yes | +| profile | string | The AWS profile name. Default value is `default`. | No | +| credentialsFile | string | The path to AWS credential file. Requires only if you want to auth by specified credential file, by default PipeCD will use `$HOME/.aws/credentials` file. | No | +| roleARN | string | The IAM role arn to use when assuming an role. Requires only if you want to auth by `WebIdentity` pattern. | No | +| tokenFile | string | The path to the WebIdentity token PipeCD should use to assume a role with. Requires only if you want to auth by `WebIdentity` pattern. | No | + +### FileStoreMinioConfig + +| Field | Type | Description | Required | +|-|-|-|-| +| endpoint | string | The address of Minio. | Yes | +| bucket | string | The bucket name. | Yes | +| accessKeyFile | string | The path to the access key file. | No | +| secretKeyFile | string | The path to the secret key file. | No | +| autoCreateBucket | bool | Whether the given bucket should be made automatically if not exists. | No | + +## Cache + +| Field | Type | Description | Required | +|-|-|-|-| +| ttl | duration | The time that in-memory cache items are stored before they are considered as stale. | Yes | + +## Project + +| Field | Type | Description | Required | +|-|-|-|-| +| id | string | The unique identifier of the project. | Yes | +| desc | string | The description about the project. | No | +| staticAdmin | [ProjectStaticUser](#projectstaticuser) | Static admin account of the project. | Yes | + +## ProjectStaticUser + +| Field | Type | Description | Required | +|-|-|-|-| +| username | string | The username string. | Yes | +| passwordHash | string | The bcrypt hashed value of the password string. | Yes | + +## InsightCollector + +| Field | Type | Description | Required | +|-|-|-|-| +| application | [InsightCollectorApplication](#insightcollectorapplication) | Application metrics collector. | No | +| deployment | [InsightCollectorDeployment](#insightcollectordeployment) | Deployment metrics collector. | No | + +## InsightCollectorApplication + +| Field | Type | Description | Required | +|-|-|-|-| +| enabled | bool | Whether to enable. Default is `true` | No | +| schedule | string | When collector will be executed. Default is `0 * * * *` | No | + +## InsightCollectorDeployment + +| Field | Type | Description | Required | +|-|-|-|-| +| enabled | bool | Whether to enable. Default is `true` | No | +| schedule | string | When collector will be executed. Default is `30 * * * *` | No | +| chunkMaxCount | int | The maximum number of deployment items could be stored in a chunk. Default is `1000` | No | + +## SharedSSOConfig + +| Field | Type | Description | Required | +|-|-|-|-| +| name | string | The unique name of the configuration. | Yes | +| provider | string | The SSO service provider. Currently, only `GITHUB` and `OIDC` is supported. | Yes | +| sessionTtl | int | The time to live of session for SSO login. Unit is `hour`. Default is 7 * 24 hours. | No | +| github | [SSOConfigGitHub](#ssoconfiggithub) | GitHub sso configuration. | No | +| oidc | [SSOConfigOIDC](#ssoconfigoidc) | OIDC sso configuration. | No | + +## SSOConfigGitHub + +| Field | Type | Description | Required | +|-|-|-|-| +| clientId | string | The client id string of GitHub oauth app. | Yes | +| clientSecret | string | The client secret string of GitHub oauth app. | Yes | +| baseUrl | string | The address of GitHub service. Required if enterprise. | No | +| uploadUrl | string | The upload url of GitHub service. | No | +| proxyUrl | string | The address of the proxy used while communicating with the GitHub service. | No | + +## SSOConfigOIDC + +| Field | Type | Description | Required | +|-|-|-|-| +| clientId | string | The client id string of OpenID Connect oauth app. | Yes | +| clientSecret | string | The client secret string of OpenID Connect oauth app. | Yes | +| issuer | string | The address of OpenID Connect service. | Yes | +| redirectUri | string | The address of the redirect URI. | Yes | +| authorizationEndpoint | string | The address of the authorization endpoint. Only set if you want to use custom authorization endpoint (still need issuer discovery). | No | +| tokenEndpoint | string | The address of the token endpoint. Only set if you want to use custom token endpoint (still need issuer discovery). | No | +| userInfoEndpoint | string | The address of the user info endpoint. Only set if you want to use custom user info endpoint (still need issuer discovery). | No | +| proxyUrl | string | The address of the proxy used while communicating with the OpenID Connect service. | No | +| scopes | []string | Scopes to request from the OpenID Connect service. Default is `openid`. Some providers may require other scopes. | No | +| usernameClaimKey | string | The key name of the claim that contains the username. If not set, the default value will be chosen in the following order: `username`, `preferred_username`, `name`, `cognito:username`. | No | +| rolesClaimKey | string | The key name of the claim that contains the roles. If not set, the default value will be chosen in the following order: `groups`, `roles`, `custom:roles`, `custom:groups`. | No | +| avatarUrlClaimKey | string | The key name of the claim that contains the avatar url. If not set, the default value will be chosen in the following order: `picture`, `avatar_url`. | No | diff --git a/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/registering-a-piped.md b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/registering-a-piped.md new file mode 100644 index 0000000000..9719f26f8d --- /dev/null +++ b/docs/content/en/docs-v1.0.x/user-guide/managing-controlplane/registering-a-piped.md @@ -0,0 +1,16 @@ +--- +title: "Registering a piped" +linkTitle: "Registering a piped" +weight: 4 +description: > + This page describes how to register a new piped to a project. +--- + +The list of pipeds are shown in the Settings page. Anyone who has the project admin role can register a new piped by clicking on the `+ADD` button. + +
+ +![](/images/settings-add-piped.png) +

+Registering a new piped +