diff --git a/daprdocs/content/en/developing-applications/building-blocks/jobs/jobs-overview.md b/daprdocs/content/en/developing-applications/building-blocks/jobs/jobs-overview.md index 24ebf808b2b..486cfc5d623 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/jobs/jobs-overview.md +++ b/daprdocs/content/en/developing-applications/building-blocks/jobs/jobs-overview.md @@ -16,11 +16,6 @@ Jobs in Dapr consist of: [See example scenarios.]({{< ref "#scenarios" >}}) -{{% alert title="Warning" color="warning" %}} -By default, job data is not resilient to [Scheduler]({{< ref scheduler.md >}}) service restarts. -A persistent volume must be provided to Scheduler to ensure job data is not lost in either [Kubernetes]({{< ref kubernetes-persisting-scheduler.md >}}) or [Self-hosted]({{< ref self-hosted-persisting-scheduler.md >}}) mode. -{{% /alert %}} - Diagram showing the Scheduler control plane service and the jobs API ## How it works diff --git a/daprdocs/content/en/operations/hosting/kubernetes/kubernetes-persisting-scheduler.md b/daprdocs/content/en/operations/hosting/kubernetes/kubernetes-persisting-scheduler.md index 130122192d6..9172a28feb9 100644 --- a/daprdocs/content/en/operations/hosting/kubernetes/kubernetes-persisting-scheduler.md +++ b/daprdocs/content/en/operations/hosting/kubernetes/kubernetes-persisting-scheduler.md @@ -6,12 +6,15 @@ weight: 50000 description: "Configure Scheduler to persist its database to make it resilient to restarts" --- -The [Scheduler]({{< ref scheduler.md >}}) service is responsible for writing jobs to its embedded database and scheduling them for execution. -By default, the Scheduler service database writes this data to an in-memory ephemeral tempfs volume, meaning that **this data is not persisted across restarts**. Job data will be lost during these events. +The [Scheduler]({{< ref scheduler.md >}}) service is responsible for writing jobs to its embedded Etcd database and scheduling them for execution. +By default, the Scheduler service database writes this data to a Persistent Volume Claim of 1Gb of size using the cluster's default [storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/). This means that there is no additional parameter required to run the scheduler service reliably on most Kubernetes deployments, although you will need additional configuration in some deployments or for a production environment. -To make the Scheduler data resilient to restarts, a persistent volume must be mounted to the Scheduler `StatefulSet`. -This persistent volume is backed by a real disk that is provided by the hosted Cloud Provider or Kubernetes infrastructure platform. -Disk size is determined by how many jobs are expected to be persisted at once; however, 64Gb should be more than sufficient for most use cases. +## Production Setup + +In case your Kubernetes deployment does not have a default storage class or you are configuring a production cluster, defining a storage class is required. + +A persistent volume is backed by a real disk that is provided by the hosted Cloud Provider or Kubernetes infrastructure platform. +Disk size is determined by how many jobs are expected to be persisted at once; however, 64Gb should be more than sufficient for most production scenarios. Some Kubernetes providers recommend using a [CSI driver](https://kubernetes.io/docs/concepts/storage/volumes/#csi) to provision the underlying disks. Below are a list of useful links to the relevant documentation for creating a persistent disk for the major cloud providers: - [Google Cloud Persistent Disk](https://cloud.google.com/compute/docs/disks) @@ -23,7 +26,7 @@ Below are a list of useful links to the relevant documentation for creating a pe - [Alibaba Cloud Disk Storage](https://www.alibabacloud.com/help/ack/ack-managed-and-ack-dedicated/user-guide/create-a-pvc) -Once the persistent volume class is available, you can install Dapr using the following command, with Scheduler configured to use the persistent volume class (replace `my-storage-class` with the name of the storage class): +Once the storage class is available, you can install Dapr using the following command, with Scheduler configured to use the storage class (replace `my-storage-class` with the name of the storage class): {{% alert title="Note" color="primary" %}} If Dapr is already installed, the control plane needs to be completely [uninstalled]({{< ref dapr-uninstall.md >}}) in order for the Scheduler `StatefulSet` to be recreated with the new persistent volume. @@ -53,3 +56,37 @@ helm upgrade --install dapr dapr/dapr \ {{% /codetab %}} {{< /tabs >}} + +## Ephemeral Storage + +Scheduler can be optionally made to use Ephemeral storage, which is in-memory storage which is **not** resilient to restarts, i.e. all Job data will be lost after a Scheduler restart. +This is useful for deployments where storage is not available or required, or for testing purposes. + +{{% alert title="Note" color="primary" %}} +If Dapr is already installed, the control plane needs to be completely [uninstalled]({{< ref dapr-uninstall.md >}}) in order for the Scheduler `StatefulSet` to be recreated without the persistent volume. +{{% /alert %}} + +{{< tabs "Dapr CLI" "Helm" >}} + +{{% codetab %}} + +```bash +dapr init -k --set dapr_scheduler.cluster.inMemoryStorage=true +``` + +{{% /codetab %}} + + +{{% codetab %}} + +```bash +helm upgrade --install dapr dapr/dapr \ +--version={{% dapr-latest-version short="true" %}} \ +--namespace dapr-system \ +--create-namespace \ +--set dapr_scheduler.cluster.inMemoryStorage=true \ +--wait +``` + +{{% /codetab %}} +{{< /tabs >}} diff --git a/daprdocs/content/en/reference/api/jobs_api.md b/daprdocs/content/en/reference/api/jobs_api.md index 37332b848c4..3a04ed1a9d4 100644 --- a/daprdocs/content/en/reference/api/jobs_api.md +++ b/daprdocs/content/en/reference/api/jobs_api.md @@ -10,11 +10,6 @@ weight: 1300 The jobs API is currently in alpha. {{% /alert %}} -{{% alert title="Warning" color="warning" %}} -By default, job data is not resilient to [Scheduler]({{< ref scheduler.md >}}) service restarts. -A persistent volume must be provided to Scheduler to ensure job data is not lost in either [Kubernetes]({{< ref kubernetes-persisting-scheduler.md >}}) or [Self-Hosted]({{< ref self-hosted-persisting-scheduler.md >}}) mode. -{{% /alert %}} - With the jobs API, you can schedule jobs and tasks in the future. > The HTTP APIs are intended for development and testing only. For production scenarios, the use of the SDKs is strongly