From 83f980bcfdb086d3af124e14dca26e55fa36ab1b Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Fri, 11 Oct 2024 11:26:47 -0400 Subject: [PATCH 01/26] draft --- modules/manage/partials/tiered-storage.adoc | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index 80697192b..e8c0191eb 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -688,7 +688,10 @@ Amazon S3:: TIP: If deploying Redpanda on an AWS Auto-Scaling group (ASG), keep in mind that the ASG controller terminates nodes and spins up replacements if the nodes saturate and are unable to heartbeat the controller (based on the EC2 health check). For more information, see the https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-system-instance-status-check.html#types-of-instance-status-checks[AWS documentation^]. Redpanda recommends deploying on Linux or Kubernetes. For more information, see xref:deploy:deployment-option/self-hosted/index.adoc[Deploy Redpanda]. -Configure access to Amazon S3 with either an IAM role attached to the instance or with access keys. +Configure access to Amazon S3 with either an IAM role attached to the instance, or with access keys. + +TIP: Optionally, if you need to manage and store encryption keys separately from your cloud provider, you can +. <>. To configure access to an S3 bucket with an IAM role: @@ -726,6 +729,10 @@ Replace `` with your own values. + CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. To reset a property to its default value, run `rpk cluster config force-reset ` or remove that line from the cluster configuration with `rpk cluster config edit`. +===== Configure access to AWS S3 using an AWS KMS key + +abasda + -- Google Cloud Storage:: + From 9ef5d197d71a5527e17c591798c681417e096a60 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Fri, 11 Oct 2024 17:32:06 -0400 Subject: [PATCH 02/26] first draft --- modules/manage/partials/tiered-storage.adoc | 37 ++++++++++++++++++--- 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index e8c0191eb..c3d35b013 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -688,10 +688,10 @@ Amazon S3:: TIP: If deploying Redpanda on an AWS Auto-Scaling group (ASG), keep in mind that the ASG controller terminates nodes and spins up replacements if the nodes saturate and are unable to heartbeat the controller (based on the EC2 health check). For more information, see the https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-system-instance-status-check.html#types-of-instance-status-checks[AWS documentation^]. Redpanda recommends deploying on Linux or Kubernetes. For more information, see xref:deploy:deployment-option/self-hosted/index.adoc[Deploy Redpanda]. -Configure access to Amazon S3 with either an IAM role attached to the instance, or with access keys. +Configure access to Amazon S3 with either an IAM role attached to the instance, or with access keys. TIP: Optionally, if you need to manage and store encryption keys separately from your cloud provider, you can -. <>. + <> instead of the default AWS S3-managed key (SSE-S3). To configure access to an S3 bucket with an IAM role: @@ -729,9 +729,38 @@ Replace `` with your own values. + CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. To reset a property to its default value, run `rpk cluster config force-reset ` or remove that line from the cluster configuration with `rpk cluster config edit`. -===== Configure access to AWS S3 using an AWS KMS key +[discrete] +==== Configure access to AWS S3 using an AWS KMS key -abasda +When there are strict data compliance requirements and you must manage and store encryption keys separately +from your cloud provider, you can configure an Amazon S3 bucket that Tiered Storage uses to leverage +your customer-provided key (SSE-KMS) instead of the default AWS-managed key (SSE-S3). + +To convert an existing S3 bucket and its contents, you must: + +. Create a new KMS key +. Configure the S3 bucket to use the new KMS key +. (Optional) Re-encrypt existing objects to use the new KMS key + +[discrete] +==== Prerequisites + +- The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. +- The S3 bucket must be assigned Key user permission. Without this permission, Redpanda will be unable to write new objects to Tiered Storage. +- If you intend to retroactively re-encrypt existing data with the new KMS key, record the ARN identifier of the key upon creation, as it will be required later when running an AWS CLI command. + +To create a new KMS key in the AWS Console, see the AWS documentation for https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html[Creating a symmetric encryption KMS key^]. + +. After selecting the Encrypt and decrypt option, select Advanced options. If you plan to use read +replicas or have Redpanda across regions, select the Multi-Region option. See https://docs.aws.amazon.com/kms/latest/developerguide/multi-region-keys-overview.html[Multi-Region keys in AWS KMS^] for more details. +. Be sure to grant a user account permission to manage this key using the AWS CLI. +. Assign the S3 bucket as a Key user. This is required for the S3 bucket to encrypt and decrypt. + +To configure the S3 bucket to use the new KMS key, see https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-bucket-key.html[Enable an S3 Bucket Key for an existing bucket^]. + +. Be sure to select "Server-side encryption with AWS Key Management Service keys (SSE-KMS)" as the encryption type when you edit the Properties for default encryption. +. Select your AWS KMS Key either from the dropdown or using the key ARN identifier. +. Be sure to enable the S3 bucket key, as it reduces KMS costs through caching. -- Google Cloud Storage:: From 7af31e9b8fb0276e726b7dfc88c83284804f2cba Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Tue, 29 Oct 2024 15:25:41 -0400 Subject: [PATCH 03/26] draft in progress --- modules/manage/partials/tiered-storage.adoc | 30 ++++++++++++++------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index c3d35b013..ec0c28d64 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -732,9 +732,7 @@ CAUTION: Do not set an object storage property to an empty string `""` or to `nu [discrete] ==== Configure access to AWS S3 using an AWS KMS key -When there are strict data compliance requirements and you must manage and store encryption keys separately -from your cloud provider, you can configure an Amazon S3 bucket that Tiered Storage uses to leverage -your customer-provided key (SSE-KMS) instead of the default AWS-managed key (SSE-S3). +When there are strict data compliance requirements and you must manage and store encryption keys separately from your cloud provider, you can configure an Amazon S3 bucket that Tiered Storage can use to leverage your customer-provided key (SSE-KMS) instead of the default AWS-managed key (SSE-S3). To convert an existing S3 bucket and its contents, you must: @@ -749,14 +747,28 @@ To convert an existing S3 bucket and its contents, you must: - The S3 bucket must be assigned Key user permission. Without this permission, Redpanda will be unable to write new objects to Tiered Storage. - If you intend to retroactively re-encrypt existing data with the new KMS key, record the ARN identifier of the key upon creation, as it will be required later when running an AWS CLI command. -To create a new KMS key in the AWS Console, see the AWS documentation for https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html[Creating a symmetric encryption KMS key^]. +To create a new KMS key in the AWS Console: -. After selecting the Encrypt and decrypt option, select Advanced options. If you plan to use read -replicas or have Redpanda across regions, select the Multi-Region option. See https://docs.aws.amazon.com/kms/latest/developerguide/multi-region-keys-overview.html[Multi-Region keys in AWS KMS^] for more details. -. Be sure to grant a user account permission to manage this key using the AWS CLI. -. Assign the S3 bucket as a Key user. This is required for the S3 bucket to encrypt and decrypt. +. In the AWS Console, search for “Key Management Service”. +. Click **Create a key**. +. On the Configure key page, select the Symmetric key type, then select Encrypt and decrypt. +. Click the **Advanced options** tab and configure Key material origin and Regionality as needed. For example, if you are using xref:manage/remote-read-replicas.adoc[Remote Read Replicas] or have Redpanda spanning across regions, select Multi-Region key. +. Click **Next**. +. On the Add labels page, specify an alias name and description for the key. Do not include sensitive information in these fields. +. Click **Next**. +. On the Define key administrative permissions page, specify a user who can administer this key through the KMS API. +. Click **Next**. +. On the Define key usage permissions page, assign the S3 bucket as a Key user. This is required for the S3 bucket to encrypt and decrypt. +. Click **Next**. +. Review your KMS key configuration and click **Finish**. -To configure the S3 bucket to use the new KMS key, see https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-bucket-key.html[Enable an S3 Bucket Key for an existing bucket^]. +For more information, see the AWS documentation https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html[Creating a symmetric encryption KMS key^]. + +To configure the S3 bucket to use the new KMS key: + +. In the AWS Console, search for "S3". + +For more information, see the AWS documentation https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-bucket-key.html[Enable an S3 Bucket Key for an existing bucket^]. . Be sure to select "Server-side encryption with AWS Key Management Service keys (SSE-KMS)" as the encryption type when you edit the Properties for default encryption. . Select your AWS KMS Key either from the dropdown or using the key ARN identifier. From 2aac9a35451ba16352b706d4f7a9ae2037399220 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Wed, 30 Oct 2024 14:38:20 -0400 Subject: [PATCH 04/26] WIP --- modules/manage/partials/tiered-storage.adoc | 103 ++++++++++++++++++-- 1 file changed, 94 insertions(+), 9 deletions(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index ec0c28d64..81004aacb 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -749,10 +749,10 @@ To convert an existing S3 bucket and its contents, you must: To create a new KMS key in the AWS Console: -. In the AWS Console, search for “Key Management Service”. +. In AWS Console, search for “Key Management Service”. . Click **Create a key**. . On the Configure key page, select the Symmetric key type, then select Encrypt and decrypt. -. Click the **Advanced options** tab and configure Key material origin and Regionality as needed. For example, if you are using xref:manage/remote-read-replicas.adoc[Remote Read Replicas] or have Redpanda spanning across regions, select Multi-Region key. +. Click the **Advanced options** tab and configure Key material origin and Regionality as needed. For example, if you are using xref:manage:remote-read-replicas.adoc[Remote Read Replicas] or have Redpanda spanning across regions, select Multi-Region key. . Click **Next**. . On the Add labels page, specify an alias name and description for the key. Do not include sensitive information in these fields. . Click **Next**. @@ -764,21 +764,32 @@ To create a new KMS key in the AWS Console: For more information, see the AWS documentation https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html[Creating a symmetric encryption KMS key^]. -To configure the S3 bucket to use the new KMS key: +To configure the S3 bucket to use the new KMS key (and reduce KMS costs through caching): -. In the AWS Console, search for "S3". +. In AWS Console, search for "S3". +. Select the bucket used by Redpanda. +. Click the **Properties** tab. +. In Default encryption, click **Edit**. +. For Encryption type, select “Server-side encryption with AWS Key Management Service keys (SSE-KMS)”. +. Locate and select your AWS KMS key ARN identifier. +. Click **Save changes**. -For more information, see the AWS documentation https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-bucket-key.html[Enable an S3 Bucket Key for an existing bucket^]. +(Optional) To re-encrypt existing data using the new KMS key: + +Existing data in your S3 bucket continues to be read using the AWS-managed key, while moving forward, new objects are encrypted using the new KMS key. If you wish to re-encrypt all S3 bucket data to use the KMS key, run the following AWS CLI command: + +[,bash] +---- +aws s3 cp s3://{BUCKET_NAME}/ s3://{BUCKET_NAME}/ --recursive --sse-kms-key-id {KMS_KEY_ARN} --sse aws:kms +---- -. Be sure to select "Server-side encryption with AWS Key Management Service keys (SSE-KMS)" as the encryption type when you edit the Properties for default encryption. -. Select your AWS KMS Key either from the dropdown or using the key ARN identifier. -. Be sure to enable the S3 bucket key, as it reduces KMS costs through caching. +For more information, see the AWS documentation https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-bucket-key.html[Enable an S3 Bucket Key for an existing bucket^]. -- Google Cloud Storage:: + -- -Configure access to Google Cloud Storage with either an IAM role attached to the instance or with access keys. +Configure access to Google Cloud Storage with either an IAM role attached to the instance, with access keys, or customer-managed keys. To configure access to Google Cloud Storage with an IAM role: @@ -820,6 +831,80 @@ Replace `` with your own values. + CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. To reset a property to its default value, run `rpk cluster config force-reset ` or remove that line from the cluster configuration with `rpk cluster config edit`. +To configure the Google Cloud bucket used by Redpanda Tiered Storage to leverage a customer-managed key using the Cloud Key Management Service API instead of the default Google-managed key, you must: + +. Create a KMS key. +. Configure the bucket to use the KMS key. +. Optionally, re-encrypt existing data with the new KMS key. + +To manage Google Cloud KMS using the command line, first install or upgrade to the latest version of https://cloud.google.com/sdk/docs/install[Google Cloud CLI^]. + +To create a KMS key: + +. In Google Cloud Console, search for "Cloud Key Managment Service API". Click **Enable** if the service is not already enabled. +. Using the Google Cloud CLI, create a new keyring in the https://cloud.google.com/kms/docs/locations^[region] where the bucket used by Redpanda is located. Note that region is case-sensitive. ++ +[,bash, indent] +---- +gcloud kms keyrings create "redpanda-keyring" --location="{REGION}" +---- ++ +. Create a new key for the keyring in the same region as the bucket: ++ +[,bash, indent] +---- +gcloud kms keys create "redpanda-key" \ + --location="{REGION}" \ + --keyring="redpanda-keyring" \ + --purpose="encryption" +---- ++ +. Get the key identifier: ++ +[,bash] +---- +gcloud kms keys list \ + --location="REGION" \ + --keyring="redpanda-keyring" +---- ++ +The result should look like the following. Be sure to record the name, as this is used to assign and manage the key, and use this as the {KEY_RESOURCE} placeholder in subsequent commands. ++ +[,bash] +---- +NAME PURPOSE ALGORITHM PROTECTION_LEVEL LABELS PRIMARY_ID PRIMARY_STATE +projects/{PROJECT_NAME}/locations/us/keyRings/redpanda-keyring/cryptoKeys/redpanda-key ENCRYPT_DECRYPT GOOGLE_SYMMETRIC_ENCRYPTION SOFTWARE 1 ENABLED +---- + +To configure the GCP bucket to use the KMS key: + +. Assign the key to a service agent: ++ +[,bash] +---- +gcloud storage service-agent \ + --project={PROJECT_ID_STORING_OBJECTS} \ + --authorize-cmek={KEY_RESOURCE} +---- ++ +. Set the bucket default encryption key to the KMS key: ++ +[,bash] +---- +gcloud storage buckets update gs://{BUCKET_NAME} \ + --default-encryption-key={KEY_RESOURCE} +---- + +(Optional) To re-encrypt existing data using the new KMS key: + +Existing data in the bucket will continue to be read using the Google-managed key, while moving forward, new objects are encrypted using the new KMS key. If you wish to re-encrypt all data in the bucket to use the KMS key, run the following command: + +[,bash] +---- +gcloud storage objects update gs://{BUCKET_NAME}/ --recursive \ + --encryption-key={KEY_RESOURCE} +---- + -- Microsoft ABS/ADLS:: + From 320fb453fdfe0ca76874ee61af2d775a4ed71e7a Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Tue, 5 Nov 2024 11:27:59 -0500 Subject: [PATCH 05/26] additional details from feature card --- modules/manage/partials/tiered-storage.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index 81004aacb..63445d0bc 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -691,7 +691,7 @@ TIP: If deploying Redpanda on an AWS Auto-Scaling group (ASG), keep in mind that Configure access to Amazon S3 with either an IAM role attached to the instance, or with access keys. TIP: Optionally, if you need to manage and store encryption keys separately from your cloud provider, you can - <> instead of the default AWS S3-managed key (SSE-S3). + <> instead of the default AWS S3-managed key (SSE-S3). This enables you to segregate data from different teams or departments and remove that data at will by removing the encryption keys. You cannot configure a cloud-provider managed encryption key at the topic level or in Dedicated clusters. For topic-level control, each CLI Get or Put for a partition must use the correct key as configured on the topic. To configure access to an S3 bucket with an IAM role: From d13d236d82501d3358e5fe552d1ba5e816e99afe Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Tue, 5 Nov 2024 13:16:55 -0600 Subject: [PATCH 06/26] Update modules/manage/partials/tiered-storage.adoc --- modules/manage/partials/tiered-storage.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index 63445d0bc..104e3e678 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -872,7 +872,7 @@ The result should look like the following. Be sure to record the name, as this i + [,bash] ---- -NAME PURPOSE ALGORITHM PROTECTION_LEVEL LABELS PRIMARY_ID PRIMARY_STATE +NAME PURPOSE ALGORITHM PROTECTION_LEVEL LABELS PRIMARY_ID PRIMARY_STATE projects/{PROJECT_NAME}/locations/us/keyRings/redpanda-keyring/cryptoKeys/redpanda-key ENCRYPT_DECRYPT GOOGLE_SYMMETRIC_ENCRYPTION SOFTWARE 1 ENABLED ---- From 944cb291e52f37b4805a850771c466fe7ede1ca3 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Tue, 5 Nov 2024 13:17:46 -0600 Subject: [PATCH 07/26] Update modules/manage/partials/tiered-storage.adoc --- modules/manage/partials/tiered-storage.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index 104e3e678..dd4f4cfac 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -873,7 +873,8 @@ The result should look like the following. Be sure to record the name, as this i [,bash] ---- NAME PURPOSE ALGORITHM PROTECTION_LEVEL LABELS PRIMARY_ID PRIMARY_STATE -projects/{PROJECT_NAME}/locations/us/keyRings/redpanda-keyring/cryptoKeys/redpanda-key ENCRYPT_DECRYPT GOOGLE_SYMMETRIC_ENCRYPTION SOFTWARE 1 ENABLED +projects/{PROJECT_NAME}/locations/us/keyRings/redpanda-keyring/cryptoKeys/redpanda-key +ENCRYPT_DECRYPT GOOGLE_SYMMETRIC_ENCRYPTION SOFTWARE 1 ENABLED ---- To configure the GCP bucket to use the KMS key: From 8b7227b6e5aefada27e311210579fc5e3c3280a0 Mon Sep 17 00:00:00 2001 From: JakeSCahill Date: Tue, 5 Nov 2024 20:33:57 +0000 Subject: [PATCH 08/26] Single-source into Kubernetes doc --- modules/manage/partials/tiered-storage.adoc | 148 ++---------------- .../partials/tiered-storage/aws-kms-key.adoc | 67 ++++++++ .../partials/tiered-storage/gcp-kms-key.adoc | 68 ++++++++ 3 files changed, 151 insertions(+), 132 deletions(-) create mode 100644 modules/manage/partials/tiered-storage/aws-kms-key.adoc create mode 100644 modules/manage/partials/tiered-storage/gcp-kms-key.adoc diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index dd4f4cfac..7cfd6aac1 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -58,6 +58,11 @@ TIP: If deploying Redpanda on an AWS Auto-Scaling group (ASG), keep in mind that You can configure access to Amazon S3 with either an IAM role attached to the instance or with access keys. +[TIP] +==== +If you need to manage and store encryption keys separately from your cloud provider, you can <> instead of the default AWS S3-managed key (SSE-S3). This option enables you to segregate data from different teams or departments and remove that data at will by removing the encryption keys. +==== + ===== Use IAM roles To configure access to an S3 bucket with an IAM role: @@ -258,9 +263,11 @@ Replace the following placeholders: + CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. +include::manage:partial$tiered-storage/aws-kms-key.adoc[leveloffset=+4] + ==== Google Cloud Storage -You can configure access to Google Cloud Storage with either an IAM role attached to the instance or with access keys. +Configure access to Google Cloud Storage with either an IAM role attached to the instance, with access keys, or customer-managed keys. ===== Use IAM roles @@ -462,6 +469,8 @@ Replace the following placeholders: + CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. +include::manage:partial$tiered-storage/gcp-kms-key.adoc[leveloffset=+4] + ==== Microsoft ABS/ADLS You can configure access to Azure Blob Storage with either account access keys or Azure's managed identities system to securely interact with Azure Blob Storage. Account access keys, as static credentials, require manual management and vigilant security practices to prevent breaches due to their unchanging nature. In contrast, managed identities provide a more secure and maintenance-free solution by automating credential management and rotation, though they are exclusive to the Azure ecosystem. @@ -690,8 +699,10 @@ TIP: If deploying Redpanda on an AWS Auto-Scaling group (ASG), keep in mind that Configure access to Amazon S3 with either an IAM role attached to the instance, or with access keys. -TIP: Optionally, if you need to manage and store encryption keys separately from your cloud provider, you can - <> instead of the default AWS S3-managed key (SSE-S3). This enables you to segregate data from different teams or departments and remove that data at will by removing the encryption keys. You cannot configure a cloud-provider managed encryption key at the topic level or in Dedicated clusters. For topic-level control, each CLI Get or Put for a partition must use the correct key as configured on the topic. +[TIP] +==== +If you need to manage and store encryption keys separately from your cloud provider, you can <> instead of the default AWS S3-managed key (SSE-S3). This option enables you to segregate data from different teams or departments and remove that data at will by removing the encryption keys. +==== To configure access to an S3 bucket with an IAM role: @@ -729,61 +740,7 @@ Replace `` with your own values. + CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. To reset a property to its default value, run `rpk cluster config force-reset ` or remove that line from the cluster configuration with `rpk cluster config edit`. -[discrete] -==== Configure access to AWS S3 using an AWS KMS key - -When there are strict data compliance requirements and you must manage and store encryption keys separately from your cloud provider, you can configure an Amazon S3 bucket that Tiered Storage can use to leverage your customer-provided key (SSE-KMS) instead of the default AWS-managed key (SSE-S3). - -To convert an existing S3 bucket and its contents, you must: - -. Create a new KMS key -. Configure the S3 bucket to use the new KMS key -. (Optional) Re-encrypt existing objects to use the new KMS key - -[discrete] -==== Prerequisites - -- The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. -- The S3 bucket must be assigned Key user permission. Without this permission, Redpanda will be unable to write new objects to Tiered Storage. -- If you intend to retroactively re-encrypt existing data with the new KMS key, record the ARN identifier of the key upon creation, as it will be required later when running an AWS CLI command. - -To create a new KMS key in the AWS Console: - -. In AWS Console, search for “Key Management Service”. -. Click **Create a key**. -. On the Configure key page, select the Symmetric key type, then select Encrypt and decrypt. -. Click the **Advanced options** tab and configure Key material origin and Regionality as needed. For example, if you are using xref:manage:remote-read-replicas.adoc[Remote Read Replicas] or have Redpanda spanning across regions, select Multi-Region key. -. Click **Next**. -. On the Add labels page, specify an alias name and description for the key. Do not include sensitive information in these fields. -. Click **Next**. -. On the Define key administrative permissions page, specify a user who can administer this key through the KMS API. -. Click **Next**. -. On the Define key usage permissions page, assign the S3 bucket as a Key user. This is required for the S3 bucket to encrypt and decrypt. -. Click **Next**. -. Review your KMS key configuration and click **Finish**. - -For more information, see the AWS documentation https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html[Creating a symmetric encryption KMS key^]. - -To configure the S3 bucket to use the new KMS key (and reduce KMS costs through caching): - -. In AWS Console, search for "S3". -. Select the bucket used by Redpanda. -. Click the **Properties** tab. -. In Default encryption, click **Edit**. -. For Encryption type, select “Server-side encryption with AWS Key Management Service keys (SSE-KMS)”. -. Locate and select your AWS KMS key ARN identifier. -. Click **Save changes**. - -(Optional) To re-encrypt existing data using the new KMS key: - -Existing data in your S3 bucket continues to be read using the AWS-managed key, while moving forward, new objects are encrypted using the new KMS key. If you wish to re-encrypt all S3 bucket data to use the KMS key, run the following AWS CLI command: - -[,bash] ----- -aws s3 cp s3://{BUCKET_NAME}/ s3://{BUCKET_NAME}/ --recursive --sse-kms-key-id {KMS_KEY_ARN} --sse aws:kms ----- - -For more information, see the AWS documentation https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-bucket-key.html[Enable an S3 Bucket Key for an existing bucket^]. +include::manage:partial$tiered-storage/aws-kms-key.adoc[leveloffset=+3] -- Google Cloud Storage:: @@ -831,80 +788,7 @@ Replace `` with your own values. + CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. To reset a property to its default value, run `rpk cluster config force-reset ` or remove that line from the cluster configuration with `rpk cluster config edit`. -To configure the Google Cloud bucket used by Redpanda Tiered Storage to leverage a customer-managed key using the Cloud Key Management Service API instead of the default Google-managed key, you must: - -. Create a KMS key. -. Configure the bucket to use the KMS key. -. Optionally, re-encrypt existing data with the new KMS key. - -To manage Google Cloud KMS using the command line, first install or upgrade to the latest version of https://cloud.google.com/sdk/docs/install[Google Cloud CLI^]. - -To create a KMS key: - -. In Google Cloud Console, search for "Cloud Key Managment Service API". Click **Enable** if the service is not already enabled. -. Using the Google Cloud CLI, create a new keyring in the https://cloud.google.com/kms/docs/locations^[region] where the bucket used by Redpanda is located. Note that region is case-sensitive. -+ -[,bash, indent] ----- -gcloud kms keyrings create "redpanda-keyring" --location="{REGION}" ----- -+ -. Create a new key for the keyring in the same region as the bucket: -+ -[,bash, indent] ----- -gcloud kms keys create "redpanda-key" \ - --location="{REGION}" \ - --keyring="redpanda-keyring" \ - --purpose="encryption" ----- -+ -. Get the key identifier: -+ -[,bash] ----- -gcloud kms keys list \ - --location="REGION" \ - --keyring="redpanda-keyring" ----- -+ -The result should look like the following. Be sure to record the name, as this is used to assign and manage the key, and use this as the {KEY_RESOURCE} placeholder in subsequent commands. -+ -[,bash] ----- -NAME PURPOSE ALGORITHM PROTECTION_LEVEL LABELS PRIMARY_ID PRIMARY_STATE -projects/{PROJECT_NAME}/locations/us/keyRings/redpanda-keyring/cryptoKeys/redpanda-key -ENCRYPT_DECRYPT GOOGLE_SYMMETRIC_ENCRYPTION SOFTWARE 1 ENABLED ----- - -To configure the GCP bucket to use the KMS key: - -. Assign the key to a service agent: -+ -[,bash] ----- -gcloud storage service-agent \ - --project={PROJECT_ID_STORING_OBJECTS} \ - --authorize-cmek={KEY_RESOURCE} ----- -+ -. Set the bucket default encryption key to the KMS key: -+ -[,bash] ----- -gcloud storage buckets update gs://{BUCKET_NAME} \ - --default-encryption-key={KEY_RESOURCE} ----- - -(Optional) To re-encrypt existing data using the new KMS key: - -Existing data in the bucket will continue to be read using the Google-managed key, while moving forward, new objects are encrypted using the new KMS key. If you wish to re-encrypt all data in the bucket to use the KMS key, run the following command: - -[,bash] ----- -gcloud storage objects update gs://{BUCKET_NAME}/ --recursive \ - --encryption-key={KEY_RESOURCE} ----- +include::manage:partial$tiered-storage/gcp-kms-key.adoc[leveloffset=+3] -- Microsoft ABS/ADLS:: diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc new file mode 100644 index 000000000..be11359df --- /dev/null +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -0,0 +1,67 @@ +[discrete] += Configure access to AWS S3 using an AWS KMS key + +When there are strict data compliance requirements and you must manage and store encryption keys separately from your cloud provider, you can configure an Amazon S3 bucket that Tiered Storage can use to leverage your customer-provided key (SSE-KMS) instead of the default AWS-managed key (SSE-S3). + +To convert an existing S3 bucket and its contents, you must: + +. Create a new KMS key +. Configure the S3 bucket to use the new KMS key +. (Optional) Re-encrypt existing objects to use the new KMS key + +[NOTE] +==== +ifdef::env-cloud[] +You cannot configure a cloud-provider managed encryption key at the topic level or in Redpanda Cloud Dedicated clusters. +endif::[] +ifndef::env-cloud[] +You cannot configure a cloud-provider managed encryption key at the topic level. +endif::[] + +For topic-level control, each CLI Get or Put for a partition must use the correct key as configured on the topic. +==== + +[discrete] +== Prerequisites + +- The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. +- The S3 bucket must be assigned Key user permission. Without this permission, Redpanda will be unable to write new objects to Tiered Storage. +- If you intend to retroactively re-encrypt existing data with the new KMS key, record the ARN identifier of the key upon creation, as it will be required later when running an AWS CLI command. + +To create a new KMS key in the AWS Console: + +. In AWS Console, search for “Key Management Service”. +. Click **Create a key**. +. On the Configure key page, select the Symmetric key type, then select Encrypt and decrypt. +. Click the **Advanced options** tab and configure Key material origin and Regionality as needed. For example, if you are using xref:manage:remote-read-replicas.adoc[Remote Read Replicas] or have Redpanda spanning across regions, select Multi-Region key. +. Click **Next**. +. On the Add labels page, specify an alias name and description for the key. Do not include sensitive information in these fields. +. Click **Next**. +. On the Define key administrative permissions page, specify a user who can administer this key through the KMS API. +. Click **Next**. +. On the Define key usage permissions page, assign the S3 bucket as a Key user. This is required for the S3 bucket to encrypt and decrypt. +. Click **Next**. +. Review your KMS key configuration and click **Finish**. + +For more information, see the AWS documentation https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html[Creating a symmetric encryption KMS key^]. + +To configure the S3 bucket to use the new KMS key (and reduce KMS costs through caching): + +. In AWS Console, search for "S3". +. Select the bucket used by Redpanda. +. Click the **Properties** tab. +. In Default encryption, click **Edit**. +. For Encryption type, select “Server-side encryption with AWS Key Management Service keys (SSE-KMS)”. +. Locate and select your AWS KMS key ARN identifier. +. Click **Save changes**. + +(Optional) To re-encrypt existing data using the new KMS key: + +Existing data in your S3 bucket continues to be read using the AWS-managed key, while moving forward, new objects are encrypted using the new KMS key. If you wish to re-encrypt all S3 bucket data to use the KMS key, run the following AWS CLI command: + +[,bash] +---- +aws s3 cp s3://{BUCKET_NAME}/ s3://{BUCKET_NAME}/ --recursive --sse-kms-key-id {KMS_KEY_ARN} --sse aws:kms +---- + +For more information, see the AWS documentation https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-bucket-key.html[Enable an S3 Bucket Key for an existing bucket^]. \ No newline at end of file diff --git a/modules/manage/partials/tiered-storage/gcp-kms-key.adoc b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc new file mode 100644 index 000000000..a1a3b431d --- /dev/null +++ b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc @@ -0,0 +1,68 @@ +[discrete] += Configure access to data in GCP using a KMS key + +To configure the Google Cloud bucket used by Redpanda Tiered Storage to leverage a customer-managed key using the Cloud Key Management Service API instead of the default Google-managed key, you must: +. Create a KMS key. +. Configure the bucket to use the KMS key. +. Optionally, re-encrypt existing data with the new KMS key. +To manage Google Cloud KMS using the command line, first install or upgrade to the latest version of https://cloud.google.com/sdk/docs/install[Google Cloud CLI^]. +To create a KMS key: +. In Google Cloud Console, search for "Cloud Key Managment Service API". Click **Enable** if the service is not already enabled. +. Using the Google Cloud CLI, create a new keyring in the https://cloud.google.com/kms/docs/locations^[region] where the bucket used by Redpanda is located. Note that region is case-sensitive. ++ +[,bash, indent] +---- +gcloud kms keyrings create "redpanda-keyring" --location="{REGION}" +---- ++ +. Create a new key for the keyring in the same region as the bucket: ++ +[,bash, indent] +---- +gcloud kms keys create "redpanda-key" \ + --location="{REGION}" \ + --keyring="redpanda-keyring" \ + --purpose="encryption" +---- ++ +. Get the key identifier: ++ +[,bash] +---- +gcloud kms keys list \ + --location="REGION" \ + --keyring="redpanda-keyring" +---- ++ +The result should look like the following. Be sure to record the name, as this is used to assign and manage the key, and use this as the \{KEY_RESOURCE} placeholder in subsequent commands. ++ +[,bash] +---- +NAME PURPOSE ALGORITHM PROTECTION_LEVEL LABELS PRIMARY_ID PRIMARY_STATE +projects/{PROJECT_NAME}/locations/us/keyRings/redpanda-keyring/cryptoKeys/redpanda-key +ENCRYPT_DECRYPT GOOGLE_SYMMETRIC_ENCRYPTION SOFTWARE 1 ENABLED +---- +To configure the GCP bucket to use the KMS key: +. Assign the key to a service agent: ++ +[,bash] +---- +gcloud storage service-agent \ + --project={PROJECT_ID_STORING_OBJECTS} \ + --authorize-cmek={KEY_RESOURCE} +---- ++ +. Set the bucket default encryption key to the KMS key: ++ +[,bash] +---- +gcloud storage buckets update gs://{BUCKET_NAME} \ + --default-encryption-key={KEY_RESOURCE} +---- +(Optional) To re-encrypt existing data using the new KMS key: +Existing data in the bucket will continue to be read using the Google-managed key, while moving forward, new objects are encrypted using the new KMS key. If you wish to re-encrypt all data in the bucket to use the KMS key, run the following command: +[,bash] +---- +gcloud storage objects update gs://{BUCKET_NAME}/ --recursive \ + --encryption-key={KEY_RESOURCE} +---- \ No newline at end of file From 002001984d06e866ce400d90220436ce3667a7e5 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 08:58:17 -0600 Subject: [PATCH 09/26] Update modules/manage/partials/tiered-storage.adoc Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index 7cfd6aac1..18deabbba 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -267,7 +267,7 @@ include::manage:partial$tiered-storage/aws-kms-key.adoc[leveloffset=+4] ==== Google Cloud Storage -Configure access to Google Cloud Storage with either an IAM role attached to the instance, with access keys, or customer-managed keys. +Configure access to Google Cloud Storage with either an IAM role attached to the instance, with access keys, or with customer-managed keys. ===== Use IAM roles From 1b56be23b3c51cbed1fa25e6479ecbd093288c50 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 09:00:09 -0600 Subject: [PATCH 10/26] Update modules/manage/partials/tiered-storage/aws-kms-key.adoc Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index be11359df..cdf9bcdfb 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -1,7 +1,7 @@ [discrete] = Configure access to AWS S3 using an AWS KMS key -When there are strict data compliance requirements and you must manage and store encryption keys separately from your cloud provider, you can configure an Amazon S3 bucket that Tiered Storage can use to leverage your customer-provided key (SSE-KMS) instead of the default AWS-managed key (SSE-S3). +When there are strict data compliance requirements and you must manage and store encryption keys separate from your cloud provider, you can configure an Amazon S3 bucket that Tiered Storage can use to leverage your customer-provided key (SSE-KMS) instead of the default AWS-managed key (SSE-S3). To convert an existing S3 bucket and its contents, you must: From 0e336da78c67b72b20f2140198c901759969c21c Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 09:44:09 -0600 Subject: [PATCH 11/26] Update modules/manage/partials/tiered-storage.adoc Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index 18deabbba..7a34f80a6 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -746,7 +746,7 @@ include::manage:partial$tiered-storage/aws-kms-key.adoc[leveloffset=+3] Google Cloud Storage:: + -- -Configure access to Google Cloud Storage with either an IAM role attached to the instance, with access keys, or customer-managed keys. +Configure access to Google Cloud Storage with either an IAM role attached to the instance, with access keys, or with customer-managed keys. To configure access to Google Cloud Storage with an IAM role: From 9ddf9b9657d5cb73043a1f60604699c61cf8da88 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 10:26:09 -0600 Subject: [PATCH 12/26] Update modules/manage/partials/tiered-storage/aws-kms-key.adoc Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index cdf9bcdfb..e405188eb 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -15,7 +15,7 @@ ifdef::env-cloud[] You cannot configure a cloud-provider managed encryption key at the topic level or in Redpanda Cloud Dedicated clusters. endif::[] ifndef::env-cloud[] -You cannot configure a cloud-provider managed encryption key at the topic level. +You cannot configure a cloud provider-managed encryption key at the topic level. endif::[] For topic-level control, each CLI Get or Put for a partition must use the correct key as configured on the topic. From 8c0e1f917635e8a7f8d19095ab96f50b1c54ac98 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 10:28:34 -0600 Subject: [PATCH 13/26] Update modules/manage/partials/tiered-storage/aws-kms-key.adoc Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index e405188eb..c981ad4fd 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -5,9 +5,9 @@ When there are strict data compliance requirements and you must manage and store To convert an existing S3 bucket and its contents, you must: -. Create a new KMS key -. Configure the S3 bucket to use the new KMS key -. (Optional) Re-encrypt existing objects to use the new KMS key +. Create a new KMS key. +. Configure the S3 bucket to use the new KMS key. +. (Optional) Re-encrypt existing objects to use the new KMS key. [NOTE] ==== From 9530e53fcb0ee4de7e5aafb6850e4d2fe7a7ee3c Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 10:29:00 -0600 Subject: [PATCH 14/26] Update modules/manage/partials/tiered-storage/aws-kms-key.adoc --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index c981ad4fd..83cd22761 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -21,8 +21,7 @@ endif::[] For topic-level control, each CLI Get or Put for a partition must use the correct key as configured on the topic. ==== -[discrete] -== Prerequisites +*Prerequisites* - The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. - The S3 bucket must be assigned Key user permission. Without this permission, Redpanda will be unable to write new objects to Tiered Storage. From 496ce9b54468413b014fa14dc3b6658e22a012d7 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 10:45:07 -0600 Subject: [PATCH 15/26] Update modules/manage/partials/tiered-storage/aws-kms-key.adoc Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index 83cd22761..746ce4b85 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -24,7 +24,7 @@ For topic-level control, each CLI Get or Put for a partition must use the correc *Prerequisites* - The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. -- The S3 bucket must be assigned Key user permission. Without this permission, Redpanda will be unable to write new objects to Tiered Storage. +- The S3 bucket must be assigned the Key user permission. Without this permission, Redpanda is unable to write new objects to Tiered Storage. - If you intend to retroactively re-encrypt existing data with the new KMS key, record the ARN identifier of the key upon creation, as it will be required later when running an AWS CLI command. To create a new KMS key in the AWS Console: From 3e09e962a374a51f5bb1b791d01a29aa21fda205 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 10:45:27 -0600 Subject: [PATCH 16/26] Update modules/manage/partials/tiered-storage/aws-kms-key.adoc Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index 746ce4b85..8b93990fc 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -25,7 +25,7 @@ For topic-level control, each CLI Get or Put for a partition must use the correc - The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. - The S3 bucket must be assigned the Key user permission. Without this permission, Redpanda is unable to write new objects to Tiered Storage. -- If you intend to retroactively re-encrypt existing data with the new KMS key, record the ARN identifier of the key upon creation, as it will be required later when running an AWS CLI command. +- If you intend to retroactively re-encrypt existing data with the new KMS key, store the ARN identifier of the key upon creation. It is required for AWS CLI commands. To create a new KMS key in the AWS Console: From 83f95bb025fcb54ede387bb3f4c7735604c92887 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 10:46:04 -0600 Subject: [PATCH 17/26] Update modules/manage/partials/tiered-storage/aws-kms-key.adoc Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index 8b93990fc..3fcb756e5 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -32,7 +32,7 @@ To create a new KMS key in the AWS Console: . In AWS Console, search for “Key Management Service”. . Click **Create a key**. . On the Configure key page, select the Symmetric key type, then select Encrypt and decrypt. -. Click the **Advanced options** tab and configure Key material origin and Regionality as needed. For example, if you are using xref:manage:remote-read-replicas.adoc[Remote Read Replicas] or have Redpanda spanning across regions, select Multi-Region key. +. Click the **Advanced options** tab and configure Key material origin and Regionality as needed. For example, if you are using xref:manage:remote-read-replicas.adoc[Remote Read Replicas] or have Redpanda spanning across regions, select **Multi-Region key**. . Click **Next**. . On the Add labels page, specify an alias name and description for the key. Do not include sensitive information in these fields. . Click **Next**. From 3fd5a42f6b367e753299a43b98706a349eeb5891 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 10:46:22 -0600 Subject: [PATCH 18/26] Update modules/manage/partials/tiered-storage/aws-kms-key.adoc Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index 3fcb756e5..23416e960 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -42,7 +42,7 @@ To create a new KMS key in the AWS Console: . Click **Next**. . Review your KMS key configuration and click **Finish**. -For more information, see the AWS documentation https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html[Creating a symmetric encryption KMS key^]. +For more information, see the https://docs.aws.amazon.com/kms/latest/developerguide/create-symmetric-cmk.html[AWS documentation^]. To configure the S3 bucket to use the new KMS key (and reduce KMS costs through caching): From 44301dd856b08673d8b4856eab5c912877a7b6dc Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Mon, 11 Nov 2024 10:50:01 -0600 Subject: [PATCH 19/26] Apply suggestions from code review Doc PR review updates Co-authored-by: Michele Cyran --- .../manage/partials/tiered-storage/gcp-kms-key.adoc | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/modules/manage/partials/tiered-storage/gcp-kms-key.adoc b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc index a1a3b431d..829ff7fa3 100644 --- a/modules/manage/partials/tiered-storage/gcp-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc @@ -2,13 +2,17 @@ = Configure access to data in GCP using a KMS key To configure the Google Cloud bucket used by Redpanda Tiered Storage to leverage a customer-managed key using the Cloud Key Management Service API instead of the default Google-managed key, you must: + . Create a KMS key. . Configure the bucket to use the KMS key. . Optionally, re-encrypt existing data with the new KMS key. + To manage Google Cloud KMS using the command line, first install or upgrade to the latest version of https://cloud.google.com/sdk/docs/install[Google Cloud CLI^]. + To create a KMS key: + . In Google Cloud Console, search for "Cloud Key Managment Service API". Click **Enable** if the service is not already enabled. -. Using the Google Cloud CLI, create a new keyring in the https://cloud.google.com/kms/docs/locations^[region] where the bucket used by Redpanda is located. Note that region is case-sensitive. +. Using the Google Cloud CLI, create a new keyring in the https://cloud.google.com/kms/docs/locations^[region] where the bucket used by Redpanda is located. Note that region is case sensitive. + [,bash, indent] ---- @@ -34,7 +38,7 @@ gcloud kms keys list \ --keyring="redpanda-keyring" ---- + -The result should look like the following. Be sure to record the name, as this is used to assign and manage the key, and use this as the \{KEY_RESOURCE} placeholder in subsequent commands. +The result should look like the following. Be sure to store the name, as this is used to assign and manage the key. Use this as the \{KEY_RESOURCE} placeholder in subsequent commands. + [,bash] ---- @@ -43,6 +47,7 @@ projects/{PROJECT_NAME}/locations/us/keyRings/redpanda-keyring/cryptoKeys/redpan ENCRYPT_DECRYPT GOOGLE_SYMMETRIC_ENCRYPTION SOFTWARE 1 ENABLED ---- To configure the GCP bucket to use the KMS key: + . Assign the key to a service agent: + [,bash] @@ -60,7 +65,8 @@ gcloud storage buckets update gs://{BUCKET_NAME} \ --default-encryption-key={KEY_RESOURCE} ---- (Optional) To re-encrypt existing data using the new KMS key: -Existing data in the bucket will continue to be read using the Google-managed key, while moving forward, new objects are encrypted using the new KMS key. If you wish to re-encrypt all data in the bucket to use the KMS key, run the following command: + +Existing data in the bucket continues to be read using the Google-managed key, while new objects are encrypted using the new KMS key. If you wish to re-encrypt all data in the bucket to use the KMS key, run: [,bash] ---- gcloud storage objects update gs://{BUCKET_NAME}/ --recursive \ From f66707188f062aaebbe03e3abfbfb620af07ead3 Mon Sep 17 00:00:00 2001 From: Michele Cyran Date: Mon, 11 Nov 2024 16:28:03 -0700 Subject: [PATCH 20/26] fix formatting --- modules/manage/partials/tiered-storage/gcp-kms-key.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/modules/manage/partials/tiered-storage/gcp-kms-key.adoc b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc index 829ff7fa3..34b5133fc 100644 --- a/modules/manage/partials/tiered-storage/gcp-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc @@ -46,6 +46,7 @@ NAME PURPOSE ALGORITHM PROTECTION_LEVEL LABELS PRIMARY_ID projects/{PROJECT_NAME}/locations/us/keyRings/redpanda-keyring/cryptoKeys/redpanda-key ENCRYPT_DECRYPT GOOGLE_SYMMETRIC_ENCRYPTION SOFTWARE 1 ENABLED ---- + To configure the GCP bucket to use the KMS key: . Assign the key to a service agent: @@ -64,6 +65,7 @@ gcloud storage service-agent \ gcloud storage buckets update gs://{BUCKET_NAME} \ --default-encryption-key={KEY_RESOURCE} ---- + (Optional) To re-encrypt existing data using the new KMS key: Existing data in the bucket continues to be read using the Google-managed key, while new objects are encrypted using the new KMS key. If you wish to re-encrypt all data in the bucket to use the KMS key, run: From 80e73d8b4da4e47cf65818eae252600d7fcc70b2 Mon Sep 17 00:00:00 2001 From: Michele Cyran Date: Mon, 11 Nov 2024 17:37:49 -0700 Subject: [PATCH 21/26] standardize bold subheadings for readability --- modules/manage/partials/tiered-storage.adoc | 34 +++++++------------ .../partials/tiered-storage/aws-kms-key.adoc | 4 +-- .../partials/tiered-storage/gcp-kms-key.adoc | 2 +- 3 files changed, 15 insertions(+), 25 deletions(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index 7a34f80a6..c3ef67fa5 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -63,9 +63,7 @@ You can configure access to Amazon S3 with either an IAM role attached to the in If you need to manage and store encryption keys separately from your cloud provider, you can <> instead of the default AWS S3-managed key (SSE-S3). This option enables you to segregate data from different teams or departments and remove that data at will by removing the encryption keys. ==== -===== Use IAM roles - -To configure access to an S3 bucket with an IAM role: +===== **Configure access with an IAM role** . Configure an xref:manage:security/iam-roles.adoc#configuring-iam-roles[IAM role]. @@ -143,9 +141,7 @@ Replace the following placeholders: + CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. -===== Use access keys - -To configure access to an S3 bucket with access keys instead of an IAM role: +===== **Configure access with access keys** . Grant an IAM user the following permissions to read and create objects in your buckets: - `GetObject` @@ -269,9 +265,7 @@ include::manage:partial$tiered-storage/aws-kms-key.adoc[leveloffset=+4] Configure access to Google Cloud Storage with either an IAM role attached to the instance, with access keys, or with customer-managed keys. -===== Use IAM roles - -To configure access to Google Cloud Storage with an IAM role: +===== **Configure access with an IAM role** . Configure an xref:manage:security/iam-roles.adoc#configuring-iam-roles[IAM role]. . Override the following required cluster properties in the Helm chart: @@ -350,7 +344,7 @@ Replace the following placeholders: CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. -===== Use access keys +===== **Configure access with access keys** To configure access to Google Cloud Storage with access keys instead of an IAM role: @@ -477,9 +471,7 @@ You can configure access to Azure Blob Storage with either account access keys o include::manage:partial$azure-blob-limitations.adoc[] -===== Use managed identities - -To configure access to an Azure container with a managed identity in AKS: +===== **Configure access with a managed identity** . Configure an xref:manage:security/iam-roles.adoc#configuring-iam-roles[Azure managed identity]. . Override the following required cluster properties in the Helm chart: @@ -579,9 +571,7 @@ NOTE: The `serviceAccount` annotations and the `statefulset` Pod labels are esse CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. -===== Use access keys - -To configure access to ABS/ADLS with account access keys: +===== **Configure access with account access keys** . Get an account access key for the Azure container that Redpanda will run on. For information on how to view your account access keys, see the https://learn.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?toc=%2Fazure%2Fstorage%2Fblobs%2Ftoc.json&bc=%2Fazure%2Fstorage%2Fblobs%2Fbreadcrumb%2Ftoc.json&tabs=azure-portal#view-account-access-keys[Azure documentation^]. . Create a Secret in which to store the access key. @@ -704,7 +694,7 @@ Configure access to Amazon S3 with either an IAM role attached to the instance, If you need to manage and store encryption keys separately from your cloud provider, you can <> instead of the default AWS S3-managed key (SSE-S3). This option enables you to segregate data from different teams or departments and remove that data at will by removing the encryption keys. ==== -To configure access to an S3 bucket with an IAM role: +**Configure access with an IAM role** . Configure an xref:manage:security/iam-roles.adoc#configuring-iam-roles[IAM role]. . Run the `rpk cluster config edit` command, then edit the following required properties: @@ -721,7 +711,7 @@ Replace `` with your own values. CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. To reset a property to its default value, run `rpk cluster config force-reset ` or remove that line from the cluster configuration with `rpk cluster config edit`. -To configure access to an S3 bucket with access keys instead of an IAM role: +**Configure access with access keys** . Grant a user the following permissions to read and create objects on the bucket to be used with the cluster (or on all buckets): `GetObject`, `DeleteObject`, `PutObject`, `PutObjectTagging`, `ListBucket`. . Copy the access key and secret key for the `cloud_storage_access_key` and `cloud_storage_secret_key` cluster properties. @@ -748,7 +738,7 @@ Google Cloud Storage:: -- Configure access to Google Cloud Storage with either an IAM role attached to the instance, with access keys, or with customer-managed keys. -To configure access to Google Cloud Storage with an IAM role: +**Configure access with an IAM role** . Configure an xref:manage:security/iam-roles.adoc#configuring-iam-roles[IAM role]. . Run the `rpk cluster config edit` command, then edit the following required properties: @@ -766,7 +756,7 @@ Replace `` with your own values. + CAUTION: Do not set an object storage property to an empty string `""` or to `null` as a way to reset it to its default value. To reset a property to its default value, run `rpk cluster config force-reset ` or remove that line from the cluster configuration with `rpk cluster config edit`. -To configure access to Google Cloud Storage with access keys instead of an IAM role: +**Configure access with access keys** . Choose a uniform access control when you create the bucket. . Use a Google-managed encryption key. @@ -797,7 +787,7 @@ Microsoft ABS/ADLS:: include::manage:partial$azure-blob-limitations.adoc[] -To configure access to an Azure container with a managed identity: +**Configure access with managed identities** . Configure an xref:manage:security/iam-roles.adoc#azure-prerequisites[Azure managed identity]. + @@ -823,7 +813,7 @@ cloud_storage_azure_container: + Replace `` with your own values. -To configure access to Azure Blob Storage with account access keys: +**Configure access with account access keys** . Copy an account access key for the Azure container you want Redpanda to use and enter it in the `cloud_storage_azure_shared_key` property. For information on how to view your account access keys, see the https://learn.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?toc=%2Fazure%2Fstorage%2Fblobs%2Ftoc.json&bc=%2Fazure%2Fstorage%2Fblobs%2Fbreadcrumb%2Ftoc.json&tabs=azure-portal#view-account-access-keys[Azure documentation^]. . Run the `rpk cluster config edit` command, then edit the following required properties: diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index 23416e960..99293c3eb 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -1,5 +1,5 @@ [discrete] -= Configure access to AWS S3 using an AWS KMS key += **Configure access with an AWS KMS key** When there are strict data compliance requirements and you must manage and store encryption keys separate from your cloud provider, you can configure an Amazon S3 bucket that Tiered Storage can use to leverage your customer-provided key (SSE-KMS) instead of the default AWS-managed key (SSE-S3). @@ -21,7 +21,7 @@ endif::[] For topic-level control, each CLI Get or Put for a partition must use the correct key as configured on the topic. ==== -*Prerequisites* +Prerequisites - The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. - The S3 bucket must be assigned the Key user permission. Without this permission, Redpanda is unable to write new objects to Tiered Storage. diff --git a/modules/manage/partials/tiered-storage/gcp-kms-key.adoc b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc index 34b5133fc..788b312cd 100644 --- a/modules/manage/partials/tiered-storage/gcp-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc @@ -1,5 +1,5 @@ [discrete] -= Configure access to data in GCP using a KMS key += **Configure access with a KMS key** To configure the Google Cloud bucket used by Redpanda Tiered Storage to leverage a customer-managed key using the Cloud Key Management Service API instead of the default Google-managed key, you must: From 145b6a7a07d97993e56386a2ecb9329998bed3a2 Mon Sep 17 00:00:00 2001 From: Michele Cyran Date: Mon, 11 Nov 2024 20:37:17 -0700 Subject: [PATCH 22/26] minor edit for consistency with AWS & GCS --- modules/manage/partials/tiered-storage.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage.adoc b/modules/manage/partials/tiered-storage.adoc index c3ef67fa5..61b024018 100644 --- a/modules/manage/partials/tiered-storage.adoc +++ b/modules/manage/partials/tiered-storage.adoc @@ -687,7 +687,7 @@ Amazon S3:: TIP: If deploying Redpanda on an AWS Auto-Scaling group (ASG), keep in mind that the ASG controller terminates nodes and spins up replacements if the nodes saturate and are unable to heartbeat the controller (based on the EC2 health check). For more information, see the https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-system-instance-status-check.html#types-of-instance-status-checks[AWS documentation^]. Redpanda recommends deploying on Linux or Kubernetes. For more information, see xref:deploy:deployment-option/self-hosted/index.adoc[Deploy Redpanda]. -Configure access to Amazon S3 with either an IAM role attached to the instance, or with access keys. +Configure access to Amazon S3 with either an IAM role attached to the instance, with access keys, or with customer-managed keys. [TIP] ==== From 0ac501129bca5ebdaed9649a96b461b6f5e18865 Mon Sep 17 00:00:00 2001 From: Jake Cahill <45230295+JakeSCahill@users.noreply.github.com> Date: Tue, 12 Nov 2024 08:57:23 +0000 Subject: [PATCH 23/26] Show cloud providers in ToC --- .../pages/kubernetes/tiered-storage/k-tiered-storage.adoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/modules/manage/pages/kubernetes/tiered-storage/k-tiered-storage.adoc b/modules/manage/pages/kubernetes/tiered-storage/k-tiered-storage.adoc index f4f6eba1b..d21e4a8d6 100644 --- a/modules/manage/pages/kubernetes/tiered-storage/k-tiered-storage.adoc +++ b/modules/manage/pages/kubernetes/tiered-storage/k-tiered-storage.adoc @@ -1,10 +1,9 @@ = Use Tiered Storage in Kubernetes :description: Configure your Redpanda cluster to offload log segments to object storage and save storage costs. -:page-context-links: [{"name": "Linux", "to": "manage:tiered-storage.adoc" },{"name": "Kubernetes", "to": "manage:kubernetes/storage/tiered-storage/k-tiered-storage.adoc" } ] :page-categories: Management, High Availability, Data Replication :env-kubernetes: true -:tags: ["Kubernetes", "Helm configuration"] +:page-toclevels: 4 :page-aliases: manage:kubernetes/tiered-storage.adoc, manage:kubernetes/storage/tiered-storage.adoc, manage:kubernetes/data-archiving.adoc, manage:kubernetes/storage/data-archiving.adoc, manage:kubernetes/storage/tiered-storage/k-data-archiving.adoc, manage:kubernetes/storage/tiered-storage/k-tiered-storage.adoc -include::manage:partial$tiered-storage.adoc[] \ No newline at end of file +include::manage:partial$tiered-storage.adoc[] From 11bc1bfcfbc1a24de318e1369b1f5f5a48f64212 Mon Sep 17 00:00:00 2001 From: JakeSCahill Date: Tue, 12 Nov 2024 09:43:21 +0000 Subject: [PATCH 24/26] Make heading discrete only on the Linux page --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 4 +++- modules/manage/partials/tiered-storage/gcp-kms-key.adoc | 2 ++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index 99293c3eb..eee67f1d9 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -1,4 +1,6 @@ +ifndef::env-kubernetes[] [discrete] +endif::[] = **Configure access with an AWS KMS key** When there are strict data compliance requirements and you must manage and store encryption keys separate from your cloud provider, you can configure an Amazon S3 bucket that Tiered Storage can use to leverage your customer-provided key (SSE-KMS) instead of the default AWS-managed key (SSE-S3). @@ -21,7 +23,7 @@ endif::[] For topic-level control, each CLI Get or Put for a partition must use the correct key as configured on the topic. ==== -Prerequisites +== **Prerequisites** - The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. - The S3 bucket must be assigned the Key user permission. Without this permission, Redpanda is unable to write new objects to Tiered Storage. diff --git a/modules/manage/partials/tiered-storage/gcp-kms-key.adoc b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc index 788b312cd..90c8bc777 100644 --- a/modules/manage/partials/tiered-storage/gcp-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/gcp-kms-key.adoc @@ -1,4 +1,6 @@ +ifndef::env-kubernetes[] [discrete] +endif::[] = **Configure access with a KMS key** To configure the Google Cloud bucket used by Redpanda Tiered Storage to leverage a customer-managed key using the Cloud Key Management Service API instead of the default Google-managed key, you must: From 0d59119c47b443de382db3a3762b302b69cee88a Mon Sep 17 00:00:00 2001 From: JakeSCahill Date: Tue, 12 Nov 2024 09:53:47 +0000 Subject: [PATCH 25/26] Make prereqs heading discrete for KMS instructions --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index eee67f1d9..1c9cd2f81 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -23,6 +23,9 @@ endif::[] For topic-level control, each CLI Get or Put for a partition must use the correct key as configured on the topic. ==== +ifndef::env-kubernetes[] +[discrete] +endif::[] == **Prerequisites** - The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. From 81a5ee9c52be3d4afbacf3e886e00496d286ef63 Mon Sep 17 00:00:00 2001 From: Joyce Fee <102751339+Feediver1@users.noreply.github.com> Date: Thu, 14 Nov 2024 07:39:28 -0600 Subject: [PATCH 26/26] Apply suggestions from code review Doc review feedback Co-authored-by: Michele Cyran --- modules/manage/partials/tiered-storage/aws-kms-key.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/manage/partials/tiered-storage/aws-kms-key.adoc b/modules/manage/partials/tiered-storage/aws-kms-key.adoc index 1c9cd2f81..a7b34787e 100644 --- a/modules/manage/partials/tiered-storage/aws-kms-key.adoc +++ b/modules/manage/partials/tiered-storage/aws-kms-key.adoc @@ -28,7 +28,7 @@ ifndef::env-kubernetes[] endif::[] == **Prerequisites** -- The user configuring S3 bucket encryption must be assigned Key admin permission. Without this permission, you will be unable to re-encrypt existing bucket objects to use the KMS key. +- The user configuring S3 bucket encryption must be assigned the Key admin permission. Without this permission, the user is unable to re-encrypt existing bucket objects to use the KMS key. - The S3 bucket must be assigned the Key user permission. Without this permission, Redpanda is unable to write new objects to Tiered Storage. - If you intend to retroactively re-encrypt existing data with the new KMS key, store the ARN identifier of the key upon creation. It is required for AWS CLI commands. @@ -36,7 +36,7 @@ To create a new KMS key in the AWS Console: . In AWS Console, search for “Key Management Service”. . Click **Create a key**. -. On the Configure key page, select the Symmetric key type, then select Encrypt and decrypt. +. On the Configure key page, select the **Symmetric** key type, then select **Encrypt and decrypt**. . Click the **Advanced options** tab and configure Key material origin and Regionality as needed. For example, if you are using xref:manage:remote-read-replicas.adoc[Remote Read Replicas] or have Redpanda spanning across regions, select **Multi-Region key**. . Click **Next**. . On the Add labels page, specify an alias name and description for the key. Do not include sensitive information in these fields. @@ -61,11 +61,11 @@ To configure the S3 bucket to use the new KMS key (and reduce KMS costs through (Optional) To re-encrypt existing data using the new KMS key: -Existing data in your S3 bucket continues to be read using the AWS-managed key, while moving forward, new objects are encrypted using the new KMS key. If you wish to re-encrypt all S3 bucket data to use the KMS key, run the following AWS CLI command: +Existing data in your S3 bucket continues to be read using the AWS-managed key, while new objects are encrypted using the new KMS key. If you wish to re-encrypt all S3 bucket data to use the KMS key, run: [,bash] ---- aws s3 cp s3://{BUCKET_NAME}/ s3://{BUCKET_NAME}/ --recursive --sse-kms-key-id {KMS_KEY_ARN} --sse aws:kms ---- -For more information, see the AWS documentation https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-bucket-key.html[Enable an S3 Bucket Key for an existing bucket^]. \ No newline at end of file +For more information, see the https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-bucket-key.html[AWS documentation^]. \ No newline at end of file