diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 84dd16c6e..06aa4d2ec 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -47,8 +47,8 @@ **** xref:networking:azure-private-link.adoc[] *** xref:networking:dedicated/gcp/index.adoc[GCP] **** xref:networking:dedicated/gcp/vpc-peering-gcp.adoc[Add a Peering Connection] -**** xref:networking:configure-private-service-connect-in-cloud-ui.adoc[Configure Private Service Connect in the Cloud UI] -**** xref:networking:gcp-private-service-connect.adoc[Configure Private Service Connect with the Cloud API] +**** xref:networking:dedicated/gcp/configure-psc-in-ui.adoc[Configure Private Service Connect in the Cloud UI] +**** xref:networking:dedicated/gcp/configure-psc-in-api.adoc[Configure Private Service Connect with the Cloud API] * xref:security:index.adoc[Security] ** xref:security:cloud-authentication.adoc[Authentication] diff --git a/modules/networking/pages/aws-privatelink.adoc b/modules/networking/pages/aws-privatelink.adoc index 022692283..091faf647 100644 --- a/modules/networking/pages/aws-privatelink.adoc +++ b/modules/networking/pages/aws-privatelink.adoc @@ -45,7 +45,7 @@ export RESOURCE_GROUP_ID= . Call xref:api:ROOT:cloud-controlplane-api.adoc#post-/v1/networks[`POST /v1/networks`] to create a network. + -Make sure to supply your own values in the following example request. Store the network ID (`network_id`) after the network is created to check whether you can proceed to cluster creation. +Make sure to supply your own values in the following example request. The example uses a BYOC cluser. For a Dedicated cluster, set `"cluster_type": "TYPE_DEDICATED"`. Store the network ID (`network_id`) after the network is created to check whether you can proceed to cluster creation. + -- - `name` @@ -84,11 +84,11 @@ In the example below, make sure to set your own values for the following fields: + -- - `zones`: for example, `"usw2-az1","usw2-az2","usw2-az3"` -- `type`: `TYPE_DEDICATED` for Dedicated Cloud clusters, or `TYPE_BYOC` for BYOC clusters -- `tier`: for example, `tier-1-aws-v2-arm` +- `type`: `"TYPE_BYOC"` or `"TYPE_DEDICATED"` +- `tier`: for example, `"tier-1-aws-v2-arm"` - `name` - `connect_console`: Whether to enable connections to Redpanda Console (boolean) -- `allowed_principals`: Amazon Resource Names (ARNs) for the AWS principals allowed to access the endpoint service. For example, for all principals in an account, use `arn:aws:iam::account_id:root`. See https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#add-remove-permission[Configure an endpoint service^] for details. +- `allowed_principals`: Amazon Resource Names (ARNs) for the AWS principals allowed to access the endpoint service. For example, for all principals in an account, use `"arn:aws:iam::account_id:root"`. See https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#add-remove-permission[Configure an endpoint service^] for details. -- + [,bash] @@ -120,7 +120,7 @@ CLUSTER_ID=`curl -vv -X POST \ echo $CLUSTER_ID ---- + -**BYOC clusters:** Check that the cluster operation is completed by calling xref:api:ROOT:cloud-controlplane-api.adoc#get-/v1/operations/-id-[`GET /v1/operations/\{id}`], and passing the operation ID returned from the Create Cluster call. +**BYOC clusters only:** Check that the cluster operation is completed by calling xref:api:ROOT:cloud-controlplane-api.adoc#get-/v1/operations/-id-[`GET /v1/operations/\{id}`], and passing the operation ID returned from the Create Cluster call. + When the Create Cluster operation is completed (`STATE_COMPLETED`), run the following `rpk cloud` command to finish setting up your BYOC cluster: + @@ -146,7 +146,7 @@ In the example below, make sure to set your own value for the following field: + -- - `connect_console`: Whether to enable connections to Redpanda Console (boolean) -- `allowed_principals`: Amazon Resource Names (ARNs) for the AWS principals allowed to access the endpoint service. For example, for all principals in an account, use `arn:aws:iam::account_id:root`. See https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#add-remove-permission[Configure an endpoint service^] for details. +- `allowed_principals`: Amazon Resource Names (ARNs) for the AWS principals allowed to access the endpoint service. For example, for all principals in an account, use `"arn:aws:iam::account_id:root"`. See https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#add-remove-permission[Configure an endpoint service^] for details. -- + [,bash] @@ -217,7 +217,7 @@ xref:networking:byoc/aws/vpc-peering-aws.adoc[VPC peering] and PrivateLink will VPC peering and PrivateLink can both be used at the same time if Kafka clients are connecting from distinct VPCs. For example, in a private Redpanda cluster, you can connect your internal Kafka clients over VPC peering, and enable PrivateLink for external services. ==== -The VPC region must be the same region where the Redpanda cluster is deployed. Run the following command to create the VPC: +The VPC region must be the same region where the Redpanda cluster is deployed. To create the VPC, run: [,bash] ---- @@ -235,7 +235,7 @@ You can also use an existing VPC. You need the VPC ID to <>. Run the following command to create a security group: +You need the security group ID `security_group_id` from the command output to <>. To create a security group, run: [,bash] ---- @@ -369,5 +369,5 @@ include::networking:partial$private-links-test-connection.adoc[] include::shared:partial$suggested-reading.adoc[] * xref:manage:api/cloud-api-overview.adoc[] -* xref:networking:dedicated/vpc-peering.adoc[] * xref:networking:byoc/aws/vpc-peering-aws.adoc[] +* xref:networking:dedicated/vpc-peering.adoc[] diff --git a/modules/networking/pages/azure-private-link.adoc b/modules/networking/pages/azure-private-link.adoc index 5c47d43b6..ab5debdae 100644 --- a/modules/networking/pages/azure-private-link.adoc +++ b/modules/networking/pages/azure-private-link.adoc @@ -19,7 +19,7 @@ After <>, you can <>. @@ -139,7 +139,7 @@ CLUSTER_ID=`curl -vv -X POST \ echo $CLUSTER_ID ---- -. **BYOC clusters:** Check that the cluster operation is completed by calling xref:api:ROOT:cloud-controlplane-api.adoc#get-/v1/operations/-id-[`GET /v1/operations/\{id}`], and passing the operation ID returned from the Create Cluster call. +. **BYOC clusters only:** Check that the cluster operation is completed by calling xref:api:ROOT:cloud-controlplane-api.adoc#get-/v1/operations/-id-[`GET /v1/operations/\{id}`], and passing the operation ID returned from the Create Cluster call. + When the Create Cluster operation is completed (`STATE_COMPLETED`), run the following `rpk cloud` command to finish setting up your BYOC cluster with Private Link enabled: + diff --git a/modules/networking/pages/configure-private-service-connect-in-cloud-ui.adoc b/modules/networking/pages/configure-private-service-connect-in-cloud-ui.adoc index 416807585..b0fe7aad6 100644 --- a/modules/networking/pages/configure-private-service-connect-in-cloud-ui.adoc +++ b/modules/networking/pages/configure-private-service-connect-in-cloud-ui.adoc @@ -1,109 +1,6 @@ = Configure GCP Private Service Connect in the Cloud UI :description: Set up GCP Private Service Connect in the Redpanda Cloud UI. :page-aliases: deploy:deployment-option/cloud/configure-private-service-connect-in-cloud-ui.adoc +:env-byoc: true -[NOTE] -==== - -* This guide is for configuring GCP Private Service Connect using the Redpanda Cloud UI. To configure and manage Private Service on an existing public cluster, you must use the xref:networking:gcp-private-service-connect.adoc[Redpanda Cloud API]. -* The latest version of Redpanda GCP Private Service Connect (available March, 2025) supports AZ affinity. This allows requests from Private Service Connect endpoints to stay within the same availability zone, avoiding additional networking costs. -* DEPRECATION: The original Redpanda GCP Private Service Connect is deprecated and will be removed in a future release. For more information, see xref:manage:maintenance.adoc#deprecated-features[Deprecated features]. -==== - - -The Redpanda GCP Private Service Connect service provides secure access to Redpanda Cloud from your own VPC. Traffic over Private Service Connect does not go through the public internet because these connections are treated as their own private GCP service. While your VPC has access to the Redpanda VPC, Redpanda cannot access your VPC. - -Consider using the endpoint services if you have multiple VPCs and could benefit from a more simplified approach to network management. - -[NOTE] -==== -* Each client VPC can have one endpoint connected to Private Service Connect. -* Private Service Connect allows overlapping xref:networking:cidr-ranges.adoc[CIDR ranges] in VPC networks. -* The number of connections is limited only by your Redpanda usage tier. Private Service Connect does not add extra connection limits. -* You control from which GCP projects connections are allowed. -==== - -== Requirements - -* Use the https://cloud.google.com/sdk/docs/install[gcloud^] command-line interface (CLI) to create the consumer-side resources, such as a client VPC and forwarding rule, or to modify existing resources to use the Private Service Connect attachment created for your cluster. -* The client VPC must be in the same region as your Redpanda cluster. - -== Enable endpoint service for existing clusters - -. In the Redpanda Cloud UI, open your https://cloud.redpanda.com/clusters[cluster^], and click **Cluster settings**. -. Under Private Service Connect, click **Enable**. -. For xref:get-started:cluster-types/byoc/gcp/vpc-byo-gcp.adoc[BYOVPC clusters], you need a NAT subnet with `purpose` set to `PRIVATE_SERVICE_CONNECT`. You also need to create VPC firewall rules to allow Private Service Connect traffic. You can use the `gcloud` CLI: -+ -NOTE: The firewall rules support up to 20 Redpanda brokers. If you have more than 20 brokers, or for help enabling Private Service Connect, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^]. -+ -[,bash] ----- -gcloud compute networks subnets create \ - --project= \ - --network= \ - --region= \ - --range= \ - --purpose=PRIVATE_SERVICE_CONNECT ----- -+ -[,bash] ----- -gcloud compute firewall-rules create redpanda-psc \ - --description="Allow access to Redpanda PSC endpoints" \ - --network="" \ - --project="" \ - --direction="INGRESS" \ - --target-tags="redpanda-node" \ - --source-ranges="10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10" \ - --allow="tcp:30181,tcp:30282,tcp:30292,tcp:31004,tcp:31082-31101,tcp:31182-31201,tcp:31282-31301,tcp:32092-32111,tcp:32192-32211,tcp:32292-32311" ----- -+ -Provide your values for the following placeholders: -+ -- ``: The name of the NAT subnet. -- ``: The host GCP project ID. -- ``: The name of the VPC being used for your Redpanda Cloud cluster. -- ``: The region of the Redpanda Cloud cluster. -- ``: The CIDR range of the subnet. The mask should be at least `/29`. Each Private Service Connect connection takes up one IP address from the NAT subnet, so the CIDR must be able to accommodate all projects from which connections to the service attachment will be issued. -+ -See the GCP documentation for https://cloud.google.com/vpc/docs/configure-private-service-connect-producer#add-subnet-psc[creating a subnet for Private Service Connect^]. -. For the accepted consumers list, you need the GCP project IDs from which incoming connections will be accepted. -. It may take several minutes for your cluster to update. When the update is complete, the Private Service Connect status in **Cluster settings** changes from **In progress** to **Enabled**. - -=== Deploy consumer-side resources - -For each VPC network, you must complete the following steps to successfully connect to the service and use Kafka API and other Redpanda services such as HTTP Proxy. - -. In **Cluster settings**, copy the **Service attachment URL** under **Private Service Connect**. Use this URL to create the Private Service Connect endpoint in GCP. - -. Create a private DNS zone. Use the cluster **DNS zone** value as the DNS name. -+ -[,bash] ----- -gcloud dns --project= managed-zones create --description="" --dns-name="" --visibility="private" --networks="" ----- - -. In the newly-created DNS zone, create a wildcard DNS record using the cluster **DNS record** value. -+ -[,bash] ----- -gcloud dns --project= record-sets create '*.' --zone="" --type="A" --ttl="300" --rrdatas="" ----- - -. Confirm that your GCP VPC firewall allows traffic to and from the Private Service Connect forwarding rule IP address, on the expected ports. - -== Access Redpanda services through VPC endpoint - -After you have enabled Private Service Connect for your cluster, your connection URLs are available in the *How to Connect* section of the cluster overview in the Redpanda Cloud UI. - -include::networking:partial$private-links-access-rp-services-through-vpc.adoc[] - -== Test the connection - -You can test the connection to the endpoint service from any VM or container in the consumer VPC. If configuring a client isn't possible right away, you can do these checks using `rpk` or cURL: - -include::networking:partial$private-links-test-connection.adoc[] - -== Disable endpoint service - -In **Cluster settings**, click **Disable**. Existing connections are closed after GCP Private Service Connect is disabled. To connect using Private Service Connect again, you must re-enable the service. \ No newline at end of file +include::networking:partial$psc-ui.adoc[] \ No newline at end of file diff --git a/modules/networking/pages/dedicated/gcp/configure-psc-in-api.adoc b/modules/networking/pages/dedicated/gcp/configure-psc-in-api.adoc new file mode 100644 index 000000000..cb9d29517 --- /dev/null +++ b/modules/networking/pages/dedicated/gcp/configure-psc-in-api.adoc @@ -0,0 +1,158 @@ += Configure GCP Private Service Connect with the Cloud API +:description: Set up GCP Private Service Connect to securely access Redpanda Cloud. +:env-dedicated: true + +include::networking:partial$psc-api.adoc[] + +== Create a new cluster with Private Service Connect + +. In the https://cloud.redpanda.com/[Redpanda Cloud UI], go to **Resource groups** and select the resource group in which you want to create a cluster. ++ +Copy and store the resource group ID (UUID) from the URL in the browser. ++ +[,bash] +---- +export RESOURCE_GROUP_ID= +---- + +. Create VPC firewall rules to allow Private Service Connect traffic. Use the `gcloud` CLI to create the firewall rules: ++ +NOTE: The firewall rules support up to 20 Redpanda brokers. If you have more than 20 brokers, or for help enabling Private Service Connect, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^]. ++ +``` +gcloud compute firewall-rules create redpanda-psc \ + --description="Allow access to Redpanda PSC endpoints" \ + --network="" \ + --project="" \ + --direction="INGRESS" \ + --target-tags="redpanda-node" \ + --source-ranges="10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10" \ + --allow="tcp:30181,tcp:30282,tcp:30292,tcp:31004,tcp:31082-31101,tcp:31182-31201,tcp:31282-31301,tcp:32092-32111,tcp:32192-32211,tcp:32292-32311" +``` + +. Make a request to the xref:api:ROOT:cloud-controlplane-api.adoc#post-/v1/networks[`POST /v1/networks`] endpoint to create a network. ++ +[,bash] +---- +network_post_body=`cat << EOF +{ + "cloud_provider": "CLOUD_PROVIDER_GCP", + "cluster_type": "DEDICATED", + "name": "", + "resource_group_id": "$RESOURCE_GROUP_ID", + "region": "", + "customer_managed_resources": { + "gcp": { + "network_name": "", + "network_project_id": "", + "management_bucket": { "name" : "" } + } + } +} +EOF` + +curl -vv -X POST \ +-H "Content-Type: application/json" \ +-H "Authorization: Bearer $AUTH_TOKEN" \ +-d "$network_post_body" $PUBLIC_API_ENDPOINT/v1/networks +---- ++ +Replace the following placeholder variables for the request body: ++ +-- +- ``: The name for the network. +- ``: The GCP region where the network will be created. +- ``: The ID of the GCP project where your VPC is created. +- ``: The name of your VPC. +- ``: The name of the Google Storage bucket you created for the cluster. +-- + + +. Store the network ID (`metadata.network_id`) returned in the response to the Create Network request. ++ +[,bash] +---- +export NETWORK_ID= +---- + +. Make a request to the xref:api:ROOT:cloud-controlplane-api.adoc#post-/v1/clusters[`POST /v1/clusters`] endpoint to create a Redpanda Cloud cluster with Private Service Connect enabled. ++ +[,bash] +---- +export CLUSTER_POST_BODY=`cat << EOF +{ + "cloud_provider": "CLOUD_PROVIDER_GCP", + "connection_type": "CONNECTION_TYPE_PRIVATE", + "type": "DEDICATED", + "name": "", + "resource_group_id": "$RESOURCE_GROUP_ID", + "network_id": "$NETWORK_ID", + "region": "", + "zones": , + "throughput_tier": "", + "redpanda_version": "", + "gcp_private_service_connect": { + "enabled": true, + "consumer_accept_list": + }, +} +EOF` + +curl -vv -X POST \ +-H "Content-Type: application/json" \ +-H "Authorization: Bearer $AUTH_TOKEN" \ +-d "$CLUSTER_POST_BODY" $PUBLIC_API_ENDPOINT/v1/clusters +---- ++ +-- +- ``: Provide a name for the new cluster. +- ``: Choose a GCP region where the network will be created. +- ``: Provide the list of GCP zones where the brokers will be deployed. Format: `["", "", ""]` +- ``: Choose a Redpanda Cloud cluster tier. For example, `tier-1-gcp-v2-x86`. +- ``: Choose the Redpanda Cloud version. +- ``: The list of IDs of GCP projects from which Private Service Connect connection requests are accepted. Format: `[{"source": ""}, {"source": ""}, {"source": ""}]` +-- + +== Enable Private Service Connect on an existing cluster + +CAUTION: As soon as Private Service Connect is available on your VPC, all communication on existing Redpanda bootstrap server and broker ports is interrupted due to the change on the private DNS resolution. Make sure all applications running in your VPC are ready to start using the corresponding Private Service Connect ports. + +. In the Redpanda Cloud UI, go to the cluster overview and copy the cluster ID from the **Details** section. ++ +[,bash] +---- +export CLUSTER_ID= +---- + + +. Make a xref:api:ROOT:cloud-controlplane-api.adoc#patch-/v1/clusters/-cluster.id-[`PATCH /v1/clusters/{cluster.id}`] request to update the cluster to enable Private Service Connect. ++ +[,bash] +---- +CLUSTER_PATCH_BODY=`cat << EOF +{ + "gcp_private_service_connect": { + "enabled": true, + "consumer_accept_list": + } +} +EOF` +curl -v -X PATCH \ +-H "Content-Type: application/json" \ +-H "Authorization: Bearer $AUTH_TOKEN" \ +-d "$CLUSTER_PATCH_BODY" $PUBLIC_API_ENDPOINT/v1/clusters/$CLUSTER_ID +---- ++ +Replace the following placeholder: ++ +``: a JSON list specifying the projects from which incoming connections will be accepted. All other sources. For example, `[{"source": "consumer-project-ID-1"},{"source": "consumer-project-ID-2"}]`. ++ +Wait for the cluster to apply the new configuration (around 15 minutes). The Private Service Connect attachment is available when the cluster update is complete. To monitor the service attachment creation, run the following `gcloud` command with the project ID: ++ +[,bash] +---- +gcloud compute service-attachments list --project '' +---- + + +include::networking:partial$psc-api2.adoc[] \ No newline at end of file diff --git a/modules/networking/pages/dedicated/gcp/configure-psc-in-ui.adoc b/modules/networking/pages/dedicated/gcp/configure-psc-in-ui.adoc new file mode 100644 index 000000000..7ae01b511 --- /dev/null +++ b/modules/networking/pages/dedicated/gcp/configure-psc-in-ui.adoc @@ -0,0 +1,5 @@ += Configure GCP Private Service Connect in the Cloud UI +:description: Set up GCP Private Service Connect in the Redpanda Cloud UI. +:env-dedicated: true + +include::networking:partial$psc-ui.adoc[] \ No newline at end of file diff --git a/modules/networking/pages/dedicated/gcp/vpc-peering-gcp.adoc b/modules/networking/pages/dedicated/gcp/vpc-peering-gcp.adoc index 8a300debb..6721375e9 100644 --- a/modules/networking/pages/dedicated/gcp/vpc-peering-gcp.adoc +++ b/modules/networking/pages/dedicated/gcp/vpc-peering-gcp.adoc @@ -1,8 +1,6 @@ = Add a Dedicated VPC Peering Connection :description: Use the Redpanda Cloud UI to set up VPC peering. -*Update for GCP* - A VPC peering connection is a networking connection between two VPCs. This connection allows the VPCs to communicate with each other as if they were within the same network. A route table routes traffic between the two VPCs using private IPv4 addresses. NOTE: Traffic is _not_ routed over the public internet. diff --git a/modules/networking/pages/gcp-private-service-connect.adoc b/modules/networking/pages/gcp-private-service-connect.adoc index eefc43c88..ebfb3b353 100644 --- a/modules/networking/pages/gcp-private-service-connect.adoc +++ b/modules/networking/pages/gcp-private-service-connect.adoc @@ -1,40 +1,9 @@ = Configure GCP Private Service Connect with the Cloud API :description: Set up GCP Private Service Connect to securely access Redpanda Cloud. :page-aliases: deploy:deployment-option/cloud/gcp-private-service-connect.adoc +:env-byoc: true -[NOTE] -==== - -* This guide is for configuring GCP Private Service Connect using the Redpanda Cloud API. To configure and manage Private Service Connect on an existing public cluster, you must use the Cloud API. See xref:networking:configure-private-service-connect-in-cloud-ui.adoc[Configure Private Service Connect in the Cloud UI] to set up the endpoint service using the Redpanda Cloud UI. -* The latest version of Redpanda GCP Private Service Connect (available March, 2025) supports AZ affinity. This allows requests from Private Service Connect endpoints to stay within the same availability zone, avoiding additional networking costs. -* DEPRECATION: The original Redpanda GCP Private Service Connect is deprecated and will be removed in a future release. For more information, see xref:manage:maintenance.adoc#deprecated-features[Deprecated features]. -==== - -The Redpanda GCP Private Service Connect service provides secure access to Redpanda Cloud from your own VPC. Traffic over Private Service Connect does not go through the public internet because a Private Service Connect connection is treated as its own private GCP service. While your VPC has access to the Redpanda VPC, Redpanda cannot access your VPC. - -As of February 2025 ability to allow requests from Private Service Connect endpoints to stay within the same AZ, avoiding additional networking costs. To upgrade, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda Support^]. - -Consider using Private Service Connect if you have multiple VPCs and could benefit from a more simplified approach to network management. - -[NOTE] -==== -* Each client VPC can have one endpoint connected to Private Service Connect. -* Private Service Connect allows overlapping xref:networking:cidr-ranges.adoc[CIDR ranges] in VPC networks. -* The number of connections is limited only by your Redpanda usage tier. Private Service Connect does not add extra connection limits. -* You control from which GCP projects connections are allowed. -==== - -After <>, you can <>, or you can <>. - -== Requirements - -* In this guide, you use the xref:manage:api/cloud-api-overview.adoc[Redpanda Cloud API] to enable the Redpanda endpoint service for your clusters. Follow the steps on this page to <>. -* Use the https://cloud.google.com/sdk/docs/install[gcloud^] command-line interface (CLI) to create the consumer-side resources, such as a VPC and forwarding rule, or to modify existing resources to use the Private Service Connect attachment created for your cluster. - -== Get a Cloud API access token - -include::networking:partial$private-links-api-access-token.adoc[] - +include::networking:partial$psc-api.adoc[] == Create a new BYOVPC cluster with Private Service Connect @@ -288,33 +257,5 @@ Wait for the cluster to apply the new configuration (around 15 minutes). The Pri gcloud compute service-attachments list --project '' ---- -== Access Redpanda services through VPC endpoint - -After you have enabled Private Service Connect for your cluster, your connection URLs are available in the *How to Connect* section of the cluster overview in the Redpanda Cloud UI. - -include::networking:partial$private-links-access-rp-services-through-vpc.adoc[] - -== Test the connection - -You can test the Private Service Connect connection from any VM or container in the consumer VPC. If configuring a client isn't possible right away, you can do these checks using `rpk` or curl: - -include::networking:partial$private-links-test-connection.adoc[] +include::networking:partial$psc-api2.adoc[] -== Disable Private Service Connect - -Make a xref:api:ROOT:cloud-controlplane-api.adoc#patch-/v1/clusters/-cluster.id-[`PATCH /v1/clusters/{cluster.id}`] request to update the cluster to disable Private Service Connect. - -[,bash] ----- -CLUSTER_PATCH_BODY=`cat << EOF -{ - "gcp_private_service_connect": { - "enabled": false, - } -} -EOF` -curl -v -X PATCH \ --H "Content-Type: application/json" \ --H "Authorization: Bearer $AUTH_TOKEN" \ --d "$CLUSTER_PATCH_BODY" $PUBLIC_API_ENDPOINT/v1/clusters/$CLUSTER_ID ----- diff --git a/modules/networking/partials/psc-api.adoc b/modules/networking/partials/psc-api.adoc new file mode 100644 index 000000000..5208add9a --- /dev/null +++ b/modules/networking/partials/psc-api.adoc @@ -0,0 +1,28 @@ +[NOTE] +==== + +* This guide is for configuring GCP Private Service Connect using the Redpanda Cloud API. To configure and manage Private Service Connect on an existing *public* cluster, you must use the Cloud API. See xref:networking:configure-private-service-connect-in-cloud-ui.adoc[Configure Private Service Connect in the Cloud UI] to set up the endpoint service using the Redpanda Cloud UI. +* The latest version of Redpanda GCP Private Service Connect (available March, 2025) supports AZ affinity. This allows requests from Private Service Connect endpoints to stay within the same availability zone, avoiding additional networking costs. +* DEPRECATION: The original Redpanda GCP Private Service Connect is deprecated and will be removed in a future release. For more information, see xref:manage:maintenance.adoc#deprecated-features[Deprecated features]. +==== + +The Redpanda GCP Private Service Connect service provides secure access to Redpanda Cloud from your own VPC. Traffic over Private Service Connect does not go through the public internet because a Private Service Connect connection is treated as its own private GCP service. While your VPC has access to the Redpanda VPC, Redpanda cannot access your VPC. + +Consider using Private Service Connect if you have multiple VPCs and could benefit from a more simplified approach to network management. + +[NOTE] +==== +* Each client VPC can have one endpoint connected to Private Service Connect. +* Private Service Connect allows overlapping xref:networking:cidr-ranges.adoc[CIDR ranges] in VPC networks. +* The number of connections is limited only by your Redpanda usage tier. Private Service Connect does not add extra connection limits. +* You control from which GCP projects connections are allowed. +==== + +== Requirements + +* In this guide, you use the xref:manage:api/cloud-api-overview.adoc[Redpanda Cloud API] to enable the Redpanda endpoint service for your clusters. Follow the steps on this page to <>. +* Use the https://cloud.google.com/sdk/docs/install[gcloud^] command-line interface (CLI) to create the consumer-side resources, such as a VPC and forwarding rule, or to modify existing resources to use the Private Service Connect attachment created for your cluster. + +== Get a Cloud API access token + +include::networking:partial$private-links-api-access-token.adoc[] \ No newline at end of file diff --git a/modules/networking/partials/psc-api2.adoc b/modules/networking/partials/psc-api2.adoc new file mode 100644 index 000000000..3015f1a07 --- /dev/null +++ b/modules/networking/partials/psc-api2.adoc @@ -0,0 +1,52 @@ +== Deploy consumer-side resources + +For each VPC network, you must complete the following steps to successfully connect to the service and use Kafka API and other Redpanda services such as HTTP Proxy. + +. In **Cluster settings**, copy the **Service attachment URL** under **Private Service Connect**. Use this URL to create the Private Service Connect endpoint in GCP. + +. Create a private DNS zone. Use the cluster **DNS zone** value as the DNS name. ++ +[,bash] +---- +gcloud dns --project= managed-zones create --description="" --dns-name="" --visibility="private" --networks="" +---- + +. In the newly-created DNS zone, create a wildcard DNS record using the cluster **DNS record** value. ++ +[,bash] +---- +gcloud dns --project= record-sets create '*.' --zone="" --type="A" --ttl="300" --rrdatas="" +---- + +. Confirm that your GCP VPC firewall allows traffic to and from the Private Service Connect forwarding rule IP address, on the expected ports. + +== Access Redpanda services through VPC endpoint + +After you have enabled Private Service Connect for your cluster, your connection URLs are available in the *How to Connect* section of the cluster overview in the Redpanda Cloud UI. + +include::networking:partial$private-links-access-rp-services-through-vpc.adoc[] + +== Test the connection + +You can test the Private Service Connect connection from any VM or container in the consumer VPC. If configuring a client isn't possible right away, you can do these checks using `rpk` or curl: + +include::networking:partial$private-links-test-connection.adoc[] + +== Disable Private Service Connect + +Make a xref:api:ROOT:cloud-controlplane-api.adoc#patch-/v1/clusters/-cluster.id-[`PATCH /v1/clusters/{cluster.id}`] request to update the cluster to disable Private Service Connect. + +[,bash] +---- +CLUSTER_PATCH_BODY=`cat << EOF +{ + "gcp_private_service_connect": { + "enabled": false, + } +} +EOF` +curl -v -X PATCH \ +-H "Content-Type: application/json" \ +-H "Authorization: Bearer $AUTH_TOKEN" \ +-d "$CLUSTER_PATCH_BODY" $PUBLIC_API_ENDPOINT/v1/clusters/$CLUSTER_ID +---- \ No newline at end of file diff --git a/modules/networking/partials/psc-ui.adoc b/modules/networking/partials/psc-ui.adoc new file mode 100644 index 000000000..d5d9d41fd --- /dev/null +++ b/modules/networking/partials/psc-ui.adoc @@ -0,0 +1,108 @@ + +[NOTE] +==== + +* This guide is for configuring GCP Private Service Connect using the Redpanda Cloud UI. To configure and manage Private Service Connect on an existing *public* cluster, you must use the xref:networking:gcp-private-service-connect.adoc[Cloud API for BYOC] or the xref:networking:dedicated/gcp/configure-psc-in-api.adoc[Cloud API for Dedicated]. +* The latest version of Redpanda GCP Private Service Connect (available March, 2025) supports AZ affinity. This allows requests from Private Service Connect endpoints to stay within the same availability zone, avoiding additional networking costs. +* DEPRECATION: The original Redpanda GCP Private Service Connect is deprecated and will be removed in a future release. For more information, see xref:manage:maintenance.adoc#deprecated-features[Deprecated features]. +==== + + +The Redpanda GCP Private Service Connect service provides secure access to Redpanda Cloud from your own VPC. Traffic over Private Service Connect does not go through the public internet because these connections are treated as their own private GCP service. While your VPC has access to the Redpanda VPC, Redpanda cannot access your VPC. + +Consider using the endpoint services if you have multiple VPCs and could benefit from a more simplified approach to network management. + +[NOTE] +==== +* Each client VPC can have one endpoint connected to Private Service Connect. +* Private Service Connect allows overlapping xref:networking:cidr-ranges.adoc[CIDR ranges] in VPC networks. +* The number of connections is limited only by your Redpanda usage tier. Private Service Connect does not add extra connection limits. +* You control from which GCP projects connections are allowed. +==== + +== Requirements + +* Use the https://cloud.google.com/sdk/docs/install[gcloud^] command-line interface (CLI) to create the consumer-side resources, such as a client VPC and forwarding rule, or to modify existing resources to use the Private Service Connect attachment created for your cluster. +* The client VPC must be in the same region as your Redpanda cluster. + +== Enable endpoint service for existing clusters + +. In the Redpanda Cloud UI, open your https://cloud.redpanda.com/clusters[cluster^], and click **Cluster settings**. +. Under Private Service Connect, click **Enable**. +ifdef::env-byoc[] +. For xref:get-started:cluster-types/byoc/gcp/vpc-byo-gcp.adoc[BYOVPC clusters], you need a NAT subnet with `purpose` set to `PRIVATE_SERVICE_CONNECT`. You also need to create VPC firewall rules to allow Private Service Connect traffic. You can use the `gcloud` CLI: ++ +NOTE: The firewall rules support up to 20 Redpanda brokers. If you have more than 20 brokers, or for help enabling Private Service Connect, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^]. ++ +[,bash] +---- +gcloud compute networks subnets create \ + --project= \ + --network= \ + --region= \ + --range= \ + --purpose=PRIVATE_SERVICE_CONNECT +---- ++ +[,bash] +---- +gcloud compute firewall-rules create redpanda-psc \ + --description="Allow access to Redpanda PSC endpoints" \ + --network="" \ + --project="" \ + --direction="INGRESS" \ + --target-tags="redpanda-node" \ + --source-ranges="10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10" \ + --allow="tcp:30181,tcp:30282,tcp:30292,tcp:31004,tcp:31082-31101,tcp:31182-31201,tcp:31282-31301,tcp:32092-32111,tcp:32192-32211,tcp:32292-32311" +---- ++ +Provide your values for the following placeholders: ++ +- ``: The name of the NAT subnet. +- ``: The host GCP project ID. +- ``: The name of the VPC being used for your Redpanda Cloud cluster. +- ``: The region of the Redpanda Cloud cluster. +- ``: The CIDR range of the subnet. The mask should be at least `/29`. Each Private Service Connect connection takes up one IP address from the NAT subnet, so the CIDR must be able to accommodate all projects from which connections to the service attachment will be issued. ++ +See the GCP documentation for https://cloud.google.com/vpc/docs/configure-private-service-connect-producer#add-subnet-psc[creating a subnet for Private Service Connect^]. +endif::[] +. For the accepted consumers list, you need the GCP project IDs from which incoming connections will be accepted. +. It may take several minutes for your cluster to update. When the update is complete, the Private Service Connect status in **Cluster settings** changes from **In progress** to **Enabled**. + +== Deploy consumer-side resources + +For each VPC network, you must complete the following steps to successfully connect to the service and use Kafka API and other Redpanda services such as HTTP Proxy. + +. In **Cluster settings**, copy the **Service attachment URL** under **Private Service Connect**. Use this URL to create the Private Service Connect endpoint in GCP. + +. Create a private DNS zone. Use the cluster **DNS zone** value as the DNS name. ++ +[,bash] +---- +gcloud dns --project= managed-zones create --description="" --dns-name="" --visibility="private" --networks="" +---- + +. In the newly-created DNS zone, create a wildcard DNS record using the cluster **DNS record** value. ++ +[,bash] +---- +gcloud dns --project= record-sets create '*.' --zone="" --type="A" --ttl="300" --rrdatas="" +---- + +. Confirm that your GCP VPC firewall allows traffic to and from the Private Service Connect forwarding rule IP address, on the expected ports. + +== Access Redpanda services through VPC endpoint + +After you have enabled Private Service Connect for your cluster, your connection URLs are available in the *How to Connect* section of the cluster overview in the Redpanda Cloud UI. + +include::networking:partial$private-links-access-rp-services-through-vpc.adoc[] + +== Test the connection + +You can test the connection to the endpoint service from any VM or container in the consumer VPC. If configuring a client isn't possible right away, you can do these checks using `rpk` or cURL: + +include::networking:partial$private-links-test-connection.adoc[] + +== Disable endpoint service + +In **Cluster settings**, click **Disable**. Existing connections are closed after GCP Private Service Connect is disabled. To connect using Private Service Connect again, you must re-enable the service. \ No newline at end of file