Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -415,9 +415,11 @@
*** xref:manage:iceberg/about-iceberg-topics.adoc[]
*** xref:manage:iceberg/specify-iceberg-schema.adoc[]
*** xref:manage:iceberg/use-iceberg-catalogs.adoc[]
*** xref:manage:iceberg/rest-catalog/index.adoc[]
**** xref:manage:iceberg/iceberg-topics-aws-glue.adoc[AWS Glue]
**** xref:manage:iceberg/iceberg-topics-databricks-unity.adoc[Databricks Unity Catalog]
**** xref:manage:iceberg/redpanda-topics-iceberg-snowflake-catalog.adoc[Snowflake and Open Catalog]
*** xref:manage:iceberg/query-iceberg-topics.adoc[]
*** xref:manage:iceberg/iceberg-topics-databricks-unity.adoc[Query Iceberg Topics with Databricks Unity Catalog]
*** xref:manage:iceberg/redpanda-topics-iceberg-snowflake-catalog.adoc[Query Iceberg Topics with Snowflake and Open Catalog]
** xref:manage:schema-reg/index.adoc[Schema Registry]
*** xref:manage:schema-reg/schema-reg-overview.adoc[]
*** xref:manage:schema-reg/schema-reg-ui.adoc[]
Expand Down
8 changes: 8 additions & 0 deletions modules/get-started/pages/whats-new-cloud.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@

This page lists new features added to Redpanda Cloud.

== August 2025

=== Iceberg topics with AWS Glue

A new xref:manage:iceberg/iceberg-topics-aws-glue.adoc[integration with AWS Glue Data Catalog] allows you to add Redpanda topics as Iceberg tables in your data lakehouse. The AWS Glue catalog integration is available in BYOC clusters with Redpanda version 25.2 and later.

See xref:manage:iceberg/rest-catalog/index.adoc[] for supported Iceberg REST catalog integrations.

== July 2025

=== Iceberg topics in Redpanda Cloud: GA
Expand Down
6 changes: 6 additions & 0 deletions modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
= Query Iceberg Topics using AWS Glue
:description: Add Redpanda topics as Iceberg tables that you can access through the AWS Glue Data Catalog.
:page-categories: Iceberg, Tiered Storage, Management, High Availability, Data Replication, Integration
:page-beta: true

include::ROOT:manage:iceberg/iceberg-topics-aws-glue.adoc[tag=single-source]
3 changes: 3 additions & 0 deletions modules/manage/pages/iceberg/rest-catalog/index.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
= Integrate with REST Catalogs
:description: Integrate Redpanda topics with managed Iceberg REST Catalogs.
:page-layout: index
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,78 @@
:page-aliases: deploy:deployment-option/cloud/configure-private-service-connect-in-cloud-ui.adoc
:env-byoc: true

include::networking:partial$psc-ui.adoc[]
[NOTE]
====

* This guide is for configuring GCP Private Service Connect using the Redpanda Cloud UI. To configure and manage Private Service Connect on an existing cluster with *public* networking, you must use the xref:networking:gcp-private-service-connect.adoc[Cloud API for BYOC] or the xref:networking:dedicated/gcp/configure-psc-in-api.adoc[Cloud API for Dedicated].
* The latest version of Redpanda GCP Private Service Connect (available March, 2025) supports zone affinity. This allows requests from Private Service Connect endpoints to stay within the same availability zone, avoiding additional networking costs.
* DEPRECATION: The original Redpanda GCP Private Service Connect is deprecated and will be removed in a future release. For more information, see xref:manage:maintenance.adoc#deprecated-features[Deprecated features].
====


The Redpanda GCP Private Service Connect service provides secure access to Redpanda Cloud from your own VPC network. Traffic over Private Service Connect does not go through the public internet because these connections are treated as their own private GCP service. While your VPC network has access to the Redpanda VPC network, Redpanda cannot access your VPC network.

Consider using Private Service Connect if you have multiple VPC networks and could benefit from a more simplified approach to network management.

[NOTE]
====
* Each consumer VPC network can have one Private Service Connect endpoint connected to the Redpanda service attachment.
* Private Service Connect allows overlapping xref:networking:cidr-ranges.adoc[CIDR ranges] in VPC networks.
* The number of connections is limited only by your Redpanda xref:reference:tiers/index.adoc[usage tier]. Private Service Connect does not add extra connection limits.
* You control from which GCP projects connections are allowed.
====

== Requirements

* Use the https://cloud.google.com/sdk/docs/install[gcloud^] command-line interface (CLI) to create the consumer-side resources, such as a consumer VPC network and forwarding rule, or to modify existing resources to use the Private Service Connect service attachment created for your cluster.
* The consumer VPC network must be in the same region as your Redpanda cluster.

== Enable Private Service Connect for existing clusters

. In the Redpanda Cloud UI, open your https://cloud.redpanda.com/clusters[cluster^], and click **Cluster settings**.
. Under Private Service Connect, click **Enable**.
ifdef::env-byoc[]
. For xref:get-started:cluster-types/byoc/gcp/vpc-byo-gcp.adoc[BYOVPC clusters], you need a PSC NAT subnet with `purpose` set to `PRIVATE_SERVICE_CONNECT`. You also need to create VPC network firewall rules to allow Private Service Connect traffic. You can use the `gcloud` CLI:
+
NOTE: The firewall rules support up to 20 Redpanda brokers. If you have more than 20 brokers, or for help enabling Private Service Connect, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^].
+
[source,bash]
----
gcloud compute networks subnets create <psc-nat-subnet-name> \
--project=<host-project-id> \
--network=<shared-vpc-name> \
--region=<region> \
--range=<psc-nat-subnet-range> \
--purpose=PRIVATE_SERVICE_CONNECT
----
+
[source,bash]
----
gcloud compute firewall-rules create redpanda-psc-ingress \
--description="Allow access to Redpanda PSC endpoints" \
--network="<shared-vpc-name>" \
--project="<host-project-id>" \
--direction="INGRESS" \
--target-tags="redpanda-node" \
--source-ranges="10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10" \
--allow="tcp:30181,tcp:30282,tcp:30292,tcp:31004,tcp:31082-31101,tcp:31182-31201,tcp:31282-31301,tcp:32092-32111,tcp:32192-32211,tcp:32292-32311"
----
+
Provide your values for the following placeholders:
+
- `<psc-nat-subnet-name>`: The name of the PSC NAT subnet.
- `<host-project-id>`: The host GCP project ID.
- `<shared-vpc-name>`: The name of the VPC network being used for your Redpanda Cloud cluster.
- `<region>`: The region of the Redpanda Cloud cluster.
- `<psc-nat-subnet-range>`: The CIDR range of the subnet. The mask should be at least `/29`. Each Private Service Connect connection takes up one IP address from the PSC NAT subnet, so the CIDR must be able to accommodate all projects from which connections to the service attachment will be issued.
+
See the GCP documentation for https://cloud.google.com/vpc/docs/configure-private-service-connect-producer#add-subnet-psc[creating a subnet for Private Service Connect^].
endif::[]
. For the accepted consumers list, you need the GCP project IDs from which incoming connections will be accepted.
. It may take several minutes for your cluster to update. When the update is complete, the Private Service Connect status in **Cluster settings** changes from **In progress** to **Enabled**.

include::networking:partial$psc-ui.adoc[]

== Disable Private Service Connect

In **Cluster settings**, click **Disable**. Existing connections are closed after it is disabled. To connect using Private Service Connect again, you must re-enable it.
48 changes: 26 additions & 22 deletions modules/networking/pages/dedicated/gcp/configure-psc-in-api.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,26 +15,11 @@ Copy and store the resource group ID (UUID) from the URL in the browser.
export RESOURCE_GROUP_ID=<uuid>
----

. Create VPC firewall rules to allow Private Service Connect traffic. Use the `gcloud` CLI to create the firewall rules:
+
NOTE: The firewall rules support up to 20 Redpanda brokers. If you have more than 20 brokers, or for help enabling Private Service Connect, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^].
+
```
gcloud compute firewall-rules create redpanda-psc \
--description="Allow access to Redpanda PSC endpoints" \
--network="<shared-vpc-name>" \
--project="<host-project-id>" \
--direction="INGRESS" \
--target-tags="redpanda-node" \
--source-ranges="10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10" \
--allow="tcp:30181,tcp:30282,tcp:30292,tcp:31004,tcp:31082-31101,tcp:31182-31201,tcp:31282-31301,tcp:32092-32111,tcp:32192-32211,tcp:32292-32311"
```

. Make a request to the xref:api:ROOT:cloud-controlplane-api.adoc#post-/v1/networks[`POST /v1/networks`] endpoint to create a network.
+
[,bash]
----
network_post_body=`cat << EOF
NETWORK_POST_BODY=`cat << EOF
{
"cloud_provider": "CLOUD_PROVIDER_GCP",
"cluster_type": "TYPE_DEDICATED",
Expand All @@ -54,7 +39,7 @@ EOF`
curl -vv -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-d "$network_post_body" $PUBLIC_API_ENDPOINT/v1/networks
-d "$NETWORK_POST_BODY" $PUBLIC_API_ENDPOINT/v1/networks
----
+
Replace the following placeholder variables for the request body:
Expand Down Expand Up @@ -94,7 +79,7 @@ export CLUSTER_POST_BODY=`cat << EOF
"gcp_private_service_connect": {
"enabled": true,
"consumer_accept_list": <consumer-accept-list>
},
}
}
EOF`

Expand All @@ -110,7 +95,7 @@ curl -vv -X POST \
- `<zones>`: Provide the list of GCP zones where the brokers will be deployed. Format: `["<zone 1>", "<zone 2>", "<zone N>"]`
- `<throughput-tier>`: Choose a Redpanda Cloud cluster tier. For example, `tier-1-gcp-v2-x86`.
- `<redpanda-version>`: Choose the Redpanda Cloud version.
- `<consumer-accept-list>`: The list of IDs of GCP projects from which Private Service Connect connection requests are accepted. Format: `[{"source": "<GCP-project-ID-1>"}, {"source": "<GCP-project-I-2>"}, {"source": "<GCP-project-ID-N>"}]`
- `<consumer-accept-list>`: The list of IDs of GCP projects from which Private Service Connect connection requests are accepted. Format: `[{"source": "<GCP-project-ID-1>"}, {"source": "<GCP-project-ID-2>"}, {"source": "<GCP-project-ID-N>"}]`
--

== Enable Private Service Connect on an existing cluster
Expand All @@ -133,7 +118,7 @@ CLUSTER_PATCH_BODY=`cat << EOF
{
"gcp_private_service_connect": {
"enabled": true,
"consumer_accept_list": <accept-list>
"consumer_accept_list": <consumer-accept-list>
}
}
EOF`
Expand All @@ -145,7 +130,7 @@ curl -v -X PATCH \
+
Replace the following placeholder:
+
`<accept-list>`: a JSON list specifying the projects from which incoming connections will be accepted. All other sources. For example, `[{"source": "consumer-project-ID-1"},{"source": "consumer-project-ID-2"}]`.
`<consumer-accept-list>`: A JSON list specifying the projects from which incoming connections will be accepted. All other sources are rejected. For example, `[{"source": "consumer-project-ID-1"},{"source": "consumer-project-ID-2"}]`.
+
Wait for the cluster to apply the new configuration (around 15 minutes). The Private Service Connect attachment is available when the cluster update is complete. To monitor the service attachment creation, run the following `gcloud` command with the project ID:
+
Expand All @@ -155,4 +140,23 @@ gcloud compute service-attachments list --project '<service-project-id>'
----


include::networking:partial$psc-api2.adoc[]
include::networking:partial$psc-ui.adoc[]

== Disable Private Service Connect

Make a xref:api:ROOT:cloud-controlplane-api.adoc#patch-/v1/clusters/-cluster.id-[`PATCH /v1/clusters/{cluster.id}`] request to update the cluster to disable Private Service Connect.

[,bash]
----
CLUSTER_PATCH_BODY=`cat << EOF
{
"gcp_private_service_connect": {
"enabled": false
}
}
EOF`
curl -v -X PATCH \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-d "$CLUSTER_PATCH_BODY" $PUBLIC_API_ENDPOINT/v1/clusters/$CLUSTER_ID
----
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,78 @@
:description: Set up GCP Private Service Connect in the Redpanda Cloud UI.
:env-dedicated: true

include::networking:partial$psc-ui.adoc[]
[NOTE]
====

* This guide is for configuring GCP Private Service Connect using the Redpanda Cloud UI. To configure and manage Private Service Connect on an existing cluster with *public* networking, you must use the xref:networking:gcp-private-service-connect.adoc[Cloud API for BYOC] or the xref:networking:dedicated/gcp/configure-psc-in-api.adoc[Cloud API for Dedicated].
* The latest version of Redpanda GCP Private Service Connect (available March, 2025) supports AZ affinity. This allows requests from Private Service Connect endpoints to stay within the same availability zone, avoiding additional networking costs.
* DEPRECATION: The original Redpanda GCP Private Service Connect is deprecated and will be removed in a future release. For more information, see xref:manage:maintenance.adoc#deprecated-features[Deprecated features].
====


The Redpanda GCP Private Service Connect service provides secure access to Redpanda Cloud from your VPC network. Traffic over Private Service Connect remains within GCP's private network, avoiding the public internet. Your VPC network can access the Redpanda VPC network, but Redpanda cannot access your VPC network.

Consider using Private Service Connect if you have multiple VPC networks and could benefit from a more simplified approach to network management.

[NOTE]
====
* Each consumer VPC network can have one Private Service Connect endpoint connected to the Redpanda service attachment.
* Private Service Connect allows overlapping xref:networking:cidr-ranges.adoc[CIDR ranges] in VPC networks.
* The number of connections is limited only by your Redpanda usage tier. Private Service Connect does not add extra connection limits.
* You control from which GCP projects connections are allowed.
====

== Prerequisites

* Use the https://cloud.google.com/sdk/docs/install[gcloud^] command-line interface (CLI) to create the consumer-side resources, such as a consumer VPC network and forwarding rule, or to modify existing resources to use the Private Service Connect service attachment created for your cluster.
* The consumer VPC network must be in the same region as your Redpanda cluster.

== Enable Private Service Connect for existing clusters

. In the Redpanda Cloud UI, open your https://cloud.redpanda.com/clusters[cluster^], and click **Cluster settings**.
. Under Private Service Connect, click **Enable**.
ifdef::env-byoc[]
. For xref:get-started:cluster-types/byoc/gcp/vpc-byo-gcp.adoc[BYOVPC clusters], you need a NAT subnet with `purpose` set to `PRIVATE_SERVICE_CONNECT`. You also need to create VPC network firewall rules to allow Private Service Connect traffic. You can use the `gcloud` CLI:
+
NOTE: The firewall rules support up to 20 Redpanda brokers. If you have more than 20 brokers, or for help enabling Private Service Connect, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^].
+
[,bash]
----
gcloud compute networks subnets create <psc-nat-subnet-name> \
--project=<host-project-id> \
--network=<shared-vpc-name> \
--region=<region> \
--range=<psc-nat-subnet-range> \
--purpose=PRIVATE_SERVICE_CONNECT
----
+
[,bash]
----
gcloud compute firewall-rules create redpanda-psc-ingress \
--description="Allow access to Redpanda PSC endpoints" \
--network="<shared-vpc-name>" \
--project="<host-project-id>" \
--direction="INGRESS" \
--target-tags="redpanda-node" \
--source-ranges="10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10" \
--allow="tcp:30181,tcp:30282,tcp:30292,tcp:31004,tcp:31082-31101,tcp:31182-31201,tcp:31282-31301,tcp:32092-32111,tcp:32192-32211,tcp:32292-32311"
Comment on lines +57 to +59
Copy link
Contributor

@paulohtb6 paulohtb6 Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are these fixed? if no, we should add them as replaceable snippet and provide these values as an example.

----
+
Provide your values for the following placeholders:
+
- `<psc-nat-subnet-name>`: The name of the NAT subnet.
- `<host-project-id>`: The host GCP project ID.
- `<shared-vpc-name>`: The name of the VPC network being used for your Redpanda Cloud cluster.
- `<region>`: The region of the Redpanda Cloud cluster.
- `<psc-nat-subnet-range>`: The CIDR range of the subnet. The mask should be at least `/29`. Each Private Service Connect connection takes up one IP address from the NAT subnet, so the CIDR must be able to accommodate all projects from which connections to the service attachment will be issued.
+
See the GCP documentation for https://cloud.google.com/vpc/docs/configure-private-service-connect-producer#add-subnet-psc[creating a subnet for Private Service Connect^].
endif::[]
. For the accepted consumers list, you need the GCP project IDs from which incoming connections will be accepted.
. It may take several minutes for your cluster to update. When the update is complete, the Private Service Connect status in **Cluster settings** changes from **In progress** to **Enabled**.

include::networking:partial$psc-ui.adoc[]

== Disable Private Service Connect

In **Cluster settings**, click **Disable**. Existing connections are closed after GCP Private Service Connect is disabled. To connect using Private Service Connect again, you must re-enable the service.
Loading