Skip to content
Merged
2 changes: 1 addition & 1 deletion local-antora-playbook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ content:
- url: https://github.com/redpanda-data/docs
branches: [v/*, api, shared, site-search,'!v-end-of-life/*']
- url: https://github.com/redpanda-data/cloud-docs
branches: 'main'
branches: 'DOC-1222-Document-feature-Iceberg-on-Cloud-BYOVPC-GA-AWS-GCP'
- url: https://github.com/redpanda-data/redpanda-labs
branches: main
start_paths: [docs,'*/docs']
Expand Down
13 changes: 9 additions & 4 deletions modules/manage/partials/iceberg/use-iceberg-catalogs.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@ ifndef::env-cloud[:about-iceberg-doc: manage:iceberg/topic-iceberg-integration.a

To read from the Redpanda-generated xref:{about-iceberg-doc}[Iceberg table], your Iceberg-compatible client or tool needs access to the catalog to retrieve the table metadata and know the current state of the table. The catalog provides the current table metadata, which includes locations for all the table's data files. You can configure Redpanda to either connect to a REST-based catalog, or use a filesystem-based catalog.

ifdef::env-cloud[]
NOTE: The Iceberg integration for Redpanda Cloud is a beta feature. It is not supported for production deployments. To configure REST catalog authentication for use with Iceberg topics in your cloud cluster, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda support^].
endif::[]

For production deployments, Redpanda recommends using an external REST catalog to manage Iceberg metadata. This enables built-in table maintenance, safely handles multiple engines and tools accessing tables at the same time, facilitates data governance, and maximizes data discovery. However, if it is not possible to use a REST catalog, you may use the filesystem-based catalog (`object_storage` catalog type), which does not require you to maintain a separate service to access the Iceberg data. In either case, you use the catalog to load, query, or refresh the Iceberg table as you produce to the Redpanda topic. See the documentation for your query engine or Iceberg-compatible tool for specific guidance on adding the Iceberg tables to your data warehouse or lakehouse using the catalog.

After you have selected a catalog type at the cluster level and xref:{about-iceberg-doc}#enable-iceberg-integration[enabled the Iceberg integration] for a topic, you cannot switch to another catalog type.
Expand All @@ -15,6 +11,15 @@ After you have selected a catalog type at the cluster level and xref:{about-iceb

Connect to an Iceberg REST catalog using the standard https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml[REST API^] supported by many catalog providers. Use this catalog integration type with REST-enabled Iceberg catalog services, such as https://docs.databricks.com/en/data-governance/unity-catalog/index.html[Databricks Unity^] and https://other-docs.snowflake.com/en/opencatalog/overview[Snowflake Open Catalog^].

ifndef::env-cloud[]
[NOTE]
====
For BYOVPC clusters, you must enable secrets management, which allows you to store and use secrets in your cluster's Iceberg catalog authentication properties.

Secrets management is enabled by default for AWS if you follow the guide to xref:get-started:cluster-types/byoc/aws/vpc-byo-aws.adoc[creating a new BYOVPC cluster]. For GCP, follow the guides to enable secrets management for a xref:get-started:cluster-types/byoc/gcp/vpc-byo-gcp.adoc[new BYOVPC cluster] or an xref:get-started:cluster-types/byoc/gcp/enable-secrets-byovpc-gcp.adoc[existing BYOVPC cluster].
====
endif::[]

To connect to a REST catalog, set the following cluster configuration properties:

* config_ref:iceberg_catalog_type,true,properties/cluster-properties[`iceberg_catalog_type`]: `rest`
Expand Down