Conversation
| title: Iceberg Rest Catalogs | ||
| --- | ||
|
|
||
| The `iceberg` extension supports attaching Iceberg Rest Catalogs. Before attaching an Iceberg Rest Catalog, you must install the `iceberg` extension by following the instructions located in the [overview]({% link docs/stable/core_extensions/iceberg/overview.md %}). |
|
|
||
| If you are attaching to an Iceberg Rest Catalog managed by Amazon, please see the instructions for attaching to [Amazon S3 tables]({% link docs/stable/core_extensions/iceberg/amazon_s3_tables.md %}) or [Amazon Sagemaker Lakehouse]({% link docs/stable/core_extensions/iceberg/amazon_sagemaker_lakehouse.md %}). | ||
|
|
||
| For all other Iceberg Rest Catalogs, you can follow the instructions below. Please see the [Examples](#examples) section for questionsabout specific catalogs. |
There was a problem hiding this comment.
questionsabout -> questions about
|
|
||
| For all other Iceberg Rest Catalogs, you can follow the instructions below. Please see the [Examples](#examples) section for questionsabout specific catalogs. | ||
|
|
||
| Most Iceberg Rest Catalogs authenticate via Oauth2. You can use the existing DuckDB secret workflow to create the oauth secret. |
There was a problem hiding this comment.
Oauth2 -> OAuth2
oauth -> oauth2 (or OAuth2)
There was a problem hiding this comment.
Also, "oauth secret", isn't a thing anymore right? We went for ICEBERG secret instead
| ); | ||
| ``` | ||
|
|
||
| If you already have a token, you can pass it directly to your `CREATE SECRET` statement |
| ); | ||
| ``` | ||
|
|
||
| You can attach the iceberg catalog with the following `Attach` statement. |
There was a problem hiding this comment.
iceberg -> Iceberg
Attach -> ATTACH, also we should probably hyperlink to the attach/detach documentation on it
| ATTACH '⟨warehouse-name⟩' AS iceberg_catalog ( | ||
| TYPE iceberg, | ||
| SECRET iceberg_secret, -- pass a specific secret name to prevent ambiguity | ||
| ENDPOINT rest-catalog-endpoint⟩ |
There was a problem hiding this comment.
rest-catalog-endpoint⟩ -> ⟨rest-catalog-endpoint⟩
| ```sql | ||
| ATTACH '⟨warehouse-name⟩' AS iceberg_catalog ( | ||
| TYPE iceberg, | ||
| SECRET iceberg_secret, -- pass a specific secret name to prevent ambiguity |
There was a problem hiding this comment.
It's not really for ambiguity, I think?
It's more that authorization options can be provided directly inside the ATTACH (in the same way as the "create secret" options), but for reusability/deduplication the secret can be made beforehand and reused in the attach afterwards
There was a problem hiding this comment.
Also that, but I can imagine users attach multiple catalogs that all require a different secret. I think it's best to go over all the options in a separate section
| Then, attach the catalog with the following commands. | ||
|
|
||
| ```sql | ||
| ATTACH '⟨warehouse-name⟩' AS my_r2_catalog ( |
There was a problem hiding this comment.
If we are omitting the secret name, relying on the lookup of the default ICEBERG secret, we should probably mention that beforehand, or link to an explanation on this behavior
Tishj
left a comment
Tishj
left a comment
There was a problem hiding this comment.
Thanks, great work!
I did have a couple comments
Perhaps we want to make a special section about the ATTACH options, as there is a bit of complexity for the available options and the default behaviors of it (duckdb/duckdb-iceberg#142)
|
Thanks! |
| ); | ||
| ``` | ||
|
|
||
| ## Limitations |
There was a problem hiding this comment.
nit: wydt of adding a sentence about catalog refreshing not being supported right now. the work around is reattach
similar to duckdb/duckdb-iceberg#247
4 participants
Describe create create and attach workflow for rest catalogs.
Also remove references to experimental and
FORCE INSTALL