Document Updated: Added GCS related Bucket Properties for vending credentials. by sakshamratra0106 · Pull Request #3066 · apache/polaris

sakshamratra0106 · 2025-11-17T02:36:14Z

==> Document Updated on 18th Nov 2025

Added GCS related Properties for vending credentials.
Added Headers which were missing.
Added line width to 120 char.

==> Edit on 20th Nov 2025

As per the comments, For the changes "Added GCS related Properties for vending credentials" i have added a new sub page "Configuring Cloud Storage" under the existing page "Production Configuration"
Reverted above changes 2 and 3.

…h to 120 char

dimas-b

Thanks for your contribution, @sakshamratra0106 !

dimas-b · 2025-11-17T16:22:24Z

site/content/in-dev/unreleased/configuring-polaris-for-production.md

 - [ ] Use a durable metastore (JDBC + PostgreSQL)
 - [ ] Bootstrap valid realms in the metastore
 - [ ] Disable local FILE storage
+- [ ] Polaris Server Header


Why is Polaris Server Header a critical point for production configuration?

This section already exists in the page but there wasnt any header for this. Thought i would just add it as header on the top of the page. But i see the point, will remove it.

Yes, the Server header is off by default due to possible security concerns. If a user wishes to enable it, it is possible. However, it does not look like every user has to make a decision about it.

dimas-b · 2025-11-17T16:24:22Z

site/content/in-dev/unreleased/configuring-polaris-for-production.md


-Polaris authentication requires specifying a token broker factory type. Two implementations are
-supported out of the box:
+Polaris authentication requires specifying a token broker factory type. Two implementations are supported out of the


I'd prefer not to change the formatting on existing lines when the text itself does not change. It complicates reviews and skews line authorship attribution.

makes reverting those changes.

dimas-b · 2025-11-17T16:29:21Z

site/content/in-dev/unreleased/configuring-polaris-for-production.md

+
+### Cloud Storage Specific Configuration
+
+GCS + Polaris: When using token vending for fine-grained access in Google Cloud Storage (GCS) with Apache Iceberg on


GCS is one of several possible cloud storage implementations. I believe it would be nicer to move this section into a sub-page under https://polaris.apache.org/in-dev/unreleased/configuring-polaris-for-production/

Other cloud storage options can get dedicated pages parallel to that one later. WDYT?

i was thinking the same thing, there could be more of cloud configuration which would come eventually in documentation. Where would it go. I will put that in sub page under this.

flyrain

Thanks @sakshamratra0106 for working on it. Left some comments.

flyrain · 2025-11-18T19:52:46Z

site/content/in-dev/unreleased/configuring-polaris-for-production.md

+[rsa-key-pair]:
+  https://github.com/apache/polaris/blob/390f1fa57bb1af24a21aa95fdbff49a46e31add7/service/common/src/main/java/org/apache/polaris/service/auth/JWTRSAKeyPairFactory.java
+[symmetric-key]:
+  https://github.com/apache/polaris/blob/390f1fa57bb1af24a21aa95fdbff49a46e31add7/service/common/src/main/java/org/apache/polaris/service/auth/JWTSymmetricKeyFactory.java


There seems be a lot of changes not related. Can we revert them?

flyrain · 2025-11-18T19:54:07Z

site/content/in-dev/unreleased/configuring-polaris-for-production.md

+
+### Cloud Storage Specific Configuration
+
+GCS + Polaris: When using token vending for fine-grained access in Google Cloud Storage (GCS) with Apache Iceberg on


I'd suggest a subtitle

Suggested change

GCS + Polaris: When using token vending for fine-grained access in Google Cloud Storage (GCS) with Apache Iceberg on

#### GCS

When using token vending for fine-grained access in Google Cloud Storage (GCS) with Apache Iceberg on

You mean title of the section could be GCS ? is it ?

oh ok! Got it!

flyrain · 2025-11-18T20:24:14Z

site/content/in-dev/unreleased/configuring-polaris-for-production.md

+
+### Cloud Storage Specific Configuration
+
+GCS + Polaris: When using token vending for fine-grained access in Google Cloud Storage (GCS) with Apache Iceberg on


Can we use the term credential vending instead of token vending to be more consistent with other places?

I'd also recommend to not mention fine-grained access to avoid any confusion with table's FGAC. I think the context is pretty clear when it comes to storage credential vending. fine-grained access isn't necessary.

make sense! Removing!

…ine width to 120 char" This reverts commit 5cd0cc9.

…-page for "Con figuring polaris for production"

dimas-b · 2025-11-20T15:55:29Z

...t/in-dev/unreleased/configuring-polaris-for-production/configuring-cloud-storage-specific.md

+# specific language governing permissions and limitations
+# under the License.
+#
+title: Configuring Cloud Storage


Suggestion: how about Configuring GCS Cloud Storage?

If AWS S3, etc. are added later, it will be a new page, not an edit to this page, which will be easier to maintain, IMHO... but current layout is ok too.

sounds good!!

dimas-b · 2025-11-20T15:56:29Z

site/content/in-dev/unreleased/configuring-polaris-for-production/_index.md

+
+### Configuring Polaris for Cloud Storages
+
+For guidance on configuring cloud storage providers (such as Google Cloud Storage, Amazon S3, and Azure Blob Storage) for use with Polaris—including credential vending, IAM roles, ACL requirements, and best practices—see [Configuring Cloud Storage](./configuring-cloud-storage-specific/).


Suggestion: For guidance on configuring cloud storage providers ... see child pages (links in the left-hand pane)

…-page for "Con figuring polaris for production"

sakshamratra0106 · 2025-11-20T16:28:44Z

@dimas-b please review again!

dimas-b · 2025-11-20T16:32:22Z

site/content/in-dev/unreleased/configuring-polaris-for-production/_index.md

+
+### Configuring Polaris for Cloud Storages
+
+For guidance on configuring cloud storage providers ... see child pages (links in the left-hand pane) for use with Polaris—including credential vending, IAM roles, ACL requirements, and best practices—see [Configuring GCS Cloud Storage](./configuring-gcs-cloud-storage-specific/).


Suggested change

For guidance on configuring cloud storage providers ... see child pages (links in the left-hand pane) for use with Polaris—including credential vending, IAM roles, ACL requirements, and best practices—see [Configuring GCS Cloud Storage](./configuring-gcs-cloud-storage-specific/).

For guidance on configuring specific cloud storage providers see child pages (links in the left-hand pane).

…-page for "Con figuring polaris for production"

flyrain

Thanks for keeping working on it. We are getting close!

flyrain · 2025-11-21T00:15:52Z

...-dev/unreleased/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md

+This page provides guidance for Configuring GCS Cloud Storage provider for use with Polaris. 
+It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration.
+
+#### GCS


We don't need this title as this page is dedicated for GCS config

fair enough!!

flyrain · 2025-11-21T00:30:44Z

...-dev/unreleased/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md

+#### GCS
+
+When using credential vending for Google Cloud Storage (GCS) with Apache Iceberg on
+Polaris, ensure that both IAM roles and HNS ACLs (if HNS is enabled) are properly configured. Even with the correct IAM


Does HNS refer to Hierarchical namespace described here, https://docs.cloud.google.com/storage/docs/hns-overview? We might add the full name and links so that reader arent' confused the by the acronym.

flyrain · 2025-11-21T00:23:58Z

...-dev/unreleased/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md

+Polaris, ensure that both IAM roles and HNS ACLs (if HNS is enabled) are properly configured. Even with the correct IAM
+role (e.g., `roles/storage.objectAdmin`), access to paths such as `gs://<bucket>/idsp_ns/sample_table4/` may fail with
+403 errors if HNS ACLs are missing for scoped tokens. The original access token may work, but scoped (vended) tokens
+require HNS ACLs on the base path or relevant subpath. Polaris does not require HNS to be enabled for basic operation,


What are basic operations? Can we clarify that? My impression is that we cannot disable credential vending when a catalog based on GCS. In that case, HNS seems mandatory.

Actually no. HNS is not mandatory with Credential Vending. We can disable HNS and use Credential vending as is with bare min permissions[object Read and Write]. And that works, i am currently doing the same thing in my project.

Where as with HNS enabled we need to another of permissions in ACLs, refference https://docs.cloud.google.com/storage/docs/uniform-bucket-level-access. i still need to explore that territory. Will keep adding more information as and when i know more about it. If thats fine ?

I have made some changes again to make more sense and details about HNS is not mandatory.

@flyrain please review again!

Correct one dead internal link to "admin-tool" page

@flyrain also could you confirm if these dead link issues to external location are bound to happen and we can merge anyways ?

https://github.com/apache/polaris/actions/runs/19544276537/job/55967194931?pr=3066

Merged, the markdown issue isn't related.

2. Corrected one broken link in "Polaris Configurtion Page"

flyrain

LGTM. Thanks @sakshamratra0106 !

dimas-b · 2025-11-21T16:30:18Z

...-dev/unreleased/configuring-polaris-for-production/configuring-gcs-cloud-storage-specific.md

+
+This page provides guidance for configuring GCS Cloud Storage provider for use with Polaris. It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration.
+
+All catalog operations in Polaris for Google Cloud Storage (GCS)—including listing, reading, and writing objects—are performed using credential vending, which issues scoped (vended) tokens for secure access.


This is not 100% accurate, the SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION flag (if set) can turn off credential vending.

* Docs: update Helm Chart page to show usage without cloning Polaris github repo (apache#2939) * Docs: update Helm Chart page to show usage without cloning Polaris github repo * Apply suggestions from code review * Add separate flows in Helm Chart doc for installing released chart and images vs building from source --------- Co-authored-by: Alexandre Dutra <adutra@apache.org> * docs: improve getting started and README documentation (apache#2267) * The outdated config snippets and health-checks are removed from the index * The binary guide stays focused and concise * Update changelog with missing 1.3.0 features (apache#3087) * Update registry.access.redhat.com/ubi9/openjdk-21-runtime Docker tag to v1.23-6.1763034977 (apache#3092) * Update docker.io/jaegertracing/all-in-one Docker tag to v1.75.0 (apache#3093) * feat: Make generate_clients.py windows compatible (apache#3084) * Make generate_clients.py windows compatible * Updated CHANGELOG.md * PRs: Remove markdown-links-check from required checks (apache#3102) The "Check Markdown links" workflow is known to produce false failures, leading to apache#3097. This change is intended to unblock PRs due to these false failures. * Update dependency com.diffplug.spotless:spotless-plugin-gradle to v8.1.0 (apache#3083) * Update dependency com.diffplug.spotless:spotless-plugin-gradle to v8.1.0 * spotlessApply --------- Co-authored-by: Robert Stupp <snazy@snazy.de> * chore(deps): update gradle to v9.2.1 (apache#3069) * chore(deps): update gradle to v9.2.1 * Fix Grale wrapper SHA --------- Co-authored-by: Robert Stupp <snazy@snazy.de> * chore(deps): update mongo docker tag to v8.2.2 (apache#3100) * chore(deps): update docker.io/mongo docker tag to v8.2.2 (apache#3099) * Source-tarball - eliminate git-gzip risk (apache#3075) Details in the `git archive` chapter in https://reproducible-builds.org/docs/archives/ * NoSQL: Allow `null` IndexKey (de)serialization (apache#3076) This change adopts the implementation to the API specification. * PRs: Re-add markdown-links-check step (apache#3103) The step was disabled in apache#3102 to pass CI and enable merging. * NoSQL: Add maintenance implementation (apache#3077) * Inject DefaultFileIOFactory in tests (apache#3043) * Inject DefaultFileIOFactory in tests also simplify `TaskFileIOSupplier` usage in tests, which allows removal of `TestFileIOFactory`. * Update Quarkus Platform and Group to v3.29.4 (apache#3094) * Site: Replace feather logo (apache#3101) The ASF has a new logo, a leaf. There is sadly no free icon that matches the new logo, so replacing the feather-ASF with "The ASF" in the top-bar navigation. * Update actions/checkout action to v6 (apache#3106) * Core: resolveAll() must be called before reading resolution results (apache#3064) * Site: Added GCS related Bucket Properties for vending credentials. (apache#3066) * Publish/pom: don't include test-fixtures dependencies as runtime (apache#3085) The list of dependencies in pom's includes the api/runtime elements of the test-fixtures, which is not what should be published, as it "pulls up" deps like junit, mockito and assertj as Maven runtime scope dependencies. This change fixes this. * Increase javadoc visibility in `persistence/nosql/persistence/cdi/weld` (apache#3110) This is to fix javadoc error: `No public or protected classes found to document` * Disable cloud storage tests that would use @tempdir (apache#3095) Disable tests from ViewCatalogTests for cloud storage integration tests (S3, ADLS, GCS) that would otherwise use @tempdir. Since @tempdir internally uses Paths.get, it cannot point to cloud storage paths. These tests remain enabled for file-based integration tests. * Remove 'beta' label for Generic Table (apache#3096) * remove beta label * address comments for change log --------- Co-authored-by: Pierre Laporte <pierre@pingtimeout.fr> * Update dependency jupyterlab to v4.5.0 (apache#3074) * chore: Fix md link check in GH action (apache#3128) * Fix md link check in GH action * Fix md link check in GH action * Update dependency pre-commit to v4.5.0 (apache#3123) * Update dependency software.amazon.awssdk:bom to v2.39.2 (apache#3127) * NoSQL: authZ API, SPI, impl and store (apache#3078) NoSQL base functionality for ACLs * NoSQL: Quarkus distributed cache invalidation (apache#3105) Adds support for distributed NoSQL cache invalidation leveraging Quarkus. * NoSQL: adjustments / merge fixes * Last merged commit e124348 --------- Co-authored-by: Oleg Soloviov <40199597+olsoloviov@users.noreply.github.com> Co-authored-by: Alexandre Dutra <adutra@apache.org> Co-authored-by: Subham <subham.sangwan@adypu.edu.in> Co-authored-by: Pierre Laporte <pierre@pingtimeout.fr> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Yong Zheng <yongzheng0809@gmail.com> Co-authored-by: Christopher Lambert <xn137@gmx.de> Co-authored-by: Yufei Gu <yufei@apache.org> Co-authored-by: Saksham Ratra <sakshamratra.0106@gmail.com> Co-authored-by: Dmitri Bourlatchkov <dmitri.bourlatchkov@gmail.com> Co-authored-by: Tamas Mate <50709850+tmater@users.noreply.github.com> Co-authored-by: Yun Zou <yunzou.colostate@gmail.com>

Added GCS related Properties for vending credentials. Added line widt…

5cd0cc9

…h to 120 char

github-project-automation bot added this to Basic Kanban Board Nov 17, 2025

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Nov 17, 2025

dimas-b reviewed Nov 17, 2025

View reviewed changes

flyrain reviewed Nov 18, 2025

View reviewed changes

saksham.ratra added 2 commits November 20, 2025 08:16

Revert "Added GCS related Properties for vending credentials. Added l…

12e150d

…ine width to 120 char" This reverts commit 5cd0cc9.

Added GCS related Properties for vending credentials. Added it as sub…

2ee7fc4

…-page for "Con figuring polaris for production"

sakshamratra0106 requested review from dimas-b and flyrain November 20, 2025 03:29

dimas-b previously approved these changes Nov 20, 2025

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Nov 20, 2025

sakshamratra0106 dismissed dimas-b’s stale review via 619188e November 20, 2025 16:27

Added GCS related Properties for vending credentials. Added it as sub…

619188e

…-page for "Con figuring polaris for production"

sakshamratra0106 requested a review from dimas-b November 20, 2025 16:28

dimas-b reviewed Nov 20, 2025

View reviewed changes

Added GCS related Properties for vending credentials. Added it as sub…

368c103

…-page for "Con figuring polaris for production"

sakshamratra0106 requested a review from dimas-b November 20, 2025 16:40

dimas-b previously approved these changes Nov 20, 2025

View reviewed changes

flyrain reviewed Nov 21, 2025

View reviewed changes

sakshamratra0106 dismissed dimas-b’s stale review via ac9cce9 November 21, 2025 03:14

1. Updated the GCS Configuration To make more sense related to HNS

ac9cce9

2. Corrected one broken link in "Polaris Configurtion Page"

sakshamratra0106 requested a review from flyrain November 21, 2025 03:17

flyrain approved these changes Nov 21, 2025

View reviewed changes

flyrain merged commit c1c9227 into apache:main Nov 21, 2025
14 of 15 checks passed

github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Nov 21, 2025

dimas-b reviewed Nov 21, 2025

View reviewed changes

sakshamratra0106 deleted the gcs_storage_properties branch November 22, 2025 04:37


		### Cloud Storage Specific Configuration

		GCS + Polaris: When using token vending for fine-grained access in Google Cloud Storage (GCS) with Apache Iceberg on


		### Configuring Polaris for Cloud Storages

		For guidance on configuring cloud storage providers (such as Google Cloud Storage, Amazon S3, and Azure Blob Storage) for use with Polaris—including credential vending, IAM roles, ACL requirements, and best practices—see [Configuring Cloud Storage](./configuring-cloud-storage-specific/).

	For guidance on configuring cloud storage providers ... see child pages (links in the left-hand pane) for use with Polaris—including credential vending, IAM roles, ACL requirements, and best practices—see [Configuring GCS Cloud Storage](./configuring-gcs-cloud-storage-specific/).
	For guidance on configuring specific cloud storage providers see child pages (links in the left-hand pane).


		This page provides guidance for configuring GCS Cloud Storage provider for use with Polaris. It covers credential vending, IAM roles, ACL requirements, and best practices to ensure secure and reliable integration.

		All catalog operations in Polaris for Google Cloud Storage (GCS)—including listing, reading, and writing objects—are performed using credential vending, which issues scoped (vended) tokens for secure access.

Conversation

sakshamratra0106 commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyrain left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sakshamratra0106 commented Nov 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyrain left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyrain left a comment

sakshamratra0106 commented Nov 17, 2025 •

edited

Loading

dimas-b Nov 17, 2025 •

edited

Loading