DOCS: OPA as External Policy Decision Point #3030

sungwy · 2025-11-12T02:59:16Z

Doc PR following up the introduction of OpaPolarisAuthorizer: #2680

Checklist

🛡️ Don't disclose security issues! (contact [email protected])
🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
🧪 Added/updated tests with good coverage, or manually tested (and explained how)
💡 Added comments for complex logic
🧾 Updated CHANGELOG.md (if needed)
📚 Updated documentation in site/content/in-dev/unreleased (if needed)

snazy

@sungwy This is a really neat piece of documentation and PR work! Thanks a lot!

Just a few minor comments, nothing serious.

site/content/in-dev/unreleased/managing-security/_index.md

snazy · 2025-11-12T07:44:20Z

site/content/in-dev/unreleased/managing-security/external-pdp/opa.md

+
+Create a policy file (e.g., `policies/polaris.rego`):
+
+```rego


Too sad - guess we need to bump the Hugo version to get syntax highlighting for Rego (not for this PR tho).

site/content/in-dev/unreleased/managing-security/external-pdp/opa.md

Co-authored-by: Robert Stupp <[email protected]>

sungwy · 2025-11-12T14:08:24Z

@sungwy This is a really neat piece of documentation and PR work! Thanks a lot!

Just a few minor comments, nothing serious.

Thanks @snazy :) I've adopted your suggested changes and it looks like Hugo Site CI is breaking as a result. I'll take another pass through this PR later today 😄

site/content/in-dev/unreleased/managing-security/_index.md

flyrain · 2025-11-13T05:59:27Z

site/content/in-dev/unreleased/managing-security/external-pdp/opa.md

+{{< alert warning "Experimental Feature" >}}
+**OPA integration is currently an experimental feature** and may undergo breaking changes in future versions. Use with caution in production environments.
+{{< /alert >}}


I’d suggest using “Preview Feature” instead of “Experimental Feature.”
“Preview” typically signals that the feature is available for early adoption and feedback, but not finalized, while sounding more stable and user-friendly than “experimental.” This may set better expectations for users. For example, we marked the event as a preview feature, https://polaris.apache.org/downloads/#120.

This is a great suggestion - thank you @flyrain !

flyrain · 2025-11-13T07:03:51Z

site/content/in-dev/unreleased/managing-security/external-pdp/opa.md

+
+The `action` field contains the operation being attempted as a string value from the `PolarisAuthorizableOperation` enum.
+
+For the complete list of available operations, see the [PolarisAuthorizableOperation enum](https://github.com/apache/polaris/blob/main/polaris-core/src/main/java/org/apache/polaris/core/auth/PolarisAuthorizableOperation.java) in the source code.


Should we recommend to handle all operations in OPA so that it makes sure that an operation won't accidentally fail? Even say, defaulting operations you don't care to deny would be helpful.

Should we also mention that the certain operations(e.g. grant privilges) should be set to deny always?

Should we recommend to handle all operations in OPA so that it makes sure that an operation won't accidentally fail? Even say, defaulting operations you don't care to deny would be helpful.

Sounds good 👍

Should we also mention that the certain operations(e.g. grant privilges) should be set to deny always?

I think that makes sense. I was just wondering if we'd want to do that explicitly in the OpaPolarisAuthorizer, but I think relying on the default behavior of default allow := false would be an easier way of achieving than maintaining a list in the opa extension

flyrain · 2025-11-13T07:17:20Z

site/content/in-dev/unreleased/managing-security/external-pdp/opa.md

+For the complete list of available operations, see the [PolarisAuthorizableOperation enum](https://github.com/apache/polaris/blob/main/polaris-core/src/main/java/org/apache/polaris/core/auth/PolarisAuthorizableOperation.java) in the source code.
+
+Common examples include:
+- Table operations: `LOAD_TABLE_WITH_READ_DELEGATION`, `LOAD_TABLE_WITH_WRITE_DELEGATION`, `CREATE_TABLE_DIRECT`, `UPDATE_TABLE`, `DROP_TABLE_WITHOUT_PURGE`


Thinking more about it, would it make sense to provide a rego example that demonstrate how it would work with all kinds of operations? It's a blocker for this PR. We could file an issue for that if that makes sense.

Did you mean to say that it's not a blocker for this PR? :)

I'm +1 for creating a follow up issue for that! Happy to create it once you clarify above

Sorry for the typo. It's NOT a blocker.

flyrain

Nice doc! Thanks a lot for working on it, @sungwy! Left some comments.

snazy · 2025-11-17T08:13:24Z

Looks like we're good on this PR, merging.

dimas-b · 2025-11-17T16:38:19Z

site/content/in-dev/unreleased/managing-security/external-pdp/_index.md

+# specific language governing permissions and limitations
+# under the License.
+#
+title: External Policy Decision Point


This PR got merged, but I do not see this page in https://polaris.apache.org/in-dev/unreleased/ 🤔

never mind... found it 😅

* Add loadEntities batch call and rename listFullEntities (apache#2508) * Add loadEntities batch call and rename listFullEntities * Changed batch call to implement loadResolvedEntities instead * Add loadResolvedEntities by id and entity cache support * Add additional test for loadResolvedEntities by id * Added additional test and updated comments in EntityCache interface * Add additional constructor to ResolvedEntitiesResult * Fixed unused method reference * Removed loadResolvedEntities method with lookup record param * Pulled out toResolvedPolarisEntity method per PR comment * Core: made the ARN role regex more generic (apache#3005) * fix(docs): Generify S3 index page (apache#2997) * Remove the mention of "cloud" since not all possible storage options are provided in "cloud". * Avoid listing specific child pages in the doc test. Rely on Hugo-general index (on the left-hand pane). --------- Co-authored-by: Alexandre Dutra <[email protected]> * fix(deps): update dependency io.prometheus:prometheus-metrics-exporter-servlet-jakarta to v1.4.3 (apache#3009) * fix(deps): update dependency com.google.cloud:google-cloud-storage-bom to v2.60.0 (apache#3011) * fix(deps): update dependency io.opentelemetry:opentelemetry-bom to v1.56.0 (apache#3012) * fix(deps): update dependency com.adobe.testing:s3mock-testcontainers to v4.10.0 (apache#3010) * fix(deps): update dependency org.agrona:agrona to v2.3.2 (apache#3014) * fix(deps): update quarkus platform and group to v3.29.2 (apache#3013) * chore(deps): update dependency pre-commit to v4.4.0 (apache#3015) * fix(deps): update dependency software.amazon.awssdk:bom to v2.38.2 (apache#3019) * Add test for TracingFilter (apache#2847) * NoSQL: Add (micro-ish) benchmarks (apache#3006) A project for JMH based benchmarks against NoSQL persistence. * Helm chart: include configmap checksum in deployment annotations (apache#3023) * fix(deps): update dependency ch.qos.logback:logback-classic to v1.5.21 (apache#3025) * NoSQL: Realms handling (apache#3007) Introduces handling for realms including realm-state management/transition. The `RealmStore` implementation for NoSQL depends on CDI components, coming in a follo-up PR. * Rename AccessConfig and AccessConfigProvider for clarity (apache#2883) * rename AccessConfig for clarity * rename getStorageAccessConfig() and add javadoc * Refactor: improve and clean up Dockerfiles (apache#2957) * Refactor: improve and clean up Dockerfiles * Refactor: improve and clean up Dockerfiles * Refactor: improve and clean up Dockerfiles * Refactor: improve and clean up Dockerfiles * Refactor: improve and clean up Dockerfiles * Refactor: improve and clean up Dockerfiles * Make StorageAccessConfigProvider request-scoped (apache#2974) - add `StorageCredentialsVendor` as request-scoped wrapper around `PolarisCredentialVendor` - make `FileIOFactory` request-scoped - make `TaskFileIOSupplier` request-scoped * Increase javadoc visibility in `nosql/realms` (apache#3029) This is to fix javadoc error: `No public or protected classes found to document` * NoSQL: Add correctness tests (apache#3027) Verifies the correctness of concurrent commits, and big index handling. These tests are intentionally _not_ part of the base-backend test suite for two reasons: 1. These tests do not run against the `Backend` interface but the `Persistence` interface, including commit and index logic. 2. These tests are intended to be runnable against a custom provisioned database cluster, not just tiny-ish test containers. * NoSQL: Add maintenance API, SPI (apache#3028) Maintenance operations include a bunch of tasks that are regularly executed against a backend database. Types of maintenance operations include: * Purging unreferenced objects and references within a catalog * Purging whole catalogs that are marked to be purged * Purging whole realms that are marked to be purged Implementation added in a follow-up PR. * Embrace request-scoped TokenBroker (apache#3024) * Embrace request-scoped TokenBroker `TokenBroker` and `CallContext` are both request-scoped, so instead of passing the former into the latter, we can do this via the `TokenBrokerFactory` and thus simplify the `TokenBroker` interface. * fix(deps): update dependency io.smallrye:jandex to v3.5.2 (apache#3032) * Fix monkey patching (apache#3016) * chore(deps): update quay.io/keycloak/keycloak docker tag to v26.4.5 (apache#3034) * chore(deps): update registry.access.redhat.com/ubi9/openjdk-21-runtime docker tag to v1.23-6.1762870925 (apache#3053) * fix(deps): update dependency org.testcontainers:testcontainers-bom to v2.0.2 (apache#3054) * chore(deps): update postgres docker tag to v18.1 (apache#3055) * Add Polaris Community Meeting 2025-11-13 (apache#3060) * Site: Rename menu "downloads" to "releases" (apache#2928) * Update dependency software.amazon.awssdk:bom to v2.38.7 (apache#3065) * Test-fix: Cleanup OPA test container on stop (apache#3041) Quarkus takes care of reusing a test-resource across tests. The current behavior leaves the container around. Plus some nit-fixes (deprecation + local var) * Update dependency org.apache.commons:commons-lang3 to v3.20.0 (apache#3063) * Build: ensure LICENSE/NOTICE is in all jars, always add pom-files to all jars (apache#3057) There are a some inconsistencies between the different kinds of jars and the included information: * LICENSE/NOTICE files are present in the "main" jar and in the sources jar, but not in the javadoc jar. * The Maven pom.xml and pom.properties files are only present for release builds or when explicitly requested. * "Additional" jar-manifest attributes that are only present in release builds. This change fixes the three mentioned issues: * Always include pom.xml and pom.properties in the built jar files. * Always include the additional jar-manifest attributes, except the Git information, which would otherwise render the Gradle build cache ineffective. * Include pom.xml + pom.properties + license/notice in literally all jar files. The Gradle logic to include the license+notice+pom files has been simplified as well. * Remove unused polarisEventListener field from IcebergCatalogHandler (apache#3045) it was added in c3f5001 but then its only usage was removed in d03c717 * fix(deps): update quarkus platform and group to v3.29.3 (apache#3052) * Site: Add Open Policy Agent (OPA) as External Policy Decision Point (apache#3030) Doc PR following up the introduction of OpaPolarisAuthorizer: apache#2680 * OPA: Tackle deprecation warnings (apache#3042) Instead of suppressing the deprecations, this change updates the code a little bit to remove the mocks (except to create a non-nullable parameter). * Use POJOs for OPA JSON schema construction and publish schema (apache#3031) Co-authored-by: Robert Stupp <[email protected]> * Use CDI for more test setups (apache#3040) this avoids a bunch of redundant manual setup. the important parts are establishing a `RealmContext` by calling `QuarkusMock.installMockForType` and then populating `polarisContext` from the injected `CallContext`. * fix(deps): update dependency com.github.dasniko:testcontainers-keycloak to v4 (apache#3070) * chore(deps): update actions/checkout digest to 93cb6ef (apache#3068) * OPA: Fail fast when OPA bearer token file is unreadable (apache#3062) * fix(deps): update immutables to v2.11.7 (apache#3072) * Skip Hugo Site workflow on forks (apache#3056) Forks usually don't have the "versioned-docs" tag and thus PRs against forks or rebasing the main branch on a fork currently always causes workflow failures. * Fix warnings around TransactionWorkspaceMetaStoreManager (apache#3044) - dont return `null` for interface methods that are `@Nonnull` - fix wrong method name parameters - dont annotate void methods as `@Nonnull` * NoSQL: Add CDI/common+testing + necessary nosql-store implementations (apache#3035) Adds common and test-specific CDI functionality. Requires the NoSQL store implementations `:polaris-persistence-nosql-realms-store-nosql` and `:polaris-nodes-store-nosql`. Those modules have cross-project dependencies for test purposes, hence those are all contained in this PR. CDI for Quarkus will be added in a follow-up. * Automate the release guide - Take 2 - Github workflows (apache#2383) The release automation is simplified to four GitHub workflows that just require the really mandatory user input: the version number. 1. workflow: Trigger the creation of the release branch 2. workflow: Upgrade the release branch with the version and build the the final change-log for that version 3. workflow: Build the RC artifacts from the release branch and push those to the various staging repositories 4. workflow: Eventually release the artifacts. See also the [email announcement](https://lists.apache.org/thread/d0smz07gnr509yj5dc6omo3cvkf1pnh7). --------- Co-authored-by: Robert Stupp <[email protected]> * Update actions/checkout digest to 93cb6ef (apache#3082) * NoSQL: adapt to conflicting changes in main * Last merged commit 8ccddc5 --------- Co-authored-by: Michael Collado <[email protected]> Co-authored-by: cccs-cat001 <[email protected]> Co-authored-by: Dmitri Bourlatchkov <[email protected]> Co-authored-by: Alexandre Dutra <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Nuoya Jiang <[email protected]> Co-authored-by: Yong Zheng <[email protected]> Co-authored-by: Christopher Lambert <[email protected]> Co-authored-by: JB Onofré <[email protected]> Co-authored-by: Yufei Gu <[email protected]> Co-authored-by: Sung Yun <[email protected]> Co-authored-by: Pierre Laporte <[email protected]>

sungwy added 2 commits November 12, 2025 02:50

add opa docs

eff0d0a

add warning

b5459ed

github-project-automation bot added this to Basic Kanban Board Nov 12, 2025

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Nov 12, 2025

snazy reviewed Nov 12, 2025

View reviewed changes

sungwy and others added 2 commits November 12, 2025 09:04

warning

1095f71

Co-authored-by: Robert Stupp <[email protected]>

bullet points

e0cc17d

Co-authored-by: Robert Stupp <[email protected]>

snazy reviewed Nov 12, 2025

View reviewed changes

site/content/in-dev/unreleased/managing-security/_index.md Show resolved Hide resolved

fix

0828043

sungwy mentioned this pull request Nov 13, 2025

Use POJOs for OPA JSON schema construction and publish schema #3031

Merged

6 tasks

flyrain reviewed Nov 13, 2025

View reviewed changes

adopt yufeis feedback

c3ad425

adutra approved these changes Nov 13, 2025

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Nov 13, 2025

flyrain approved these changes Nov 13, 2025

View reviewed changes

snazy approved these changes Nov 17, 2025

View reviewed changes

snazy merged commit 5d5828a into apache:main Nov 17, 2025
15 checks passed

github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Nov 17, 2025

sungwy deleted the opa-docs branch November 17, 2025 14:08

dimas-b reviewed Nov 17, 2025

View reviewed changes


		Create a policy file (e.g., `policies/polaris.rego`):

		```rego


		The `action` field contains the operation being attempted as a string value from the `PolarisAuthorizableOperation` enum.

		For the complete list of available operations, see the [PolarisAuthorizableOperation enum](https://github.com/apache/polaris/blob/main/polaris-core/src/main/java/org/apache/polaris/core/auth/PolarisAuthorizableOperation.java) in the source code.

DOCS: OPA as External Policy Decision Point #3030

DOCS: OPA as External Policy Decision Point #3030

Uh oh!

Conversation

sungwy commented Nov 12, 2025

Checklist

Uh oh!

snazy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sungwy commented Nov 12, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sungwy Nov 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flyrain left a comment

Choose a reason for hiding this comment

Uh oh!

snazy commented Nov 17, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sungwy Nov 13, 2025 •

edited

Loading