diff --git a/docs/access-control.md b/docs/access-control.md index fe5ceb1316..3e58638d46 100644 --- a/docs/access-control.md +++ b/docs/access-control.md @@ -18,25 +18,25 @@ This section provides information about how access control works for Polaris Catalog. -Polaris Catalog uses a role-based access control (RBAC) model, in which the Polaris administrator assigns access privileges to catalog roles, -and then grants service principals access to resources by assigning catalog roles to principal roles. +Polaris Catalog uses a role-based access control (RBAC) model in which the Polaris administrator assigns access privileges to catalog roles +and then grants access to resources to service principals by assigning catalog roles to principal roles. -The key concepts to understanding access control in Polaris are: +These are the key concepts to understanding access control in Polaris: -- **Securable object** -- **Principal role** -- **Catalog role** -- **Privilege** +- **Securable object** +- **Principal role** +- **Catalog role** +- **Privilege** ## Securable object A securable object is an object to which access can be granted. Polaris has the following securable objects: -- Catalog -- Namespace -- Iceberg table -- View +- Catalog +- Namespace +- Iceberg table +- View ## Principal role @@ -60,17 +60,17 @@ The following table shows examples of principal roles that you might configure i ## Catalog role -A catalog role belongs to a particular catalog resource in Polaris and specifies a set of permissions for actions on the catalog, or on objects +A catalog role belongs to a particular catalog resource in Polaris and specifies a set of permissions for actions on the catalog or objects in the catalog, such as catalog namespaces or tables. You can create one or more catalog roles for a catalog. -You grant privileges to a catalog role, and then grant the catalog role to a principal role to bestow the privileges to one or more service +You grant privileges to a catalog role and then grant the catalog role to a principal role to bestow the privileges to one or more service principals. -**Note** - -If you update the privileges bestowed to a service principal, the updates won\'t take effect for up to one hour. This means that if you -revoke or grant some privileges for a catalog, the updated privileges won\'t take effect on any service principal with access to that catalog -for up to one hour. +> **Note** +> +> If you update the privileges bestowed to a service principal, the updates won't take effect for up to one hour. This means that if you +> revoke or grant some privileges for a catalog, the updated privileges won't take effect on any service principal with access to that catalog +> for up to one hour. Polaris also supports a many-to-many relationship between catalog roles and principal roles. You can grant the same catalog role to one or more principal roles. Likewise, a principal role can be granted to one or more catalog roles. @@ -87,10 +87,10 @@ configure in Polaris: ## RBAC model The following diagram illustrates the RBAC model used by Polaris Catalog. For each catalog, the Polaris administrator assigns access -privileges to catalog roles, and then grants service principals access to resources by assigning catalog roles to principal roles. Polaris +privileges to catalog roles and then grants service principals access to resources by assigning catalog roles to principal roles. Polaris supports a many-to-one relationship between service principals and principal roles. -![Diagram that shows the RBAC model for Polaris Catalog.](./img/rbac-model.svg "Polaris Catalog RBAC model") +![Diagram that shows the RBAC model for Polaris Catalog.](img/rbac-model.svg "Polaris Catalog RBAC model") ## Access control privileges @@ -98,43 +98,47 @@ This section describes the privileges that are available in the Polaris access c roles are granted to principal roles, and principal roles are granted to service principals to specify the operations that service principals can perform on objects in Polaris. +> **Important** +> +> You can only grant privileges at the catalog level. Fine-grained access controls are not available. For example, you can grant read +> privileges to all tables in a catalog but not to an individual table in the catalog. + To grant the full set of privileges (drop, list, read, write, etc.) on an object, you can use the *full privilege* option. ### Table privileges -**Note** - -The TABLE_FULL_METADATA full privilege doesn't grant access to the TABLE_READ_DATA or TABLE_WRITE_DATA individual privileges. - -| Full privilege | Individual privilege | Description | -| -----------------------| ----------- | ---- | -| TABLE_FULL_METADATA | TABLE_CREATE | Enables registering a table with the catalog. | -| | TABLE_DROP | Enables dropping a table from the catalog. | -| | TABLE_LIST | Enables listing any tables in the catalog. | -| | TABLE_READ_PROPERTIES | Enables reading [properties](https://iceberg.apache.org/docs/nightly/configuration/#table-properties) of the table. | -| | TABLE_WRITE_PROPERTIES | Enables configuring [properties](https://iceberg.apache.org/docs/nightly/configuration/#table-properties) for the table. | -| N/A | TABLE_READ_DATA | Enables reading data from the table by receiving short-lived read-only storage credentials from the catalog. | -| N/A | TABLE_WRITE_DATA | Enables writing data to the table by receiving short-lived read+write storage credentials from the catalog. | +| Privilege | Description | +| --------- | ----------- | +| TABLE_CREATE | Enables registering a table with the catalog. | +| TABLE_DROP | Enables dropping a table from the catalog. | +| TABLE_LIST | Enables listing any tables in the catalog. | +| TABLE_READ_PROPERTIES | Enables reading [properties](https://iceberg.apache.org/docs/nightly/configuration/#table-properties) of the table. | +| TABLE_WRITE_PROPERTIES | Enables configuring [properties](https://iceberg.apache.org/docs/nightly/configuration/#table-properties) for the table. | +| TABLE_READ_DATA | Enables reading data from the table by receiving short-lived read-only storage credentials from the catalog. | +| TABLE_WRITE_DATA | Enables writing data to the table by receiving short-lived read+write storage credentials from the catalog. | +| TABLE_FULL_METADATA | Grants all table privileges, except TABLE_READ_DATA and TABLE_WRITE_DATA, which need to be granted individually. | ### View privileges -| Full privilege | Individual privilege | Description | -| -----------------------| ----------- | ---- | -| VIEW_FULL_METADATA | VIEW_CREATE | Enables registering a view with the catalog. | -| | VIEW_DROP | Enables dropping a view from the catalog. | -| | VIEW_LIST | Enables listing any views in the catalog. | -| | VIEW_READ_PROPERTIES | Enables reading all the view properties. | -| | VIEW_WRITE_PROPERTIES | Enables configuring view properties. | +| Privilege | Description | +| --------- | ----------- | +| VIEW_CREATE | Enables registering a view with the catalog. | +| VIEW_DROP | Enables dropping a view from the catalog. | +| VIEW_LIST | Enables listing any views in the catalog. | +| VIEW_READ_PROPERTIES | Enables reading all the view properties. | +| VIEW_WRITE_PROPERTIES | Enables configuring view properties. | +| VIEW_FULL_METADATA | Grants all view privileges. | ### Namespace privileges -| Full privilege | Individual privilege | Description | -| -----------------------| ----------- | ---- | -| NAMESPACE_FULL_METADATA | NAMESPACE_CREATE | Enables creating a namespace in a catalog. | -| | NAMESPACE_DROP | Enables dropping the namespace from the catalog. | -| | NAMESPACE_LIST | Enables listing any object in the namespace, including nested namespaces and tables. | -| | NAMESPACE_READ_PROPERTIES | Enables reading all the namespace properties. | -| | NAMESPACE_WRITE_PROPERTIES | Enables configuring namespace properties. | +| Privilege | Description | +| --------- | ----------- | +| NAMESPACE_CREATE | Enables creating a namespace in a catalog. | +| NAMESPACE_DROP | Enables dropping the namespace from the catalog. | +| NAMESPACE_LIST | Enables listing any object in the namespace, including nested namespaces and tables. | +| NAMESPACE_READ_PROPERTIES | Enables reading all the namespace properties. | +| NAMESPACE_WRITE_PROPERTIES | Enables configuring namespace properties. | +| NAMESPACE_FULL_METADATA | Grants all namespace privileges. | ### Catalog privileges @@ -142,47 +146,42 @@ The TABLE_FULL_METADATA full privilege doesn't grant access to the TABLE_READ_DA | -----------------------| ----------- | | CATALOG_MANAGE_ACCESS | Includes the ability to grant or revoke privileges on objects in a catalog to catalog roles, and the ability to grant or revoke catalog roles to or from principal roles. | | CATALOG_MANAGE_CONTENT | Enables full management of content for the catalog. This privilege encompasses the following privileges: | -| CATALOG_MANAGE_METADATA | Enables full management of the catalog, as well as catalog roles, namespaces, and tables. | +| CATALOG_MANAGE_METADATA | Enables full management of the catalog, catalog roles, namespaces, and tables. | | CATALOG_READ_PROPERTIES | Enables listing catalogs and reading properties of the catalog. | | CATALOG_WRITE_PROPERTIES | Enables configuring catalog properties. | ## RBAC example -The following diagram illustrates how RBAC works in Polaris, and +The following diagram illustrates how RBAC works in Polaris and includes the following users: -- **Alice**: A service admin who signs up for Polaris. Alice can +- **Alice:** A service admin who signs up for Polaris. Alice can create service principals. She can also create catalogs and - namespaces, and configure access control for Polaris resources. - -> **Note** -> -> The service principal for Alice is not visible in the Polaris Catalog -> user interface. + namespaces and configure access control for Polaris resources. -- **Bob**: A data engineer who uses Snowpipe Streaming (in Snowflake) - and Apache Spark connections to interact with Polaris. +- **Bob:** A data engineer who uses Snowpipe Streaming (in Snowflake) + and Apache Spark™ connections to interact with Polaris. - - Alice has created a service principal for Bob. It has been + - Alice has created a service principal for Bob. It has been granted the Data_engineer principal role, which in turn has been granted the following catalog roles: Catalog contributor and Data administrator (for both the Silver and Gold zone catalogs in the following diagram). - - The Catalog contributor role grants permission to create + - The Catalog contributor role grants permission to create namespaces and tables in the Bronze zone catalog. - - The Data administrator roles grant full administrative rights to + - The Data administrator roles grant full administrative rights to the Silver zone catalog and Gold zone catalog. -- **Mark**: A data scientist who uses Snowflake AI services to +- **Mark:** A data scientist who uses Snowflake AI services to interact with Polaris. - - Alice has created a service principal for Mark. It has been + - Alice has created a service principal for Mark. It has been granted the Data_scientist principal role, which in turn has been granted the catalog role named Catalog reader. - - The Catalog reader role grants read-only access for a catalog + - The Catalog reader role grants read-only access for a catalog named Gold zone catalog. -![Diagram that shows an example of how RBAC works in Polaris Catalog.](./img/rbac-example.svg "Polaris Catalog RBAC example") +![Diagram that shows an example of how RBAC works in Polaris Catalog.](img/rbac-example.svg "Polaris Catalog RBAC example") diff --git a/docs/index.html b/docs/index.html index 8c25542041..95906f366e 100644 --- a/docs/index.html +++ b/docs/index.html @@ -430,7 +430,7 @@ -
Polaris Catalog Logo
API docs by Redocly
Polaris Catalog Logo
API docs by Redocly
Connecting with Spark --> -<p>Polaris Catalog is a catalog implementation for Apache Iceberg built on the open source Apache Iceberg REST protocol.</p> -<p>With Polaris Catalog, you can provide centralized, secure read and write access across different REST-compatible query engines to your Iceberg tables.</p> -<p><img src="./img/overview.svg" alt="Conceptual diagram of Polaris Catalog." title="Polaris Catalog overview"></p> +<p>Polaris Catalog is a catalog implementation for Apache Iceberg&trade; tables and is built on the open source Apache Iceberg&trade; REST protocol.</p> +<p>With Polaris Catalog, you can provide centralized, secure read and write access to your Iceberg tables across different REST-compatible query engines.</p> +<p><img src="img/overview.svg" alt="Conceptual diagram of Polaris Catalog." title="Polaris Catalog overview"></p> "> -

Polaris Catalog is a catalog implementation for Apache Iceberg built on the open source Apache Iceberg REST protocol.

-

With Polaris Catalog, you can provide centralized, secure read and write access across different REST-compatible query engines to your Iceberg tables.

-

Conceptual diagram of Polaris Catalog.

+

Polaris Catalog is a catalog implementation for Apache Iceberg™ tables and is built on the open source Apache Iceberg™ REST protocol.

+

With Polaris Catalog, you can provide centralized, secure read and write access to your Iceberg tables across different REST-compatible query engines.

+

Conceptual diagram of Polaris Catalog.

Key concepts

Connecting with Spark <thead> <tr> <th>Service connection name</th> -<th>Description</th> +<th>Purpose</th> </tr> </thead> <tbody><tr> <td>Flink ingestion</td> -<td>For Apache Flink to ingest streaming data into Iceberg tables.</td> +<td>For Apache Flink&reg; to ingest streaming data into Apache Iceberg&trade; tables.</td> </tr> <tr> <td>Spark ETL pipeline</td> -<td>For Apache Spark to run ETL pipeline jobs on Iceberg tables.</td> +<td>For Apache Spark&trade; to run ETL pipeline jobs on Iceberg tables.</td> </tr> <tr> <td>Snowflake data pipelines</td> -<td>For Snowflake to run data pipelines for transforming data in Iceberg tables.</td> +<td>For Snowflake to run data pipelines for transforming data in Apache Iceberg&trade; tables.</td> </tr> <tr> <td>Trino BI dashboard</td> @@ -1020,23 +1008,23 @@

Connecting with Spark

</tr> <tr> <td>Snowflake AI team</td> -<td>For Snowflake to run AI jobs on data in Iceberg tables.</td> +<td>For Snowflake to run AI jobs on data in Apache Iceberg&trade; tables.</td> </tr> </tbody></table> <h3 id="service-connection">Service connection</h3> -<p>A service connection represents a REST-compatible engine (such as Apache Spark, Apache Flink, or Trino) that can read from and write to Polaris +<p>A service connection represents a REST-compatible engine (such as Apache Spark&trade;, Apache Flink&reg;, or Trino) that can read from and write to Polaris Catalog. When creating a new service connection, the Polaris administrator grants the service principal that is created with the new service -connection with either a new or existing principal role. A principal role is a resource in Polaris that you can use to logically group Polaris -service principals together and grant privileges on securable objects. For more information, see <a href="./access-control.md#principal-role" title="Principal role">Principal role</a>. Polaris Catalog uses a role-based access control (RBAC) model to grant service principals access to resources. For more information, -see <a href="./access-control.md" title="Access control">Access control</a>. For a diagram of this model, see <a href="./access-control.md#rbac-model" title="RBAC model">RBAC model</a>.</p> -<p>If the Polaris administrator grants the service principal for the new service connection with a new principal role, the service principal +connection either a new or existing principal role. A principal role is a resource in Polaris that you can use to logically group Polaris +service principals together and grant privileges on securable objects. For more information, see <a href="access-control.md#principal-role" title="Principal role">Principal role</a>. Polaris Catalog uses a role-based access control (RBAC) model to grant service principals access to resources. For more information, +see <a href="access-control.md" title="Access control">Access control</a>. For a diagram of this model, see <a href="access-control.md#rbac-model" title="RBAC model">RBAC model</a>.</p> +<p>If the Polaris administrator grants the service principal for the new service connection a new principal role, the service principal doesn&#39;t have any privileges granted to it yet. When securing the catalog that the new service connection will connect to, the Polaris administrator grants privileges to catalog roles and then grants these catalog roles to the new principal role. As a result, the service -principal for the new service connection is bestowed with these privileges. For more information about catalog roles, see <a href="./access-control.md#catalog-role" title="Catalog role">Catalog role</a>.</p> +principal for the new service connection has these privileges. For more information about catalog roles, see <a href="access-control.md#catalog-role" title="Catalog role">Catalog role</a>.</p> <p>If the Polaris administrator grants an existing principal role to the service principal for the new service connection, the service principal -is bestowed with the privileges granted to the catalog roles that are granted to the existing principal role. If needed, the Polaris +has the same privileges granted to the catalog roles that are granted to the existing principal role. If needed, the Polaris administrator can grant additional catalog roles to the existing principal role or remove catalog roles from it to adjust the privileges -bestowed to the service principal. For an example of how RBAC works in Polaris, see <a href="./access-control.md#rbac-example" title="RBAC example">RBAC example</a>.</p> +bestowed to the service principal. For an example of how RBAC works in Polaris, see <a href="access-control.md#rbac-example" title="RBAC example">RBAC example</a>.</p> <h3 id="storage-configuration">Storage configuration</h3> <p>A storage configuration stores a generated identity and access management (IAM) entity for your external cloud storage and is created when you create a catalog. The storage configuration is used to set the values to connect Polaris Catalog to your cloud storage. During the @@ -1056,7 +1044,7 @@

Connecting with Spark

</tr> <tr> <td>Google Cloud Storage (GCS)</td> -<td><ul><li>Default base location for your GCS bucket</li><li>Locations for your Amazon GCS bucket</li></ul></td> +<td><ul><li>Default base location for your GCS bucket</li><li>Locations for your GCS bucket</li></ul></td> </tr> <tr> <td>Azure</td> @@ -1064,74 +1052,62 @@

Connecting with Spark

</tr> </tbody></table> ">

This section introduces key concepts associated with using Polaris Catalog.

-

In the following diagram, a sample Polaris Catalog structure with nested namespaces is shown for Catalog1. No tables -or namespaces have been created yet for Catalog2 or Catalog3:

-

Diagram that shows an example Polaris Catalog structure.

+

In the following diagram, a sample Polaris Catalog structure with nested namespaces is shown for Catalog1. No tables +or namespaces have been created yet for Catalog2 or Catalog3.

+

Diagram that shows an example Polaris Catalog structure.

Catalog

In Polaris Catalog, you can create one or more catalog resources to organize Iceberg tables.

Configure your catalog by setting values in the storage configuration for S3, Azure, or Google Cloud Storage. An Iceberg catalog enables a -query engine to manage and organize tables. The catalog forms the first architectural layer in the Iceberg table specification and must support:

+query engine to manage and organize tables. The catalog forms the first architectural layer in the Apache Iceberg™ table specification and must support the following tasks:

  • Storing the current metadata pointer for one or more Iceberg tables. A metadata pointer maps a table name to the location of that table's -current metadata file.

    + current metadata file.

  • Performing atomic operations so that you can update the current metadata pointer for a table to the metadata pointer of a new version of the table.

-

To learn more about Iceberg catalogs, see the Apache Iceberg documentation.

+

To learn more about Iceberg catalogs, see the Apache Iceberg™ documentation.

Catalog types

A catalog can be one of the following two types:

  • Internal: The catalog is managed by Polaris. Tables from this catalog can be read and written in Polaris.

  • External: The catalog is externally managed by another Iceberg catalog provider (for example, Snowflake, Glue, Dremio Arctic). Tables from -this catalog are synced to Polaris. These tables are read-only in Polaris. In the current release, only Snowflake external catalog is provided.

    +this catalog are synced to Polaris. These tables are read-only in Polaris. In the current release, only a Snowflake external catalog is provided.

A catalog is configured with a storage configuration that can point to S3, Azure storage, or GCS.

-

To create a new catalog, see Create a catalog.

Namespace

-

You create namespaces to logically group Iceberg tables within a catalog. A catalog can have one or more namespaces. You can also create +

You create namespaces to logically group Iceberg tables within a catalog. A catalog can have multiple namespaces. You can also create nested namespaces. Iceberg tables belong to namespaces.

-

Iceberg tables & catalogs

+

Apache Iceberg™ tables and catalogs

In an internal catalog, an Iceberg table is registered in Polaris Catalog, but read and written via query engines. The table data and metadata is stored in your external cloud storage. The table uses Polaris Catalog as the Iceberg catalog.

If you have tables that use Snowflake as the Iceberg catalog (Snowflake-managed tables), you can sync these tables to an external catalog in Polaris Catalog. If you sync this catalog to Polaris Catalog, it appears as an external catalog in Polaris Catalog. The table data and metadata is stored in your external cloud storage. The Snowflake query engine can read from or write to these tables. However, the other query engines can only read from these tables.

+

Important

-

To ensure that the access privileges defined for a catalog are enforced -correctly, you must:

+

For the access privileges defined for a catalog to be enforced correctly, the following conditions must be met:

    -

  • Ensure a directory only contains the data files that belong to a -single table.

    -
    • -

  • Create a directory hierarchy that matches the namespace hierarchy -for the catalog.

    -
    • +
  • The directory only contains the data files that belong to a single table.
    • +
  • The directory hierarchy matches the namespace hierarchy for the catalog.
-

For example, if a catalog includes:

+

For example, if a catalog includes the following items:

    -

  • Top-level namespace namespace1

    -
    • -

  • Nested namespace namespace1a

    -
    • -

  • A customers table, which is grouped under nested namespace -namespace1a

    -
    • -

  • An orders table, which is grouped under nested namespace namespace1a

    -
    • +
  • Top-level namespace namespace1
    • +
  • Nested namespace namespace1a
    • +
  • A customers table, which is grouped under nested namespace namespace1a
    • +
  • An orders table, which is grouped under nested namespace namespace1a
-

The directory hierarchy for the catalog must be:

+

The directory hierarchy for the catalog must follow this structure:

    -

  • /namespace1/namespace1a/customers/<files for the customers table -*only*>

    -
    • -

  • /namespace1/namespace1a/orders/<files for the orders table *only*>

    -
    • +
  • /namespace1/namespace1a/customers/<files for the customers table *only*>
    • +
  • /namespace1/namespace1a/orders/<files for the orders table *only*>
+

Service principal

A service principal is an entity that you create in Polaris Catalog. Each service principal encapsulates credentials that you use to connect to Polaris Catalog.

@@ -1142,20 +1118,20 @@

Service principal

Service connection name -Description +Purpose Flink ingestion -For Apache Flink to ingest streaming data into Iceberg tables. +For Apache Flink® to ingest streaming data into Apache Iceberg™ tables. Spark ETL pipeline -For Apache Spark to run ETL pipeline jobs on Iceberg tables. +For Apache Spark™ to run ETL pipeline jobs on Iceberg tables. Snowflake data pipelines -For Snowflake to run data pipelines for transforming data in Iceberg tables. +For Snowflake to run data pipelines for transforming data in Apache Iceberg™ tables. Trino BI dashboard @@ -1163,23 +1139,23 @@

Service principal

Snowflake AI team -For Snowflake to run AI jobs on data in Iceberg tables. +For Snowflake to run AI jobs on data in Apache Iceberg™ tables.

Service connection

-

A service connection represents a REST-compatible engine (such as Apache Spark, Apache Flink, or Trino) that can read from and write to Polaris +

A service connection represents a REST-compatible engine (such as Apache Spark™, Apache Flink®, or Trino) that can read from and write to Polaris Catalog. When creating a new service connection, the Polaris administrator grants the service principal that is created with the new service -connection with either a new or existing principal role. A principal role is a resource in Polaris that you can use to logically group Polaris -service principals together and grant privileges on securable objects. For more information, see Principal role. Polaris Catalog uses a role-based access control (RBAC) model to grant service principals access to resources. For more information, -see Access control. For a diagram of this model, see RBAC model.

-

If the Polaris administrator grants the service principal for the new service connection with a new principal role, the service principal +connection either a new or existing principal role. A principal role is a resource in Polaris that you can use to logically group Polaris +service principals together and grant privileges on securable objects. For more information, see Principal role. Polaris Catalog uses a role-based access control (RBAC) model to grant service principals access to resources. For more information, +see Access control. For a diagram of this model, see RBAC model.

+

If the Polaris administrator grants the service principal for the new service connection a new principal role, the service principal doesn't have any privileges granted to it yet. When securing the catalog that the new service connection will connect to, the Polaris administrator grants privileges to catalog roles and then grants these catalog roles to the new principal role. As a result, the service -principal for the new service connection is bestowed with these privileges. For more information about catalog roles, see Catalog role.

+principal for the new service connection has these privileges. For more information about catalog roles, see Catalog role.

If the Polaris administrator grants an existing principal role to the service principal for the new service connection, the service principal -is bestowed with the privileges granted to the catalog roles that are granted to the existing principal role. If needed, the Polaris +has the same privileges granted to the catalog roles that are granted to the existing principal role. If needed, the Polaris administrator can grant additional catalog roles to the existing principal role or remove catalog roles from it to adjust the privileges -bestowed to the service principal. For an example of how RBAC works in Polaris, see RBAC example.

+bestowed to the service principal. For an example of how RBAC works in Polaris, see RBAC example.

Storage configuration

A storage configuration stores a generated identity and access management (IAM) entity for your external cloud storage and is created when you create a catalog. The storage configuration is used to set the values to connect Polaris Catalog to your cloud storage. During the @@ -1199,58 +1175,58 @@

Storage configuration

Google Cloud Storage (GCS) -
  • Default base location for your GCS bucket
  • Locations for your Amazon GCS bucket
+
  • Default base location for your GCS bucket
  • Locations for your GCS bucket
Azure
  • Default base location for your Microsoft Azure container
  • Locations for your Microsoft Azure container
  • Azure tenant ID
-

Example workflow

Example workflow

In the following example workflow, Bob creates an Iceberg table named Table1 and Alice reads data from Table1.

+<p><img src="img/example-workflow.svg" alt="Diagram that shows an example workflow for Polaris Catalog" title="Example workflow for Polaris Catalog"></p> +">

In the following example workflow, Bob creates an Apache Iceberg™ table named Table1 and Alice reads data from Table1.

    -

  1. Bob uses Apache Spark to create the Table1 table under the +

  2. Bob uses Apache Spark™ to create the Table1 table under the Namespace1 namespace in the Catalog1 catalog and insert values into Table1.

    -

    Bob can create Table1 and insert data into it, because he is using a -service connection with a service principal that is bestowed with +

    Bob can create Table1 and insert data into it because he is using a +service connection with a service principal that has the privileges to perform these actions.

  3. Alice uses Snowflake to read data from Table1.

    -

    Alice can read data from Table1, because she is using a service +

    Alice can read data from Table1 because she is using a service connection with a service principal with a catalog integration that -is bestowed with the privileges to perform this action. Alice +has the privileges to perform this action. Alice creates an unmanaged table in Snowflake to read data from Table1.

-

Diagram that shows an example workflow for Polaris Catalog

+

Diagram that shows an example workflow for Polaris Catalog

Security and access control

Storage configuration service principals.</p> </li> <li><p><strong>Catalog roles:</strong> Configured with certain privileges on Polaris -catalog resources, and granted to principal roles.</p> +catalog resources and granted to principal roles.</p> </li> </ul> -<p>For more information, see <a href="./access-control.md" title="Access control">Access control</a>.</p> +<p>For more information, see <a href="access-control.md" title="Access control">Access control</a>.</p> ">

This section describes security and access control.

Credential vending

To secure interactions with service connections, Polaris Catalog vends temporary storage credentials to the query engine during query -execution. These credentials allow the query engine to run the query without needing to have access to your external cloud storage for +execution. These credentials allow the query engine to run the query without requiring access to your external cloud storage for Iceberg tables. This process is called credential vending.

Identity and access management (IAM)

Polaris Catalog uses the identity and access management (IAM) entity to securely connect to your storage for accessing table data, Iceberg metadata, and manifest files that store the table schema, partitions, and other metadata. Polaris Catalog retains the IAM entity for your storage location.

Access control

-

Polaris Catalog enforces the access control that you configure across all tables registered with the service, and governs security for all +

Polaris Catalog enforces the access control that you configure across all tables registered with the service and governs security for all queries from query engines in a consistent manner.

Polaris uses a role-based access control (RBAC) model that lets you centrally configure access for Polaris service principals to catalogs, namespaces, and tables.

@@ -1286,10 +1262,12 @@

Access control

service principals.

  • Catalog roles: Configured with certain privileges on Polaris -catalog resources, and granted to principal roles.

    +catalog resources and granted to principal roles.

    • -

    For more information, see Access control.

    +

    For more information, see Access control.

    +

    Legal Notices

    Apache®, Apache Iceberg™, Apache Spark™, Apache Flink®, and Flink® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.

    Polaris Catalog Entities

    Storage Type --> <p>This section provides information about how access control works for Polaris Catalog.</p> -<p>Polaris Catalog uses a role-based access control (RBAC) model, in which the Polaris administrator assigns access privileges to catalog roles, -and then grants service principals access to resources by assigning catalog roles to principal roles.</p> -<p>The key concepts to understanding access control in Polaris are:</p> +<p>Polaris Catalog uses a role-based access control (RBAC) model in which the Polaris administrator assigns access privileges to catalog roles +and then grants access to resources to service principals by assigning catalog roles to principal roles.</p> +<p>These are the key concepts to understanding access control in Polaris:</p> <ul> <li><strong>Securable object</strong></li> <li><strong>Principal role</strong></li> @@ -1425,9 +1403,9 @@

    Storage Type

    -->

    This section provides information about how access control works for Polaris Catalog.

    -

    Polaris Catalog uses a role-based access control (RBAC) model, in which the Polaris administrator assigns access privileges to catalog roles, -and then grants service principals access to resources by assigning catalog roles to principal roles.

    -

    The key concepts to understanding access control in Polaris are:

    +

    Polaris Catalog uses a role-based access control (RBAC) model in which the Polaris administrator assigns access privileges to catalog roles +and then grants access to resources to service principals by assigning catalog roles to principal roles.

    +

    These are the key concepts to understanding access control in Polaris:

    • Securable object
    • Principal role
      • @@ -1500,14 +1478,16 @@

      Storage Type

      A role that is granted to multiple service principals for running data science or AI jobs. -

    Catalog role

    Catalog role

    Storage Type <td>A role that has been granted read and write access privileges to all tables that belong to the catalog.<br /><br />Principal roles that have been granted this role are allowed to perform read and write operations on tables in the catalog.</td> </tr> </tbody></table> -">

    A catalog role belongs to a particular catalog resource in Polaris and specifies a set of permissions for actions on the catalog, or on objects +">

    A catalog role belongs to a particular catalog resource in Polaris and specifies a set of permissions for actions on the catalog or objects in the catalog, such as catalog namespaces or tables. You can create one or more catalog roles for a catalog.

    -

    You grant privileges to a catalog role, and then grant the catalog role to a principal role to bestow the privileges to one or more service +

    You grant privileges to a catalog role and then grant the catalog role to a principal role to bestow the privileges to one or more service principals.

    +

    Note

    If you update the privileges bestowed to a service principal, the updates won't take effect for up to one hour. This means that if you revoke or grant some privileges for a catalog, the updated privileges won't take effect on any service principal with access to that catalog for up to one hour.

    +

    Polaris also supports a many-to-many relationship between catalog roles and principal roles. You can grant the same catalog role to one or more principal roles. Likewise, a principal role can be granted to one or more catalog roles.

    The following table displays examples of catalog roles that you might @@ -1565,133 +1547,128 @@

    Storage Type

    RBAC model

    The following diagram illustrates the RBAC model used by Polaris Catalog. For each catalog, the Polaris administrator assigns access -privileges to catalog roles, and then grants service principals access to resources by assigning catalog roles to principal roles. Polaris +privileges to catalog roles and then grants service principals access to resources by assigning catalog roles to principal roles. Polaris supports a many-to-one relationship between service principals and principal roles.

    -

    Diagram that shows the RBAC model for Polaris Catalog.

    +

    Diagram that shows the RBAC model for Polaris Catalog.

    Access control privileges

    Storage Type </tr> <tr> <td>CATALOG_MANAGE_METADATA</td> -<td>Enables full management of the catalog, as well as catalog roles, namespaces, and tables.</td> +<td>Enables full management of the catalog, catalog roles, namespaces, and tables.</td> </tr> <tr> <td>CATALOG_READ_PROPERTIES</td> @@ -1725,123 +1702,118 @@

    Storage Type

    ">

    This section describes the privileges that are available in the Polaris access control model. Privileges are granted to catalog roles, catalog roles are granted to principal roles, and principal roles are granted to service principals to specify the operations that service principals can perform on objects in Polaris.

    +
    +

    Important

    +

    You can only grant privileges at the catalog level. Fine-grained access controls are not available. For example, you can grant read +privileges to all tables in a catalog but not to an individual table in the catalog.

    +

    To grant the full set of privileges (drop, list, read, write, etc.) on an object, you can use the full privilege option.

    Table privileges

    -

    Note

    -

    The TABLE_FULL_METADATA full privilege doesn't grant access to the TABLE_READ_DATA or TABLE_WRITE_DATA individual privileges.

    - - + - - - - - - - + + + +
    Full privilegeIndividual privilegePrivilege Description
    TABLE_FULL_METADATA TABLE_CREATE Enables registering a table with the catalog.
    TABLE_DROP Enables dropping a table from the catalog.
    TABLE_LIST Enables listing any tables in the catalog.
    TABLE_READ_PROPERTIES Enables reading properties of the table.
    TABLE_WRITE_PROPERTIES Enables configuring properties for the table.
    N/A TABLE_READ_DATA Enables reading data from the table by receiving short-lived read-only storage credentials from the catalog.
    N/A TABLE_WRITE_DATA Enables writing data to the table by receiving short-lived read+write storage credentials from the catalog.
    TABLE_FULL_METADATAGrants all table privileges, except TABLE_READ_DATA and TABLE_WRITE_DATA, which need to be granted individually.

    View privileges

    - - + - - - - - + + + +
    Full privilegeIndividual privilegePrivilege Description
    VIEW_FULL_METADATA VIEW_CREATE Enables registering a view with the catalog.
    VIEW_DROP Enables dropping a view from the catalog.
    VIEW_LIST Enables listing any views in the catalog.
    VIEW_READ_PROPERTIES Enables reading all the view properties.
    VIEW_WRITE_PROPERTIES Enables configuring view properties.
    VIEW_FULL_METADATAGrants all view privileges.

    Namespace privileges

    - - + - - - - - + + + +
    Full privilegeIndividual privilegePrivilege Description
    NAMESPACE_FULL_METADATA NAMESPACE_CREATE Enables creating a namespace in a catalog.
    NAMESPACE_DROP Enables dropping the namespace from the catalog.
    NAMESPACE_LIST Enables listing any object in the namespace, including nested namespaces and tables.
    NAMESPACE_READ_PROPERTIES Enables reading all the namespace properties.
    NAMESPACE_WRITE_PROPERTIES Enables configuring namespace properties.
    NAMESPACE_FULL_METADATAGrants all namespace privileges.

    Catalog privileges

    @@ -1861,7 +1833,7 @@

    Catalog privileges

    - + @@ -1872,94 +1844,82 @@

    Catalog privileges

    CATALOG_MANAGE_METADATAEnables full management of the catalog, as well as catalog roles, namespaces, and tables.Enables full management of the catalog, catalog roles, namespaces, and tables.
    CATALOG_READ_PROPERTIES Enables configuring catalog properties.
    -

    RBAC example

    RBAC example

    The following diagram illustrates how RBAC works in Polaris, and +<p><img src="img/rbac-example.svg" alt="Diagram that shows an example of how RBAC works in Polaris Catalog." title="Polaris Catalog RBAC example"></p> +">

    The following diagram illustrates how RBAC works in Polaris and includes the following users:

      -
    • Alice: A service admin who signs up for Polaris. Alice can -create service principals. She can also create catalogs and -namespaces, and configure access control for Polaris resources.
      • -
    -
    -

    Note

    -

    The service principal for Alice is not visible in the Polaris Catalog -user interface.

    -
    -
      -

    • Bob: A data engineer who uses Snowpipe Streaming (in Snowflake) -and Apache Spark connections to interact with Polaris.

      +

    • Alice: A service admin who signs up for Polaris. Alice can + create service principals. She can also create catalogs and + namespaces and configure access control for Polaris resources.

      +
      • +

    • Bob: A data engineer who uses Snowpipe Streaming (in Snowflake) + and Apache Spark™ connections to interact with Polaris.

      • Alice has created a service principal for Bob. It has been -granted the Data_engineer principal role, which in turn has been -granted the following catalog roles: Catalog contributor and -Data administrator (for both the Silver and Gold zone catalogs -in the following diagram).

        + granted the Data_engineer principal role, which in turn has been + granted the following catalog roles: Catalog contributor and + Data administrator (for both the Silver and Gold zone catalogs + in the following diagram).

      • The Catalog contributor role grants permission to create -namespaces and tables in the Bronze zone catalog.

        + namespaces and tables in the Bronze zone catalog.

      • The Data administrator roles grant full administrative rights to -the Silver zone catalog and Gold zone catalog.

        + the Silver zone catalog and Gold zone catalog.

      • -

    • Mark: A data scientist who uses Snowflake AI services to -interact with Polaris.

      +

    • Mark: A data scientist who uses Snowflake AI services to + interact with Polaris.

      • Alice has created a service principal for Mark. It has been -granted the Data_scientist principal role, which in turn has -been granted the catalog role named Catalog reader.

        + granted the Data_scientist principal role, which in turn has + been granted the catalog role named Catalog reader.

      • The Catalog reader role grants read-only access for a catalog -named Gold zone catalog.

        + named Gold zone catalog.

    -

    Diagram that shows an example of how RBAC works in Polaris Catalog.

    +

    Diagram that shows an example of how RBAC works in Polaris Catalog.

    Configuring Polaris for Production

    featureConfiguratio " class="sc-euGpHm sc-exayXG fwfkcU jYGAQp">

    Generic base server URL, with all parts configurable

    {scheme}://{host}:{port}/{basePath}/v1/{prefix}/views/rename

    Request samples

    Content type
    application/json
    {
    • "source": {
      },
    • "destination": {
      }
    }

    Response samples

    Content type
    application/json
    {
    • "error": {
      }
    }