-
Notifications
You must be signed in to change notification settings - Fork 495
docs: Add Polaris Evolution page
#1890
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 16 commits
aa09063
1cf4310
28838b5
82b57b4
a8cb09b
fe8511b
588eccc
51231fd
166f48a
e4f07af
c71dacc
cd0800d
4deb83c
6567835
a968666
eb54ece
018f203
269aa1b
41e6d92
1edd258
7a4062a
281c117
6951cc1
41df096
746cf3d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| --- | ||
| # | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
| # | ||
| title: Polaris Evolution | ||
| type: docs | ||
| weight: 1000 | ||
| --- | ||
|
|
||
| This page discusses what can be expected from Apache Polaris as the project evolves. | ||
|
|
||
| ## Using Polaris as a Catalog | ||
|
|
||
| Polaris is primarily intended to be used as a Catalog of Tables and Views. As such, | ||
| it implements the [Iceberg REST API, Generic Tables API](../polaris-catalog-service/), | ||
| and [Polaris Management API](../polaris-management-service/). | ||
|
|
||
| Revisions of the Iceberg REST API are controlled by the [Apache Iceberg](https://iceberg.apache.org/) | ||
| community. Polaris attempts to accurately implement this specification. Nonetheless, | ||
| optional REST Catalog features may or may not be supported immediately. In general, | ||
| there is no guarantee that Polaris releases always implement the latest version of | ||
| the Iceberg REST Catalog API. | ||
|
|
||
| Any API under Polaris control (e.g. the Management API) is maintained as a versioned REST | ||
| API. New releases of Polaris may include changes to the current version of the API. When | ||
| that happens those changes are intended to be compatible with prior versions of Polaris | ||
| clients. Certain endpoints and parameters may be deprecated. | ||
|
|
||
| In case a major change is required to the Management API that cannot be implemented in a | ||
|
dimas-b marked this conversation as resolved.
Outdated
|
||
| backward-compatible way, new endpoints (URI paths) may be introduced. New URI "roots" may | ||
| be introduced too (e.g. `api/catalog/v2`). | ||
|
|
||
| Note that those "v1", "v2", etc. URI path segments are not meant to be 1:1 with Polaris | ||
| releases or Polaris project version numbers (e.g. a "v2" path segment does not mean that | ||
| it is added in Polaris 2.0). | ||
|
|
||
| Polaris servers will support deprecated API endpoints / parameters / versions / etc. | ||
| for some transition period to allow clients to migrate. | ||
|
|
||
| ### Managing Polaris Database | ||
|
|
||
| Polaris stores its data in a database, which is sometimes referred to as "Metastore" or | ||
| "Persistence" in other docs. | ||
|
|
||
| Each Polaris release may support multiple Persistence [implementations](../metastores), | ||
| for example, "EclipseLink" (deprecated) and "JDBC" (current). | ||
|
eric-maynard marked this conversation as resolved.
|
||
|
|
||
| Each type of Persistence evolves individually. Within each Persistence type, Polaris | ||
| attempts to support rolling upgrades (both version X and X + 1 servers running at the | ||
| same time). | ||
|
|
||
| However, migrating between different Persistence types is not supported in a rolling | ||
| upgrade manner (for example, migrating from "EclipseLink" to "JDBC"). Polaris provides | ||
| [tools](https://github.com/apache/polaris-tools/) for migrating between different | ||
| catalogs and those tools may be used to migrate between different Persistence types | ||
| as well. Service interruption (downtime) should be expected in those cases. | ||
|
|
||
| ## Using Polaris as a Build-Time Dependency | ||
|
|
||
| Polaris produces several jars. These jars or custom builds of Polaris code may be used in | ||
| downstream projects according to the terms of the license included into Polaris distributions. | ||
|
|
||
| This means that major version increments should be expected often (in the SemVer sense, see | ||
| below). | ||
|
|
||
| The minimal version of the JRE required by Polaris code (compilation target) may be updated in | ||
| any release. Different Polaris jars may have different minimal JRE version requirements. | ||
|
|
||
| This approach is not meant to discourage the use of Polaris code in downstream projects, but | ||
| to allow more flexibility in evolving the codebase to support new catalog-level features | ||
|
Comment on lines
+84
to
+85
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm a bit confused by
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this means the approach of having frequent breaking changes isn't explicitly intended to discourage the use of Polaris, but to allow ... I agree with this sentiment, but I think the comments above about a need to clearly define which APIs are actually considered public under SemVer are fair. |
||
| and improve code efficiency. Maintainers of downstream projects are encouraged to join Polaris | ||
| mailing lists to monitor project changes, suggest improvements, and engage with the Polaris | ||
| community in case of specific compatibility concerns. | ||
|
|
||
| ## Semantic Versioning | ||
|
|
||
| Polaris strives to follow [Semantic Versioning](https://semver.org/) conventions both with | ||
| respect to Java code and REST APIs. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same comment here that REST APIs doesn't follow semver. |
||
|
|
||
| The API surface for the purpose of Semantic Versioning is defined as follows. | ||
|
|
||
| **Client-Facing API** | ||
|
|
||
| * Iceberg REST Catalog API and Generic Tables API (refer to this [link](../polaris-catalog-service/) | ||
| for their combined Open API definition). | ||
| * Note: Polaris implementing an optional Iceberg REST Catalog feature that was unimplemented | ||
| in the previous release is not considered a major change. | ||
| * Supporting a new revision of the Iceberg REST Catalog spec in a backward-compatible way | ||
| is not considered a major change. | ||
| * Changing the implementation of an Iceberg REST Catalog feature / endpoint in a non-backward | ||
| compatible way is a major change. | ||
| * [Polaris Management API](../polaris-management-service/) | ||
| * [Polaris Policies](http://localhost:1313/in-dev/unreleased/policy/) | ||
|
flyrain marked this conversation as resolved.
Outdated
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Again, as I said here, #1890 (comment). I don't think the REST APIs follow Semver. Can we remove them here?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure I follow your suggestion, @flyrain . This is not about identifying a particular API spec revision (v1, v2, etc.) but about changes in Polaris server behaviour. IMHO, adding support for a new API spec version (e.g. v2) is not a major change from the SemVer perspective, because it does not break backward-compatibility. In that regard, Polaris may support several API spec versions in the same release of Polaris. On the other hand, if Polaris had a bug in implementing the Catalog REST API (for example) such that fixing it introduced a backward incompatible change in behaviour, why would we not do a major Polaris version bump? I think it would make sense. WDYT? |
||
| * Configuration | ||
| * Public (not internal) properties inside Polaris [Entities](../entities/) that affect Polaris behaviour. | ||
| * Runtime [configuration](../configuration/) properties with the `polaris.` name prefix. | ||
|
|
||
| **Build-Time Dependencies** | ||
|
|
||
| * The `polaris-core` module (`public` classes and interfaces) | ||
|
|
||
| **Non-API Modules** | ||
|
|
||
| Build-time dependencies other than the `polaris-core` module are considered non-API | ||
| and are not subject to tracking in semantic version number changes. Their `public` | ||
| Java interfaces may change without notice even in minor releases. | ||
Uh oh!
There was an error while loading. Please reload this page.