Skip to content

Add 0.12.0 release notes pt 2 #2986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cwsteinbach merged 8 commits into from
Aug 19, 2021
Merged

Add 0.12.0 release notes pt 2 #2986

Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
fadf368
Add 0.12.0 release notes pt 2
cwsteinbach Aug 17, 2021
aedee9a
Add more blurbs and fix formatting.
cwsteinbach Aug 17, 2021
fa6023e
Add blurb for #2613 Hive Vectorized Reader
cwsteinbach Aug 17, 2021
6c798f5
Reword blurbs for #2565 and #2365
cwsteinbach Aug 17, 2021
00480e8
More changes based on review comments
cwsteinbach Aug 17, 2021
8e50cb3
More updates to the 0.12.0 release notes
cwsteinbach Aug 17, 2021
6e0625d
Add blurb for #2232 fix parquet row group filters
cwsteinbach Aug 17, 2021
fdc2811
Add blurb for #2308
cwsteinbach Aug 19, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 21 additions & 13 deletions site/docs/releases.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,28 +64,36 @@ High-level features:

* **Core**
* Allow Iceberg schemas to specify one or more columns as row identifers [[\#2465](https://github.com/apache/iceberg/pull/2465)]. Note that this is a prerequisite for supporting upserts in Flink.
* Added JDBC [[\#1870](https://github.com/apache/iceberg/pull/1870)] and DynamoDB [[\#2688](https://github.com/apache/iceberg/pull/2688)] Iceberg catalog implementations.
* Added JDBC [[\#1870](https://github.com/apache/iceberg/pull/1870)] and DynamoDB [[\#2688](https://github.com/apache/iceberg/pull/2688)] catalog implementations.
* Added predicate pushdown for partitions and files metadata tables [[\#2358](https://github.com/apache/iceberg/pull/2358), [\#2926](https://github.com/apache/iceberg/pull/2926)].
* Added a new, more flexible compaction action for Spark that can support different strategies such as bin packing and sorting. [[\#2501](https://github.com/apache/iceberg/pull/2501), [\#2609](https://github.com/apache/iceberg/pull/2609)].
* Added the ability to upgrade to v2 or create a v2 table using the table property format-version=2 [[\#2887](https://github.com/apache/iceberg/pull/2887)].
* **Flink**
* Added support for SQL primary keys [[\#2410](https://github.com/apache/iceberg/pull/2410)].
* **Hive**
* Added the ability to set the catalog at the table level in the Hive Metastore. This makes it possible to write queries that reference tables from multiple catalogs [[\#2129](https://github.com/apache/iceberg/pull/2129)].
* Deprecated the configuration property `iceberg.mr.catalog` is now deprecated [[\#2565](https://github.com/apache/iceberg/pull/2565)].
* Enabled dropping HMS tables despite metadata problems [[\#2583](https://github.com/apache/iceberg/pull/2583)].
* Added table-level JVM lock on commits[[\#2547](https://github.com/apache/iceberg/pull/2547)].
* Use Hive's vectorized ORC reader [[\#2613](https://github.com/apache/iceberg/pull/2613)].
* **Spark**
* [[\#2560](https://github.com/apache/iceberg/pull/2560)] added extensions DDL to set identifier fields.
* [[\#2365](https://github.com/apache/iceberg/pull/2365)] added support for `ALTER TABLE table CHANGE PARTITION FIELD transform TO transform` DDL.
* [[\#2660](https://github.com/apache/iceberg/pull/2660)] added support for micro-batch streaming reads for structured streaming in Spark3.
* [[\#2777](https://github.com/apache/iceberg/pull/2777)] improves the performance of importing a Hive table by not loading all partitions from Hive and instead pushing the partition filter to the Metastore.
* [[\#2193](https://github.com/apache/iceberg/pull/2193), [\#2206](https://github.com/apache/iceberg/pull/2206)] added support for UPDATE statements in Spark.
* **Flink** now support SQL primary keys [[\#2410](https://github.com/apache/iceberg/pull/2410)].
* Added extensions DDL to set identifier fields [[\#2560](https://github.com/apache/iceberg/pull/2560)].
* Added support for `ALTER TABLE table CHANGE PARTITION FIELD transform TO transform` DDL [[\#2365](https://github.com/apache/iceberg/pull/2365)].
* Added support for micro-batch streaming reads for structured streaming in Spark3 [[\#2660](https://github.com/apache/iceberg/pull/2660)].
* Improved the performance of importing a Hive table by not loading all partitions from Hive and instead pushing the partition filter to the Metastore [[\#2777](https://github.com/apache/iceberg/pull/2777)].
* Added support for UPDATE statements in Spark [[\#2193](https://github.com/apache/iceberg/pull/2193), [\#2206](https://github.com/apache/iceberg/pull/2206)].


Important bug fixes:

* **Core**
* [\#2849](https://github.com/apache/iceberg/pull/2849) fixes string bucketing with non-BMP characters
* [\#2551](https://github.com/apache/iceberg/pull/2551) fixes Parquet dictionary filter not handling fixed length byte arrays.
* [\#2550](https://github.com/apache/iceberg/pull/2550) fixes a problem with the configuration of HiveCatalog.
* Fixed string bucketing with non-BMP characters [[\#2849](https://github.com/apache/iceberg/pull/2849)].
* Fixed Parquet dictionary filter not handling fixed length byte arrays [[\#2551](https://github.com/apache/iceberg/pull/2551)].
* Fixed a problem with the configuration of HiveCatalog [[\#2550](https://github.com/apache/iceberg/pull/2550)].
* **Spark**
* [\#2584](https://github.com/apache/iceberg/pull/2584) fixes MERGE INTO in Spark when used with SinglePartition partitioning.
* [\#2877](https://github.com/apache/iceberg/pull/2877) fixes nested struct pruning in Spark
* [\#2757](https://github.com/apache/iceberg/pull/2757) fixes Timestamp without Timezone type support in Spark
* Fixed `MERGE INTO` in Spark when used with SinglePartition partitioning [[\#2584](https://github.com/apache/iceberg/pull/2584)].
* Fixed nested struct pruning in Spar [[\#2877](https://github.com/apache/iceberg/pull/2877)].
* Fixed Timestamp without Timezone type support in Spark [[\#2757](https://github.com/apache/iceberg/pull/2757)].

Other notable changes:

Expand Down