Add release notes for 0.260 by varungajjala · Pull Request #16619 · prestodb/presto

varungajjala · 2021-08-16T21:32:14Z

Missing Release Notes

Bin Fan

Fix CachingFileSystem to include missing override #16576 Fix CachingFileSystem to include missing override (Merged by: Bin Fan)

vaishnavibatni

Updating release notes for 0.259 and 0.259.1 #16606 Updating release notes for 0.259 and 0.259.1 (Merged by: Rongrong Zhong)
Revert "Supports pushing down part of the filters when all filters cannot be pushed down" #16604 Revert "Supports pushing down part of the filters when all filters cannot be pushed down" (Merged by: Maria Basmanova)

Extracted Release Notes

Add support for Prometheus plugin #15194 (Author: George Wang): Add support for Prometheus plugin
- Add support for Prometheus connector plugin.
Allow multiple or missing Hive bucket files #16456 (Author: v-jizhang): Allow multiple or missing Hive bucket files
- Allow multiple or missing Hive bucket files This can be configured by using Hive Configuration Property hive.create-empty-bucket-files. Changes are also made to use Hive naming convention for bucket file names when computing bucket file name.
Rewrite AGG(IF()) to AGG() FILTER() #16534 (Author: Zhan Yuan): Rewrite AGG(IF()) to AGG() FILTER()
- Introduce a new config optimizer.aggregation-if-to-filter-rewrite-enabled and its corresponding session property aggregation_if_to_filter_rewrite_enabled to enable or disable an optimizer rule to improve the query performance of IF expressions inside aggregation functions.
Fix multithread writing for fragment result cache #16546 (Author: superqtqt): Fix multithread writing for fragment result cache
- Fix multithread writing for fragment result cache fragment cache bug #16455.
- Add config fragment-result-cache.max-single-pages-size to control the max fragement cache file size.
Set experimental.join-spill-enabled to true by default #16560 (Author: Rebecca Schlussel): Set experimental.join-spill-enabled to true by default
- When spill is enabled, enable join spilling by default. This can be disabled by setting the configuration property experimental.join-spill-enabled or session property join_spill_enabled to false.
Improvements to disabling spill for distinct and order by aggregations #16564 (Author: Rebecca Schlussel): Improvements to disabling spill for distinct and order by aggregations
- Add session property query_max_revocable_memory_per_node to override existing configuration property experimental.max-revocable-memory-per-node.
Add UUID type and operators #16573 (Author: v-jizhang): Add UUID type and operators
- Add UUID type and operators.
Fix lambda desugar for sql functions #16577 (Author: Rongrong Zhong): Fix lambda desugar for sql functions
- Fix a bug in desugar lambda expression in sql functions that would cause compiler error when sql function references input in lambda expression when session property inline_sql_function is set to false.
Add configs to control aggregate/Window/Orderby Spilling #16581 (Author: Arjun Gupta): Add configs to control aggregate/Window/Orderby Spilling
- Add a new configuration property experimental.aggregation-spill-enabled to control aggregate spills explicitly. This can be overridden by aggregation_spill_enabled session property.
- Add a new configuration property experimental.window-spill-enabled to control window spills explicitly. This can be overridden by window_spill_enabled session property.
- Add a new configuration property experimental.order-by-spill-enabled to control order by spills explicitly. This can be overridden by order_by_spill_enabled session property.
Upgrade Iceberg version to 0.11.1 #16603 (Author: Jack Ye): Upgrade Iceberg version to 0.11.1
- Upgrade Iceberg version to 0.11.1.

All Commits

a6f1d0f Add custom headers to JDBC driver (Basar Hamdi Onat)
ccfb095 Fix session property removed in verifier (Mayank Garg)
247a020 Revert "Skip empty bucket creation by default" (Ariel Weisberg)
495e4d0 Updating release notes of 0.259 (vaishnavibatni)
ebeb8c6 Upgrade Iceberg version to 0.11.1 (Jack Ye)
d28a1fc Revert "Support partial pushdown of JDBC filters." (vaishnavibatni)
fda59c3 Fix multithread writing for fragment result cache (superqtqt)
2b4d50c Add configs to control aggregate/Window/Orderby Spilling (Arjun Gupta)
a1f14ed Enhance unit tests by validating query plan (Zac Wen)
77d7a0c Return TIMESTAMP WITH TIME ZONE from Prometheus connector (George Wang)
8805ba9 Add support for Prometheus plugin (George Wang)
84247f7 Miscellaneous formatting changes (Rebecca Schlussel)
a778192 Always create hive target directory if not present (Rebecca Schlussel)
0138bb4 Validate table directory exists in FileHiveMetastore (Rebecca Schlussel)
f14cbe1 Use absolute uris for QueryRunners using the FileHiveMetastore (Rebecca Schlussel)
dbc3d75 Fix OutOfBounds error when strip has above MAX_INT rows (Arunachalam Thirupathi)
f06f460 Improve window function documentation (Rebecca Schlussel)
ce71526 Disable metadata query rewrite for incomplete stats (Zhan Yuan)
bf8b8a3 Add UUID type and operators (v-jizhang)
6386e9b Update CachingFileSystem.java (Bin Fan)
6cd2b9d Fix CachingFileSystem to include missing override (Zhongting Hu)
d4d852b Fix Out Of Bound error in selective reader (Arunachalam Thirupathi)
3cc496d Fix memory accounting of DictionaryRowGroup (linzebing)
69f63c5 Document Hive create empty buckets config (v-jizhang)
0e23ca5 Skip empty bucket creation by default (v-jizhang)
bd666e3 Allow multiple or missing Hive bucket files (v-jizhang)
ff78660 Fix lambda desugar for sql functions (Rongrong Zhong)
b8e105f Disable AGG(IF()) rewrite for nondeterministic functions (Zhan Yuan)
72644f6 Disable AGG(IF()) rewrite for AGG applied on NULLs (Zhan Yuan)
16b910e Update documentation for spill (Rebecca Schlussel)
1aeef5b Fix tests for disabling distinct + order by aggregation spill (Rebecca Schlussel)
f6b301f Remove unnecessary session property from spill test setup (Rebecca Schlussel)
066db66 Add session property of max revocable memory per-node (Rebecca Schlussel)
220210b Improve how spill is disabled for disinct/order by aggregations (Rebecca Schlussel)
b62c3ce Restrict functions for Materialized view optimization (Julian Zhuoran Zhao)
ceee614 Fix compile error of upgrading alluxio to 2.6.1 (Beinan Wang)
4ac8e65 Support exclusively positive integer columns (gxin@fb.com)
029e3fc Add queryId to Hive temporary table names (shenh062326)
3127c28 Set experimental.join-spill-enabled to true by default (Rebecca Schlussel)
e67f97c Introduce commit API for Spiller (Arjun Gupta)
18542b5 Get materialized views from base table metadata (Zac Wen)
5cb2b91 Add a helper to parse SchemaTableName (Zac Wen)
39cc942 Add session property and config to control AGG IF rewrite (Zhan Yuan)
9ab0285 Rewrite AGG(IF()) to AGG() FILTER() (Zhan Yuan)
28b43b7 Allow and/or to take subclasses of RowExpression (Zhan Yuan)
b1e2292 Fix aggregation mask matcher (Zhan Yuan)
f880608 Add filter validation for query optimization (Julian Zhuoran Zhao)

varungajjala · 2021-08-17T17:12:21Z

presto-docs/src/main/sphinx/release/release-0.260.rst

How about changing this to -
Enable join spilling by default when spill is enabled. This can be disabled by setting the configuration property experimental.join-spill-enabled or session property join_spill_enabled to false.

ajaygeorge · 2021-08-17T18:44:44Z

presto-docs/src/main/sphinx/release/release-0.260.rst

I think this needs to go into the Connectors section.
see https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines#order-of-sections

This is adding a new connector and not a change in the connector. I am not sure where it needs to go.

You need to make a new section Prometheus Connector Changes add to that -

* Added Prometheus Connector. See :doc:`connector/prometheus`

ajaygeorge · 2021-08-17T18:48:11Z

presto-docs/src/main/sphinx/release/release-0.260.rst

Add ``UUID`` type and operators.

Add ``UUID`` type. See :doc:`types`

The fact that we added operators like LIKE, CAST is not that important (rather expected I'd guess) - so we can skip those.

ajaygeorge · 2021-08-17T18:50:06Z

presto-docs/src/main/sphinx/release/release-0.260.rst

double quotes for the property.

ajaygeorge · 2021-08-17T18:57:05Z

presto-docs/src/main/sphinx/release/release-0.260.rst

can we reword this to make it more clear.
trinodb/trino#776 (comment) this reads so much better.

Yeah! I will just copy them

mayankgarg1990 · 2021-08-17T23:05:29Z

presto-docs/src/main/sphinx/release/release-0.260.rst

lets remove this if there are no highlights

mayankgarg1990 · 2021-08-17T23:05:51Z

presto-docs/src/main/sphinx/release/release-0.260.rst

sql -> SQL

mayankgarg1990 · 2021-08-17T23:10:29Z

presto-docs/src/main/sphinx/release/release-0.260.rst

This is too low level. We should write it in a way such that it is clear what is the end impact. Did this lead to a query failure ? Did this lead to a performance regression?

@shixuan-fan Can you provide more details here since you reviewed this code?

How about: Fix a bug in fragment result cache that might cause query failure?

Fix a bug in fragment result cache that caused query failures

mayankgarg1990 · 2021-08-17T23:17:38Z

presto-docs/src/main/sphinx/release/release-0.260.rst

Add ``UUID`` type. See :doc:`types`

The fact that we added operators like LIKE, CAST is not that important (rather expected I'd guess) - so we can skip those.

mayankgarg1990 · 2021-08-17T23:24:05Z

presto-docs/src/main/sphinx/release/release-0.260.rst

You need to make a new section Prometheus Connector Changes add to that -

* Added Prometheus Connector. See :doc:`connector/prometheus`

mayankgarg1990 · 2021-08-17T23:24:44Z

presto-docs/src/main/sphinx/release/release-0.260.rst

look at how to phrase the line when we have both configuration and session property - https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines#phrasing

mayankgarg1990 · 2021-08-17T23:24:55Z

presto-docs/src/main/sphinx/release/release-0.260.rst

https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines#phrasing

to toggle the optimization which improves performance of ``IF`` expressions inside aggregation functions

mayankgarg1990 · 2021-08-17T23:32:31Z

presto-docs/src/main/sphinx/release/release-0.260.rst

Hive Connector Changes

mayankgarg1990 · 2021-08-17T23:32:44Z

presto-docs/src/main/sphinx/release/release-0.260.rst

Iceberg Connector Changes

Yeah I was just talking to @ajaygeorge about adding connector in the names. Good that you called it out.

mayankgarg1990 · 2021-08-19T00:52:56Z

presto-docs/src/main/sphinx/release/release-0.260.rst

improves the performance of instead of query performance

mayankgarg1990 · 2021-08-19T00:53:34Z

presto-docs/src/main/sphinx/release/release-0.260.rst

This is unclear to me. Is there some documentation from Hive or something that can make this clear?

mayankgarg1990 · 2021-08-19T00:54:30Z

presto-docs/src/main/sphinx/release/release-0.260.rst

Add support to create files for empty buckets when writing data

varungajjala commented Aug 17, 2021

View reviewed changes

varungajjala force-pushed the release-notes-0.260 branch from 4458ef5 to f556993 Compare August 17, 2021 17:57

ajaygeorge reviewed Aug 17, 2021

View reviewed changes

presto-docs/src/main/sphinx/release/release-0.260.rst Outdated

Copy link

Contributor

ajaygeorge Aug 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

double quotes for the property.

ajaygeorge reviewed Aug 17, 2021

View reviewed changes

varungajjala marked this pull request as draft August 17, 2021 19:05

mayankgarg1990 suggested changes Aug 17, 2021

View reviewed changes

varungajjala force-pushed the release-notes-0.260 branch from f556993 to d317cd2 Compare August 18, 2021 22:41

varungajjala marked this pull request as ready for review August 18, 2021 22:44

varungajjala requested a review from mayankgarg1990 August 18, 2021 22:45

mayankgarg1990 reviewed Aug 19, 2021

View reviewed changes

Add release notes for 0.260

ac60281

varungajjala force-pushed the release-notes-0.260 branch from d317cd2 to ac60281 Compare August 19, 2021 19:01

varungajjala requested review from highker and mayankgarg1990 August 19, 2021 19:19

mayankgarg1990 approved these changes Aug 19, 2021

View reviewed changes

highker approved these changes Aug 19, 2021

View reviewed changes

highker merged commit 8acec0b into prestodb:master Aug 19, 2021

Conversation

varungajjala commented Aug 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Missing Release Notes

Bin Fan

vaishnavibatni

Extracted Release Notes

All Commits

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ajaygeorge Aug 17, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

varungajjala commented Aug 16, 2021 •

edited

Loading

ajaygeorge Aug 17, 2021 •

edited

Loading