Add release notes for 0.239#14908
Conversation
|
Can you comment with the missing release notes for the PRs that you authored/merged - @caithagoras , @highker , @rschlussel , @mbasmanova , @zhenxiao , @arhimondr |
| * Add local disk spilling support for aggregation functions with `ORDER BY` or `DISTINCT` syntax. | ||
| * Add optimization for cursor projection & filter by extract and compute common subexpressions among all projections & filter first. This optimization can be turned off by session property ``optimize_common_sub_expressions``. | ||
| * Add support for 2 security modes for views. The default `DEFINER` security mode is the same as the previous behavior. Tables referenced in the view are accessed using the permissions of the view owner (the **creator** or. | ||
| * Add support for warning on unfiltered partition keys using `partition-keys-to-warn-on-no-filtering` system property. |
There was a problem hiding this comment.
I updated the description with the release note for this PR for #14800 but I can't seem to open a PR against your repo to update it here.
|
I'm not sure if this needs a release note, but it is a dependency change. Also, maybe it should be under the accumulo and kafka release notes since that is where the dependency is actually used. General Changes
|
| * Remove experimental feature to perform grouped execution for eligible table scans and its associated configuration property ``experimental.grouped-execution-for-elligible-table-scans`` and session property ``grouped_execution_for_eligible_table_scans``. | ||
| * Definer** of the view) rather than the user executing the query. In the `INVOKER` security mode, tables referenced in the view are accessed using the permissions of the query user (the **invoker** of the view). | ||
| * Enable ``dynamic-schedule-for-grouped-execution`` by default. In future releases, we will remove this property, and grouped execution will always use dynamic scheduling. | ||
| * Enable ``grouped-execution-for-aggregation`` and ``experimental.grouped-execution-for-eligible-table-scans`` by default. |
There was a problem hiding this comment.
sorry for the confusion. The commit for Remove experimental feature to perform grouped execution for eligible table scans ... was merged after this one, so this note should just say
Enable ``grouped-execution-for-aggregation`` by default.
| * Enable ``dynamic-schedule-for-grouped-execution`` by default. In future releases, we will remove this property, and grouped execution will always use dynamic scheduling. | ||
| * Enable ``grouped-execution-for-aggregation`` and ``experimental.grouped-execution-for-eligible-table-scans`` by default. | ||
| * Enable async page transport with non-blocking IO by default. This can be disabled by setting ``exchange.async-page-transport-enabled`` configuration property to false. | ||
| * Introduce new configuration property ``grouped-execution-enabled`` and session property ``grouped_execution`` to turn grouped execution on or off. This property is true by default. If set to false, it is equivalent to setting all of ``grouped-execution-for-aggregation``, ``grouped-execution-for-join``, and ``experimental.grouped-execution-for-eligible-table-scans`` to false. In future releases we will remove these other properties and only have a single switch for enabling and disabling grouped execution. |
There was a problem hiding this comment.
remove experimental.grouped-execution-for-eligilbe-table-scans from this list, as we decided to get rid of this feature entirely instead.
| * Add support for warning on unfiltered partition keys using `partition-keys-to-warn-on-no-filtering` system property. | ||
| * Add support to optimize min/max only metadata query. This is controlled by existing config ``optimizer.optimize-metadata-queries`` and session property ``optimize_metadata_queries``. Note that enabling this config/session property might change query result if there are metadata that refers to empty data, e.g. empty hive partition. | ||
| * Remove experimental feature to perform grouped execution for eligible table scans and its associated configuration property ``experimental.grouped-execution-for-elligible-table-scans`` and session property ``grouped_execution_for_eligible_table_scans``. | ||
| * Definer** of the view) rather than the user executing the query. In the `INVOKER` security mode, tables referenced in the view are accessed using the permissions of the query user (the **invoker** of the view). |
There was a problem hiding this comment.
this should be part of the note Add support for 2 security modes for views. Looks like somehow some other bullets came in between.
| * Definer** of the view) rather than the user executing the query. In the `INVOKER` security mode, tables referenced in the view are accessed using the permissions of the query user (the **invoker** of the view). | ||
| * Enable ``dynamic-schedule-for-grouped-execution`` by default. In future releases, we will remove this property, and grouped execution will always use dynamic scheduling. | ||
| * Enable ``grouped-execution-for-aggregation`` and ``experimental.grouped-execution-for-eligible-table-scans`` by default. | ||
| * Enable async page transport with non-blocking IO by default. This can be disabled by setting ``exchange.async-page-transport-enabled`` configuration property to false. |
There was a problem hiding this comment.
can we get more information about what this means?
| * Enable ``grouped-execution-for-aggregation`` and ``experimental.grouped-execution-for-eligible-table-scans`` by default. | ||
| * Enable async page transport with non-blocking IO by default. This can be disabled by setting ``exchange.async-page-transport-enabled`` configuration property to false. | ||
| * Introduce new configuration property ``grouped-execution-enabled`` and session property ``grouped_execution`` to turn grouped execution on or off. This property is true by default. If set to false, it is equivalent to setting all of ``grouped-execution-for-aggregation``, ``grouped-execution-for-join``, and ``experimental.grouped-execution-for-eligible-table-scans`` to false. In future releases we will remove these other properties and only have a single switch for enabling and disabling grouped execution. | ||
| * Reverting Reliable Resource Group Versioning. |
There was a problem hiding this comment.
needs more information and to be in proper release note format.
| * Enable async page transport with non-blocking IO by default. This can be disabled by setting ``exchange.async-page-transport-enabled`` configuration property to false. | ||
| * Introduce new configuration property ``grouped-execution-enabled`` and session property ``grouped_execution`` to turn grouped execution on or off. This property is true by default. If set to false, it is equivalent to setting all of ``grouped-execution-for-aggregation``, ``grouped-execution-for-join``, and ``experimental.grouped-execution-for-eligible-table-scans`` to false. In future releases we will remove these other properties and only have a single switch for enabling and disabling grouped execution. | ||
| * Reverting Reliable Resource Group Versioning. | ||
| * Specify allowed roles for HTTP endpoints. |
There was a problem hiding this comment.
needs to be fleshed out
There was a problem hiding this comment.
I will introduce this point once we have the proper documentation for this feature.
| * Introduce new configuration property ``grouped-execution-enabled`` and session property ``grouped_execution`` to turn grouped execution on or off. This property is true by default. If set to false, it is equivalent to setting all of ``grouped-execution-for-aggregation``, ``grouped-execution-for-join``, and ``experimental.grouped-execution-for-eligible-table-scans`` to false. In future releases we will remove these other properties and only have a single switch for enabling and disabling grouped execution. | ||
| * Reverting Reliable Resource Group Versioning. | ||
| * Specify allowed roles for HTTP endpoints. | ||
| * The worker page's thread snapshot UI does not work (no stack trace displayed on click) when there is active query load (tested under Chrome). This patch fixes an uninitialized variable in client JS that was causing this UI behavior. |
There was a problem hiding this comment.
Something like
Fix a bug where the UI for the worker's thread snapshot wouldn't display the stack trace.
Also, should be in a separate Web UI section.
|
|
||
| Hive Changes | ||
| ____________ | ||
| * Add support for caching the Glue metastore. |
There was a problem hiding this comment.
We should add a release note for encryption at rest @mayankgarg1990
- Add support for reading and writing DWRF files with encryption. To create a table with encrypted columns....
There was a problem hiding this comment.
Given that encryption cannot be used readily and neither do we have a user guide for how to set it up - my opinion is that we don't publish it in the release notes.
| * Enable ``dynamic-schedule-for-grouped-execution`` by default. In future releases, we will remove this property, and grouped execution will always use dynamic scheduling. | ||
| * Enable ``grouped-execution-for-aggregation`` and ``experimental.grouped-execution-for-eligible-table-scans`` by default. | ||
| * Enable async page transport with non-blocking IO by default. This can be disabled by setting ``exchange.async-page-transport-enabled`` configuration property to false. | ||
| * Introduce new configuration property ``grouped-execution-enabled`` and session property ``grouped_execution`` to turn grouped execution on or off. This property is true by default. If set to false, it is equivalent to setting all of ``grouped-execution-for-aggregation``, ``grouped-execution-for-join``, and ``experimental.grouped-execution-for-eligible-table-scans`` to false. In future releases we will remove these other properties and only have a single switch for enabling and disabling grouped execution. |
There was a problem hiding this comment.
also, can this be moved above the previous note so that all the grouped execution property changes are next to each other.
There was a problem hiding this comment.
I have done some overall re-formatting - will let you review the final version.
|
A quick note - I have not cleaned up the release notes yet - I am still working on cleaning up and reorganizing them. |
mayankgarg1990
force-pushed
the
release-notes-0.239
branch
from
df474ca to
d24cea7
Compare
mayankgarg1990
force-pushed
the
release-notes-0.239
branch
from
d24cea7 to
855901d
Compare
|
@tdcmeehan , @rschlussel - the release notes are ready for review |
There was a problem hiding this comment.
just realized the property is called grouped-execution-for-aggregation-enabled.
There was a problem hiding this comment.
Can you also mention here that grouped-execution-for-aggregation-enabled and grouped-execution-for-join-enabled will be removed in a future release.
There was a problem hiding this comment.
mentioned grouped-execution-for-aggregation-enabled next to the point below where I say that it is enabled by default. Added the point about join one here
There was a problem hiding this comment.
what are the configurations?
There was a problem hiding this comment.
elasticsearch.max-http-connections : Maximum number of persistent HTTP connections to Elasticsearch.
elasticsearch.http-thread-count : Number of threads handling HTTP connections to Elasticsearch
There was a problem hiding this comment.
added the detailed configuration information
There was a problem hiding this comment.
what's a composite publish_address? (maybe this is just because I don't know elasticsearch)
There was a problem hiding this comment.
publish_address field can contain addresses of the following form:
cname/ip:port
ip:port
If the CNAME is present, elasticsearch connector will use the CNAME and port.
Otherwise, it will use the IP and port.
There was a problem hiding this comment.
Updated publish_address to be back quoted and also linked to the PR for context
There was a problem hiding this comment.
This description is a bit too technical. If this is accurate, maybe say
Optimize queries with repeated expressions in filters or projections by computing the common expressions only once. This can be disabled by the session property
``optimize_common_sub_expressions``.
There was a problem hiding this comment.
what's a min/max only metadata query?
There was a problem hiding this comment.
Queries that are doing min(), max() only kind of operations
There was a problem hiding this comment.
I have reworded this to be more clear (hopefully)
mayankgarg1990
force-pushed
the
release-notes-0.239
branch
2 times, most recently
from
508c7bb to
aa23eed
Compare
There was a problem hiding this comment.
| * Fix ``NullPointerException`` in ``/v1/thread`` end point. | |
| * Fix error in ``/v1/thread`` end point. |
There was a problem hiding this comment.
| for certain queries that had filters pushed down to the table scan. | |
| for certain queries. |
There was a problem hiding this comment.
| * Fix missing query completion events for queries which fail prior to dispatching. | |
| * Fix missing query completion events for queries which fail prior to execution. |
There was a problem hiding this comment.
| * Fix potential infinite loop when the setting ``use_legacy_scheduler`` is set to ``false``. | |
| * Fix potential performance regression when setting ``use_legacy_scheduler`` is set to ``false``. |
There was a problem hiding this comment.
| * Fix a bug where the UI for the worker's thread snapshot wouldn't display the stack trace. | |
| * Fix worker thread snapshot UI to correctly display the stack trace. |
There was a problem hiding this comment.
What is the default setting?
There was a problem hiding this comment.
Generally when we say enable using, my assumption is that it is disabled by default.
mayankgarg1990
force-pushed
the
release-notes-0.239
branch
from
aa23eed to
f119c4d
Compare
There was a problem hiding this comment.
- Fix incorrect results from :func:
classification_miss_rate, :func:classification_fall_out, and :func:classification_precision(:pr:14740).
rschlussel
left a comment
rschlussel
left a comment
There was a problem hiding this comment.
Please fix the merge conflict
mayankgarg1990
force-pushed
the
release-notes-0.239
branch
from
f119c4d to
d6f2c37
Compare
mayankgarg1990
force-pushed
the
release-notes-0.239
branch
from
d6f2c37 to
12429af
Compare
There was a problem hiding this comment.
I think you need
:pr:`14740`
Could you visually check the other content of the html by running
cd presto-docs
make clean
make html
12429af to
50b5c92
Compare
7 participants
Missing Release Notes
Leiqing Cai
Nikhil Collooru
Ravion
Rebecca Schlussel
Tim Meehan
Weidong Duan
Wenlei Xie
Zhenxiao Luo
Zhi Wen
tgorthi
Extracted Release Notes
ORDER BYorDISTINCTsyntax.optimize_common_sub_expressions.classification_miss_rateand :func:classification_fall_outfunctions (:pr:14740).DEFINERsecurity mode is the same as the previous behavior. Tables referenced in the view are accessed using the permissions of the view owner (the creator or.INVOKERsecurity mode, tables referenced in the view are accessed using the permissions of the query user (the invoker of the view)./v1/threadend point.partition-keys-to-warn-on-no-filteringsystem property.DistinctLimitNodetopresto-spimodule for connectors to push down.ORDER BYsyntax.optimizer.optimize-metadata-queriesand session propertyoptimize_metadata_queries. Note that enabling this config/session property might change query result if there are metadata that refers to empty data, e.g. empty hive partition.ignore_stats_calculator_failureswould not be honored for certain queries that had filters pushed down to the table scan.use_legacy_scheduleris set to false.exchange.async-page-transport-enabledconfiguration property to false.dynamic-schedule-for-grouped-executionby default. In future releases, we will remove this property, and grouped execution will always use dynamic scheduling.grouped-execution-for-aggregationandexperimental.grouped-execution-for-eligible-table-scansby default.grouped-execution-enabledand session propertygrouped_executionto turn grouped execution on or off. This property is true by default. If set to false, it is equivalent to setting all ofgrouped-execution-for-aggregation,grouped-execution-for-join, andexperimental.grouped-execution-for-eligible-table-scansto false. In future releases we will remove these other properties and only have a single switch for enabling and disabling grouped execution.experimental.grouped-execution-for-elligible-table-scansand session propertygrouped_execution_for_eligible_table_scans.All Commits