Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
<packaging>pom</packaging>

<name>presto-root</name>
Expand Down
2 changes: 1 addition & 1 deletion presto-accumulo/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-accumulo</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-array/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-array</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-atop/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-atop</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-base-jdbc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-base-jdbc</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-benchmark-driver/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-benchmark-driver</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-benchmark/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>presto-root</artifactId>
<groupId>io.prestosql</groupId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-benchmark</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-benchto-benchmarks/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-benchto-benchmarks</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-blackhole/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-blackhole</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-cassandra/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-cassandra</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-cli/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-cli</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-client</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-docs/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-docs</artifactId>
Expand Down
18 changes: 10 additions & 8 deletions presto-docs/src/main/sphinx/connector/hive.rst
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ Property Name Description
``hive.s3select-pushdown.enabled`` Enable query pushdown to AWS S3 Select service. ``false``

``hive.s3select-pushdown.max-connections`` Maximum number of simultaneously open connections to S3 for 500
S3SelectPushdown.
:ref:`s3selectpushdown`.
================================================== ============================================================ ============

Amazon S3 Configuration
Expand Down Expand Up @@ -346,23 +346,25 @@ the ``org.apache.hadoop.conf.Configurable`` interface from the Hadoop Java API,
will be passed in after the object instance is created and before it is asked to provision or retrieve any
encryption keys.

S3SelectPushdown
^^^^^^^^^^^^^^^^
.. _s3selectpushdown:

S3SelectPushdown enables pushing down projection (SELECT) and predicate (WHERE)
S3 Select Pushdown
^^^^^^^^^^^^^^^^^^

S3 Select Pushdown enables pushing down projection (SELECT) and predicate (WHERE)
processing to `S3 Select <https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectSELECTContent.html>`_.
With S3SelectPushdown Presto only retrieves the required data from S3 instead of
entire S3 objects reducing both latency and network usage.
With S3 Select Pushdown, Presto only retrieves the required data from S3 instead
of entire S3 objects, reducing both latency and network usage.

Is S3 Select a good fit for my workload?
########################################

Performance of S3SelectPushdown depends on the amount of data filtered by the
Performance of S3 Select Pushdown depends on the amount of data filtered by the
query. Filtering a large number of rows should result in better performance. If
the query doesn't filter any data then pushdown may not add any additional value
and user will be charged for S3 Select requests. Thus, we recommend that you
benchmark your workloads with and without S3 Select to see if using it may be
suitable for your workload. By default, S3SelectPushdown is disabled and you
suitable for your workload. By default, S3 Select Pushdown is disabled and you
should enable it in production after proper benchmarking and cost analysis. For
more information on S3 Select request cost, please see
`Amazon S3 Cloud Storage Pricing <https://aws.amazon.com/s3/pricing/>`_.
Expand Down
39 changes: 39 additions & 0 deletions presto-docs/src/main/sphinx/functions/aggregate.rst
Original file line number Diff line number Diff line change
Expand Up @@ -316,3 +316,42 @@ Statistical Aggregate Functions
.. function:: var_samp(x) -> double

Returns the sample variance of all input values.

Lambda Aggregate Functions
--------------------------

.. function:: reduce_agg(inputValue T, initialState S, inputFunction(S, T, S), combineFunction(S, S, S)) -> S

Reduces all input values into a single value. ``inputFunction`` will be invoked
for each non-null input value. In addition to taking the input value, ``inputFunction``
takes the current state, initially ``initialState``, and returns the new state.
``combineFunction`` will be invoked to combine two states into a new state.
The final state is returned::

SELECT id, reduce_agg(value, 0, (a, b) -> a + b, (a, b) -> a + b)
FROM (
VALUES
(1, 3),
(1, 4),
(1, 5),
(2, 6),
(2, 7)
) AS t(id, value)
GROUP BY id;
-- (1, 12)
-- (2, 13)

SELECT id, reduce_agg(value, 1, (a, b) -> a * b, (a, b) -> a * b)
FROM (
VALUES
(1, 3),
(1, 4),
(1, 5),
(2, 6),
(2, 7)
) AS t(id, value)
GROUP BY id;
-- (1, 60)
-- (2, 42)

The state type must be a boolean, integer, floating-point, or date/time/interval.
1 change: 1 addition & 0 deletions presto-docs/src/main/sphinx/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Release Notes
.. toctree::
:maxdepth: 1

release/release-300
release/release-0.215
release/release-0.214
release/release-0.213
Expand Down
114 changes: 114 additions & 0 deletions presto-docs/src/main/sphinx/release/release-300.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
===========
Release 300
===========

General Changes
---------------

* Fix :func:`array_intersect` and :func:`array_distinct`
skipping zeros when input also contains nulls.
* Fix ``count(*)`` aggregation returning null on empty relation
when ``optimize_mixed_distinct_aggregation`` is enabled.
* Improve table scan performance for structured types.
* Improve performance for :func:`array_intersect`.
* Improve performance of window functions by filtering partitions early.
* Add :func:`reduce_agg` aggregate function.
* Add :func:`millisecond` function.
* Remove ``ON`` keyword from :doc:`/sql/show-stats` (use ``FOR`` instead).
* Restrict ``WHERE`` clause in :doc:`/sql/show-stats`
to filters that can be pushed down to connectors.
* Return final results to clients immediately for failed queries.

JMX MBean Naming Changes
------------------------

* The base domain name for server MBeans is now ``presto``. The old names can be
used by setting the configuration property ``jmx.base-name`` to ``com.facebook.presto``.
* The base domain name for the Hive, Raptor, and Thrift connectors is ``presto.plugin``.
The old names can be used by setting the catalog configuration property
``jmx.base-name`` to ``com.facebook.presto.hive``, ``com.facebook.presto.raptor``,
or ``com.facebook.presto.thrift``, respectively.

Web UI
------

* Fix rendering of live plan view for queries involving index joins.

JDBC Driver Changes
-------------------

* Change driver class name to ``io.prestosql.jdbc.PrestoDriver``.

System Connector Changes
------------------------

* Remove ``node_id`` column from ``system.runtime.queries`` table.

Hive Connector Changes
----------------------

* Fix accounting of time spent reading Parquet data.
* Fix corner case where the ORC writer fails with integer overflow when writing
highly compressible data using dictionary encoding (:issue:`x11930`).
* Fail queries reading Parquet files if statistics in those files are corrupt
(e.g., min > max). To disable this behavior, set the configuration
property ``hive.parquet.fail-on-corrupted-statistics``
or session property ``parquet_fail_with_corrupted_statistics`` to false.
* Add support for :ref:`s3selectpushdown`, which enables pushing down
column selection and range filters into S3 for text files.

Kudu Connector Changes
----------------------

* Add ``number_of_replicas`` table property to ``SHOW CREATE TABLE`` output.

Cassandra Connector Changes
---------------------------

* Add ``cassandra.splits-per-node`` and ``cassandra.protocol-version`` configuration
properties to allow connecting to Cassandra servers older than 2.1.5.

MySQL Connector Changes
-----------------------

* Add support for predicate pushdown for columns of ``char(x)`` type.

PostgreSQL Connector Changes
----------------------------

* Add support for predicate pushdown for columns of ``char(x)`` type.

Redshift Connector Changes
---------------------------

* Add support for predicate pushdown for columns of ``char(x)`` type.

SQL Server Connector Changes
----------------------------

* Add support for predicate pushdown for columns of ``char(x)`` type.

Raptor Legacy Connector Changes
-------------------------------

* Change name of connector to ``raptor-legacy``.

Verifier Changes
----------------

* Add ``run-teardown-on-result-mismatch`` configuration property to facilitate debugging.
When set to false, temporary tables will not be dropped after checksum failures.

SPI Changes
-----------

* Change base package to ``io.prestosql.spi``.
* Move connector related classes to package ``io.prestosql.spi.connector``.
* Make ``ConnectorBucketNodeMap`` a top level class.
* Use list instead of map for bucket-to-node mapping.

.. note::

These are backwards incompatible changes with the previous SPI.
If you have written a plugin, you will need to update your code
before deploying this release.
23 changes: 11 additions & 12 deletions presto-docs/src/main/sphinx/sql/show-stats.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,15 @@ Description
Returns approximated statistics for the named table or for the results of a (limited) query.

Statistics are returned for each column, along with a summary row.
(column_name will be ``NULL`` for the summary row).

===================== ============
Column Description
===================== ============
column_name The name of the column
data_size The total size in bytes of all of the values in the column
distinct_values_count The number of distinct values in the column
nulls_fractions The portion of the values in the column that are ``NULL``
row_count The number of rows (only returned for the summary row)
low_value The lowest value found in this column (only for some types)
high_value The highest value found in this column (only for some types)
===================== ============
========================== =============================================================
Column Description
========================== =============================================================
``column_name`` The name of the column (``NULL`` for the sumary row)
``data_size`` The total size in bytes of all of the values in the column
``distinct_values_count`` The number of distinct values in the column
``nulls_fractions`` The portion of the values in the column that are ``NULL``
``row_count`` The number of rows (only returned for the summary row)
``low_value`` The lowest value found in this column (only for some types)
``high_value`` The highest value found in this column (only for some types)
========================== =============================================================
2 changes: 1 addition & 1 deletion presto-example-http/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-example-http</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-geospatial-toolkit/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-geospatial-toolkit</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-geospatial/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-geospatial</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-hive-hadoop2/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-hive-hadoop2</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion presto-hive/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>io.prestosql</groupId>
<artifactId>presto-root</artifactId>
<version>0.216-SNAPSHOT</version>
<version>300-SNAPSHOT</version>
</parent>

<artifactId>presto-hive</artifactId>
Expand Down
Loading