CBO documentation by findepi · Pull Request #127 · trinodb/trino

findepi · 2019-02-01T10:31:01Z

See https://docs.starburstdata.com/latest/optimizer.html how this looks like.

cla-bot · 2019-02-01T10:31:03Z

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@prestosql.io. If you are contributing on behalf of someone else (e.g., your employer), the individual CLA may not be sufficient and your employer may need the Corporate CLA signed.

cla-bot · 2019-02-01T10:32:27Z

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@prestosql.io. If you are contributing on behalf of someone else (e.g., your employer), the individual CLA may not be sufficient and your employer may need the Corporate CLA signed.

cla-bot · 2019-02-01T11:29:05Z

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@prestosql.io. If you are contributing on behalf of someone else (e.g., your employer), the individual CLA may not be sufficient and your employer may need the Corporate CLA signed.

findepi · 2019-02-01T14:34:29Z

CLA check is red, but the commits are annotated with Extracted-From:. @electrum ready for review.

cla-bot · 2019-02-12T09:08:02Z

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@prestosql.io. If you are contributing on behalf of someone else (e.g., your employer), the individual CLA may not be sufficient and your employer may need the Corporate CLA signed.

electrum

Please squash the fixups in the last commit into the previous commits as appropriate

electrum · 2019-02-27T21:29:30Z

presto-docs/src/main/sphinx/optimizer.rst

Use dashes in file names (like other files)

electrum · 2019-02-27T21:30:41Z

presto-docs/src/main/sphinx/index.rst

Maybe put this after admin?

electrum · 2019-02-27T21:31:28Z

presto-docs/src/main/sphinx/optimizer/cost-in-explain.rst

Please wrap documentation text at 80-100 characters. These line that wrap in GitHub are too long. Pre-formatted text like explain plans are fine to be as along as necessary, of course.

electrum · 2019-02-27T21:33:10Z

presto-docs/src/main/sphinx/optimizer/cost-in-explain.rst

What is "root" here? Maybe just remove that word.

electrum · 2019-02-27T21:33:44Z

presto-docs/src/main/sphinx/optimizer/cost-in-explain.rst

Use a link

:doc:`/sql/explain`

electrum · 2019-02-27T22:52:28Z

presto-docs/src/main/sphinx/optimizer/statistics.rst

t does not exist in the example. How about

will show statistics for the table layout representing a subset of data after applying the given filtering condition

electrum · 2019-02-27T22:54:24Z

presto-docs/src/main/sphinx/optimizer/statistics.rst

Change since we don't support a column list in the select clause

The filtering condition used in the ``WHERE`` clause can reference table columns.

Although I'm not sure this sentence adds any value. What else would it reference or why else would it be used?

electrum · 2019-02-27T23:03:49Z

presto-docs/src/main/sphinx/optimizer/statistics.rst

How about

If the ``WHERE`` clause filters out all of the data in the table, then ``SHOW STATS`` will return no statistics, as shown in the example below.

electrum · 2019-02-27T23:05:01Z

presto-docs/src/main/sphinx/optimizer/statistics.rst

Remove this section since it is no longer supported

electrum · 2019-02-27T23:06:19Z

presto-docs/src/main/sphinx/optimizer/statistics.rst

Please update this section to use Presto's ANALYZE

I am dropping this whole section (below for reference). I added For the Hive connector, refer to :ref:`Hive connector <hive_analyze>` documentation if you want to update table statistics. instead.

Updating Statistics For Hive Tables ----------------------------------- For the Hive connector, Presto uses the statistics that are managed by Hive and exposed via the Hive metastore API. Depending on the Hive configuration, table statistics may not be updated automatically. If statistics are not updated automatically, the user needs to trigger a statistics update via the Hive CLI. The following command can be used in the Hive CLI to update table statistics for non-partitioned table ``t``:: hive> ANALYZE TABLE t COMPUTE STATISTICS; hive> ANALYZE TABLE t COMPUTE STATISTICS FOR COLUMNS; For partitioned tables, partitioning information must be specified in the command. Assuming table ``t`` has two partitioning keys ``a`` and ``b``, the following command would update the table statistics for all partitions:: hive> ANALYZE TABLE t PARTITION (a, b) COMPUTE STATISTICS FOR COLUMNS; It is also possible to update statistics for just a subset of partitions. This command will update statistics for all partitions for which partitioning key ``a`` is equal to ``1``:: hive> ANALYZE TABLE t PARTITION (a=1, b) COMPUTE STATISTICS FOR COLUMNS; And this command will update statistics for just one partition:: hive> ANALYZE TABLE t PARTITION (a=1, b=5) COMPUTE STATISTICS FOR COLUMNS; For documentation on Hive's statistics mechanism see https://cwiki.apache.org/confluence/display/Hive/StatsDev

@mattsfuller

findepi · 2019-03-06T12:21:45Z

ac

electrum

A few nits, otherwise looks great!

electrum · 2019-03-08T19:26:10Z

presto-docs/src/main/sphinx/optimizer/statistics.rst

Nit: don't wrap a single word. We wrap to improve readability, but this is less readable

electrum · 2019-03-08T19:26:47Z

presto-docs/src/main/sphinx/optimizer/statistics.rst

Add "the" at end

refer to the ...

electrum · 2019-03-08T19:27:25Z

presto-docs/src/main/sphinx/optimizer/statistics.rst

How about

refer to the ... documentation to learn how to update table statistics.

electrum · 2019-03-08T19:28:08Z

presto-docs/src/main/sphinx/optimizer/cost-based-optimizations.rst

Comma after "available"

electrum · 2019-03-08T19:30:43Z

presto-docs/src/main/sphinx/optimizer/cost-based-optimizations.rst

The sentence sounds tautological. How about

With cost based join distribution selection, Presto automatically chooses to use a partitioned or broadcast join.

electrum · 2019-03-08T19:32:48Z

presto-docs/src/main/sphinx/optimizer/cost-based-optimizations.rst

Remove code quotes and lowercase on the description part, since we are describing the type of join, not referencing a config option

* ``BROADCAST`` - broadcast join distribution is used for all joins

electrum · 2019-03-08T19:34:19Z

presto-docs/src/main/sphinx/optimizer/cost-based-optimizations.rst

Let's change this to

* Partitioned: each node participating in the query builds a hash table from only fraction of the data * Broadcast: each node participating in the query builds a hash table from all of the data (data is replicated to each node)

Since we are talking about logical operations, not config options. Also a few minor wording changes.

cla-bot · 2019-03-12T10:40:24Z

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@prestosql.io. If you are contributing on behalf of someone else (e.g., your employer), the individual CLA may not be sufficient and your employer may need the Corporate CLA signed.

findepi · 2019-03-12T10:40:41Z

ac

Extracted-From: https://github.com/starburstdata/presto

cla-bot · 2019-03-13T08:48:52Z

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@prestosql.io. If you are contributing on behalf of someone else (e.g., your employer), the individual CLA may not be sufficient and your employer may need the Corporate CLA signed.

Extracted-From: https://github.com/starburstdata/presto Cherry-pick of trinodb/trino#127 Co-authored-by: Ariel Weisberg <aweisberg@fb.com>

Extracted-From: https://github.com/starburstdata/presto Cherry-pick of trinodb/trino#127 Co-authored-by: Łukasz Osipiuk <lukasz@osipiuk.net>

Extracted-From: https://github.com/starburstdata/presto Cherry-pick of trinodb/trino#127 Co-authored-by: Ariel Weisberg <aweisberg@fb.com>

Extracted-From: https://github.com/starburstdata/presto Cherry-pick of trinodb/trino#127 Co-authored-by: Łukasz Osipiuk <lukasz@osipiuk.net>

Extracted-From: https://github.com/starburstdata/presto Cherry-pick of trinodb/trino#127 Co-authored-by: Ariel Weisberg <aweisberg@fb.com>

Extracted-From: https://github.com/starburstdata/presto Cherry-pick of trinodb/trino#127 Co-authored-by: Łukasz Osipiuk <lukasz@osipiuk.net>

findepi force-pushed the epic/cbo/pr/cbo-docs branch from 1e3e043 to 565f7f0 Compare February 1, 2019 10:32

findepi requested a review from electrum February 1, 2019 10:33

findepi force-pushed the epic/cbo/pr/cbo-docs branch from 565f7f0 to 7e28f25 Compare February 1, 2019 11:29

rzeyde-varada mentioned this pull request Feb 18, 2019

Add a comment about ColumnStatistics' unknown estimates semantics #254

Merged

electrum reviewed Feb 27, 2019

View reviewed changes

findepi force-pushed the epic/cbo/pr/cbo-docs branch 2 times, most recently from bb09b12 to 0d33efe Compare March 6, 2019 12:21

trinodb deleted a comment from cla-bot bot Mar 6, 2019

electrum approved these changes Mar 8, 2019

View reviewed changes

electrum assigned findepi Mar 8, 2019

findepi force-pushed the epic/cbo/pr/cbo-docs branch from 0d33efe to 2d98e04 Compare March 12, 2019 10:40

findepi assigned electrum and unassigned findepi Mar 12, 2019

losipiuk added 2 commits March 13, 2019 09:47

Add documentation for cost and statistics

6ced267

Extracted-From: https://github.com/starburstdata/presto

Add high level CBO documentation

6d48e0f

Extracted-From: https://github.com/starburstdata/presto

findepi force-pushed the epic/cbo/pr/cbo-docs branch from 2d98e04 to 6d48e0f Compare March 13, 2019 08:48

findepi merged commit f79058e into trinodb:master Mar 13, 2019

findepi deleted the epic/cbo/pr/cbo-docs branch March 13, 2019 10:23

findepi mentioned this pull request Mar 15, 2019

Optimize join ordering in query execution #483

Closed

aweisberg mentioned this pull request Jul 30, 2019

Cherry-pick documentation for cost, statistics, CBO prestodb/presto#13155

Merged

Conversation

findepi commented Feb 1, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cla-bot bot commented Feb 1, 2019

Uh oh!

cla-bot bot commented Feb 1, 2019

Uh oh!

cla-bot bot commented Feb 1, 2019

Uh oh!

findepi commented Feb 1, 2019

Uh oh!

cla-bot bot commented Feb 12, 2019

Uh oh!

electrum left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

findepi commented Mar 6, 2019

Uh oh!

electrum left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

electrum Mar 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cla-bot bot commented Mar 12, 2019

Uh oh!

findepi commented Mar 12, 2019

Uh oh!

cla-bot bot commented Mar 13, 2019

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

findepi commented Feb 1, 2019 •

edited

Loading

electrum Mar 8, 2019 •

edited

Loading