Skip to content

Do not enforce fixed partitioning in Hive connector#13340

Merged
sopel39 merged 2 commits intotrinodb:masterfrom
starburstdata:ks/do_not_enforce
Jul 26, 2022
Merged

Do not enforce fixed partitioning in Hive connector#13340
sopel39 merged 2 commits intotrinodb:masterfrom
starburstdata:ks/do_not_enforce

Conversation

@sopel39
Copy link
Member

@sopel39 sopel39 commented Jul 25, 2022

Engine will ensure that consecutive buckets are assigned
to shuffled nodes, therefore it's not needed to explicity
provide that by connector

Description

Is this change a fix, improvement, new feature, refactoring, or other?

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

How would you describe this change to a non-technical end user or system administrator?

Related issues, pull requests, and links

Documentation

( ) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

( ) No release notes entries required.
( ) Release notes entries required with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@sopel39
Copy link
Member Author

sopel39 commented Jul 25, 2022

This essentially reverts 9558ff4. I honestly cannot remember why I've added that in the first place

Only io.trino.sql.planner.NodePartitioningManager#getNodePartitioningMap and io.trino.sql.planner.NodePartitioningManager#getBucketNodeMap use io.trino.spi.connector.ConnectorBucketNodeMap#hasFixedMapping and they assign same shuffle mapping when when explicit connector mapping is not present

@sopel39 sopel39 force-pushed the ks/do_not_enforce branch from 17a33af to 0f1f9f1 Compare July 26, 2022 08:18
sopel39 added 2 commits July 26, 2022 14:32
Engine will ensure that consecutive buckets are assigned
to shuffled nodes, therefore it's not needed to explicity
provide that by connector
@sopel39 sopel39 force-pushed the ks/do_not_enforce branch from 0f1f9f1 to 4570f09 Compare July 26, 2022 12:32
@sopel39 sopel39 merged commit 662de67 into trinodb:master Jul 26, 2022
@sopel39 sopel39 deleted the ks/do_not_enforce branch July 26, 2022 15:53
@github-actions github-actions bot added this to the 392 milestone Jul 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants