Skip to content

Use decimals by default in tpch connector#17344

Closed
raunaqmorarka wants to merge 2 commits intotrinodb:masterfrom
raunaqmorarka:tpch-update
Closed

Use decimals by default in tpch connector#17344
raunaqmorarka wants to merge 2 commits intotrinodb:masterfrom
raunaqmorarka:tpch-update

Conversation

@raunaqmorarka
Copy link
Copy Markdown
Member

@raunaqmorarka raunaqmorarka commented May 4, 2023

Description

Use decimals by default in tpch connector
Use simplified naming convention in tpch dev catalog

Additional context and related issues

Release notes

( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# TPC-H
* Use SQL `decimal` type for floating point values. ({issue}`17344`)

Copy link
Copy Markdown
Member

@findepi findepi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as long as tests pass.

@raunaqmorarka raunaqmorarka force-pushed the tpch-update branch 3 times, most recently from c6d9422 to d242a06 Compare May 4, 2023 18:24
@github-actions github-actions bot added delta-lake Delta Lake connector iceberg Iceberg connector mongodb MongoDB connector labels May 4, 2023
@raunaqmorarka raunaqmorarka force-pushed the tpch-update branch 3 times, most recently from 665826e to 5735f10 Compare May 6, 2023 02:44
@raunaqmorarka
Copy link
Copy Markdown
Member Author

LGTM as long as tests pass.

@findepi tests are green now, but it required a few more tweaks. Please take another look at it.

@raunaqmorarka raunaqmorarka requested a review from findepi May 6, 2023 07:54
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW if this is inserted in pretty much every query runner, how can i know that ATF would pass for a new test class that doesn't have this property?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to do this because all these connectors have written test assertions based on double types in tpch. It was requiring a lot of trial and error to uncover and change every such instance. In some cases the test actually needs a double data type. In other cases decimal type is not supported by the connector.
You don't need to set this property for any new tests, it's there to avoid breaking tests which were written when tpch tables had double type.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which BCT & BCST subclasses exercise BCT / BCST with tpch with decimals?

when will the existing tests migrated to use tpch with decimals?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which BCT & BCST subclasses exercise BCT / BCST with tpch with decimals?

I've updated TestTpchConnectorTest to use decimals now. I'm guessing that you're concerned about whether some BCT is run with tpch+decimals rather than which connectors are specifically doing what.

when will the existing tests migrated to use tpch with decimals?

I think it is legitimate for other connectors and integration tests to configure tpch with doubles and continue to use that in their tests. There are tests which are specifically about double type and there are connectors which don't support decimals, so we definitely can't migrate everything. Whether we should migrate the tests which can be migrated depends on what's the benefit of making those changes.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated TestTpchConnectorTest to use decimals now.

tpch doesn't support create table, so i guess this test class has many tests skipped.

when will the existing tests migrated to use tpch with decimals?

I think it is legitimate for other connectors and integration tests to configure tpch with doubles and continue to use that in their tests.

absolutely.

but the end state should be consistent: all connectors that can use decimals, should use decimals
and exceptions to the rule should be have rationale documented

if we cannot achieve consistency that way, maybe we should revert 0c58983 instead

@raunaqmorarka
Copy link
Copy Markdown
Member Author

^ Just rebased to master

@github-actions
Copy link
Copy Markdown

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

@github-actions github-actions bot added the stale label Jan 16, 2024
@mosabua
Copy link
Copy Markdown
Member

mosabua commented Jan 16, 2024

@raunaqmorarka is this still in progress?

@github-actions github-actions bot removed the stale label Jan 18, 2024
@github-actions
Copy link
Copy Markdown

github-actions bot commented Feb 9, 2024

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

@github-actions github-actions bot added the stale label Feb 9, 2024
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 1, 2024

Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.

@github-actions github-actions bot closed this Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed delta-lake Delta Lake connector hive Hive connector iceberg Iceberg connector mongodb MongoDB connector stale

Development

Successfully merging this pull request may close these issues.

3 participants