Skip to content

File-metastore do not allow createing view when schema is not exist#12453

Closed
Emor-nj wants to merge 14 commits intotrinodb:masterfrom
Emor-nj:fixhivefilemetastore
Closed

File-metastore do not allow createing view when schema is not exist#12453
Emor-nj wants to merge 14 commits intotrinodb:masterfrom
Emor-nj:fixhivefilemetastore

Conversation

@Emor-nj
Copy link
Copy Markdown
Member

@Emor-nj Emor-nj commented May 18, 2022

Description

Fix file-metastore allows creating views in a schema where it doesn't exist

Related issues, pull requests, and links

Fixes #10857

Documentation

No documentation is needed.

Release notes

No release notes entries required.

@cla-bot
Copy link
Copy Markdown

cla-bot bot commented May 18, 2022

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@trino.io. For more information, see https://github.com/trinodb/cla.

@Emor-nj
Copy link
Copy Markdown
Member Author

Emor-nj commented May 18, 2022

cc @ebyhr Pelase check and give me some advice. Thank you very much.

Copy link
Copy Markdown
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a test to BaseConnectorTest? You can refer to existing testCreateTableSchemaNotFound() test.

@Emor-nj Emor-nj force-pushed the fixhivefilemetastore branch from b307b3b to 62f1e71 Compare May 18, 2022 09:26
@cla-bot
Copy link
Copy Markdown

cla-bot bot commented May 18, 2022

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@trino.io. For more information, see https://github.com/trinodb/cla.

@Emor-nj
Copy link
Copy Markdown
Member Author

Emor-nj commented May 18, 2022

Could you add a test to BaseConnectorTest? You can refer to existing testCreateTableSchemaNotFound() test.
@ebyhr Thank you for your guidance, i have added the test.

Copy link
Copy Markdown
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you confirm CI failure except for TestMySqlTableStatisticsMySql8Histograms?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move to after testView().

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ebyhr
I found that the reason ci failure is trino-accumulo and trino-raptor-legacy have the same issue with HiveFileMetastore.
They both allow createing view when schema is not exist, so it caused the unit test testCreateTableSchemaNotFound() failed.
I think we can overwrite function testCreateTableSchemaNotFound() in class TestAccumuloConnectorTest and BaseRaptorConnectorTest to skip.
Then i will open new issue and pr about accumulo and raptor to solve this bug.
What do you think?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Please file an issue for Accumulo. There's no need to file an issue for Raptor since it's same as #11110

Copy link
Copy Markdown
Member Author

@Emor-nj Emor-nj May 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, something conflict with this pr and i resolve it failed,so i had opened another #12482

@cla-bot
Copy link
Copy Markdown

cla-bot bot commented May 19, 2022

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@trino.io. For more information, see https://github.com/trinodb/cla.

Emor-nj and others added 14 commits May 20, 2022 10:20
The reason is that condition `replace` is always 'true' when reached.
The reason is that condition `b.getClass().isArray()` is always 'true' when reached.
The reason is that the method `all` is annotated as `@NonNull`.
Not partitioned lookup source has better performance
than partitioned one so for small build side the overall
join performance is better even if the lookup source is
created using a single thread.
Statement ordering was incorrect previously and routine to process
pending acquires was woken up before whole state related to released
node was updated. It could still be observed by routine that released node is
occupied and pending acquire was not fulfilled.
Input and output aggregation methods require a specific parameter order
but this not was being enforces by the annotation parser.  Instead
this error causes confusing runtime errors.

Additionally, it appears that in older versions of Trino alternative
parameter orders were allowed, and there was a test for that. This test
is invalid and had been adjusted to ensure proper exceptions are thrown.
The same table name was used in testMismatchedBucketing.
@Emor-nj Emor-nj force-pushed the fixhivefilemetastore branch from 7671d64 to bbb5f46 Compare May 20, 2022 02:21
@cla-bot
Copy link
Copy Markdown

cla-bot bot commented May 20, 2022

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@trino.io. For more information, see https://github.com/trinodb/cla.

@github-actions github-actions bot added the docs label May 20, 2022
@Emor-nj Emor-nj closed this May 20, 2022
@Emor-nj Emor-nj deleted the fixhivefilemetastore branch May 20, 2022 02:22
@ebyhr
Copy link
Copy Markdown
Member

ebyhr commented May 20, 2022

@Emor-nj I can share some git commands in Slack. Please feel free to ping me in https://trino.io/slack.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

file-metastore allows creating views in a schema where it doesn't exist

6 participants