Skip to content

Add tests for AS OF syntax for tables with redirects#12831

Merged
ebyhr merged 1 commit intotrinodb:masterfrom
homar:homar/test_iceberg_time_travel_with_redirections
Jun 20, 2022
Merged

Add tests for AS OF syntax for tables with redirects#12831
ebyhr merged 1 commit intotrinodb:masterfrom
homar:homar/test_iceberg_time_travel_with_redirections

Conversation

@homar
Copy link
Copy Markdown
Member

@homar homar commented Jun 13, 2022

Description

Fixes: #12634

Is this change a fix, improvement, new feature, refactoring, or other?

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

How would you describe this change to a non-technical end user or system administrator?

Related issues, pull requests, and links

Documentation

(X) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

(X) No release notes entries required.
( ) Release notes entries required with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AsOfVersion is a bit misleading because the test verifies also the functionality of FOR TIMESTAMP ... syntax.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right initially I wanted to have 2 different tests then I changed my mind

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to add Thread.sleep(1) (same as in

public void setUp()
throws InterruptedException
{
assertQuerySucceeds("CREATE TABLE test_iceberg_read_versioned_table(a_string varchar, an_integer integer)");
assertQuerySucceeds("INSERT INTO test_iceberg_read_versioned_table VALUES ('a', 1)");
v1SnapshotId = getLatestSnapshotId("test_iceberg_read_versioned_table");
v1EpochMillis = getCommittedAtInEpochMilliSeconds("test_iceberg_read_versioned_table", v1SnapshotId);
Thread.sleep(1);
assertQuerySucceeds("INSERT INTO test_iceberg_read_versioned_table VALUES ('b', 2)");
) in order to avoid version collisions based on the same timestamp.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw that but I thought this is unnecessary. I will add it if you think it makes sense

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may very well happen that in the same millisecond there are two snapshots created and we can't distinguish then with the syntax FOR TIMESTAMP AS OF ... which snapshot we want to retrieve.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the problem I just doubt they will happen in the same milisecond.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the added value of the queries having WHERE clauses in this context?
We're checking the table redirection functionality and not the table filter functionality here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just wanted to make sure that the row I inserted is there. But you are right that doesn't bring much value

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we extract these methods to a utility class?
I think they are already multiple times duplicated.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will do that as a separate pr

@homar homar force-pushed the homar/test_iceberg_time_travel_with_redirections branch from bc594c4 to bce85db Compare June 14, 2022 14:07
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not modify the nation table directly, make a copy first

Copy link
Copy Markdown
Member

@alexjo2144 alexjo2144 Jun 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have been clearer in the issue, but I think the more interesting thing to test here is that queries on iceberg tables going through hive_with_redirections work.

Also that versioned queries on hive tables going through the iceberg_with_redirections catalog should not work. Something like select * from iceberg_with_redirection.schema.a_hive_tabl for timestamp as of ... should fail

@homar homar force-pushed the homar/test_iceberg_time_travel_with_redirections branch 3 times, most recently from e2f1c09 to 2e8b189 Compare June 15, 2022 08:26
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this table affects other tests using information_schema. testReadInformationSchema & testShowTables will fail if they are executed at the same time.

@homar homar force-pushed the homar/test_iceberg_time_travel_with_redirections branch from 2e8b189 to 0e2e789 Compare June 20, 2022 09:10
@ebyhr ebyhr added the no-release-notes This pull request does not require release notes entry label Jun 20, 2022
@ebyhr ebyhr merged commit dc3efd9 into trinodb:master Jun 20, 2022
@ebyhr
Copy link
Copy Markdown
Member

ebyhr commented Jun 20, 2022

Merged, thanks!

@github-actions github-actions bot added this to the 387 milestone Jun 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed no-release-notes This pull request does not require release notes entry

Development

Successfully merging this pull request may close these issues.

Test AS OF syntax with table redirections

4 participants