Skip to content

Iceberg connector support for Materialized Views#4832

Merged
electrum merged 5 commits intotrinodb:masterfrom
anjalinorwood:matview_iceberg
Nov 9, 2020
Merged

Iceberg connector support for Materialized Views#4832
electrum merged 5 commits intotrinodb:masterfrom
anjalinorwood:matview_iceberg

Conversation

@anjalinorwood
Copy link
Member

@anjalinorwood anjalinorwood commented Aug 14, 2020

This PR is for Iceberg connector changes to support materialized views. Note that only the top commit (
25678de) needs to be reviewed as a part of this PR. Other commits are being reviewed here: #3283

@cla-bot cla-bot bot added the cla-signed label Aug 14, 2020
@anjalinorwood anjalinorwood requested a review from martint August 14, 2020 06:13
@anjalinorwood anjalinorwood force-pushed the matview_iceberg branch 3 times, most recently from a196b29 to 4e43768 Compare August 20, 2020 16:01
@anjalinorwood anjalinorwood force-pushed the matview_iceberg branch 4 times, most recently from b97176c to 3d35309 Compare August 31, 2020 20:32
@martint martint requested a review from electrum August 31, 2020 21:15
@electrum
Copy link
Member

electrum commented Sep 1, 2020

@anjalinorwood can you rebase now that the materialized view code is merged?

@anjalinorwood
Copy link
Member Author

anjalinorwood commented Sep 1, 2020 via email •
edited
Loading

@anjalinorwood
Copy link
Member Author

anjalinorwood commented Sep 8, 2020

@martint and @electrum Did you get a chance to review this PR? Thanks.

anjalinorwood
anjalinorwood commented Sep 9, 2020
View reviewed changes
electrum
electrum reviewed Sep 15, 2020
View reviewed changes
Copy link
Member

@electrum electrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial comments, mostly superficial. Still reviewing

electrum
electrum reviewed Sep 17, 2020
View reviewed changes
electrum
electrum reviewed Sep 17, 2020
View reviewed changes
@anjalinorwood anjalinorwood force-pushed the matview_iceberg branch 2 times, most recently from 85c58a6 to e7298a3 Compare October 1, 2020 05:45
@anjalinorwood anjalinorwood force-pushed the matview_iceberg branch 2 times, most recently from 03b2126 to ccf250b Compare October 7, 2020 20:39
electrum
electrum reviewed Oct 16, 2020
View reviewed changes
Copy link
Member

@electrum electrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bunch of minor comments. Overall looks good

@anjalinorwood
Copy link
Member Author

anjalinorwood commented Oct 21, 2020

@martint Did you want to give this a look? I think @electrum is winding up his review. Thank you both.

martint
martint reviewed Oct 28, 2020
View reviewed changes
Copy link
Member

@martint martint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed all commits except for "Iceberg connector support for Materialized Views". Just a couple of minor comments.

@electrum, can you finish reviewing that other commit?

electrum
electrum reviewed Oct 29, 2020
View reviewed changes
@anjalinorwood
Disallow 'CREATE OR REPLACE' and 'IF NOT EXISTS'...
f7215ec 
... clauses together in 'CREATE MATERIALIZED VIEW' statement.

The optional 'IF NOT EXISTS' clause causes the error to be suppressed
if a materialized view already exists. This clause was silently ignored
if 'OR REPLACE' was specified.
This can be confusing to users, so this commit explicitly throws an error
when the two clauses are specified together.
@anjalinorwood
Materialized view connector API changes
4effcf7 
Even though materialized views are accessible through a connector and can be said to
'belong' to a connector, they need to be treated separately from tables. Connector
APIs for materialized views that modeled them as TableHandles are being changed to
pass in materialized view name rather than handles.
@anjalinorwood
Send storage table handle to finishRefreshMaterializedView
ecbb659
@anjalinorwood
Iceberg connector support for materialized views
2081b4d 
Iceberg connector allows creation, refresh and drop of materialized views.
It provides methods to determine if a given materialized view is current
with respect to underlying base table(s).
Freshness of a materialized view is determined by comparing the snapshot ID
of base table(s) at the time the materialized view was refreshed with
snapshot ID of base table(s) at the time of the check.
@anjalinorwood
Fix typo in 'assertInvalidStatemennt'
e473e33
@electrum electrum merged commit a8f94f0 into trinodb:master Nov 9, 2020
@electrum
Copy link
Member

electrum commented Nov 9, 2020

Thanks!

@electrum electrum mentioned this pull request Nov 10, 2020
Closed
10 tasks
@electrum electrum added this to the 346 milestone Nov 10, 2020
@findepi
Copy link
Member

findepi commented Nov 10, 2020

#5892

This was referenced Nov 10, 2020
Closed
Merged
@bitsondatadev bitsondatadev mentioned this pull request Dec 28, 2021
Merged
@findepi findepi mentioned this pull request Aug 4, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@electrum electrum Awaiting requested review from electrum

@martint martint Awaiting requested review from martint

Assignees

No one assigned

Labels

cla-signed

Milestone

346

Development

Successfully merging this pull request may close these issues.

4 participants

@anjalinorwood @electrum @findepi @martint