Skip to content

Improve Iceberg documentation for refresh new materialized view #17309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mosabua merged 1 commit into from
May 1, 2023
Merged

Improve Iceberg documentation for refresh new materialized view #17309

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 19 additions & 16 deletions docs/src/main/sphinx/connector/iceberg.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1385,22 +1385,25 @@ view definition. The ``iceberg.materialized-views.storage-schema`` catalog
configuration property or ``storage_schema`` materialized view property can be
used to specify the schema where the storage table is created.

Updating the data in the materialized view with
:doc:`/sql/refresh-materialized-view` deletes the data from the storage table,
and inserts the data that is the result of executing the materialized view
query into the existing table. Data is replaced atomically, so users can
continue to query the materialized view while it is being refreshed.
Refreshing a materialized view also stores
the snapshot-ids of all Iceberg tables that are part of the materialized
view's query in the materialized view metadata. When the materialized
view is queried, the snapshot-ids are used to check if the data in the storage
table is up to date. If the data is outdated, the materialized view behaves
like a normal view, and the data is queried directly from the base tables.
Detecting outdated data is possible only when the materialized view uses
Iceberg tables only, or when it uses mix of Iceberg and non-Iceberg tables
but some Iceberg tables are outdated. When the materialized view is based
on non-Iceberg tables, querying it can return outdated data, since the connector
has no information whether the underlying non-Iceberg tables have changed.
Creating a materialized view does not automatically populate it with data. You
must run :doc:`/sql/refresh-materialized-view` to
populate data in the materialized view.

Updating the data in the materialized view with ``REFRESH MATERIALIZED VIEW``
deletes the data from the storage table, and inserts the data that is the result
of executing the materialized view query into the existing table. Data is
replaced atomically, so users can continue to query the materialized view while
it is being refreshed. Refreshing a materialized view also stores the
snapshot-ids of all Iceberg tables that are part of the materialized view's
query in the materialized view metadata. When the materialized view is queried,
the snapshot-ids are used to check if the data in the storage table is up to
date. If the data is outdated, the materialized view behaves like a normal view,
and the data is queried directly from the base tables. Detecting outdated data
is possible only when the materialized view uses Iceberg tables only, or when it
uses mix of Iceberg and non-Iceberg tables but some Iceberg tables are outdated.
When the materialized view is based on non-Iceberg tables, querying it can
return outdated data, since the connector has no information whether the
underlying non-Iceberg tables have changed.

Dropping a materialized view with :doc:`/sql/drop-materialized-view` removes
the definition and the storage table.
Expand Down