Allow Iceberg materialized views to federate

[findepi]

Probably fixes #13131

[ebyhr]

Looks good to me. I leave the approval to other reviewers who're familiar with materialized views.

[findepi]

thank you @ebyhr for your review. AC. PTAL

[findepi]

@raunaqmorarka added tests reproducing the problem and addressed

We have effectively three different states an MV can be in

• "fresh" -- can be queried, and REFRESH is a no-op
• "stale" -- cannot be queried (needs to be inlined), and REFRESH should refresh it
• "in between" -- e.g. federated mviews -- can be queried, but REFRESH should refresh it

To support this, i changed MaterializedViewFreshness. It's boolean field could not convey those different meanings, so replaced it with an enum.

cc @losipiuk @sopel39 @martint @alexjo2144 @romanvainb

[findinpath]

testing/bin/ptl env up --environment 'singlenode'  --config 'config-default' --without-trino

create table iceberg.default.itable1 (x integer);

insert into iceberg.default.itable1 values (1),(2),(3);

create materialized view iceberg.default.mvtest1
as select nationkey, name
from iceberg.default.itable1 itable1 
inner join tpch.sf1.nation tnation on itable1.x = tnation.nationkey;


refresh maerialized view  iceberg.default.mvtest1;

select * from  iceberg.default.mvtest1 ;

insert into iceberg.default.itable1 values (4);

select * from  iceberg.default.mvtest1 ;

I see the country corresponding to the nationkey 4 when selecting from iceberg.default.mvtest1 even though the materialized view was not refreshed. Is this supposed to be happening?

SELECT * FROM system.metadata.materialized_views;

The freshness for mvtest1 is always STALE even right after refreshing the materialized view. Wasn't is supposed to be unknown due to the fact that it is "federated" with a tpch table?

[findepi]

(just rebased)

[findepi]

I see the country corresponding to the nationkey 4 when selecting from iceberg.default.mvtest1 even though the materialized view was not refreshed. Is this supposed to be happening?

Yes.

In this case, there are two base tables

1. iceberg table
2. non-iceberg table (tpch)

When you have a modification in the iceberg table, the MV is known to be stale, so the freshness is STALE.
A STALE MV gets inlined by the engine. This is the same behavior that existed before my changes.

[findepi]

The freshness for mvtest1 is always STALE even right after refreshing the materialized view. Wasn't is supposed to be unknown due to the fact that it is "federated" with a tpch table?

can this be related to a typo in the command:

refresh maerialized view  iceberg.default.mvtest1;

?

otherwise I couldn't reproduce this.

[findepi]

AC, thank you @findinpath for thorough review.

[raunaqmorarka]

We should probably update iceberg or MV docs to explain what REFRESH MV does when all the source tables are iceberg tables vs when there is a non-iceberg table involved.

[findepi]

@raunaqmorarka good point, updated the docs now.

@mosabua i am going to merge this when build passes. if you have comments, i will happily apply them, but it's also ok to improve the wording as a follow-up.

[mosabua]

So AWESOME @findepi

[mosabua]

Filed #15199 to follow up on the docs with some more clarifications.

[findinpath]

can this be related to a typo in the command:

refresh maerialized view iceberg.default.mvtest1;

Indeed. It was apparently a typo.
I've retested the scenario now and see that the freshness is set to "UNKNOWN" (due to the dependency on the non versioned tpch table).