Make Iceberg Dynamic Filter test more resilient#10997
Conversation
|
I'll remove the |
There was a problem hiding this comment.
I guess those are stats which correspond to totalprice
There was a problem hiding this comment.
Yes, 4 is the fieldId for totalPrice. I looked through the $ metadata tables to see if there was a way to query for that value but couldn't find a good way of doing it
There was a problem hiding this comment.
But why are we sure gonna find it?
If we have two files with non overlaping ranges we get empty result range.
E.g:
- f1: [10, 20]
- f2: [3,7]
We get range from the first query: [10, 7]. Which yields empty result from second query.
I am probably missing sth.
There was a problem hiding this comment.
That is possible, and would break the test, but seems unlikely. The overlapping range when I ran the test was most of the range of the column. Unless the data is explicitly ordered I think there should be some overlap.
If this still feels too flaky we can take it out and try testing the DF a different way.
There was a problem hiding this comment.
It is guaranteed by the query now, and driver count stats are kinda flaky.
There was a problem hiding this comment.
I had same issue when adding MinIO test's for Iceberg. (this assertion was removed)
Would be nice to merge this PR earlier: #10894 (comment)
than https://github.com/trinodb/trino/pull/10894/checks
raunaqmorarka
left a comment
There was a problem hiding this comment.
lgtm % @findepi comments
7e24b12 to
0b127ec
Compare
Description
Fixes: #10932
General information
Is this change a fix, improvement, new feature, refactoring, or other?
Flaky test fix
Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)
Test only
How would you describe this change to a non-technical end user or system administrator?
n/a
Related issues, pull requests, and links
Documentation
(x) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
(x) No release notes entries required.
( ) Release notes entries required with the following suggested text: