Add BaseConnectorTest for Glue catalog in Iceberg#11746
Closed
Add BaseConnectorTest for Glue catalog in Iceberg#11746
Conversation
ebyhr
added
test
no-release-notes
ebyhr
force-pushed
the
ebi/iceberg-glue-bct
branch
2 times, most recently
from
c3cd734 to
83d5496
Compare
findepi
reviewed
Apr 1, 2022
Comment on lines
+129
to
+130
| Logging logging = Logging.initialize(); | ||
| logging.setLevel("org.apache.iceberg.BaseTableScan", Level.WARN); // To suppress huge "Scanning table ... filter" message |
findepi
reviewed
Apr 1, 2022
| assertQuery("SELECT * FROM test_rollback ORDER BY col0", "VALUES (123, CAST(987 AS BIGINT)), (456, CAST(654 AS BIGINT))"); | ||
|
|
||
| assertUpdate(format("CALL system.rollback_to_snapshot('tpch', 'test_rollback', %s)", afterFirstInsertId)); | ||
| assertUpdate(format("CALL system.rollback_to_snapshot('%s', 'test_rollback', %s)", getSession().getSchema().orElseThrow(), afterFirstInsertId)); |
findepi
reviewed
Apr 1, 2022
| * on ways to set your AWS credentials which will be needed to run this test. | ||
| */ | ||
| public class TestIcebergGlueConnectorTest | ||
| extends BaseIcebergConnectorTest |
Member
There was a problem hiding this comment.
BaseIcebergConnectorTest doesn't randomize names and doesn't drop tables in try-finally, because it has not been used on shared resources until now.
Member
Member
Author
|
Managing only BCST looks good to me. Sadly, the Glue BCT took more than 1h with 1 thread in my laptop. |
Member
|
wow. How much does it take on the CI? |
Member
Author
|
I can't see the additional CI time because it's failing with OOM on the way (around |
ebyhr
force-pushed
the
ebi/iceberg-glue-bct
branch
4 times, most recently
from
c0336bd to
3ac5513
Compare
ebyhr
force-pushed
the
ebi/iceberg-glue-bct
branch
2 times, most recently
from
4849bec to
16ded08
Compare
ebyhr
force-pushed
the
ebi/iceberg-glue-bct
branch
3 times, most recently
from
403381a to
8885f23
Compare
ebyhr
force-pushed
the
ebi/iceberg-glue-bct
branch
from
8885f23 to
81e5562
Compare
For upcoming Glue test in Iceberg.
ebyhr
force-pushed
the
ebi/iceberg-glue-bct
branch
4 times, most recently
from
2924c3d to
d66e93a
Compare
ebyhr
force-pushed
the
ebi/iceberg-glue-bct
branch
2 times, most recently
from
85845e6 to
4a40138
Compare
ebyhr
force-pushed
the
ebi/iceberg-glue-bct
branch
3 times, most recently
from
5438d26 to
5840afd
Compare
Member
Author
|
Glue connector test adds about 19 minutes in CI
|
ebyhr
force-pushed
the
ebi/iceberg-glue-bct
branch
from
5840afd to
2da7cfe
Compare
Member
That's quite something. |
Member
Author
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add BaseConnectorTest for Glue catalog in Iceberg
Documentation
(x) No documentation is needed.
Release notes
(x) No release notes entries required.