Skip to content

Add support for Databricks runtime 14.3 & 15.4 LTS#21700

Merged
ebyhr merged 2 commits intomasterfrom
ebi/delta-143-lts
Oct 1, 2024
Merged

Add support for Databricks runtime 14.3 & 15.4 LTS#21700
ebyhr merged 2 commits intomasterfrom
ebi/delta-143-lts

Conversation

@ebyhr
Copy link
Copy Markdown
Member

@ebyhr ebyhr commented Apr 25, 2024

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

@cla-bot cla-bot bot added the cla-signed label Apr 25, 2024
@github-actions github-actions bot added the docs label Apr 25, 2024
Copy link
Copy Markdown
Member

@findepi findepi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@ebyhr ebyhr marked this pull request as ready for review April 25, 2024 23:00
@ebyhr
Copy link
Copy Markdown
Member Author

ebyhr commented Apr 26, 2024

It seems 14.3 LTS enabled deletion vectors by default: https://github.com/trinodb/trino/actions/runs/8840347798/job/24275694250?pr=21700. Some tests started failing due to unsupported writer feature.

@ebyhr ebyhr marked this pull request as draft April 26, 2024 02:57
@findinpath findinpath requested a review from pajaks May 3, 2024 09:01
@ebyhr ebyhr changed the base branch from master to ebi/delta-write-deletion-vector May 26, 2024 23:35
@ebyhr ebyhr force-pushed the ebi/delta-143-lts branch from 16da623 to 0efda1c Compare May 26, 2024 23:35
@github-actions github-actions bot added hudi Hudi connector iceberg Iceberg connector delta-lake Delta Lake connector hive Hive connector labels May 26, 2024
@ebyhr ebyhr changed the base branch from ebi/delta-write-deletion-vector to master May 26, 2024 23:35
@benbauer-draeger
Copy link
Copy Markdown

Hey, is someone workign on this? Support of deletion vectors would be highly appreciated

@ebyhr ebyhr force-pushed the ebi/delta-143-lts branch from 0efda1c to 9bb5b06 Compare August 19, 2024 08:29
@ebyhr
Copy link
Copy Markdown
Member Author

ebyhr commented Aug 19, 2024

@benbauer-draeger This PR just adds a new CI environment of Databricks 14.3 LTS. #22102 to support writing deletion vectors was already merged in version 454.

@ebyhr ebyhr force-pushed the ebi/delta-143-lts branch 4 times, most recently from 00eb4ff to 762c709 Compare October 1, 2024 05:03
@ebyhr ebyhr changed the title Add support for Databricks runtime 14.3 LTS Add support for Databricks runtime 14.3 & 15.4 LTS Oct 1, 2024
@ebyhr ebyhr force-pushed the ebi/delta-143-lts branch from a794f63 to c5df088 Compare October 1, 2024 05:45
}

@Test(groups = {DELTA_LAKE_DATABRICKS, PROFILE_SPECIFIC_TESTS})
@Test(groups = {DELTA_LAKE_DATABRICKS_104, PROFILE_SPECIFIC_TESTS})
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this change made?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The connector doesn't support VACUUM with deletion vector, so I specified the old version here. Added a comment to revert this change in #22809.

@ebyhr ebyhr marked this pull request as ready for review October 1, 2024 08:57
@ebyhr ebyhr merged commit 94d1b50 into master Oct 1, 2024
@ebyhr ebyhr deleted the ebi/delta-143-lts branch October 1, 2024 08:57
@github-actions github-actions bot added this to the 460 milestone Oct 1, 2024
@mosabua
Copy link
Copy Markdown
Member

mosabua commented Oct 1, 2024

Are you sure we dont want a release notes entry for this @ebyhr and @findinpath ?

@mosabua mosabua mentioned this pull request Oct 1, 2024
1 task
@ebyhr
Copy link
Copy Markdown
Member Author

ebyhr commented Oct 1, 2024

@mosabua We don't need a release note entry because this PR changed only tests.

@mosabua
Copy link
Copy Markdown
Member

mosabua commented Oct 1, 2024

ok .. thanks @ebyhr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed delta-lake Delta Lake connector docs hive Hive connector hudi Hudi connector iceberg Iceberg connector

Development

Successfully merging this pull request may close these issues.

5 participants