Skip to content

Don't overwrite reader and writer versions during COMMENT and ADD COLUMN in Delta#14612

Merged
ebyhr merged 1 commit intomasterfrom
ebi/delta-reader-writer-versions
Oct 14, 2022
Merged

Don't overwrite reader and writer versions during COMMENT and ADD COLUMN in Delta#14612
ebyhr merged 1 commit intomasterfrom
ebi/delta-reader-writer-versions

Conversation

@ebyhr
Copy link
Copy Markdown
Member

@ebyhr ebyhr commented Oct 13, 2022

Description

Fixes #14611

Non-technical explanation

Release notes

(x) Release notes are required, with the following suggested text:

# Delta Lake
* Don't overwrite reader and writer versions when executing `COMMENT` and `ALTER TABLE ... ADD COLUMN` statements. ({issue}`14611`)

@ebyhr ebyhr requested a review from findepi October 13, 2022 06:46
}

@Test(groups = {DELTA_LAKE_DATABRICKS, DELTA_LAKE_OSS, PROFILE_SPECIFIC_TESTS})
public void testTrinoAlterTablePreservesReaderAndWriterVersions()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we good for inserts, checkpoints, etc.?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those operations look fine. Added some statements to this test.

@alexjo2144
Copy link
Copy Markdown
Member

Can we put the minWriterVersion and minReaderVersion in the DeltaLakeTableHandle?

Either way, thanks for spotting this

@ebyhr ebyhr force-pushed the ebi/delta-reader-writer-versions branch from 72524e2 to 5ab5ab4 Compare October 14, 2022 00:56
@ebyhr ebyhr changed the title Don't overwrite reader and writer versions during during COMMENT and ADD COLUMN in Delta Don't overwrite reader and writer versions during COMMENT and ADD COLUMN in Delta Oct 14, 2022
@ebyhr
Copy link
Copy Markdown
Member Author

ebyhr commented Oct 14, 2022

Can we put the minWriterVersion and minReaderVersion in the DeltaLakeTableHandle?

Actually, I considered this during fixing, but we avoid adding fields to DeltaLakeTableHandle as much as possible. Let me keep as-is because the affected operations are only COMMENT and ADD COLUMN statements.

@ebyhr ebyhr merged commit 1e0e2d9 into master Oct 14, 2022
@ebyhr ebyhr deleted the ebi/delta-reader-writer-versions branch October 14, 2022 03:47
@ebyhr ebyhr mentioned this pull request Oct 14, 2022
@github-actions github-actions bot added this to the 401 milestone Oct 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

COMMENT and ADD COLUMN statements change reader and writer versions in Delta Lake connector

4 participants