Skip to content

Update Spark Iceberg image to v0.13.1#123

Merged
findepi merged 1 commit intotrinodb:masterfrom
alexjo2144:upgrade-iceberg-spark-version
Mar 24, 2022
Merged

Update Spark Iceberg image to v0.13.1#123
findepi merged 1 commit intotrinodb:masterfrom
alexjo2144:upgrade-iceberg-spark-version

Conversation

@alexjo2144
Copy link
Copy Markdown
Member

Required for testing v2 tables with merge-on-read deletes.

@cla-bot cla-bot bot added the cla-signed label Mar 22, 2022
@alexjo2144 alexjo2144 requested review from findepi and phd3 March 22, 2022 21:04
ARG HADOOP_VERSION=3.2
ARG ICEBERG_VERSION=0.12.1
ARG ICEBERG_VERSION=0.13.1
ARG ICEBERG_JAR_VERSION=3.2_2.12
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spark 3.2.1 vs iceberg-spark 3.2.2 -- should these match?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2.12 is the Scala version number, not the spark minor version. They're both referring to Spark 3.2.1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worth adding a code comment?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment

@alexjo2144 alexjo2144 force-pushed the upgrade-iceberg-spark-version branch from 4f1b731 to 95e74b9 Compare March 23, 2022 18:14
@alexjo2144 alexjo2144 requested a review from findepi March 23, 2022 18:15
@findepi findepi merged commit 5540859 into trinodb:master Mar 24, 2022
@findepi
Copy link
Copy Markdown
Member

findepi commented Mar 24, 2022

released as 55

@findepi
Copy link
Copy Markdown
Member

findepi commented Oct 3, 2022

@alexjo2144 can you please also rename the image to spark3-... ?
it looks like an omission in #119, but I would be grateful if you could fix that

@alexjo2144 alexjo2144 deleted the upgrade-iceberg-spark-version branch October 5, 2022 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants