Skip to content

Conversation

@danielcweeks
Copy link
Contributor

This PR relocates the parquet implementation from the iceberg-parquet project to iceberg-core so that parquet can be used as part of metadata operations.

The project and artifacts are left so that dependencies still resolve and we can deprecate in a future release.

@danielcweeks danielcweeks changed the title Relocate parquet to core Core: Relocate parquet to core Dec 6, 2024
@ajantha-bhat
Copy link
Member

ajantha-bhat commented Dec 7, 2024

Already we have a PR and issue tracker for the same
#8500

The project and artifacts are left so that dependencies still resolve and we can deprecate in a future release.

Since, Iceberg-parquet users will definitely use Iceberg-core, I assumed that we don't have to keep dummy modules and deprecate it in future. So, I removed them directly.

@nastra
Copy link
Contributor

nastra commented Dec 9, 2024

is there anything left in iceberg-parquet or can the gradle module be removed too?

@ajantha-bhat
Copy link
Member

@nastra: He kept dummy module for compatibility. But I think it can be removed as mentioned in #11716 (comment)

My PR has end to end work, I can revive it.
#8500

@github-actions
Copy link

github-actions bot commented Jan 9, 2025

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

@github-actions github-actions bot added the stale label Jan 9, 2025
@ajantha-bhat
Copy link
Member

Currently introducing internal writers for parquet: #11904

Once, it is merged, I will revive/rework the original #8500

@github-actions github-actions bot removed the stale label Jan 10, 2025
@ajantha-bhat
Copy link
Member

@danielcweeks: I have updated my old PR as per latest code. Please take a look.
#8500

Maybe this can go in for 1.8.0 too.

@ajantha-bhat
Copy link
Member

Aah, I got the same Spark smoke test failure too. Let me debug this.

@danielcweeks
Copy link
Contributor Author

Closing in favor of #12060

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants