Skip to content

Conversation

@raunaqmorarka
Copy link
Member

@raunaqmorarka raunaqmorarka commented Mar 29, 2024

Description

Minimum split sizes in parquet are determined by parquet writer row group size.
The default parquet row group size is 128MB which makes any initial splits smaller than that unncessary.
Concept of initial splits is intended for formats like TEXT

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Delta Lake
* Catalog configuration properties `delta.max-initial-splits` and `delta.max-initial-split-size` are now defunct and must be removed from cluster configuration properties to successfully start the server. The catalog session property `max_initial_split_size` has been removed. ({issue}`21320`)

@cla-bot cla-bot bot added the cla-signed label Mar 29, 2024
@github-actions github-actions bot added the delta-lake Delta Lake connector label Mar 29, 2024
Minimum split sizes in parquet are determined by parquet writer row group size.
The default parquet row group size is 128MB which makes any initial splits smaller
than that unncessary.
Concept of initial splits is intended for formats like TEXT
@sopel39 sopel39 merged commit c8629fc into trinodb:master Mar 29, 2024
@github-actions github-actions bot added this to the 444 milestone Mar 29, 2024
@raunaqmorarka raunaqmorarka deleted the remove-delta-initial branch July 9, 2024 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

3 participants