Skip to content

implement target max file size for delta lake#12820

Merged
ebyhr merged 1 commit intotrinodb:masterfrom
homar:homar/implement_target_maex_file_size_for_delta_lake
Jun 14, 2022
Merged

implement target max file size for delta lake#12820
ebyhr merged 1 commit intotrinodb:masterfrom
homar:homar/implement_target_maex_file_size_for_delta_lake

Conversation

@homar
Copy link
Copy Markdown
Member

@homar homar commented Jun 13, 2022

Description

Is this change a fix, improvement, new feature, refactoring, or other?

improvement

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

delta lake connector

How would you describe this change to a non-technical end user or system administrator?

It allows you to control maximum size of the result file(to some extent)

Related issues, pull requests, and links

Documentation

( ) No documentation is needed.
(x) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

( ) No release notes entries required.
( ) Release notes entries required with the following suggested text:

# Delta Lake
* Limit maximum file size to 1 GB by default. The limit is not exact and is performed best effort.
  The limit can be set with the `target_max_file_size` session property, or the `target-max-file-size` configuration property. ({issue}`12820`)

@cla-bot cla-bot bot added the cla-signed label Jun 13, 2022
@github-actions github-actions bot added the docs label Jun 13, 2022
@homar homar force-pushed the homar/implement_target_maex_file_size_for_delta_lake branch from e5cc993 to 95d286b Compare June 13, 2022 13:21
@findepi findepi requested review from alexjo2144 and ebyhr June 13, 2022 20:50
Copy link
Copy Markdown
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good except for minor comments.

@homar homar force-pushed the homar/implement_target_maex_file_size_for_delta_lake branch from 95d286b to e2a30f3 Compare June 14, 2022 07:42
Add docs for delta.target-max-file-size property
@homar homar force-pushed the homar/implement_target_maex_file_size_for_delta_lake branch from e2a30f3 to c804cd0 Compare June 14, 2022 07:43
@ebyhr ebyhr merged commit 1f8f92c into trinodb:master Jun 14, 2022
@ebyhr
Copy link
Copy Markdown
Member

ebyhr commented Jun 14, 2022

Merged, thanks!

@ebyhr ebyhr mentioned this pull request Jun 14, 2022
@github-actions github-actions bot added this to the 386 milestone Jun 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants