Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(5851): ArrowWriter memory usage #5967

Merged
merged 15 commits into from
Jul 2, 2024
Merged

feat(5851): ArrowWriter memory usage #5967

merged 15 commits into from
Jul 2, 2024

Commits on Jun 27, 2024

  1. refactor(5851): delineate the different memory estimates APIs for the… 

    … ArrowWriter and column writers
    @wiedld
    wiedld committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    b796982 View commit details
    Browse the repository at this point in the history

  2. feat(5851): add memory size estimates to the ColumnValueEncoder imple… 

    …mentations and the DictEncoder
    @wiedld
    wiedld committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    b9a002f View commit details
    Browse the repository at this point in the history

  3. test(5851): add memory_size() to in-progress test

    @wiedld
    wiedld committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    23775bd View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. chore(5851): update docs to make it more explicit what is the differe… 

    …nce btwn memory_size vs get_estimated_total_byte
    @wiedld
    wiedld committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    0702756 View commit details
    Browse the repository at this point in the history

  2. feat(5851): clarify the ColumnValueEncoder::estimated_memory_size int… 

    …erface, and update impls to account for bloom filter size
    @wiedld
    wiedld committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    3f8681b View commit details
    Browse the repository at this point in the history

  3. feat(5851): account for stats array size in the ByteArrayEncoder

    @wiedld
    wiedld committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    dcefe9e View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Refine documentation

    @alamb
    alamb committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    678766b View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'apache/master' into 5851/arrow-writer-m… 

    …emory-usage
    @alamb
    alamb committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    1d289e6 View commit details
    Browse the repository at this point in the history

  3. More accurate memory estimation

    @alamb
    alamb committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    aced4a1 View commit details
    Browse the repository at this point in the history

  4. Improve tests

    @alamb
    alamb committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    3ca8839 View commit details
    Browse the repository at this point in the history

  5. Update accounting for non dict encoded data

    @alamb
    alamb committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    81e5efb View commit details
    Browse the repository at this point in the history

  6. Include more memory size calculations

    @alamb
    alamb committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    27ceff8 View commit details
    Browse the repository at this point in the history

  7. clean up async writer

    @alamb
    alamb committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    9043f20 View commit details
    Browse the repository at this point in the history

  8. clippy

    @alamb
    alamb committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    6485b3c View commit details
    Browse the repository at this point in the history

  9. tweak

    @alamb
    alamb committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    099b00e View commit details
    Browse the repository at this point in the history