Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PARQUET-1508: [C++] Read ByteArray data directly into arrow::BinaryBuilder and BinaryDictionaryBuilder. Refactor encoders/decoders to use cleaner virtual interfaces #3492

Closed
wants to merge 7 commits into from
Closed

PARQUET-1508: [C++] Read ByteArray data directly into arrow::BinaryBuilder and BinaryDictionaryBuilder. Refactor encoders/decoders to use cleaner virtual interfaces #3492

wants to merge 7 commits into from

Commits on Jan 27, 2019

  1. Refactor encoder and decoder classes to facilitate type-level extensi… 

    …bility

Refactor DictionaryDecoder to be more easily extensible

Add missing file

windows hell

Do not use __declspec(dllexport) when building parquet_static. does not fix problems though

gcc does not like using the DictEncoder name inside EncoderTraits

Change-Id: I67b51494da3e76f8e856a672cdfd29c9e257efbe

Add Ben's windows fixes

Refactor encoders to use virtual interface, with ability to add additional methods for ByteArray, FixedLenByteArray

Factor decoder code into encoding.cc

Tests passing again

Make DictEncoder templated too for consistency

Draft direct read into ChunkedBinaryBuilder

Finish implementing DecodeArrow methods for ByteArrayType

Fix release build

Use dynamic_cast in a couple places that MSVC doesn't like

Don't export interfaces that we don't need to

debug

Change-Id: Ida18b93c1ea1e51216ee007b258635fcac6f54b5

Clearer names, comments, fix segfault that appeared when ZSTD was turned off

IWYU cleaning

Revert changes to schema.cc to fix msvc build
    @wesm
    wesm committed Jan 27, 2019
    Configuration menu
    Copy the full SHA
    453ecbd View commit details
    Browse the repository at this point in the history

  2. Delete a couple commented-out methods, add code comments about unimpl… 

    …emented DecodeArrowNonNull method for DictionaryBuilder

Change-Id: Id75eed26fd057aaf3f7ba6d8664c240a33a6cbd7
    @wesm
    wesm committed Jan 27, 2019
    Configuration menu
    Copy the full SHA
    daac8a6 View commit details
    Browse the repository at this point in the history

  3. lint 

    Change-Id: Icd4fdd9c9f6358a76e25d0eeb994773a707e6e3b
    @wesm
    wesm committed Jan 27, 2019
    Configuration menu
    Copy the full SHA
    06e2c23 View commit details
    Browse the repository at this point in the history

  4. verbose makefile

    @wesm
    wesm committed Jan 27, 2019
    Configuration menu
    Copy the full SHA
    c4fcf74 View commit details
    Browse the repository at this point in the history

  5. Fix public compile definition on windows

    @wesm
    wesm committed Jan 27, 2019
    Configuration menu
    Copy the full SHA
    4bafc55 View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2019

  1. Update cpp/src/parquet/arrow/record_reader.cc 

    Co-Authored-By: wesm <[email protected]>
    @xhochy @wesm
    xhochy and wesm authored Jan 29, 2019
    Configuration menu
    Copy the full SHA
    f3fadcb View commit details
    Browse the repository at this point in the history

  2. lint 

    Change-Id: Ic7faef17bc54e81d4975eece5d2f7d316d87f03d
    @wesm
    wesm committed Jan 29, 2019
    Configuration menu
    Copy the full SHA
    df1bfc0 View commit details
    Browse the repository at this point in the history