ARROW-10438: [C++][Dataset] Partitioning::Format on nulls#9323

Closed

westonpace wants to merge 33 commits intoapache:masterfrom

westonpace:feature/arrow-10438

Commits on Feb 23, 2021

Merge/rebase
westonpace
committed
WIP commit
westonpace
committed
Added tests of vector_hash for inputs with nulls. Added ability to specify encoded nulls when encoding a dictionary
westonpace
committed
Prevent using dictionary columns as partition columns. It wouldn't work.
westonpace
committed
Addressing PR comments
westonpace
committed
Taking out an extraneous using that I missed in the last commit
westonpace
committed
WIP
westonpace
committed
Adding the null fallback logic to the python half
westonpace
committed
WIP
westonpace
committed
WIP
westonpace
committed
WIP
westonpace
committed
Improved null handling in expression/partition a bit
westonpace
committed
Added the python half of the new extract known values
westonpace
committed
Lint
westonpace
committed
Missed a test case
westonpace
committed
Re-lint, it appears my IDE is using the wrong style file
westonpace
committed
Final lint pass. Turns out I was relying on black which was messing up everything
westonpace
committed
Added more tests, rounded out a few behaviors
westonpace
committed
Added tests for SetDefaultValues to ensure it does the correct thing on null
westonpace
committed
Cleaned up logic for valid but not known case
westonpace
committed
Fixing compiler warning
westonpace
committed
Python lint
westonpace
committed
Addressing PR comments
westonpace
committed
Update cpp/src/arrow/compute/kernels/vector_hash.cc

westonpace
and
bkietz
committed
Update cpp/src/arrow/compute/kernels/vector_hash.cc

westonpace
and
bkietz
committed
Update cpp/src/arrow/compute/kernels/vector_hash_test.cc

westonpace
and
bkietz
committed
Update cpp/src/arrow/dataset/partition_test.cc

westonpace
and
bkietz
committed
Update python/pyarrow/_dataset.pyx

westonpace
and
bkietz
committed
Added test case to probe what happens when inferring a partition column that is only null. Changed it to an error to match directory partitioning
westonpace
committed
Use null scalars for known-null fields

bkietz
authored and
westonpace
committed
constexpr not supported in this context in all gcc versions due to gcc bugs. Pulling out for a second
westonpace
committed
Missed one of the merge conflicts
westonpace
committed
Putting in suggestion from Ben. It got lost on rebase / force-push
westonpace
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ARROW-10438: [C++][Dataset] Partitioning::Format on nulls#9323

ARROW-10438: [C++][Dataset] Partitioning::Format on nulls#9323
westonpace wants to merge 33 commits intoapache:masterfrom
westonpace:feature/arrow-10438

Commits on Feb 23, 2021

Merge/rebase

WIP commit

Added tests of vector_hash for inputs with nulls. Added ability to specify encoded nulls when encoding a dictionary

Prevent using dictionary columns as partition columns. It wouldn't work.

Addressing PR comments

Taking out an extraneous using that I missed in the last commit

WIP

Adding the null fallback logic to the python half

WIP

WIP

WIP

Improved null handling in expression/partition a bit

Added the python half of the new extract known values

Lint

Missed a test case

Re-lint, it appears my IDE is using the wrong style file

Final lint pass. Turns out I was relying on black which was messing up everything

Added more tests, rounded out a few behaviors

Added tests for SetDefaultValues to ensure it does the correct thing on null

Cleaned up logic for valid but not known case

Fixing compiler warning

Python lint

Addressing PR comments

Update cpp/src/arrow/compute/kernels/vector_hash.cc

Update cpp/src/arrow/compute/kernels/vector_hash.cc

Update cpp/src/arrow/compute/kernels/vector_hash_test.cc

Update cpp/src/arrow/dataset/partition_test.cc

Update python/pyarrow/_dataset.pyx

Added test case to probe what happens when inferring a partition column that is only null. Changed it to an error to match directory partitioning

Use null scalars for known-null fields

constexpr not supported in this context in all gcc versions due to gcc bugs. Pulling out for a second

Missed one of the merge conflicts

Putting in suggestion from Ben. It got lost on rebase / force-push