Skip to content

Add owning types to hold Arrow data #18084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rapids-bot merged 59 commits into from
Mar 17, 2025
Merged

Add owning types to hold Arrow data #18084

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
015ddbd
First attempt
vyasr Feb 3, 2025
7661570
Get basic conversion working starting with a cudf column
vyasr Feb 19, 2025
f01a07b
Enable construction from an ArrowDeviceArray
vyasr Feb 19, 2025
851d4dc
CMake testing changes
vyasr Feb 19, 2025
a09e87d
Use nanoarray for copying schema
vyasr Feb 19, 2025
e9865bb
Fix reference bug
vyasr Feb 19, 2025
a07b2ae
More CMake testing changes
vyasr Feb 19, 2025
bd65da6
Enable getting views and try asserting equivalence
vyasr Feb 19, 2025
a55e508
Fully passing first round of tests.
vyasr Feb 20, 2025
943eb2b
Add explicit test of lifetime management
vyasr Feb 20, 2025
461c6c8
Basic arrow table
vyasr Feb 20, 2025
362529d
Add arrow conversions
vyasr Feb 20, 2025
fda4ca4
Support construction from device array
vyasr Feb 20, 2025
7c07058
Some cleanup
vyasr Feb 20, 2025
11ec3d9
Add documentation
vyasr Feb 20, 2025
2c7e78a
Fix some bugs
vyasr Feb 21, 2025
f75b17b
More cleanup
vyasr Feb 21, 2025
951914d
More CMake testing
vyasr Feb 21, 2025
31f134e
Get tests passing with complex nanoarrow host tables
vyasr Feb 21, 2025
a76c3b9
More CMake testing
vyasr Feb 21, 2025
6a59f41
Support single columns from arrow host data
vyasr Feb 21, 2025
9600c3c
Also test nanoarrow device data
vyasr Feb 21, 2025
68fb9d8
Implement conversion to host array
vyasr Feb 21, 2025
36bbd36
Support direct ingestion of ArrowArray data
vyasr Feb 21, 2025
0bc9389
Support construction from streams
vyasr Feb 22, 2025
2b2e77f
Make ownership semantics consistent across types
vyasr Feb 24, 2025
89b34c3
Make sure stream and mr are forwarded everywhere
vyasr Feb 24, 2025
eb93262
Centralize as much logic as possible
vyasr Feb 25, 2025
374e7ac
Dictionary behavior is correct since we are just pointing back to exi…
vyasr Feb 25, 2025
7014f30
Update comments
vyasr Feb 25, 2025
4410262
Revert debugging changes
vyasr Feb 25, 2025
dab19cf
Rename file
vyasr Feb 25, 2025
57b7cac
Rename test
vyasr Feb 26, 2025
bcfdc09
Merge remote-tracking branch 'upstream/branch-25.04' into feat/arrow_…
vyasr Feb 27, 2025
e3fce98
Use protocol for dlpack instead of deprecated function
vyasr Feb 28, 2025
0a059ec
Merge remote-tracking branch 'upstream/branch-25.04' into feat/arrow_…
vyasr Mar 4, 2025
fd5b3cc
Address most PR comments
vyasr Mar 4, 2025
03e049b
Stop accepting ArrowSchemas as const and move from them
vyasr Mar 4, 2025
ea5bcdd
CI fixes
vyasr Mar 4, 2025
be89dc6
Merge remote-tracking branch 'upstream/branch-25.04' into feat/arrow_…
vyasr Mar 4, 2025
45918c0
Update cpp/include/cudf/interop.hpp
vyasr Mar 4, 2025
da5a6db
Switch to rvalues
vyasr Mar 5, 2025
30c686a
Put all new functions into a new namespace
vyasr Mar 5, 2025
41d22d4
Merge branch 'branch-25.04' into feat/arrow_column
vyasr Mar 5, 2025
ecd2ce1
Merge branch 'branch-25.04' into feat/arrow_column
vyasr Mar 6, 2025
de138bd
Apply suggestions from code review
vyasr Mar 7, 2025
b0399ff
Some additional fixes
vyasr Mar 7, 2025
e31c04a
Merge remote-tracking branch 'upstream/branch-25.04' into feat/arrow_…
vyasr Mar 7, 2025
791eefa
Fix some typos
vyasr Mar 7, 2025
2e230fe
Merge remote-tracking branch 'upstream/branch-25.04' into feat/arrow_…
vyasr Mar 8, 2025
9c7c8ae
Merge remote-tracking branch 'upstream/branch-25.04' into feat/arrow_…
vyasr Mar 12, 2025
f9091bb
Merge remote-tracking branch 'upstream/branch-25.04' into feat/arrow_…
vyasr Mar 12, 2025
e34356e
Update for new header location
vyasr Mar 13, 2025
5f36e12
Fix bug introduced in GH review application
vyasr Mar 13, 2025
42c68be
Merge remote-tracking branch 'upstream/branch-25.04' into feat/arrow_…
vyasr Mar 13, 2025
bb7ce4b
Appease linter
vyasr Mar 14, 2025
e401959
Add description of the memory management logic and update some comments.
vyasr Mar 17, 2025
83e4b59
Make to_arrow_schema const
vyasr Mar 17, 2025
0d9df33
Check for error
vyasr Mar 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cpp/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -465,6 +465,7 @@ add_library(
src/hash/xxhash_64.cu
src/interop/dlpack.cpp
src/interop/arrow_utilities.cpp
src/interop/arrow_data_structures.cpp
src/interop/to_arrow_device.cu
src/interop/to_arrow_host.cu
src/interop/from_arrow_device.cu
Expand Down
Loading