Repository navigation
[cudf] Adding memory_resources support for column wrappers
#23581
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 20 commits
d11a8bb
3f518af
cb35fb0
4a02ab8
4d2d6b8
8479c3d
20eddfd
2e4d5a8
32b153d
3cab3a6
b839630
19857eb
877ffc3
cd3466f
e3cc3db
40155c3
f4293ad
25e3b24
e701a38
1a5fa38
aa0742e
58544b7
00ef7bd
c7a70e3
47c4eb4
128c1c9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -489,8 +489,8 @@ std::string stringify_column_differences(cudf::device_span<int const> difference | |
| buffer << depth_str << "differences:" << std::endl; | ||
|
|
||
| auto source_table = cudf::table_view({lhs, rhs}); | ||
| auto diff_column = | ||
| fixed_width_column_wrapper<int32_t>(h_differences.begin(), h_differences.end()); | ||
| auto diff_column = fixed_width_column_wrapper<int32_t>( | ||
| h_differences.begin(), h_differences.end(), stream, mr.get_temporary_mr()); | ||
| auto diff_table = cudf::gather(source_table, | ||
| diff_column, | ||
| cudf::out_of_bounds_policy::DONT_CHECK, | ||
|
|
@@ -541,6 +541,8 @@ struct column_comparator_impl { | |
| auto lhs_tview = table_view{{lhs}}; | ||
| auto rhs_tview = table_view{{rhs}}; | ||
|
|
||
| // TODO: Pass `mr` once two_table_comparator / equality preprocessed_table::create accept | ||
| // memory_resources instead of allocating from the current device resource. | ||
|
Comment on lines
+545
to
+546
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It doesn't seem like this is included in #23027. Is this a requirement to get to the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Its already drafted #23617
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We need the comparators and preprocessed table changes, to fully make the column utils like |
||
| auto const comparator = | ||
| cudf::detail::row::equality::two_table_comparator{lhs_tview, rhs_tview, stream}; | ||
| auto const has_nulls = cudf::has_nulls(lhs_tview) or cudf::has_nulls(rhs_tview); | ||
|
|
@@ -884,6 +886,9 @@ bool expect_columns_equal(cudf::column_view const& lhs, | |
| rmm::cuda_stream_view stream, | ||
| cudf::memory_resources mr) | ||
| { | ||
| // TODO: equality row preprocessing (two_table_comparator / preprocessed_table::create) still | ||
| // allocates from the current device resource; pass `mr` through once that path accepts | ||
| // memory_resources so callers need not disable failing current-resource scopes. | ||
| check_non_empty_nulls(lhs, rhs, stream); | ||
| auto lhs_indices = generate_all_row_indices(lhs.size(), stream, mr); | ||
| auto rhs_indices = generate_all_row_indices(rhs.size(), stream, mr); | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did we need to make this movable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wanted to be able to move an obj to
std::optional. Either way, I think moveable is a correct semantic forscoped_current_device_resource, is it not?