Repository navigation
Occupancy improvement for Hash table build #15700
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 38 commits
Commits
Show all changes
50 commits
Select commit
Hold shift + click to select a range
3df7dce
nested template instantiation for hiding types
tgujar 088042c
hasher conditional type dispatch works
tgujar 34bad1e
delete dead comment block
tgujar 40f291e
Fix docs
PointKernel b56bf75
fix type logic, minor refactor
tgujar 078f53d
refactor
tgujar 14785e4
added template specialization for equality comparator
tgujar b88b60b
added template specialized calls to comparator
tgujar 1b89198
fix for register usage discrepancy
tgujar ca63201
fix for register usage discrepancy
tgujar ff5e0d4
revert edited comment blocks
tgujar 80a9f61
added variant visitor pattern to avoid duplication for hasher
tgujar b8ad315
added variant visitor pattern for comparator
tgujar e29e346
add template specialization for hash joins
tgujar 27fa7a0
add template specialization for hash join size
tgujar 073a328
Merge branch 'branch-24.08' into hash-occupancy
PointKernel 04f3906
minor refactor
tgujar 90f33b4
fixup naming and docs
tgujar a016cd2
separate mixed semi join into multiple TU
tgujar 8fc47df
Merge branch 'branch-24.08' into hash-occupancy
PointKernel 0dc17ac
compile fails because traits not satisfied
tgujar ac5391d
compile fails with no instance of constructor
tgujar 83ded25
compile fails with constructor not found
tgujar 5306d1d
fix HasNested interaction, compile still fails
tgujar 62c60fa
fails with illegal access
tgujar fbcf50f
fix illegal mem access
tgujar 946d56f
fix index type
tgujar fe2ff8b
Merge branch 'branch-24.08' into distinct-join-occupancy
tgujar 3febec0
Merge branch 'branch-24.08' into distinct-join-occupancy
tgujar 77be694
Merge branch 'rapidsai:branch-24.10' into distinct-join-occupancy
tgujar 0ad9407
Merge branch 'branch-24.10' into hash-occupancy
tgujar 74219fe
use CUDF_HIDDEN
tgujar 448b14b
address review comments
tgujar b30f6ed
Merge branch 'distinct-join-occupancy' into hash-occupancy
tgujar 621e29f
Merge branch 'branch-24.10' into hash-occupancy
ttnghia 970a829
address review comments
tgujar d063dbc
Merge branch 'branch-24.10' into hash-occupancy
tgujar 50341bd
Merge branch 'branch-24.10' into hash-occupancy
PointKernel 470d6e2
fix issue with distinct hash join
tgujar 57c9b5e
address review comments
tgujar 8e6e6e5
refactor find_any
tgujar 9cd1bda
Merge branch 'branch-24.10' into hash-occupancy
tgujar 7d48f52
Merge branch 'branch-24.10' into hash-occupancy
tgujar 728478f
remove redundant SFINAE check
tgujar 926342f
use distance from thrust namespace
tgujar d82754d
update docs
tgujar c78da6b
fix spelling
tgujar daa2b40
merge branch-24.10, needs fixes
tgujar 30ab4e3
fail with instantiating correct type
tgujar f1db848
fix issue with constness
tgujar File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -29,6 +29,7 @@ | |||||
| #include <memory> | ||||||
| #include <type_traits> | ||||||
| #include <utility> | ||||||
| #include <variant> | ||||||
|
|
||||||
| namespace cudf::detail { | ||||||
|
|
||||||
|
|
@@ -85,22 +86,42 @@ struct hasher_adapter { | |||||
| template <cudf::has_nested HasNested> | ||||||
| struct distinct_hash_join { | ||||||
| private: | ||||||
| /// Device row equal type | ||||||
| using d_equal_type = cudf::experimental::row::equality::strong_index_comparator_adapter< | ||||||
| cudf::experimental::row::equality::device_row_comparator<HasNested == cudf::has_nested::YES, | ||||||
| cudf::nullate::DYNAMIC>>; | ||||||
| using row_comparator = cudf::experimental::row::equality::device_row_comparator< | ||||||
| true, | ||||||
| cudf::nullate::DYNAMIC, | ||||||
| cudf::experimental::row::equality::nan_equal_physical_equality_comparator, | ||||||
| cudf::experimental::type_identity_t>; | ||||||
|
|
||||||
| using row_comparator_no_nested = cudf::experimental::row::equality::device_row_comparator< | ||||||
| false, | ||||||
| cudf::nullate::DYNAMIC, | ||||||
| cudf::experimental::row::equality::nan_equal_physical_equality_comparator, | ||||||
| cudf::experimental::dispatch_void_if_nested_t>; | ||||||
|
|
||||||
| using row_comparator_no_compound = cudf::experimental::row::equality::device_row_comparator< | ||||||
| false, | ||||||
| cudf::nullate::DYNAMIC, | ||||||
| cudf::experimental::row::equality::nan_equal_physical_equality_comparator, | ||||||
| cudf::experimental::dispatch_void_if_compound_t>; | ||||||
|
|
||||||
| using hasher = hasher_adapter<thrust::identity<hash_value_type>>; | ||||||
| using probing_scheme_type = cuco::linear_probing<1, hasher>; | ||||||
| using cuco_storage_type = cuco::storage<1>; | ||||||
|
|
||||||
| /// Hash table type | ||||||
| using hash_table_type = cuco::static_set<cuco::pair<hash_value_type, rhs_index_type>, | ||||||
| cuco::extent<size_type>, | ||||||
| cuda::thread_scope_device, | ||||||
| comparator_adapter<d_equal_type>, | ||||||
| probing_scheme_type, | ||||||
| cudf::detail::cuco_allocator<char>, | ||||||
| cuco_storage_type>; | ||||||
| template <typename Comparator> | ||||||
| using static_set_with_comparator = cuco::static_set< | ||||||
| cuco::pair<hash_value_type, rhs_index_type>, | ||||||
| cuco::extent<size_type>, | ||||||
| cuda::thread_scope_device, | ||||||
| comparator_adapter< | ||||||
| cudf::experimental::row::equality::strong_index_comparator_adapter<Comparator>>, | ||||||
| probing_scheme_type, | ||||||
|
PointKernel marked this conversation as resolved.
|
||||||
| cudf::detail::cuco_allocator, | ||||||
| cuco_storage_type>; | ||||||
| using hash_table_type = std::variant<static_set_with_comparator<row_comparator>, | ||||||
| static_set_with_comparator<row_comparator_no_nested>, | ||||||
| static_set_with_comparator<row_comparator_no_compound>>; | ||||||
|
|
||||||
| bool _has_nulls; ///< true if nulls are present in either build table or probe table | ||||||
| cudf::null_equality _nulls_equal; ///< whether to consider nulls as equal | ||||||
|
|
@@ -109,8 +130,8 @@ struct distinct_hash_join { | |||||
| std::shared_ptr<cudf::experimental::row::equality::preprocessed_table> | ||||||
| _preprocessed_build; ///< input table preprocssed for row operators | ||||||
| std::shared_ptr<cudf::experimental::row::equality::preprocessed_table> | ||||||
| _preprocessed_probe; ///< input table preprocssed for row operators | ||||||
| hash_table_type _hash_table; ///< hash table built on `_build` | ||||||
| _preprocessed_probe; ///< input table preprocssed for row operators | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| std::unique_ptr<hash_table_type> _hash_table; ///< hash table built on `_build` | ||||||
|
|
||||||
| public: | ||||||
| distinct_hash_join() = delete; | ||||||
|
|
||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keep these filenames alphabetized. If you like, you could rename this to
mixed_join_semi_kernels.cu.