Skip to content

[REVIEW] Add nvtext::edit_distance API - #5811

Merged
ramakrishnap-nv merged 26 commits into
NVIDIA:branch-0.15from
davidwendt:fea-nvtext-edit-distance
Aug 11, 2020
Merged

ramakrishnap-nv merged 26 commits into
NVIDIA:branch-0.15from
davidwendt:fea-nvtext-edit-distance

Conversation

@davidwendt

Copy link
Copy Markdown
Contributor

This is also to help unblock NVIDIA/cuml#2476

The edit_distance API uses the Levenshtein algorithm.
This is used in NLP processes to compute the relative closeness between the characters in two words/strings.

The libcudf API looks like this:

std::unique_ptr<cudf::column> edit_distance(
  cudf::strings_column_view const& strings,
  cudf::strings_column_view const& targets,
  rmm::mr::device_memory_resource* mr)

And can be used to compute the edit distance between two columns of strings or a column of strings and a single string.

This PR will also include the appropriate Python/Cython interfaces to expose this through the cudf Python library.

@davidwendt davidwendt self-assigned this Jul 30, 2020
@davidwendt davidwendt added 2 - In Progress Currently a work in progress Python Affects Python cuDF API. libcudf Affects libcudf (C++/CUDA) code. strings strings issues (C++ and Python) labels Jul 30, 2020
@codecov

codecov Bot commented Jul 30, 2020

Copy link
Copy Markdown

Codecov Report

Merging #5811 into branch-0.15 will increase coverage by 0.32%.
The diff coverage is 87.50%.

Impacted file tree graph

@@               Coverage Diff               @@
##           branch-0.15    #5811      +/-   ##
===============================================
+ Coverage        84.11%   84.43%   +0.32%     
===============================================
  Files               80       80              
  Lines            13084    13411     +327     
===============================================
+ Hits             11006    11324     +318     
- Misses            2078     2087       +9     
Impacted Files Coverage Δ
python/cudf/cudf/core/column/string.py 86.83% <87.50%> (+0.17%) ⬆️
python/cudf/cudf/io/feather.py 100.00% <0.00%> (ø)
python/cudf/cudf/comm/serialize.py 0.00% <0.00%> (ø)
python/custreamz/custreamz/_version.py 0.00% <0.00%> (ø)
python/dask_cudf/dask_cudf/_version.py 0.00% <0.00%> (ø)
python/dask_cudf/dask_cudf/io/tests/test_orc.py 100.00% <0.00%> (ø)
python/dask_cudf/dask_cudf/io/tests/test_json.py 100.00% <0.00%> (ø)
...ython/dask_cudf/dask_cudf/io/tests/test_parquet.py 100.00% <0.00%> (ø)
python/cudf/cudf/utils/applyutils.py 98.75% <0.00%> (+0.02%) ⬆️
python/cudf/cudf/core/join/join.py 92.41% <0.00%> (+0.03%) ⬆️
... and 30 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 40dd54b...ea65a51. Read the comment docs.

@davidwendt
davidwendt marked this pull request as ready for review August 6, 2020 14:10
@davidwendt
davidwendt requested review from a team as code owners August 6, 2020 14:10
@davidwendt
davidwendt requested review from cwharris and devavret August 6, 2020 14:10
@davidwendt davidwendt changed the title [WIP] Add nvtext::edit_distance API [REVIEW] Add nvtext::edit_distance API Aug 6, 2020
@davidwendt davidwendt added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Aug 6, 2020

@kkraus14 kkraus14 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python / CMake LGTM!

Comment thread cpp/src/text/edit_distance.cu

@ramakrishnap-nv ramakrishnap-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c++ LGTM

@ramakrishnap-nv ramakrishnap-nv added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels Aug 11, 2020
@ramakrishnap-nv
ramakrishnap-nv merged commit 228d6ca into NVIDIA:branch-0.15 Aug 11, 2020
@davidwendt
davidwendt deleted the fea-nvtext-edit-distance branch August 11, 2020 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5 - Ready to Merge Testing and reviews complete, ready to merge libcudf Affects libcudf (C++/CUDA) code. Python Affects Python cuDF API. strings strings issues (C++ and Python)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants