Skip to content

Commit

Permalink
ANN warnings/tests updates (rapidsai#4101)
Browse files Browse the repository at this point in the history
With the update to FAISS 1.7, the [previously observed issue with IVFPQ](rapidsai#3318) has disappeared. However some other issues were recently identified in ANN methods. This PR updates the relevant warnings and pytests accordingly.

Authors:
  - Victor Lafargue (https://github.com/viclafargue)

Approvers:
  - Dante Gama Dessavre (https://github.com/dantegd)

URL: rapidsai#4101
  • Loading branch information
viclafargue authored Jul 28, 2021

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
1 parent dcec9b1 commit f6d9395
Showing 2 changed files with 10 additions and 21 deletions.
17 changes: 9 additions & 8 deletions python/cuml/neighbors/nearest_neighbors.pyx
Original file line number Diff line number Diff line change
@@ -276,9 +276,10 @@ class NearestNeighbors(Base,
Notes
-----
Warning: IVFPQ might be unstable in this version of cuML.
This is due to a known issue in the FAISS release that this
cuML version is linked to. (see FAISS issue #1421)
Warning: Approximate Nearest Neighbor methods might be unstable
in this version of cuML. This is due to a known issue in
the FAISS release that this cuML version is linked to.
(see cuML issue #4020)
For an additional example see `the NearestNeighbors notebook
<https://github.com/rapidsai/cuml/blob/branch-0.15/notebooks/nearest_neighbors_demo.ipynb>`_.
@@ -349,11 +350,11 @@ class NearestNeighbors(Base,
cdef handle_t* handle_ = <handle_t*><uintptr_t> self.handle.getHandle()
cdef knnIndexParam* algo_params = <knnIndexParam*> 0
if self.algorithm in ['ivfflat', 'ivfpq', 'ivfsq']:
if self.algorithm == 'ivfpq':
warnings.warn("\nWarning: IVFPQ might be unstable in this "
"version of cuML. This is due to a known issue "
"in the FAISS release that this cuML version "
"is linked to. (see FAISS issue #1421)")
warnings.warn("\nWarning: Approximate Nearest Neighbor methods "
"might be unstable in this version of cuML. "
"This is due to a known issue in the FAISS "
"release that this cuML version is linked to. "
"(see cuML issue #4020)")

if not is_dense(X):
raise ValueError("Approximate Nearest Neigbors methods "
14 changes: 1 addition & 13 deletions python/cuml/test/test_nearest_neighbors.py
Original file line number Diff line number Diff line change
@@ -160,12 +160,6 @@ def test_self_neighboring(datatype, metric_p, nrows):
("ivfsq", "numpy")])
def test_neighborhood_predictions(nrows, ncols, n_neighbors, n_clusters,
datatype, algo):
if algo == "ivfpq":
pytest.xfail("Warning: IVFPQ might be unstable in this "
"version of cuML. This is due to a known issue "
"in the FAISS release that this cuML version "
"is linked to. (see FAISS issue #1421)")

if not has_scipy():
pytest.skip('Skipping test_neighborhood_predictions because ' +
'Scipy is missing')
@@ -222,19 +216,13 @@ def test_ivfflat_pred(nrows, ncols, n_neighbors, nlist):

@pytest.mark.parametrize("nlist", [8])
@pytest.mark.parametrize("M", [16, 32])
@pytest.mark.parametrize("n_bits", [2, 4])
@pytest.mark.parametrize("n_bits", [4, 6])
@pytest.mark.parametrize("usePrecomputedTables", [False, True])
@pytest.mark.parametrize("nrows", [4000])
@pytest.mark.parametrize("ncols", [128, 512])
@pytest.mark.parametrize("n_neighbors", [8])
def test_ivfpq_pred(nrows, ncols, n_neighbors,
nlist, M, n_bits, usePrecomputedTables):

pytest.xfail("Warning: IVFPQ might be unstable in this "
"version of cuML. This is due to a known issue "
"in the FAISS release that this cuML version "
"is linked to. (see FAISS issue #1421)")

algo_params = {
'nlist': nlist,
'nprobe': int(nlist * 0.2),

0 comments on commit f6d9395

Please sign in to comment.