cuVS Cagra FP16 support by jinsolp · Pull Request #4384 · facebookresearch/faiss

jinsolp · 2025-06-10T18:30:52Z

Supporting fp16 for cuVS cagra, and introducing new extended APIs for this.
Discussions related to this issue: #4324

Added tests in faiss/gpu/test/TestGpuIndexCagra.cu and faiss/gpu/test/test_cagra.py for example usage.

facebook-github-bot · 2025-06-10T18:30:58Z

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

mnorris11 · 2025-06-11T20:15:36Z

Hi @jinsolp can you complete the CLA? Then I can import it and run internal tests.

jinsolp · 2025-06-11T23:39:02Z

@mnorris11 Sure! : ) Who should I be writing as the "Point of Contact"? What about "Schedule A" (list of designated employees)? Should I be writing myself in those sections?

mnorris11 · 2025-06-12T00:16:03Z

@mnorris11 Sure! : ) Who should I be writing as the "Point of Contact"? What about "Schedule A" (list of designated employees)? Should I be writing myself in those sections?

Hmm, @cjnolet @tarang-jain do you remember, did you fill out the Individual or Company one for NVIDIA? If Company, did you email cla@meta.com to update it with additional folks, or do you usually just direct folks to the Individual option? I think there is no preference on our side.

If there is no NVIDIA "Company" CLA yet, feel free to start it @jinsolp and add yourself as Point of Contact and under Schedule A list of employees (along with Corey, Tarang, Tamas, and any others you deem should be added)

tarang-jain · 2025-06-12T00:25:24Z

@mnorris11 we were told to sign the individual one.

jinsolp · 2025-06-12T00:44:06Z

@mnorris11 Signed!

facebook-github-bot · 2025-06-12T01:23:26Z

@mnorris11 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

mnorris11 · 2025-06-12T14:49:37Z

Seems like the ROCM build fails, are the logs visible to you @jinsolp ?

jinsolp · 2025-06-12T16:03:43Z

@mnorris11 yes I can see the logs, but I can't tell why it failed from the logs. Do you know how I can reproduce the results?

mnorris11 · 2025-06-12T16:32:26Z

Weird; seems like rocm hipification is having trouble with half syntax. But we do have other files with half being used in Faiss and those hipify fine. @jinsolp can you try including headers that other files use? You can search the codebase for half references in faiss/gpu code, then run the hipify script.

The cmake command to repro looks like this:

2025-06-12T01:51:51.2772694Z �[36;1mcmake -B build \�[0m
2025-06-12T01:51:51.2772913Z �[36;1m      -DBUILD_TESTING=ON \�[0m
2025-06-12T01:51:51.2773167Z �[36;1m      -DBUILD_SHARED_LIBS=ON \�[0m
2025-06-12T01:51:51.2773418Z �[36;1m      -DFAISS_ENABLE_GPU=ON \�[0m
2025-06-12T01:51:51.2773687Z �[36;1m      -DFAISS_ENABLE_CUVS=OFF \�[0m
2025-06-12T01:51:51.2773931Z �[36;1m      -DFAISS_ENABLE_ROCM=ON \�[0m
2025-06-12T01:51:51.2774179Z �[36;1m      -DFAISS_OPT_LEVEL=generic \�[0m
2025-06-12T01:51:51.2774427Z �[36;1m      -DFAISS_ENABLE_C_API=ON \�[0m
2025-06-12T01:51:51.2774689Z �[36;1m      -DPYTHON_EXECUTABLE=$CONDA/bin/python \�[0m
2025-06-12T01:51:51.2774959Z �[36;1m      -DCMAKE_BUILD_TYPE=Release \�[0m
2025-06-12T01:51:51.2775194Z �[36;1m      -DBLA_VENDOR=Intel10_64_dyn \�[0m
2025-06-12T01:51:51.2775490Z �[36;1m      -DCMAKE_CUDA_FLAGS="-gencode arch=compute_75,code=sm_75"

Meanwhile @ItsPitt do you have ideas on the AMD side of what to include for half to hipify? (Sorry, I would tag Johannes too but I am not finding his Github username...)

Error logs:

2025-06-12T01:52:38.8113545Z /__w/faiss/faiss/faiss/gpu-rocm/GpuIndex.hip:195:18: error: use of undeclared identifier 'half'
2025-06-12T01:52:38.8114096Z   195 |         dispatch(half{});
2025-06-12T01:52:38.8114355Z       |                  ^
2025-06-12T01:52:38.8173913Z /__w/faiss/faiss/faiss/gpu-rocm/GpuIndex.hip:252:18: error: use of undeclared identifier 'half'
2025-06-12T01:52:38.8174480Z   252 |         dispatch(half{});
2025-06-12T01:52:38.8174780Z       |                  ^
2025-06-12T01:52:38.8232056Z /__w/faiss/faiss/faiss/gpu-rocm/GpuIndex.hip:406:39: error: use of undeclared identifier 'half'
2025-06-12T01:52:38.8232642Z   406 |         auto vecs = toDeviceTemporary<half, 2>(
2025-06-12T01:52:38.8232999Z       |                                       ^
2025-06-12T01:52:38.8261155Z /__w/faiss/faiss/faiss/gpu-rocm/GpuIndex.hip:409:28: error: unknown type name 'half'
2025-06-12T01:52:38.8261785Z   409 |                 const_cast<half*>(static_cast<const half*>(x)),
2025-06-12T01:52:38.8262145Z       |                            ^
2025-06-12T01:52:38.8286932Z /__w/faiss/faiss/faiss/gpu-rocm/GpuIndex.hip:409:53: error: unknown type name 'half'
2025-06-12T01:52:38.8287481Z   409 |                 const_cast<half*>(static_cast<const half*>(x)),
2025-06-12T01:52:38.8287853Z       |                                                     ^
2025-06-12T01:52:38.8341304Z /__w/faiss/faiss/faiss/gpu-rocm/GpuIndex.hip:485:51: error: unknown type name 'half'
2025-06-12T01:52:38.8341882Z   485 |                                 static_cast<const half*>(x) + cur * this->d),
2025-06-12T01:52:38.8342244Z       |                                                   ^
2025-06-12T01:52:38.8342742Z /__w/faiss/faiss/faiss/gpu-rocm/GpuIndex.hip:485:61: error: arithmetic on a pointer to void
2025-06-12T01:52:38.8343279Z   485 |                                 static_cast<const half*>(x) + cur * this->d),
2025-06-12T01:52:38.8344246Z       |                                                          ~  ^
2025-06-12T01:52:38.8485720Z /__w/faiss/faiss/faiss/gpu-rocm/GpuIndex.hip:646:18: error: use of undeclared identifier 'half'
2025-06-12T01:52:38.8486277Z   646 |         dispatch(half{});

…into cuvs-cagra-fp16-pub

jinsolp · 2025-06-12T17:58:37Z

I've added #include <faiss/gpu/utils/Float16.cuh> which seems to be including <hip/hip_fp16.h>. : )

facebook-github-bot · 2025-06-12T18:01:39Z

@mnorris11 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

jinsolp · 2025-06-12T22:24:55Z

@mnorris11 Looks like build&tests failed with a bunch of warnings, but I don't think I have access to the log 👀
Also, how can I run the facebook internal linter to pass the linter check?

mnorris11 · 2025-06-12T23:19:52Z

@mnorris11 Looks like build&tests failed with a bunch of warnings, but I don't think I have access to the log 👀 Also, how can I run the facebook internal linter to pass the linter check?

It looks like just warnings on the internal end, so no worries, it is now just in review.

facebook-github-bot · 2025-06-14T00:24:58Z

@mnorris11 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2025-06-14T04:34:04Z

@mnorris11 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2025-06-17T02:22:04Z

@mnorris11 merged this pull request in 752b687.

mdouze

Sorry I did not follow this PR. See my comments inline.

mdouze · 2025-06-20T09:37:03Z

faiss/gpu/GpuIndex.h

+            idx_t n,
+            const void* x,
+            NumericType numeric_type,
+            const idx_t* xids) override;


Maybe we should have taken the opportunity to also make the id sizes parameterizable. There are many use cases where int32 is more appropriate than int64

We can change that in a subsequent diff, but I would like to avoid having 3 different add_with_ids implementaitons.

mdouze · 2025-06-20T09:39:54Z

faiss/python/class_wrappers.py

        n, d = x.shape
        assert d == self.d
-        x = np.ascontiguousarray(x, dtype='float32')
+        if numeric_type == faiss.Float32:


It is a bit clumsy that the Python interface is not able to directly accept np.float16 arguments because there is no way to tell if this will raise an error.

@mdouze is x expected to be a numpy array? The docs here say "array-like", but since it calls x.shape, can I assume that this is a numpy array and access the dtype instead of getting numeric_type as an argument?

It should be a numpy array, see https://github.com/facebookresearch/faiss/blob/main/faiss/python/swigfaiss.swig#L1163

Support for torch arrays is via another mechanism.

mdouze · 2025-06-20T09:40:46Z

faiss/python/class_wrappers.py

+            x = np.ascontiguousarray(x, dtype='float32')
+        else:
+            x = np.ascontiguousarray(x, dtype='float16')
        self.add_c(n, swig_ptr(x))


this will not work in the fp16 case because it will go through the regular add method, not the one with numeric_type

jinsolp · 2025-06-24T16:06:26Z

Thanks for the feedback @mdouze ! It looks like the PR is merged already. I'll open up a new PR with follow-ups.

jinsolp · 2025-06-25T17:52:23Z

@mdouze Changes and fixes for python API are reflected in this PR #4411

Summary: This PR does 2 things - Enable support fot `IndexIDMap` with Cagra fp16 (original support introduced in #4188) - Added tests in `test_cagra.py` - Reflecting feedback about python API from #4384 (comment) Pull Request resolved: #4411 Reviewed By: junjieqi Differential Revision: D78695771 Pulled By: mnorris11 fbshipit-source-id: 4b3a0869bed5d33165354f415c748812b0d4b253

Summary: This PR does 2 things - Enable support fot `IndexIDMap` with Cagra fp16 (original support introduced in facebookresearch#4188) - Added tests in `test_cagra.py` - Reflecting feedback about python API from facebookresearch#4384 (comment) Pull Request resolved: facebookresearch#4411 Reviewed By: junjieqi Differential Revision: D78695771 Pulled By: mnorris11 fbshipit-source-id: 4b3a0869bed5d33165354f415c748812b0d4b253

Summary: Supporting fp16 for cuVS cagra, and introducing new extended APIs for this. Discussions related to this issue: facebookresearch/faiss#4324 Added tests in `faiss/gpu/test/TestGpuIndexCagra.cu` and `faiss/gpu/test/test_cagra.py` for example usage. Pull Request resolved: facebookresearch/faiss#4384 Reviewed By: junjieqi Differential Revision: D76480612 Pulled By: mnorris11 fbshipit-source-id: 863d8671eab461733110f74550ffc56650f77407

Summary: This PR does 2 things - Enable support fot `IndexIDMap` with Cagra fp16 (original support introduced in facebookresearch/faiss#4188) - Added tests in `test_cagra.py` - Reflecting feedback about python API from facebookresearch/faiss#4384 (comment) Pull Request resolved: facebookresearch/faiss#4411 Reviewed By: junjieqi Differential Revision: D78695771 Pulled By: mnorris11 fbshipit-source-id: 4b3a0869bed5d33165354f415c748812b0d4b253

jinsolp added 12 commits May 30, 2025 03:07

search,train,copyFrom,copyTo baselevel

35d454f

use FAISS_THROW

c373d5c

tutorial example

581b8dc

python binding for search and trian

79bf4c1

python binding

c96e0e0

numeric type part of class

cbbe81c

python tutorial

7e6aebb

Merge branch 'facebookresearch:main' into cuvs-cagra-fp16

65eaee3

add for fp32

0612cec

cleanup

5ebb451

for (de)serialize

743024a

python test

bbc9c23

jinsolp changed the title ~~Cuvs Cagra FP 16 support~~ cuVS Cagra FP16 support Jun 10, 2025

Merge branch 'main' into cuvs-cagra-fp16-pub

aa4c5c4

facebook-github-bot added the CLA Signed label Jun 12, 2025

Merge branch 'main' into cuvs-cagra-fp16-pub

3d452fe

jinsolp added 2 commits June 12, 2025 17:56

fp16 header

2812f50

Merge branch 'cuvs-cagra-fp16-pub' of https://github.com/jinsolp/faiss …

eb4f14f

…into cuvs-cagra-fp16-pub

Merge branch 'main' into cuvs-cagra-fp16-pub

9b438ab

facebook-github-bot closed this in 752b687 Jun 17, 2025

facebook-github-bot added the Merged label Jun 17, 2025

This was referenced Jun 17, 2025

Native support for half precision #4324

Closed

Support for FP16, BF16, and INT8 Index Creation in FAISS #4397

Closed

mdouze reviewed Jun 20, 2025

View reviewed changes

jinsolp mentioned this pull request Jun 25, 2025

Add support for IndexIDMap with Cagra fp16 #4411

Closed

jinsolp deleted the cuvs-cagra-fp16-pub branch July 11, 2025 17:28

Conversation

jinsolp commented Jun 10, 2025

Uh oh!

facebook-github-bot commented Jun 10, 2025

Action Required

Process

Uh oh!

mnorris11 commented Jun 11, 2025

Uh oh!

jinsolp commented Jun 11, 2025

Uh oh!

mnorris11 commented Jun 12, 2025

Uh oh!

tarang-jain commented Jun 12, 2025

Uh oh!

jinsolp commented Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 12, 2025

Uh oh!

mnorris11 commented Jun 12, 2025

Uh oh!

jinsolp commented Jun 12, 2025

Uh oh!

mnorris11 commented Jun 12, 2025

Uh oh!

jinsolp commented Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 12, 2025

Uh oh!

jinsolp commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mnorris11 commented Jun 12, 2025

Uh oh!

facebook-github-bot commented Jun 14, 2025

Uh oh!

facebook-github-bot commented Jun 14, 2025

Uh oh!

facebook-github-bot commented Jun 17, 2025

Uh oh!

mdouze left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdouze Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

mdouze Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

mdouze Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

jinsolp Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

mdouze Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

mdouze Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

jinsolp commented Jun 24, 2025

Uh oh!

jinsolp commented Jun 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jinsolp commented Jun 12, 2025 •

edited

Loading

mdouze left a comment •

edited

Loading