int8 support for cuVS cagra#4439
Conversation
jinsolp
changed the title
int8 support for cuVS cagraint8 support for cuVS cagra
|
@mnorris11 has imported this pull request. If you are a Meta employee, you can view this in D78746656. |
| else if (auto icg = dynamic_cast<const GpuIndexCagra*>(index)) { | ||
| IndexHNSWCagra* res = new IndexHNSWCagra(); | ||
| if (icg->get_numeric_type() == faiss::NumericType::Float16) { | ||
| if (icg->get_numeric_type() != faiss::NumericType::Float32) { |
There was a problem hiding this comment.
For my learning, what happens in the 16 and int8 case?
There was a problem hiding this comment.
This is for copyTo where we copy from GpuIndexCagra to IndexHNSWCagra.
If base_level_only=false, we end up calling hnsw_add_vertices, which only support fp32 computations on CPU, so we need it to be true.
When base_level_only=true, we can use the storage(which is IndexScalarQuantizer)'s add() instead. : )
|
@mnorris11 has imported this pull request. If you are a Meta employee, you can view this in D78746656. |
|
@mnorris11 has imported this pull request. If you are a Meta employee, you can view this in D78746656. |
|
@mnorris11 has imported this pull request. If you are a Meta employee, you can view this in D78746656. |
|
Added logic for encode/decode for interop between cpu and gpu. Previously, it was directly copying the data bits causing issues between int8 and uint8. For example, in the The latest commit accounts for this situation and does the encoding/decoding step as part of the copyTo and copyFrom functions. |
|
@mnorris11 has imported this pull request. If you are a Meta employee, you can view this in D78746656. |
|
@mnorris11 has imported this pull request. If you are a Meta employee, you can view this in D78746656. |
|
@mnorris11 has imported this pull request. If you are a Meta employee, you can view this in D78746656. |
|
@mnorris11 merged this pull request in b4534d8. |
Summary: Closes facebookresearch#4430 by exposing int8 for cuVS cagra. This PR (facebookresearch#4411) needs to be merged before this one . Pull Request resolved: facebookresearch#4439 Reviewed By: junjieqi Differential Revision: D78746656 Pulled By: mnorris11 fbshipit-source-id: 79147c8196f298c443796a615fabefac34fb188d
Summary: Closes facebookresearch/faiss#4430 by exposing int8 for cuVS cagra. This PR (facebookresearch/faiss#4411) needs to be merged before this one . Pull Request resolved: facebookresearch/faiss#4439 Reviewed By: junjieqi Differential Revision: D78746656 Pulled By: mnorris11 fbshipit-source-id: 79147c8196f298c443796a615fabefac34fb188d
3 participants
Closes #4430 by exposing int8 for cuVS cagra.
This PR (#4411) needs to be merged before this one .