Skip to content

Add support for working with binary indexes.#80

Closed
aalekhpatel07 wants to merge 15 commits intoEnet4:masterfrom
aalekhpatel07:binary-indexes
Closed

Add support for working with binary indexes.#80
aalekhpatel07 wants to merge 15 commits intoEnet4:masterfrom
aalekhpatel07:binary-indexes

Conversation

@aalekhpatel07
Copy link
Contributor

This is an attempt to provide first-class support for Faiss binary indexes consuming the C API (along with a couple of utility functions introduced in facebookresearch/faiss#3318)

Note: #77 got merged so this is in lieu of #79

This should be complete in terms of functionality (at least for a default BinaryIndexImpl) so I'm hoping to learn of any major concerns before continuing down this path and adding the IndexBinaryFlat, IndexBinaryHNSW, etc, concrete impls.

aalekhpatel07 and others added 7 commits March 27, 2024 12:10
…macros for binary indexes.

Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
@aalekhpatel07
Copy link
Contributor Author

Seems like the upstream changes for cloning and creating binary indexes via the C api have landed.

Please let me know if I can contribute towards any other blockers for landing this.

Thanks!

Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
…y indexes implemented yet.

Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
…vectors.

Signed-off-by: Aalekh Patel <aalekh.gwpeck.7998@icloud.com>
@aalekhpatel07
Copy link
Contributor Author

So I enabled the CI for these changes and if we bumped upstream to use the latest revision, then the CI test suite continues to pass.

I'm leaning towards waiting for a new release of upstream so we can build against v1.8.1 and hopefully the binary index clone/read changes would be included in that release.

The CI helped discover a couple of bugs and their fixes have now been cherry-picked to this PR.

@bh1cqx
Copy link

bh1cqx commented Oct 8, 2024

@aalekhpatel07 Thanks a lot for your work. Looks like 1.9.0 is out. Do you intend to follow up on this?

@aalekhpatel07
Copy link
Contributor Author

@aalekhpatel07 Thanks a lot for your work. Looks like 1.9.0 is out. Do you intend to follow up on this?

Certainly! Thanks for the ping. I'll try to contribute as life permits.

In the meantime, I'm thinking we should probably sync up Enet4/faiss upto v1.9.0 and bump the checkout of ./faiss-sys/faiss to d243e62 so that the binary index IO is in place for CI.

@aalekhpatel07 aalekhpatel07 closed this by deleting the head repository Aug 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants