Running in non-Intel machine

[msharmavikram]

Summary

I am trying to run FAISS in IBM Power 8 system with P100 GPUs. I see that most of code is hardcoded to use Intel platform with SSE4 enabled. Are there any ways I could get this working without the help of SSE4? I see IndexHNSW files have SSE4 optimizations. Any means I can avoid file and get running?

Platform

IBM Power 8 with Nvidia P100 machines with Cuda 8.0

OS: Ubuntu 16.04LTS

Faiss version: 2dc30e1

Faiss compilation options: Not using MKL, SSE4 or Intel specific options

Running on:

• CPU
• GPU

Interface:

• C++
• Python

Reproduction instructions

Run it on IBM Machine.

[mdouze]

This is a good question. Sadly, by lack of a non-x64 dev platform we were dragged into writing non portable code.

However, all code that depends on SSE/popcount has a "slow" version that is written in generic c++. That code must be enabled, often manually in the current version.

In IndexHNSW, the only part that depends on SSE is the one handling PQ compressed HNSW which is a corner case use of HNSW. It can be safely commented out.

The function fvec_L2sqr is the most useful function for HNSW without compression, it is implemented in utils.cpp and the generic version is fvec_L2sqr_ref.

[msharmavikram]

Thanks for the information. I am going to try out these modifications that you suggested and if I find more issues, I will continue this thread.

[msharmavikram]

Are there any alternatives to these two structures?
Distance2xXPQ4 and Distance2xXPQ4

[mdouze]

Compilation for non-intel machines is supported in 1.4.0