[Native][x64] Native implementation of DiskBBQ scoring (distance w adjustment)#141332
[Native][x64] Native implementation of DiskBBQ scoring (distance w adjustment)#141332thecoop merged 43 commits intoelastic:mainfrom
Conversation
…2 for x64. Untested!
# Conflicts: # libs/simdvec/native/src/vec/c/aarch64/vec_1.cpp
libs/native/src/main/java/org/elasticsearch/nativeaccess/VectorSimilarityFunctions.java
Outdated
Show resolved
Hide resolved
...java/org/elasticsearch/simdvec/internal/vectorization/MSBitToInt4ESNextOSQVectorsScorer.java
Outdated
Show resolved
Hide resolved
...java/org/elasticsearch/simdvec/internal/vectorization/MSBitToInt4ESNextOSQVectorsScorer.java
Outdated
Show resolved
Hide resolved
|
Are we able to use these methods for D2Q4 as well, or do those need completely separate implementations? |
benwtrent
left a comment
There was a problem hiding this comment.
Could we do a rename and call the actions that apply corrections applyCorrections?
I added a index bits params for this reason - I'll double check, but the implementations looks to be identical Or in other words, it is my intention to have 1 implementation for all |
|
Is it worth just doing the D2Q4 impl here as well then? |
Yep, it was easy. Done! |
Done! |
…earch into native/diskbbq-scoring
In #140264 we introduce native functions to compute distance metrics for (Disk)BBQ. We kept the Panama implementation of the scoring, i.e. application of adjustments over the basic distance computation.
In this PR we introduce native functions that implement this second part too; this first iteration has optimized functions for x86 (AVX2 and AVX-512).
Benchmarks show an additional improvement of 25-30% (or, in other words, ~55% overall wrt the Panama implementation).
The PR also has a small refactoring on the C++ code, moving capabilities check to a separate file and some shared functionality to common headers.