Skip to content

[Native][x64] Native implementation of DiskBBQ scoring (distance w adjustment)#141332

Merged
thecoop merged 43 commits intoelastic:mainfrom
ldematte:native/diskbbq-scoring
Feb 9, 2026
Merged

[Native][x64] Native implementation of DiskBBQ scoring (distance w adjustment)#141332
thecoop merged 43 commits intoelastic:mainfrom
ldematte:native/diskbbq-scoring

Conversation

@ldematte
Copy link
Copy Markdown
Contributor

In #140264 we introduce native functions to compute distance metrics for (Disk)BBQ. We kept the Panama implementation of the scoring, i.e. application of adjustments over the basic distance computation.
In this PR we introduce native functions that implement this second part too; this first iteration has optimized functions for x86 (AVX2 and AVX-512).
Benchmarks show an additional improvement of 25-30% (or, in other words, ~55% overall wrt the Panama implementation).

The PR also has a small refactoring on the C++ code, moving capabilities check to a separate file and some shared functionality to common headers.

@ldematte ldematte changed the title [Native] Native implementation of DiskBBQ scoring (distance w adjustment) [Native][x64] Native implementation of DiskBBQ scoring (distance w adjustment) Jan 27, 2026
@ldematte ldematte marked this pull request as ready for review January 27, 2026 17:23
@ldematte ldematte requested a review from a team as a code owner January 27, 2026 17:23
@ldematte ldematte requested review from benwtrent and thecoop February 5, 2026 09:48
@thecoop
Copy link
Copy Markdown
Member

thecoop commented Feb 6, 2026

Are we able to use these methods for D2Q4 as well, or do those need completely separate implementations?

@tteofili
Copy link
Copy Markdown
Contributor

tteofili commented Feb 6, 2026

Looks like this will need some rework after #141598, but I think the native code is already OK (it already gets the bulk size as a separate parameter and handles the tail). @tteofili can you confirm?

sorry for the late reply, I guess you figured the answer was yes :)

Copy link
Copy Markdown
Contributor

@tteofili tteofili left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

Copy link
Copy Markdown
Member

@benwtrent benwtrent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we do a rename and call the actions that apply corrections applyCorrections?

@ldematte
Copy link
Copy Markdown
Contributor Author

ldematte commented Feb 6, 2026

Are we able to use these methods for D2Q4 as well, or do those need completely separate implementations?

I added a index bits params for this reason - I'll double check, but the implementations looks to be identical

Or in other words, it is my intention to have 1 implementation for all

@thecoop
Copy link
Copy Markdown
Member

thecoop commented Feb 6, 2026

Is it worth just doing the D2Q4 impl here as well then?

@ldematte
Copy link
Copy Markdown
Contributor Author

ldematte commented Feb 7, 2026

Is it worth just doing the D2Q4 impl here as well then?

Yep, it was easy. Done!
I left D4Q4 alone as I know you are working on it; better if you address it then.

@ldematte
Copy link
Copy Markdown
Contributor Author

ldematte commented Feb 7, 2026

Could we do a rename and call the actions that apply corrections applyCorrections?

Done!

Copy link
Copy Markdown
Member

@thecoop thecoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@thecoop thecoop merged commit b2c4df8 into elastic:main Feb 9, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>non-issue :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch test-arm Pull Requests that should be tested against arm agents v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants