Adds support for 7-bit quantization in BBQ index#143611
Conversation
Enables the use of 7-bit quantization for indexed vectors in BBQ, improving flexibility for disk-based vector fields. Updates validation, documentation, and tests to accommodate the new option and ensures correct parameter handling throughout the codebase. Relates to elastic#141183
elasticsearchmachine
added
needs:triage
🔍 Preview links for changed docs |
ℹ️ Important: Docs version tagging👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version. We use applies_to tags to mark version-specific features and changes. Expand for a quick overviewWhen to use applies_to tags:✅ At the page level to indicate which products/deployments the content applies to (mandatory) What NOT to do:❌ Don't remove or replace information that applies to an older version 🤔 Need help?
|
✅ Vale Linting ResultsNo issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
elasticsearchmachine
added
the
Team:Search Relevance
|
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
elasticsearchmachine
removed
the
needs:triage
| if (Build.current().isSnapshot()) { | ||
| return new ESNextDiskBBQVectorsFormat( | ||
| ESNextDiskBBQVectorsFormat.QuantEncoding.ONE_BIT_4BIT_QUERY, | ||
| ESNextDiskBBQVectorsFormat.QuantEncoding.fromId(diskbbq.getBits() >> 1), |
There was a problem hiding this comment.
We cannot do the id like this. It makes an assumption we just cannot keep. WE should either send the ID directly, or have a direct mapping.
There was a problem hiding this comment.
Fixed. Since the pattern used in DenseVectorFieldMapper.java was followed, that file has been updated as well.
Switches from using a shifted ID to directly interpreting bits for quantization encoding selection, improving accuracy and consistency in vector format initialization.
* Adds support for 7-bit quantization in BBQ index Enables the use of 7-bit quantization for indexed vectors in BBQ, improving flexibility for disk-based vector fields. Updates validation, documentation, and tests to accommodate the new option and ensures correct parameter handling throughout the codebase. Relates to elastic#141183 * Updates quantization encoding selection logic Switches from using a shifted ID to directly interpreting bits for quantization encoding selection, improving accuracy and consistency in vector format initialization.
* Adds support for 7-bit quantization in BBQ index Enables the use of 7-bit quantization for indexed vectors in BBQ, improving flexibility for disk-based vector fields. Updates validation, documentation, and tests to accommodate the new option and ensures correct parameter handling throughout the codebase. Relates to elastic#141183 * Updates quantization encoding selection logic Switches from using a shifted ID to directly interpreting bits for quantization encoding selection, improving accuracy and consistency in vector format initialization.
3 participants
Enables the use of 7-bit quantization for indexed vectors in BBQ (based on received comment), improving flexibility for disk-based vector fields. Updates validation, documentation, and tests to accommodate the new option and ensures correct parameter handling throughout the codebase.
Relates to #141183