|
| 1 | += Fine-Tuning a Vector Search Query |
| 2 | +:stem: asciimath |
| 3 | +:page-ui-name: {ui-name} |
| 4 | +:page-product-name: {product-name} |
| 5 | +:description: Add additional parameters to a Vector Search query object to tune the search for recall or accuracy. |
| 6 | + |
| 7 | +[abstract] |
| 8 | +{description} |
| 9 | + |
| 10 | + |
| 11 | + |
| 12 | +The Search Service automatically tunes your Vector Search indexes to achieve a balance between: |
| 13 | + |
| 14 | +* Recall, or the quality of your search results |
| 15 | +* Latency, or your search response time |
| 16 | +* Memory efficiency |
| 17 | +
|
| 18 | +This tuning occurs during indexing and querying. |
| 19 | +You do not need to adjust these parameters manually. |
| 20 | + |
| 21 | +Specifically, the Search Service dynamically adjusts two critical vector parameters: |
| 22 | + |
| 23 | +`nlist`, also known as `Centroid` count:: |
| 24 | +The number of clusters used for indexing. |
| 25 | +Centroids are used to quickly find the surrounding closest matches in the Vector Search index. |
| 26 | +Increasing the number of centroids will increase accuracy but will decrease the speed of the search. |
| 27 | ++ |
| 28 | +The `nlist` is determined dynamically based on the size of the dataset, or the number of vectors in a partition: |
| 29 | ++ |
| 30 | +[%header, cols="3*a"] |
| 31 | +|=== |
| 32 | +| Number of vectors in partition (`nvec`) |
| 33 | +| `Centroid count` (`nlist` calculation) |
| 34 | +| Notes |
| 35 | + |
| 36 | +| stem:["nvec " ge " 200,000"] |
| 37 | +| stem:[4 xx sqrt("nvec")] |
| 38 | +| This formula is designed to handle larger datasets |
| 39 | + where increasing the number of datasets does not yield significant improvements in recall. |
| 40 | + |
| 41 | +| stem:["1000" le "nvec" le "200,000" ] |
| 42 | +| stem:["nvec" / 100] |
| 43 | +| This formula targets approximately 100 vectors per cluster, |
| 44 | +which balances between too few and too many clusters, ensuring efficient indexing. |
| 45 | + |
| 46 | +| stem:["nvec" lt 1000] |
| 47 | +| N/A |
| 48 | +| For a number of vectors less than 1000, the Search Service will carry out a straight forward one-to-one mapping between IDs and vectors with an exact vector comparison. |
| 49 | +Vectors are directly stored without the need for additional processing for the `nlist` calculation. |
| 50 | + |
| 51 | +|=== |
| 52 | + |
| 53 | +`nprobes` (or `probes`):: |
| 54 | +This is the number of `centroids` that a Search query will check for similar vectors. |
| 55 | +The `nprobe` value is only set when the Search Service is using an Inverted File Index. The Search Service will select the best index type and comparison method depending on the size of the dataset and your `vector_index_optimized_for` setting. |
| 56 | ++ |
| 57 | +For more information on the `vector_index_optimized_for` setting, see xref:search:search-index-params.adoc#vector-index-optimized-param[Search Index JSON properties]. |
| 58 | ++ |
| 59 | ++ |
| 60 | +[%header, cols="3*a"] |
| 61 | +|=== |
| 62 | +| Query optimization |
| 63 | +| `nprobe` calculation |
| 64 | +| Notes |
| 65 | + |
| 66 | + |
| 67 | +| Default calculation |
| 68 | +| stem:[sqrt("nlist")] |
| 69 | +| This provides a balanced tradeoff between recall and latency by adjusting the number of clusters probed during queries. |
| 70 | + |
| 71 | +| Latency-optimized calculation (`vector_index_optimized_for: latency`) |
| 72 | +| stem:[sqrt("nlist") / 2] |
| 73 | +| A minimum value of 1 is enforced to avoid setting `nprobe` too low. |
| 74 | + |
| 75 | +|=== |
| 76 | + |
| 77 | +== Default `nlist` and `nprobe` calculations on a Vector Search Index |
| 78 | + |
| 79 | + |
| 80 | +The cluster maintains two dynamically adjusted parameters that will affect the speed, accuracy, and resources used during a search: |
| 81 | + |
| 82 | +`ivf_nprobe_pct`:: |
| 83 | +The percentage of clusters searched during queries, allowing for fine-tuning of the balance between recall and performance. |
| 84 | +If the value of `nprobe` is 5% of `nlist` (the centroid count), then setting the value of `ivf_nprobe_pct` higher than 5% will have the search cover a higher percentage of clusters, which will improve the accuracy of the search. |
| 85 | + |
| 86 | +`ivf_max_codes_pct`:: |
| 87 | +The value represents the percentage of `centroids` that will be visited during a search. |
| 88 | +Reducing the value reduces the number of centroids visited, which will decrease accuracy and recall, but will result in faster compute times. The default value is 100 (i.e 100% of the centroids will be visited during the search). |
| 89 | + |
| 90 | + |
| 91 | +.Default calculation |
| 92 | +==== |
| 93 | +If you have a Vector Search index with `vector_index_optimized_for` set to `"recall"` and `indexPartitions` set to `5`, then the `centroid count` (`nlist`) and `nprobe` are determined based on the current vector count in a given partition. |
| 94 | +[options="noheader", frame="none", grid="none" cols="1,1"] |
| 95 | +
|
| 96 | +|=== |
| 97 | +| Total vectors in index (optimization = recall) |
| 98 | +| 10,000,000 |
| 99 | +
|
| 100 | +| Average vectors in a partition for 5 partitions total |
| 101 | +| 2,000,000 |
| 102 | +
|
| 103 | +| centroid count (`nlist`) = stem:[4 times sqrt("total vectors in index")] |
| 104 | +| 5657 |
| 105 | +
|
| 106 | +| nprobes = stem:[sqrt(nlist)] |
| 107 | +| 75 |
| 108 | +
|
| 109 | +| Calculated default: `ivf_nprobe_pct` |
| 110 | +| 1.325% |
| 111 | +
|
| 112 | +| Calculated default: `ivf_max_codes_pct` |
| 113 | +| 100% (default value) |
| 114 | +
|
| 115 | +|=== |
| 116 | +
|
| 117 | +==== |
| 118 | + |
| 119 | +== Fine-Tuning Query Parameters |
| 120 | + |
| 121 | + |
| 122 | +You can add set the values of `ivf_nprobe_pct` and `ivf_max_codes_pct` in your Vector Search queries to tune the recall or accuracy of your search. |
| 123 | + |
| 124 | +You can add the following parameters to your query: |
| 125 | + |
| 126 | +[source, json] |
| 127 | +.Using tuning parameters |
| 128 | +---- |
| 129 | +{ |
| 130 | + "fields": ["*"], |
| 131 | + "knn": [{ |
| 132 | + "k": 10, |
| 133 | + "params": { |
| 134 | + "ivf_nprobe_pct": 1, |
| 135 | + "ivf_max_codes_pct": 0.2 |
| 136 | + }, |
| 137 | + "field": "embedding", |
| 138 | + "vector": [0.024901132253900747, . . .] |
| 139 | + }] |
| 140 | +} |
| 141 | +---- |
| 142 | + |
| 143 | +In the preceding example, the Search request returns results from any available fields in the index, but specifically searches the `embedding` field for the `10` closest matches to the vector `[0.024901132253900747, . . .]`. |
| 144 | +The vector in the Search request has been truncated to reduce the display size of the example. |
| 145 | +The parameters have been set to search 1% of the clusters, and 0.2 per cent of the centroids. |
| 146 | + |
| 147 | + |
| 148 | + |
| 149 | + |
| 150 | + |
0 commit comments