PoC - Vector rescoring in kNN

[carlosdelest]

PoC that adds a new parameter to kNN query (rescore_vector_oversample) that is used to:

• Multiply the number of candidates in the kNN query
• Perform an approximate search in the extended candidate set
• Perform an exact search over the returned results, and get the top k

This is done by overriding the approximateSearch() method in ESKnnFloatVectorQuery. It could be pushed down to the Lucene query if needed.

Usage:

GET msmarco-v2-bbq/_search
{
    "query": {
        "knn": {
            "field": "emb",
            "query_vector": [...],
            "k": 10,
            "num_candidates": 100,
            "rescore_vector_oversample": 10.0
        }
    }
}

[benwtrent]

I don't think this will work as exactSearch would use the quantized scorers, also this would be done per segment. Which would be bad.

I think what we need to do is override rewrite to return another query that can be further rewritten but scores the previously scored documents given the raw floating point vectors.

We only want to rescore for the entire shard. Doing each segment would be very expensive.

[carlosdelest]

I see, thanks for explaining Ben. I'll give it another shot 🔫

[benwtrent]

I do think an object with a new parameter is best. We will likely have separate options to provide (rescore field, rescore kind, etc.)

[carlosdelest]

Agreed - this was just me not wanting to deal with the parser yet 😁

[carlosdelest]

Closing this in favour of #116663