showMatchesPosition and arrays of objects

[damienalexandre]

I'm trying out MeiliSearch and I stuck with a functionality.

Imagine a simple product in a store, it has multiple variants:

{
    "id": 666,
    "name": "Product name",
    "variations": [
        {
            "color": "red",
            "description": "Foobar",
            "price": 5000,
            "preview": "preview-red.jpg"
        },
        {
            "color": "pink",
            "description": "Covfefe",
            "price": 4000,
            "preview": "preview-pink.jpg"
        }
    ]
}

When I search product, for example "pink", I want this product to be displayed.

But I also want to display the appropriate and most relevant variant price, color and preview.

I was hoping the showMatchesPosition option would tell me which child is matched but it's not:

{
    "id": 666,
    "name": "Product name",
    "variations": [
        {
            "color": "red",
            "description": "Foobar",
            "price": 5000,
            "preview": "preview-red.jpg"
        },
        {
            "color": "pink",
            "description": "Covfefe",
            "price": 4000,
            "preview": "preview-pink.jpg"
        }
    ],
    "_matchesPosition": {
        "variations.color": [
            {
                "start": 0,
                "length": 4
            }
        ],
        "variations.preview": [
            {
                "start": 8,
                "length": 4
            }
        ]
    }
}

At the same time, the _matchesPosition response is not usable. In the presence of an array of objets, we don't know the position of the object in the array so having string offsets of matches inside one of the object cannot be used (maybe this should be a bug report).

Back in Elasticsearch I was using the inner hits option to get my variants but that's also because there is no showMatchesPosition option available.

A proposal for MeiliSearch could be to add the object position in the path? Like:

{
    "id": 666,
    [...]
    "_matchesPosition": {
        "variations.color.1": [
            {
                "start": 0,
                "length": 4
            }
        ],
        "variations.preview.1": [
            {
                "start": 8,
                "length": 4
            }
        ]
    }
}

As a workaround maybe I should transform my array of objects in a big object?

{
    "id": 666,
    "name": "Product name",
    "variations": {
        "1": {
            "color": "red",
            "description": "Foobar",
            "price": 5000,
            "preview": "preview-red.jpg"
        },
        "2": {
            "color": "pink",
            "description": "Covfefe",
            "price": 4000,
            "preview": "preview-pink.jpg"
        }
    }
}

But that sound bad for some reasons.

WDYT?

[gmourier]

Hi @damienalexandre 👋

But I also want to display the appropriate and most relevant variant price, color and preview.

Please correct, If I'm wrong.

Do you want to display the pink variation of the search result first in your interface, or only show the pink variation? Is it possible that you want to display another variation than the one-matching most terms in some cases?

{
    "id": 666,
    "name": "Product name",
    "variations": {
        "1": {
            "color": "red",
            "description": "Foobar",
            "price": 5000,
            "preview": "preview-red.jpg"
        },
        "2": {
            "color": "pink",
            "description": "Covfefe",
            "price": 4000,
            "preview": "preview-pink.jpg"
        }
    }
}

I agree; it feels bad. Did you tried it?

Thanks!

[damienalexandre]

Is it possible that you want to display another variation than the one-matching most terms in some cases?

Yes it is possible. Another need I have where this is more relevant is video searching. I'm indexing the subtitles of the video and I want to display the parts of text (with the timecode to be able to jump right to it) that matches. So for each video, you get multiple matches.

Did you tried it?

Yes absolutely and it works 👍 I get the full path of the matching variations in _matchesPosition 👍

I'm not sure about the related cost / impact on MeiliSearch though; in Elasticsearch, sending such a JSON is a big NOGO as it create A LOT of "fields". What if I have 3000 variations on a product? Is it going to explode?

[gmourier]

Hi @damienalexandre 👋

Thanks for the input.

Yes it is possible. Another need I have where this is more relevant is video searching. I'm indexing the subtitles of the video and I want to display the parts of text (with the timecode to be able to jump right to it) that matches. So for each video, you get multiple matches.

This could be a proper discussion but let's discuss that here for now.

Do you need to know the most relevant variation? In your example with the videos, I imagine you will list all matches with a for loop for a document, but I may be wrong; please correct me if it's the case.

I'm not sure about the related cost / impact on MeiliSearch though; in Elasticsearch, sending such a JSON is a big NOGO as it create A LOT of "fields". What if I have 3000 variations on a product? Is it going to explode?

It depends, I think. What bothers me the most is that it forces you to add an extra level and lose time reworking your dataset for this need.

We are very busy at the moment I can't give any ETA, but I would like to consider a solution to this pain point.

We could consider your proposal:

{
    "id": 666,
    "name": "Product name"
    "_matchesPosition": {
        "variations.color.1": [
            {
                "start": 0,
                "length": 4
            }
        ],
        "variations.preview.1": [
            {
                "start": 8,
                "length": 4
            }
        ]
    }
}

Thanks, I'm sorry for the delay.

[damienalexandre]

Do you need to know the most relevant variation?

Oh that would be awesome yes! Having the more relevant product variation or video subtitle part first would be very nice for the end-user.
In that case, _matchesPosition entries could be sorted by relevance.

It depends, I think. What bothers me the most is that it forces you to add an extra level and lose time reworking your dataset for this need.

Ok I'm going to do some benchmark about this "number of fields" issue because it worries me a bit - some of my videos are very long, and in this setup I'm creating a new field for each dialog part.

Thanks for considering the _matchesPosition change by the way 👍 😎

[damienalexandre]

I'm sad to report that using string keys (my workaround from the original question) are destroying the performances of MeiliSearch.

I indexed 200 videos with subtitles, it may have created an very large number of fields (+1000).

• When searching in that index I get +60 seconds responses times 🤯
• Remove all the subtitles improve speed but I get 300 ms responses 💥 (looks like the index is permanently slow now)
• Deleting the index / recreating it without subtitles, I get 1 ms responses as expected 😎

I came across this page too: https://docs.meilisearch.com/learn/advanced/known_limitations.html#maximum-number-of-document-fields

I will not be able to use _matchesPosition for my implementation.

[gmourier]

Thanks, @damienalexandre, for the report!

[Sofiman]

Hey, I wanted to know if there is any update or follow up to the consideration? Thank you!

[macraig]

Hello everyone! 👋

We're currently quite busy with features that have higher priority, however, we do see a need for this feature to be considered in the future. We've created a Portal in our Productboard to gauge user interest better; feel free to upvote there or share any use cases you have in this discussion.

As always, thank you for your feedback!

[macraig]

Adding feedback from #655 by @labadcloyd

I have a document in an index like so:

{
            "id": "9ef0e1cb-0c15-4cf0-a655-6e83af162bcb",
            "ingredient_mapping": [
                {
                    "ingredient_name": "Croissants",
                    "ingredient_variant_Name": "cheese",
                },
                { 
                    "ingredient_name": "Croissants", 
                    "ingredient_variant_Name": "butter", 
                },
                { 
                    "ingredient_name": "Croissants",
                    "ingredient_variant_Name": "apple", 
                }
            ],
            "name": "Croissants",
            "name_owner": "USDA"
}

If im going to search for the term "Croissants butter" it would most likely return this entire object. But if there was a way to know at which index of the array "ingredient_mapping" the match was then it would really help a lot.

This is similar to showMatchesPosition but the index of the object in the array would be returned, not the index of the string.

EDIT: I've queried the above document with showMatcherPosition and this is what it returns:

{
  "hits": [
    {
      "_matchesPosition": {
        "ingredient_details.ingredient_variant_name": [
          {
            "length": 6,
            "start": 0
          }
        ],
        "name": [
          {
            "length": 10,
            "start": 0
          }
        ]
      },
      "id": 129,
      "ingredient_details": [
        {
          "ingredient_mapping_id": 1950,
          "ingredient_subvariant_name": "",
          "ingredient_variant_name": "cheese"
        },
        {
          "ingredient_mapping_id": 1951,
          "ingredient_subvariant_name": "",
          "ingredient_variant_name": "butter"
        },
        {
          "ingredient_mapping_id": 1952,
          "ingredient_subvariant_name": "",
          "ingredient_variant_name": "apple"
        }
      ],
      "name": "Croissants",
      "name_owner": "USDA"
    }
  ],
  "estimatedTotalHits": 1,
  "limit": 20,
  "processingTimeMs": 1,
  "query": "croissant butter"
}

Is there a way we can add the index to the array in the "_matchesPosition" field data?

[LukasKalbertodt]

When working on a PR for this, I encountered some implementation questions that I would like to check with the devs first. Hence this comment.

Another motivational example

I manage lecture recordings where each video has a metadata field speakers, which is an array as there are videos with multiple people talking. Searching through all videos should find by speaker name, of course. And I want to highlight it via _matchesPosition, but that's impossible right now. My only option is to work around this by concatenating all speaker names with some super special separator character (that I must hope never appears in speaker names...) and store it as string field. Or to make an object out of it with keys like "2". Both of these workarounds seem silly.

What I propose to change

Include the array index in the keys of _matchesPosition (e.g. "foo[1]"). Example document:

{
  "id":"123",
  "names": ["foo", "bar", "catnip"]
}

Searching for cat would return:

"_matchesPosition": {
  "names[2]": [  // <- new
    {
      "length": 3,
      "start": 0
    }
  ]
}

This is technically a breaking change, however I would argue that it's reasonable to include this change in a new anyway: currently, the _matchesPosition for array fields are completely unusable since you don't know what element it refers to. So I don't think anyone is using that right now, and thus nothing out there will break.

Implementation questions

(CC @irevoire since you coded permissive-json-pointer and are likely relevant here)

The relevant code is here: https://github.com/meilisearch/meilisearch/blob/30f3c303897e478352e69bc1f647dda383081160/meilisearch/src/search/mod.rs#L1731-L1770

permissive_json_pointer::map_leaf_values is used to walk over the document, while only looking at fields that have been requested. The key argument the closure receives is what ends up the key in _matchesPosition (see matches.insert(key.to_owned(), infos);). So the function map_leaf_values contains the logic we want to change.

One option is to change it in the permissive-json-pointer crate directly, which then needs to be released as a breaking change and then updated in Meili. However, it seems like the crate does not see much usage outside of Meilisearch. Further, Meilisearch uses map_leaf_values in only this place I linked (other than that, only select_values from the crate is used). And given that the exact logic of map_leaf_values is quite important for Meili's API, I would argue to inline this functionality in Meili directly. It's not a lot of code either. Then we can easily change it without worrying about users of the permissive-json-pointer library.

As an aside: I noticed that the format_value function (directly below format_fields, the function I linked above) is only used by format_fields in one place. And since map_leaf_values only calls the closure for, well, leaf values, the branches inside format_value that deal with arrays and objects are never used and could be removed.

When copying map_leaf_values to Meili, the actual change would be super easy. I can certainly create a PR for that. I just need to know (a) if that change would be merged, and (b) whether my implementation suggestion is fine.

[irevoire]

Hey @damienalexandre thanks for looking into it!

First of all, I already pushed permissive-json-pointer in Meilisearch here.
So you can modify it easily without re-publishing anything.
The one on crates.io works well but isn’t used in Meilisearch anymore 😔

Technically, I personally don’t see any issue with what you said and the API you propose seems on par with what I currently do in the error message (the [x] at the end of a field name).
But:

• It’s a breaking change, I believe? (not sure / we could keep the current behavior + add the new one maybe)
• It’s going to be hard to uses, isn’t it? Are people supposed to parse the string back and try to extract the index?

Maybe we could find a better API where the index is part of a json field 🤔
What do you think of something like that?

"_matchesPosition": {
  "names": [  // <- removed the `[2]`
    {
      "length": 3,
      "start": 0,
      "index": 2 // <- new
    }
  ]
}

[LukasKalbertodt]

First of all, I already pushed permissive-json-pointer in Meilisearch here.

Oh, I didn't think to check Cargo.toml :o okay that makes it easier

It’s going to be hard to uses, isn’t it? Are people supposed to parse the string back and try to extract the index?

That's a good point. I still think the [i] syntax is the most natural, but yeah, parsing it seems annoying. Also, foo[2] is a valid JSON key (not sure if Meili allows you to add such documents, but anyway). Now I feel kinda stupid not having thought about these problems :D

I like your API idea, it's straight forward and backwards-compatible. Can't think of any problems with it.

Will prepare a PR for this on Monday.

[irevoire]

not sure if Meili allows you to add such documents

Meili accept everything as far as I know, even an empty string for a field name.

That's a good point. I still think the [i] syntax is the most natural

Yeah, I was lowkey thinking of outputting it both in the field name and in the fields with the index, but I don’t think it really adds anything to the end user, and it makes everything harder to parse 😔

---

On my side, I’ll ping the product team to ask them:

• What they think of the feature => is this something we want to merge
• What they think of the API

(poke @macraig)

[macraig]

Both the API and the feature look good to me, so merge ahead 😄

Thanks @LukasKalbertodt for picking this up! 🫶

[LukasKalbertodt]

meilisearch/meilisearch#5005

Unfortunately, the simple index: 3 design does not work thanks to nested arrays. I used a similar API design, but made it indices: [1, 2], as can be seen in the PR. The PR description discusses a few alternatives as well.