Consider improving interaction between stop words and phrase search

[LukasKalbertodt]

I just tried using stop words with Meili for the first time. My main goal was to reduce the number of match positions that would be returned by the API, as I'm using showMatchesPosition. But of course also to improve relevancy, search speed, indexing speed and index size.

But I noticed that stop words basically make phrase search unusable. It still works for phrases that don't have any stop words in it, like full names. So it's still useful. But as soon as the phrase contains a stop word, two things happen:

• The phrase is not highlighted in results anymore, i.e. _matchesPosition does not contain the match.
• Way more results are returned which do not contain the exact phrase.

Here you can see an example from our production application. Left is without stop words, right is with. The second and third result on the left have an exact match in the subtitles (just in case you were confused why they appeared). The second result on the right does not contain moon anywhere in the document.

I checked Google and there, phrase search also considers stop words. Well, at least I suspect that Google uses stop words somehow when not using phrase search. Sorry for the ego-centric example, but it's the easiest I could come up with: searching for "performs the perspective divide and uses the angular radius as field of view" only brings up exactly one result (well, I suppose it will soon also bring up this very discussion...). And the exact phrase is highlighted in the result, including the. Changing the the to a does not result in any results.

---

I have not really found a way to work around this problem. From my perspective, using stop words cripples phrase search, making it completely useless in many situations and very likely confusing users. That's not really an option for us, so it seems we cannot use stop words at all?

This seems like a suboptimal solution and I wanted to open this discussion to talk about this. Whether you have considered ways to improve the situation. I think it would be best if phrase search would still work as before, not being affected by any configured stop words. But I suspect that implementing this is not trivial?

[macraig]

Hi @LukasKalbertodt , thanks for bringing this to our attention. You make a good point about stop words affecting phrase search for your use case. We’ll bring this up with the team and share an update here. As a heads-up, this will probably happen in January since most of the team will be off for the holidays in the coming weeks.

[LukasKalbertodt]

Thanks for the quick reply and for the plans to discuss this :) Happy holidays to you all!