search: improve mechanism

[lvthillo]

I know it's a bit of a vague request, but we are using this theme for an internal handbook. We recently switched to your theme instead of the old one. Thanks for your work.

Unfortunately, we receive many internal complains about the search mechanism of the theme (also when we were using the old theme, but not it starts to become a real issue). We activated the dedicated search page to make it easier but still, there are too many results shown, especially when you try to search for a combination of words or exact sentences. I'm not an Hugo expert at all, but it would be nice if the theme offers other/multiple possibilities to implement searching. The official docs describe some of them. I do understand if you see this as "overkill" but we're just looking for some improvements here. Feel free to make other recommendations.

[McShelby]

This partly plays together with #407 where more options to the search will be added.

The theme already uses Lunr to provide search capabilities. So generating an index is not the real issue here.

Instead, some research must be done on how to improve search results for multiple terms and options. This could also mean to replace the whole Lunr library in case it does not suffice all requirements.

I plan to tackle this somewhere around this year.

[tlindsay42]

One of my customers reported the opposite- results aren't showing up for indexed words. I don't have bandwidth to dive deeper on this one at the moment.

[MarciaRieferJohnston]

Hi, @McShelby. The attached screenshots show two two types of search failures I see:

• Results are returned when they shouldn't be (matches are wrong, such as line returned instead of like).
• No results are returned when they should be (matches are missing, such as like or might not returned at all). A colleague tells me that these keywords did show up in the output JavaScript file in these cases.

Obviously, this behavior compromises credibility and usability of our docs. Fixing this basic functionality will make a big difference. When do you expect to turn attention to this?

[McShelby]

@tlindsay42 @MarciaRieferJohnston I can not reproduce this with the exampleSite. Especially searching there for like and line produces the correct search results.

I need access to your source or a stripped down minimal example.

I have the feeling that there may be shortcodes involved but that's just a nerdy guess.

Anyways, search is configured to be fusy and tries to compensate typos. So there might be false positives on the result page. Currently it is configured to allow for the following number of typos per word length:

length in chars	allowed typos
<= 4	0
<= 8	1
<= 12	2
<= 16	3
<= 20	4
<= 40	3
<= 60	2
> 60	1

[tlindsay42]

Thanks for the quick response @McShelby! These are internal projects, so we're not authorized to share the source for these, but I'll put together an example for repro in the next day or so.

[MarciaRieferJohnston]

@McShelby Thanks from me too. In case it's relevant, the screenshots above are from a doc built from .adoc files. I haven't tried .md files yet.

[postb99]

I need to check out whether there could be a setting for enabling full word matching only, or maybe match the word and its plural form (when plural is just addition of some characters to the singular form). In particular, when looking for "chat" on my live site I have some results that absolutely don't contain this. It looks like it performs a proximity search. i.e. in French "chat" gives a result with "chapelet" and "chant" as close words.

[McShelby]

It looks like it performs a proximity search

That's correct. See my previous comment here. This not configurable at the moment.