Search has issues with words adjacent to puncutation characters

[Knaui]

for example: it wont find "house" in "big-house"

but it will find "big"

this is the case for book or page titles and for page content

tested with BookStack version v0.29.0

[kshitijsharma97]

I also tried same ting in my dev instance I had the same issue.
If I try the 1st word then the page will come but if I try the whole name with hyphen nothing came in search result.

But if you pull the changes and update the version to BookStack v0.29.3.
The issue with this hyphen separated search is resolved.

[ssddanbrown]

Updating the title to be more generic in the interest of merging down some issues.

• search cannot handle dots (.) correctly #2843
• Search does not find words after slash ("/") #2523

Related to #1037

[Wookbert]

@ssddanbrown

I’ve just realized that searching word parts which are combined through hyphens, doesn't work either.

Example: Searching for historian does not find the page on CCU-Historian, while searching for ccu does. Note that hyphens are a very common element in for instance German language. You often have word combinations which are connected through 2 or even 3 hyphens.

An english language example would be Remote-robot-assisted, which IMO should be retrieved when searching for any of the three words individually, but also e.g. robot-assisted, robot assisted or robotassisted. (Same applies for any spelling of the Remote robot combination).

[dweinerATL]

@ssddanbrown we are running into something similar. Running BookStack v21.05.4 for a science fiction authors book series. One of her races are called Ke!endarian. If you search for Ke!endarian, no results. If you search for Kel, you get the expected response. We have found that the search will work if you search for "Ke!endarian" however.

[ssddanbrown]

As part of #3043 I've made a change to auto-convert any search terms, that would experience this issue, into exact match terms instead which will run a direct, although less efficient, content match. Doesn't directly solve this but should provide a much better user-experience in such situations. Will be part of the next feature release.

[caius-martinus]

Hello @ssddanbrown,
I think issue isn't solved at least in 23.08.2, here is how to reproduce: create a page with the content /abc123 on a single line. Now search abc1 and you should observe it doesn't match. However /abc1 would.

[sNiXx]

I can confirm this issue is still present on 23.10.2. I also just verified on the demo instance (currently 23.10.4) and hyphenated words are not correctly found. For instance, the pages prod-linode-sparkjet or dev-internal-sparklebike on the demo instance cannot be found if the last term (i.e. sparkjet or sparklebike) is used to search.

[watschi]

Facing the same issue with hyphenated words, which are pretty common in german text.
Quick and dirty solution (needs to be applied after any update):

• Edit app/Search/SearchIndex.php, add a hyphen (-) to $delimiters (at Link)
• Run php artisan bookstack:regenerate-search
• For the word Test-Word, Test, Word and Test-Word will return the desired content

@ssddanbrown Any reason to exclude - from the delimiters? Feels like this should be included by default, maybe it's an oversight, maybe I'm missing something 🙂

[ssddanbrown]

@watschi

Any reason to exclude - from the delimiters? Feels like this should be included by default, maybe it's an oversight, maybe I'm missing something 🙂

Really it was because they felt more part of a term rather than something to split them by, but I can see the issue that would result.

I spent some time on this today to change up the indexing a bit via #5488.
I've tried to come to a compromise to help address some of the most problematic areas, in addition to adding - as a delimiter.
Now, for the text cat-dog BookStack will now index that as cat, dog and cat-dog.
That way, searching for either work will work but the full term will also work via our proper indexed term system.
The same is done for dots/periods (which I thought could be important for numbering among other things).

There will still be gaps and limitations in search due to the nature of the trying to keep content indexed, using prefix matching, and the use of custom tokenization, but this should solve some of the most common issues here reported about hyphenated words.
Therefore I'm going to close this off but new focus areas can be raised as needed (If not already open).

The mentioned changes will be part of the next feature release.
Note, that you'd need to regenerate the search index after updating to gain these index improvements.

Thanks all for your input!