Fulltextsearch seems broken under certain conditions

[kelson42]

I have loaded two "generations" of the same Gutenberg title:

$ ls *mul*
gutenberg_mul_all_2023-05.zim  gutenberg_mul_all_2023-08.zim
$ kiwix-serve --port=8080 *mul*zim
The Kiwix server is running and can be accessed in the local network at: http://192.168.0.168:8080

Once online, I have chosen one and started a fulltext search and I have got:

This looks like a bug, why I'm not able to fulltext search in my ZIM file?

[kelson42]

@veloman-yunkan I never have seen this error before, I suspect this has been introduced quite recently.

[veloman-yunkan]

This error comes from kiwix/libkiwix#838 which restricts the search to a set of books all in the same (single) language. In the reported scenario we deal with a search in a single ZIM file but the problem is that it is a multi-language ZIM file! Hence the search operation is prone to various problems discussed in kiwix/libkiwix#785, and thus the error is justified. On the other hand, this makes all multi-language ZIM files unsearchable which is plain silly. I can easily relax the check that prevents search from working on such a ZIM file but that will be only part of the full fix. For full-text search to work correctly the xapian database must be properly populated, therefore indexing of multi-language ZIM files must be enhanced in libzim. Also we will have to specify the query language when searching in a multi-language ZIM file.

@mgautierfr @kelson42

[kelson42]

@veloman-yunkan Thank you for the explanation. @mgautierfr will work very soon on openzim/libzim#734 and that should fix this bug. Postponing to the next release.