Use Stork as the search backend in HTML sites

[Seelengrab]

Hello,

at the moment, reloading the search page or reloading a page built with Documenter leads to VERY noticeable lag on the first search. Since it's kind of unusual to keep a search tab for each package open in the background, it would be very nice to improve search performance. I think the majority of this initial search lag is related to deserializing/reading an index created by Lunr, but I must admit that I don't know too much about how this works in Documenter.

In a discussion on Slack about this, a number of people mentioned stork, which is expressly built for being a fast search for static sites - which seems like the perfect application for Documenter. A small example of the experience provided can be checked out here, which really does seem to be unbelievably fast (though I don't know how long "the first 20 of the Federalist Papers" is compared to julia documentation - but I don't think our docs are longer). Is it possible to consider switching to this by default, to provide a better search experience for everyone, including users of the official Julia Documentation?

[kdheepak]

I use Stork for my personal blog and have greatly enjoyed the experience. It allows for building the index on CI and pushing that as part of a static site for GitHub.

I think some extra care needs to be taken to manage the index of documentation of different tagged versions, but otherwise I think it’d be a great fit.

[Seelengrab]

Makie just switched to stork, and I gotta say - this looks & feels extremely nice!

Take a look:

https://makie.juliaplots.org/v0.17.3/

[mortenpi]

I think the best way to contribute here would be to create a simple prototype that runs the Julia manual on Stork, to see if and by how much it improves performance. All the search-related front end stuff lives in search.js, and one could just convert the current JSON index, so it wouldn't be necessary to touch Documenter itself at all for the initial prototype.

I'll make this issue specific to stork (so it would be actionable), but there is also #1831 that takes a different approach to improve the search performance / UX.

[Seelengrab]

Stork would mostly work as part of the CI doc creation step - it doesn't really live in any search.js the way Lunr has to. The index is built in CI, with the JS library simply retrieving it. The only bit of JS required is to register the <input> for search and associate it with the given index.

I'll see if I can find some time later today to make a PR to base (it really shouldn't be too complicated - here is the diff Makie had to do, which is basically nothing - only the building of the stork config is a bit longer due to not living in Documenter already), but getting this to work by default in Documenter seems much more interesting/valuable to me. That way, Base could benefit by default.

[mortenpi]

I presume though that there is a small JS snippet that needs to run to configure the library and load the index? It is a fair point though that with Stork we probably wouldn't need a separate search page.

[Seelengrab]

A separate search page is still nice to have (makes it easier to scan a large number of results after all), but the good thing is that it looks like this can be fully customized. Stork allows for an arbitrary element to be used as the container for the search result, which is also registered, just like the <input> field.

As far as the small snippet goes, this is it, taken from the demo page linked above:

    <script src="https://files.stork-search.net/releases/v1.4.2/stork.js"></script>
    <script>
      stork.register(
        'federalist',
        'https://files.stork-search.net/releases/v1.4.2/federalist.st'
      )
    </script>

It literally just loads the library, looks for the tagged <input> and desired output container, and loads the precomputed index. That's what's powerful about this - stork can build the index in CI and is seemingly very open to what you want to do with it.

I haven't tried it yet (my day was busier than expected), but I think we can have the best of both worlds - navigating to a custom, full-on all-results search page, while also having the hover for quick results. I think it's just a matter of customizing some CSS for the search result page and making the <input> navigate to our full-result page (extracting the search parameter from the URL may require some JS, but it should be negligible). Since it's fast, I don't think it'll hurt to just restart the search on the full result page.

[mortenpi]

We went with minisearch in #2172, so I'll close this, since it's unlikely that we'll change the engine again in the near future unless a compelling case is made. That said, if anyone wants to prototype a Stork integration, it would always be interesting to see if it would work better than minisearch.