Add search feature

[mattstratton]

I thought we had an issue for this already, but we don't.

It's not terribly difficult, and there are several cases where it might be helpful.

Here's a helpful howto! https://blog.jeremylikness.com/blog/dynamic-search-in-a-static-hugo-website/

[mattstratton]

I just did this for the ADO website, and it wasn't too hard.

Here's the PR for comparison arresteddevops/ado-hugo#474

[mattstratton]

Huh. This won't actually work out - when we move stuff to static it will be removed from the index, so the search will only work for relatively current content. Hmm.

[mattstratton]

For reference, the key is creating this index.json page, which gets done with code like this:

{{- $.Scratch.Add "index" slice -}}

{{- range (where $.Site.Pages "Type" "episode") -}}
    {{- $.Scratch.Add "index" (dict "title" .Title "subtitle" .Params.subtitle "description" .Params.description "image" ( printf "/%s" .Params.episode_image ) "content" .Plain "permalink" .Permalink) -}}
{{- end -}}

{{- range (where $.Site.Pages "Type" "guest") -}}
    {{- $.Scratch.Add "index" (dict "title" .Title "subtitle" .Params.subtitle "description" .Content "image" ( printf "/%s" .Params.Thumbnail ) "content" .Plain "permalink" .Permalink) -}}
{{- end -}}
{{- $.Scratch.Get "index" | jsonify -}}

so as you can see it needs to range over Pages. Which get lost when we migrate the generated pages of older events.

[mattstratton]

We might be able to do it with google - this post is how Jeremy did his search prior to moving it into native Hugo, and it might address some of the concerns I have with using google search:

https://blog.jeremylikness.com/blog/more-hugo-migration-tips/

[mattstratton]

actually that doesn't help; he uses AdSense with the search, which isn't what we want, of course!

[mattstratton]

For some thoughts on this - I had set up a Google CSE for devopsdays a while ago (apparently!) and you can see what the search experience is like here (just look at the search box and then see the results; don't worry about page formatting):

https://cse.google.com/cse?cx=008769174023796316711:pj3syggtkz0

It embeds google ads at the top :(

[mattstratton]

we can do it with duck duck go, which is a better privacy solution, but it's not as pretty: https://duckduckgo.com/search_box

basically, we can add a search box, but they are taken off-site to get the results.

[mattstratton]

the google custom search linked above also includes results from "legacy" which is nice.

[bridgetkromhout]

Finding repeat speakers over events seems like the biggest win? I'm not sure we need to know there are x thousand uses of the word "microservices". :)

[mattstratton]

yeah, the good part is that if we used our own index, it would be much easier to weight the results, exclude stuff, etc, whereas the google one depends on the searcher to have the right google skillz

[mattstratton]

I'm wondering if this might help - it's a function on the netlify side which would generate the search index. If we do this, we would want to use the serverless function though, because the index would be way too huge to download to the client I suspect.

https://github.com/sw-yx/netlify-plugin-search-index#readme

[mattstratton]

It might be even better to use this plugin and try the community version of Algolia

https://github.com/lukeocodes/netlify-plugin-algolia-index#readme

[mattstratton]

I did a little test with Algolia; I don't think it's ideal for us. The free plan has a limit of 50K operations/month; updating the index used 10K operations. So that's not great, even if we do an atomic update as part of the prod build (only updating the changed records). If nothing else, even just testing will blow out our usage of it.

Generating the index with the plugin adds 2 minutes to the build. We would limit that plugin only to prod builds though. Might want to see if the serverless function approach works better.

[mattstratton]

Building the serverless function which we can then query seems to be the right move! It's not too bad. It appears that there is a bug in the function generation in the plugin, so we need to wait until we see updates for swyxio/netlify-plugin-search-index#19 before moving forward.

This is the branch where work is being done on the devopsdays search side - fix-8663