Add related content links to the Docs Site in the right rail

[jpvajda]

Adding dynamic content links using our related content component in the right rail under the page header links would allow us to drive users to OSS and Dev Sites and connect users to more discoverable content.

Considerations

• display logic what about really long right rail scenarios like this page. Would we want to NOT show the related content component if a page has a very long set of navigational headers in the right rail?

• cost What costs would we incur with swift type as costs would be driven up by all the docs pages needing to refresh with dynamic content links. (WIP)

• event tracking How can we measure if this is valuable to the end user? Perhaps track click events on the dynamic links to see if people use it? Or look at bounce rates on pages that have the dynamic links, It will likely increase bounce rates on those pages but increase traffic to our other sites. Measuring when users click on the related links would be very useful to understand when users feel they need to go to another site for content needs.

• Localization Impacts What about searching through a translated site like Japan? It's currently done through the proxy, so what is our approach now? - once the related content is available via a query, it shouldn't really impact swifttype cost as we'll have the related content already from the initial query.

Swift type impacts and approach

We refetch results per page once a day on a cron job. The data is available as part of the build: https://github.com/newrelic/developer-website/blob/develop/src/data/related-pages.json

That data gets rebuilt in this action once a day: https://github.com/newrelic/developer-website/blob/develop/.github/workflows/fetch-related-content.yml

We can specify whatever interval we want if we go this route. When we execute a query per page each time we fetch related content for a page. We have a limit on the number of queries we can run before we are charged more.

If we fetch once a day, and we have 3000 queries we have to run, thats 3k queries thats ~90k queries per month. We could decrease the interval and instead do once a week, or once a month.

Doing so would put costs down, but means that auto related content could become stale (maybe thats a fair tradeoff)

Right Rail Mockup

[jpvajda]

@djsauble any thoughts on this? It's come back up again. :)

[djsauble]

I still think it's a bad idea, but the right rail is flexible enough that adding a related content section should be easy.

[jpvajda]

@markweitzel confirmed with Mark Buchanan that this approach is welcome on the Doc Site, so we'll move forward into bringing the related content into the right rail. cc @austin-schaefer

[jpvajda]

Moving back to Proposed to estimate as part of catch-all estimate process.

[jpvajda]

Before implementation I think understand the potential cost increases of using Swiftype on the Docs Site to populate related content is important.

[jerelmiller]

As of right now, the docs site has 1,374 pages that we render the right rail on. That number will go up once we are 100% migrated (likely closer to 2,000), but this should give us a ballpark range of cost. Assuming we have 2,000 pages, this means we would execute 2,000 search queries to Swiftype every time we refetch the related pages.

I'm not sure which plan we are on, but pricing is as follows:

Basic
50k searches
$25/50k additional

Pro
100k searches
$25/50k additional

We refetch the dev site every day. Assuming we followed suit and updated the related pages every day, this equates to about 60k searches/month (assuming 2k searches * 30 days), just to populate the links. This is in addition to any searches users are making as well as the dev site related pages.

Hope this helps!

[jpvajda]

@jerelmiller thank you! In Oct it looks like we executed 193,210k and in Nov it looks like we executed 110,064k total searches across all site properties using Swiftype, so this means we'd be increasing our volume by around ~60k searches if we were planning on implementing the right rail and have it run daily page updates for relate links.

[jpvajda]

@austin-schaefer FYI on this, we'll want to discuss this potential cost increases with the Mark(s)

[jpvajda]

Do we know:

• Why was APM searched 25,742 a month in Nov and 29,571 in Oct?
• Why was Java searched 21,930 a month in Oct?

[zstix]

We've added the custom plugin that we created for the developer site to the theme. When we pick this work back up, we should update the developer site to use the plugin via the theme and then add it to the docs site.

[jpvajda]

@zstix do you want an issue created to do that?

[zstix]

I figured this ticket would be used to track that work, so I moved it back to proposed.

[jerelmiller]

@jpvajda How many related resources are we planning to add to a page? Are we sticking with 5?

We'll also need to figure out how best to utilize this.

[jerelmiller]

Here is a screenshot with some (dummy) resources for a page. This page only has 3 table of contents links, but you can see it already takes up almost the full height of the page. This will get worse for some of the larger pages.

[jerelmiller]

Here is a little screen recording of what it looks like on a large page. You can see it gets buried pretty quickly and leads to a double scroll:

large-toc.mov

[jerelmiller]

@djsauble not sure if you have any ideas or thoughts, but I'd love to hear them if you do :)

[jpvajda]

@jerelmiller raised some really great concerns here so we are discussing the best way to handle the experience of the right rail with related content.

[djsauble]

As I mentioned in Slack, I think the right approach here is to put "related resources" at the bottom of the content, like they are on the current docs site.

[jpvajda]

I scheduled a quick sync to align with key stakeholders on the approach to ensure we all agree.

[jerelmiller]

We've decided to go ahead and move forth with this using the following changes:

• Shorten up the "On this page" section to show 5-6 links
• Limit the "Related resources" to 3 links
• Move feedback above contributing guidelines (needs design help)
• Add a divider between the "On this page" and "Related resources" sections
• Try and tighten up the vertical spacing on the right rail

Quick mock: