rustdoc referenced search.js and settings.js not via static-root-path

[syphar]

• there was a bug in rustdoc where a toolchain specific static asset (search.js) wasn't referenced via static-root-path, but via the path for release (invocation) specific assets. ( introduced in rustdoc: Consolidate static-file replacement mechanism rust#91062 ).
• this was fixed with rustdoc: add hash to filename of toolchain files rust#101702, so all docs built after that are correct.
• our LegacySharedResourcesHandler which was a hack for Docs built during 2018-{10-12} link to the wrong url for static resources #1181 also worked for this bug, so this was never visible to users.
• after the rebuild for Docs built during 2018-{10-12} link to the wrong url for static resources #1181 was finished we dropped the LegacySharedResourcesHandler with the last piece of the axum migration (migrate the main rustdoc handlers to axum, drop iron #1963).
• we added a new workaround in refs #1979: add workaround for toolchain JS files that were referenced wrong #1980 which only handles search.js, and only at the specific location it might be requested.
• we added another workaround in refs #1979, #1992: workaround for settings.js not served from the root #1993 to also handle settings.js, same as to refs #1979: add workaround for toolchain JS files that were referenced wrong #1980
• to remove the workaround, we need to rebuild the affected releases.

initial issue

• This was initially reported as "broken search" after migrate the main rustdoc handlers to axum, drop iron #1963 was merged.
• broken settings were reported in Broken settings on crossbeam-0.8.2 #1992

reported broken releases:

• search: https://docs.rs/mipidsi/latest/mipidsi/index.html?search=init (released 2022-10-19)
• search: https://docs.rs/js-sys/latest/js_sys/index.html?search=Array (released 2022-09-12)
• search: https://docs.rs/diesel/latest/diesel/?search=tosql (released 2022-10-11)
• settings: https://docs.rs/crossbeam/0.8.2/crossbeam/channel/index.html (released 2022-07-22)

reported via:

• https://www.reddit.com/r/rust/comments/zx1j8a/is_search_broken_on_docsrs/
• https://rust-lang.zulipchat.com/#narrow/stream/356853-t-docs-rs/topic/search.20broken.20for.20some.20crates.3F

[syphar]

interestingly I found that

• this release is broken: https://docs.rs/regex/1.7.0/regex/index.html?search=test (released 2022-11-05)
• while accessing it via /latest/ works: https://docs.rs/regex/latest/regex/index.html?search=test
  Probably

the latter seems to be served from the cache

[syphar]

there are also working releases, via version and /latest/:

• https://docs.rs/itertools/0.10.5/itertools/index.html?search=chain
• https://docs.rs/itertools/latest/itertools/index.html?search=chain
  ( released 2022-09-20)

[syphar]

the JS file that gets requested:
https://docs.rs/regex/1.7.0/search-20221104-1.67.0-nightly-09508489e.js

is actually not invocation specific, but toolchain specific. I can also find it in our bucket in the root directory.

It is still requested from the release specific path, and not from the given static-root-path.

[syphar]

This feels like a bug in rustdoc, while our LegacySharedResourcesHandler previously hid this issue because it checked all static file requests with the bucket root

[syphar]

as a workaround I'll probably add back checking the toolchain specific files for this case, pending a fix in rustdoc (and then potentially a rebuild?)

[syphar]

To sum up the discussion on zulip ( topic link in issue description):

• there was a bug in rustdoc where a toolchain specific static asset wasn't referenced via static-root-path, but via the path for release (invocation) specific assets. ( rustdoc: Consolidate static-file replacement mechanism rust#91062 ).
• this was fixed in rustdoc: add hash to filename of toolchain files rust#101702, so all docs built after that are correct.
• our LegacySharedResourcesHandler which was a hack for Docs built during 2018-{10-12} link to the wrong url for static resources #1181 also worked for this bug, so this was never visible to users.
• after the rebuild for Docs built during 2018-{10-12} link to the wrong url for static resources #1181 was finished I dropped the LegacySharedResourcesHandler with the last piece of the axum migration (migrate the main rustdoc handlers to axum, drop iron #1963).

[syphar]

The fix (= hack) for this is to partially re-introduce this behaviour, pending another rebuild of the broken docs.

[syphar]

I believe a broken settings.js reference is probably coming from the same rustdoc issue, see #1992

[syphar]

Some numbers:

• rustdoc: Consolidate static-file replacement mechanism rust#91062 was merged on 2021-11-27
• rustdoc: add hash to filename of toolchain files rust#101702 was merged on 2022-11-05

( just guessing the nightly versions):

cratesfyi=> select min(id) from releases where doc_rustc_version like '%2021-11-27%';
  min
--------
 453093
(1 row)

cratesfyi=> select max(id) from releases where doc_rustc_version like '%2022-11-07%';
  max
--------
 650838
(1 row)

cratesfyi=> select count(*) from releases where id >= 453093 and id <= 650838;
 count
--------
 195720
(1 row)

cratesfyi=> select count(*) from releases inner join crates on releases.id = crates.latest_version_id where releases.id >= 453093 and releases.id <= 650838;
 count
-------
 24056
(1 row)

Which means we would have to rebuild 195720 releases to rebuild if we want to fully fix the issue, and remove the fallback static file handler.
Alternatively 24056 releases to rebuild if we only care about latest versions.