Outdated documentation on doc.rust-lang.org is exposed to search engines

[kornelski]

Is this the right place to report issues with doc.rust-lang.org?

---

https://doc.rust-lang.org/robots.txt blocks documentation pre-1.0 versions, but not outdated documentation from post 1.0.0.

This includes things like an unfinished style guide: https://doc.rust-lang.org/1.0.0/style

While it's great to have a full archive of documentation for all Rust versions, it causes problems for users using search engines. New users coming to Rust are sometimes directed by search engines to old documentation. From perspective of new users who have the latest Rust version, the old documentation is incomplete or even inaccurate.

I suggest blocking from search engines all frozen-in-time versions of the documentation. This could be achieved by adding

Disallow: /1.

rule to the robots.txt

[steveklabnik]

doc.rust-lang.org lives in the rust-lang/rust repo, under the docs folder. This is a duplicate of a bunch of issues over there. Closing!

[kornelski]

Thanks. I've found it: rust-lang/rust#44894