Rollup merge of #59930 - kornelski:first-robots, r=GuillaumeGomez

Exclude some copies of old book editions from search engines These are only stubs that confuse search engine users. There's no useful content in these locations.
rust-lang · Apr 13, 2019 · cf370b4 · cf370b4
2 parents 2413ac7 + 0a9b214
commit cf370b4
Showing 1 changed file with 8 additions and 0 deletions.
diff --git a/src/doc/robots.txt b/src/doc/robots.txt
@@ -17,3 +17,11 @@ Disallow: /1.0.0-beta.2/
 Disallow: /1.0.0-beta.3/
 Disallow: /1.0.0-beta.4/
 Disallow: /1.0.0-beta.5/
+Disallow: /book/first-edition/
+Disallow: /book/second-edition/
+Disallow: /stable/book/first-edition/
+Disallow: /stable/book/second-edition/
+Disallow: /beta/book/first-edition/
+Disallow: /beta/book/second-edition/
+Disallow: /nightly/book/first-edition/
+Disallow: /nightly/book/second-edition/