Rollup merge of rust-lang#73566 - jyn514:name-resolve-first, r=eddyb

Don't run `everybody_loops` for rustdoc; instead ignore resolution errors

r? @eddyb
cc @petrochenkov, @GuillaumeGomez, @Manishearth, @ecstatic-morse, @marmeladema

~~Blocked on rust-lang#73743 Merged.
~~Blocked on crater run.~~ Crater popped up some ICEs ([now fixed](rust-lang#73566 (comment))). See [crater run](https://crater-reports.s3.amazonaws.com/pr-73566/index.html), [ICEs](rust-lang#73566 (comment)).
~~Blocked on rust-lang#74070 so that we don't make typeck_tables_of public when it shouldn't be.~~ Merged.

Closes rust-lang#71820, closes rust-lang#71104, closes rust-lang#65863.

## What is the motivation for this change?

As seen from a lengthy trail of PRs and issues (rust-lang#73532, rust-lang#73103, rust-lang#71820, rust-lang#71104), `everybody_loops` is causing bugs in rustdoc. The main issue is that it does not preserve the validity of the `DefId` tree, meaning that operations on DefIds may unexpectedly fail when called later. This is blocking intra-doc links (see rust-lang#73101).

This PR starts by removing `everybody_loops`, fixing rust-lang#71104 and rust-lang#71820. However, that brings back the bugs seen originally in rust-lang#43348: Since libstd documents items for all platforms, the function bodies sometimes do not type check. Here are the errors from documenting `libstd` with `everybody_loops` disabled and no other changes:

```rust
error[E0433]: failed to resolve: could not find `handle` in `sys`
  --> src/libstd/sys/windows/ext/process.rs:13:27
   |
13 |         let handle = sys::handle::Handle::new(handle as *mut _);
   |                           ^^^^^^ could not find `handle` in `sys`

error[E0425]: cannot find function `symlink_inner` in module `sys::fs`
   --> src/libstd/sys/windows/ext/fs.rs:544:14
    |
544 |     sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), false)
    |              ^^^^^^^^^^^^^ not found in `sys::fs`

error[E0425]: cannot find function `symlink_inner` in module `sys::fs`
   --> src/libstd/sys/windows/ext/fs.rs:564:14
    |
564 |     sys::fs::symlink_inner(src.as_ref(), dst.as_ref(), true)
    |              ^^^^^^^^^^^^^ not found in `sys::fs`
```

## Why does this need changes to `rustc_resolve`?

Normally, this could be avoided by simply not calling the `typeck_item_bodies` pass. However, the errors above happen before type checking, in name resolution itself. Since name resolution is intermingled with macro expansion, and rustdoc needs expansion to happen before it knows all items to be documented, there needs to be someway to ignore _resolution_ errors in function bodies.

An alternative solution suggested by @petrochenkov was to not run `everybody_loops` on anything containing a nested `DefId`. This would solve some of the immediate issues, but isn't bullet-proof: the following functions still could not be documented if the items in the body failed to resolve:

- Functions containing a nested `DefId` (rust-lang#71104)
- ~~Functions returning `impl Trait` (rust-lang#43878 These ended up not resolving anyway with this PR.
- ~~`const fn`, because `loop {}` in `const fn` is unstable (rust-lang#43636 `const_loop` was just stabilized.

This also isn't exactly what rustdoc wants, which is to avoid looking at function bodies in the first place.

## What changes were made?

The hack implemented in this PR is to add an option to ignore all resolution errors in function bodies. This is enabled only for rustdoc. Since resolution errors are ignored, the MIR generated will be invalid, as can be seen in the following ICE:

```rust
error: internal compiler error: broken MIR in DefId(0:11 ~ doc_cfg[8787]::uses_target_feature[0]) ("return type"): bad type [type error]
  --> /home/joshua/src/rust/src/test/rustdoc/doc-cfg.rs:51:1
   |
51 | / pub unsafe fn uses_target_feature() {
52 | |     content::should::be::irrelevant();
53 | | }
   | |_^
```

Fortunately, rustdoc does not need to access MIR in order to generate documentation. Therefore this also removes the call to `analyze()` in `rustdoc::run_core`. This has the side effect of not generating all lints by default. Most lints are safe to ignore (does rustdoc really need to run liveness analysis?) but `missing_docs` in particular is disabled when it should not be. Re-running `missing_docs` specifically does not help, because it causes the typechecking pass to be run, bringing back the errors from rust-lang#24658:

```
error[E0599]: no method named `into_handle` found for struct `sys::unix::pipe::AnonPipe` in the current scope
  --> src/libstd/sys/windows/ext/process.rs:71:27
   |
71 |         self.into_inner().into_handle().into_raw() as *mut _
   |                           ^^^^^^^^^^^ method not found in `sys::unix::pipe::AnonPipe`
   |
```

Because of rust-lang#73743, we only run typeck on demand. So this only causes an issue for functions returning `impl Trait`, which were already special cased by `ReplaceFunctionWithBody`. However, it now considers `async fn f() -> T` to be considered `impl Future<Output = T>`, where before it was considered to have a concrete `T` type.

## How will this affect future changes to rustdoc?

- Any new changes to rustdoc will not be able to perform type checking without bringing back resolution errors in function bodies.
    + As a corollary, any new lints cannot require or perform type checking. In some cases this may require refactoring other parts of the compiler to perform type-checking only on-demand, see for example rust-lang#73743.
    + As a corollary, rustdoc can never again call `tcx.analysis()` unless this PR is reverted altogether.

## Current status

- ~~I am not yet sure how to bring back `missing_docs` without running typeck. @eddyb suggested allowing lints to opt-out of type-checking, which would probably be another rabbit hole.~~ The opt-out was implemented in rust-lang#73743. However, of the rustc lints, now _only_ missing_docs is run and no other lints: rust-lang#73566 (comment). We need a team decision on whether that's an acceptable tradeoff. Note that all rustdoc lints are still run (`intra_doc_link_resolution_failure`, etc). **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment)
- ~~The implementation of optional errors in `rustc_resolve` is very brute force, it should probably be moved from `LateResolver` to `Resolver` to avoid duplicating the logic in many places.~~ I'm mostly happy with it now.

- This no longer allows errors in `async fn f() -> T`. This caused breakage in 50 crates out of a full crater run, all of which (that I looked at) didn't compile when run with rustc directly. In other words, it used to be that they could not be compiled but could still be documented; now they can't be documented either. This needs a decision from the rustdoc team on whether this is acceptable breakage. **UPDATE**: This was deemed acceptable in rust-lang#73566 (comment)
- ~~This makes `fn typeck_tables_of` in `rustc_typeck` public. This is not desired behavior, but needs the changes from rust-lang#74070 in order to be fixed.~~ Reverted.

src/librustc_interface/passes.rs

@@ -354,13 +354,7 @@ fn configure_and_expand_inner<'a>(
         )
     });
 
-    // If we're actually rustdoc then there's no need to actually compile
-    // anything, so switch everything to just looping
-    let mut should_loop = sess.opts.actually_rustdoc;
     if let Some(PpMode::PpmSource(PpSourceMode::PpmEveryBodyLoops)) = sess.opts.pretty {
-        should_loop |= true;
-    }
-    if should_loop {
         log::debug!("replacing bodies with loop {{}}");
         util::ReplaceBodyWithLoop::new(&mut resolver).visit_crate(&mut krate);
     }

src/librustc_resolve/late.rs

@@ -394,13 +394,23 @@ struct LateResolutionVisitor<'a, 'b, 'ast> {
 
     /// Fields used to add information to diagnostic errors.
     diagnostic_metadata: DiagnosticMetadata<'ast>,
+
+    /// State used to know whether to ignore resolution errors for function bodies.
+    ///
+    /// In particular, rustdoc uses this to avoid giving errors for `cfg()` items.
+    /// In most cases this will be `None`, in which case errors will always be reported.
+    /// If it is `Some(_)`, then it will be updated when entering a nested function or trait body.
+    in_func_body: bool,
 }
 
 /// Walks the whole crate in DFS order, visiting each item, resolving names as it goes.
 impl<'a, 'ast> Visitor<'ast> for LateResolutionVisitor<'a, '_, 'ast> {
     fn visit_item(&mut self, item: &'ast Item) {
         let prev = replace(&mut self.diagnostic_metadata.current_item, Some(item));
+        // Always report errors in items we just entered.
+        let old_ignore = replace(&mut self.in_func_body, false);
         self.resolve_item(item);
+        self.in_func_body = old_ignore;
         self.diagnostic_metadata.current_item = prev;
     }
     fn visit_arm(&mut self, arm: &'ast Arm) {
@@ -497,13 +507,17 @@ impl<'a, 'ast> Visitor<'ast> for LateResolutionVisitor<'a, '_, 'ast> {
 
                 visit::walk_fn_ret_ty(this, &declaration.output);
 
+                // Ignore errors in function bodies if this is rustdoc
+                // Be sure not to set this until the function signature has been resolved.
+                let previous_state = replace(&mut this.in_func_body, true);
                 // Resolve the function body, potentially inside the body of an async closure
                 match fn_kind {
                     FnKind::Fn(.., body) => walk_list!(this, visit_block, body),
                     FnKind::Closure(_, body) => this.visit_expr(body),
                 };
 
                 debug!("(resolving function) leaving function");
+                this.in_func_body = previous_state;
             })
         });
         self.diagnostic_metadata.current_function = previous_value;
@@ -644,6 +658,8 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
             label_ribs: Vec::new(),
             current_trait_ref: None,
             diagnostic_metadata: DiagnosticMetadata::default(),
+            // errors at module scope should always be reported
+            in_func_body: false,
         }
     }
 
@@ -757,7 +773,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
                 return if self.is_label_valid_from_rib(i) {
                     Some(*id)
                 } else {
-                    self.r.report_error(
+                    self.report_error(
                         original_span,
                         ResolutionError::UnreachableLabel {
                             name: label.name,
@@ -775,7 +791,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
             suggestion = suggestion.or_else(|| self.suggestion_for_label_in_rib(i, label));
         }
 
-        self.r.report_error(
+        self.report_error(
             original_span,
             ResolutionError::UndeclaredLabel { name: label.name, suggestion },
         );
@@ -833,7 +849,11 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
             };
             let report_error = |this: &Self, ns| {
                 let what = if ns == TypeNS { "type parameters" } else { "local variables" };
-                this.r.session.span_err(ident.span, &format!("imports cannot refer to {}", what));
+                if this.should_report_errs() {
+                    this.r
+                        .session
+                        .span_err(ident.span, &format!("imports cannot refer to {}", what));
+                }
             };
 
             for &ns in nss {
@@ -1008,7 +1028,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
             if seen_bindings.contains_key(&ident) {
                 let span = seen_bindings.get(&ident).unwrap();
                 let err = ResolutionError::NameAlreadyUsedInParameterList(ident.name, *span);
-                self.r.report_error(param.ident.span, err);
+                self.report_error(param.ident.span, err);
             }
             seen_bindings.entry(ident).or_insert(param.ident.span);
 
@@ -1274,7 +1294,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
                 .is_err()
             {
                 let path = &self.current_trait_ref.as_ref().unwrap().1.path;
-                self.r.report_error(span, err(ident.name, &path_names_to_string(path)));
+                self.report_error(span, err(ident.name, &path_names_to_string(path)));
             }
         }
     }
@@ -1289,6 +1309,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
     }
 
     fn resolve_local(&mut self, local: &'ast Local) {
+        debug!("resolving local ({:?})", local);
         // Resolve the type.
         walk_list!(self, visit_ty, &local.ty);
 
@@ -1390,7 +1411,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
             if inconsistent_vars.contains_key(name) {
                 v.could_be_path = false;
             }
-            self.r.report_error(
+            self.report_error(
                 *v.origin.iter().next().unwrap(),
                 ResolutionError::VariableNotBoundInPattern(v),
             );
@@ -1400,7 +1421,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
         let mut inconsistent_vars = inconsistent_vars.iter().collect::<Vec<_>>();
         inconsistent_vars.sort();
         for (name, v) in inconsistent_vars {
-            self.r.report_error(v.0, ResolutionError::VariableBoundWithDifferentMode(*name, v.1));
+            self.report_error(v.0, ResolutionError::VariableBoundWithDifferentMode(*name, v.1));
         }
 
         // 5) Finally bubble up all the binding maps.
@@ -1550,7 +1571,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
                 // `Variant(a, a)`:
                 _ => IdentifierBoundMoreThanOnceInSamePattern,
             };
-            self.r.report_error(ident.span, error(ident.name));
+            self.report_error(ident.span, error(ident.name));
         }
 
         // Record as bound if it's valid:
@@ -1624,7 +1645,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
                 // to something unusable as a pattern (e.g., constructor function),
                 // but we still conservatively report an error, see
                 // issues/33118#issuecomment-233962221 for one reason why.
-                self.r.report_error(
+                self.report_error(
                     ident.span,
                     ResolutionError::BindingShadowsSomethingUnacceptable(
                         pat_src.descr(),
@@ -1677,18 +1698,27 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
         source: PathSource<'ast>,
         crate_lint: CrateLint,
     ) -> PartialRes {
+        log::debug!("smart_resolve_path_fragment(id={:?},qself={:?},path={:?}", id, qself, path);
         let ns = source.namespace();
         let is_expected = &|res| source.is_expected(res);
 
         let report_errors = |this: &mut Self, res: Option<Res>| {
-            let (err, candidates) = this.smart_resolve_report_errors(path, span, source, res);
-
-            let def_id = this.parent_scope.module.normal_ancestor_id;
-            let instead = res.is_some();
-            let suggestion =
-                if res.is_none() { this.report_missing_type_error(path) } else { None };
-
-            this.r.use_injections.push(UseError { err, candidates, def_id, instead, suggestion });
+            if this.should_report_errs() {
+                let (err, candidates) = this.smart_resolve_report_errors(path, span, source, res);
+
+                let def_id = this.parent_scope.module.normal_ancestor_id;
+                let instead = res.is_some();
+                let suggestion =
+                    if res.is_none() { this.report_missing_type_error(path) } else { None };
+
+                this.r.use_injections.push(UseError {
+                    err,
+                    candidates,
+                    def_id,
+                    instead,
+                    suggestion,
+                });
+            }
 
             PartialRes::new(Res::Err)
         };
@@ -1746,13 +1776,17 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
 
             let def_id = this.parent_scope.module.normal_ancestor_id;
 
-            this.r.use_injections.push(UseError {
-                err,
-                candidates,
-                def_id,
-                instead: false,
-                suggestion: None,
-            });
+            if this.should_report_errs() {
+                this.r.use_injections.push(UseError {
+                    err,
+                    candidates,
+                    def_id,
+                    instead: false,
+                    suggestion: None,
+                });
+            } else {
+                err.cancel();
+            }
 
             // We don't return `Some(parent_err)` here, because the error will
             // be already printed as part of the `use` injections
@@ -1809,7 +1843,7 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
 
             Err(err) => {
                 if let Some(err) = report_errors_for_call(self, err) {
-                    self.r.report_error(err.span, err.node);
+                    self.report_error(err.span, err.node);
                 }
 
                 PartialRes::new(Res::Err)
@@ -1843,6 +1877,21 @@ impl<'a, 'b, 'ast> LateResolutionVisitor<'a, 'b, 'ast> {
         if let Some(LexicalScopeBinding::Res(res)) = binding { res != Res::Err } else { false }
     }
 
+    /// A wrapper around [`Resolver::report_error`].
+    ///
+    /// This doesn't emit errors for function bodies if this is rustdoc.
+    fn report_error(&self, span: Span, resolution_error: ResolutionError<'_>) {
+        if self.should_report_errs() {
+            self.r.report_error(span, resolution_error);
+        }
+    }
+
+    #[inline]
+    /// If we're actually rustdoc then avoid giving a name resolution error for `cfg()` items.
+    fn should_report_errs(&self) -> bool {
+        !(self.r.session.opts.actually_rustdoc && self.in_func_body)
+    }
+
     // Resolve in alternative namespaces if resolution in the primary namespace fails.
     fn resolve_qpath_anywhere(
         &mut self,

src/librustdoc/core.rs

@@ -5,10 +5,15 @@ use rustc_driver::abort_on_err;
 use rustc_errors::emitter::{Emitter, EmitterWriter};
 use rustc_errors::json::JsonEmitter;
 use rustc_feature::UnstableFeatures;
-use rustc_hir::def::Namespace::TypeNS;
+use rustc_hir::def::{Namespace::TypeNS, Res};
 use rustc_hir::def_id::{CrateNum, DefId, DefIndex, LocalDefId, CRATE_DEF_INDEX, LOCAL_CRATE};
 use rustc_hir::HirId;
+use rustc_hir::{
+    intravisit::{self, NestedVisitorMap, Visitor},
+    Path,
+};
 use rustc_interface::interface;
+use rustc_middle::hir::map::Map;
 use rustc_middle::middle::cstore::CrateStore;
 use rustc_middle::middle::privacy::AccessLevels;
 use rustc_middle::ty::{Ty, TyCtxt};
@@ -372,7 +377,35 @@ pub fn run_core(options: RustdocOptions) -> (clean::Crate, RenderInfo, RenderOpt
         crate_name,
         lint_caps,
         register_lints: None,
-        override_queries: None,
+        override_queries: Some(|_sess, providers, _external_providers| {
+            // Most lints will require typechecking, so just don't run them.
+            providers.lint_mod = |_, _| {};
+            // Prevent `rustc_typeck::check_crate` from calling `typeck_tables_of` on all bodies.
+            providers.typeck_item_bodies = |_, _| {};
+            // hack so that `used_trait_imports` won't try to call typeck_tables_of
+            providers.used_trait_imports = |_, _| {
+                lazy_static! {
+                    static ref EMPTY_SET: FxHashSet<LocalDefId> = FxHashSet::default();
+                }
+                &EMPTY_SET
+            };
+            // In case typeck does end up being called, don't ICE in case there were name resolution errors
+            providers.typeck_tables_of = move |tcx, def_id| {
+                // Closures' tables come from their outermost function,
+                // as they are part of the same "inference environment".
+                // This avoids emitting errors for the parent twice (see similar code in `typeck_tables_of_with_fallback`)
+                let outer_def_id = tcx.closure_base_def_id(def_id.to_def_id()).expect_local();
+                if outer_def_id != def_id {
+                    return tcx.typeck_tables_of(outer_def_id);
+                }
+
+                let hir = tcx.hir();
+                let body = hir.body(hir.body_owned_by(hir.as_local_hir_id(def_id)));
+                debug!("visiting body for {:?}", def_id);
+                EmitIgnoredResolutionErrors::new(tcx).visit_body(body);
+                (rustc_interface::DEFAULT_QUERY_PROVIDERS.typeck_tables_of)(tcx, def_id)
+            };
+        }),
         registry: rustc_driver::diagnostics_registry(),
     };
 
@@ -416,10 +449,17 @@ pub fn run_core(options: RustdocOptions) -> (clean::Crate, RenderInfo, RenderOpt
             let mut global_ctxt = abort_on_err(queries.global_ctxt(), sess).take();
 
             global_ctxt.enter(|tcx| {
-                tcx.analysis(LOCAL_CRATE).ok();
-
-                // Abort if there were any errors so far
-                sess.abort_if_errors();
+                // Certain queries assume that some checks were run elsewhere
+                // (see https://github.com/rust-lang/rust/pull/73566#issuecomment-656954425),
+                // so type-check everything other than function bodies in this crate before running lints.
+                // NOTE: this does not call `tcx.analysis()` so that we won't
+                // typeck function bodies or run the default rustc lints.
+                // (see `override_queries` in the `config`)
+                let _ = rustc_typeck::check_crate(tcx);
+                tcx.sess.abort_if_errors();
+                sess.time("missing_docs", || {
+                    rustc_lint::check_crate(tcx, rustc_lint::builtin::MissingDoc::new);
+                });
 
                 let access_levels = tcx.privacy_access_levels(LOCAL_CRATE);
                 // Convert from a HirId set to a DefId set since we don't always have easy access
@@ -570,6 +610,62 @@ pub fn run_core(options: RustdocOptions) -> (clean::Crate, RenderInfo, RenderOpt
     })
 }
 
+/// Due to https://github.com/rust-lang/rust/pull/73566,
+/// the name resolution pass may find errors that are never emitted.
+/// If typeck is called after this happens, then we'll get an ICE:
+/// 'Res::Error found but not reported'. To avoid this, emit the errors now.
+struct EmitIgnoredResolutionErrors<'tcx> {
+    tcx: TyCtxt<'tcx>,
+}
+
+impl<'tcx> EmitIgnoredResolutionErrors<'tcx> {
+    fn new(tcx: TyCtxt<'tcx>) -> Self {
+        Self { tcx }
+    }
+}
+
+impl<'tcx> Visitor<'tcx> for EmitIgnoredResolutionErrors<'tcx> {
+    type Map = Map<'tcx>;
+
+    fn nested_visit_map(&mut self) -> NestedVisitorMap<Self::Map> {
+        // We need to recurse into nested closures,
+        // since those will fallback to the parent for type checking.
+        NestedVisitorMap::OnlyBodies(self.tcx.hir())
+    }
+
+    fn visit_path(&mut self, path: &'tcx Path<'_>, _id: HirId) {
+        debug!("visiting path {:?}", path);
+        if path.res == Res::Err {
+            // We have less context here than in rustc_resolve,
+            // so we can only emit the name and span.
+            // However we can give a hint that rustc_resolve will have more info.
+            let label = format!(
+                "could not resolve path `{}`",
+                path.segments
+                    .iter()
+                    .map(|segment| segment.ident.as_str().to_string())
+                    .collect::<Vec<_>>()
+                    .join("::")
+            );
+            let mut err = rustc_errors::struct_span_err!(
+                self.tcx.sess,
+                path.span,
+                E0433,
+                "failed to resolve: {}",
+                label
+            );
+            err.span_label(path.span, label);
+            err.note("this error was originally ignored because you are running `rustdoc`");
+            err.note("try running again with `rustc` or `cargo check` and you may get a more detailed error");
+            err.emit();
+        }
+        // We could have an outer resolution that succeeded,
+        // but with generic parameters that failed.
+        // Recurse into the segments so we catch those too.
+        intravisit::walk_path(self, path);
+    }
+}
+
 /// `DefId` or parameter index (`ty::ParamTy.index`) of a synthetic type parameter
 /// for `impl Trait` in argument position.
 #[derive(Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]