WSL1 does not support `fcntl`-based lock and will always report success,
therefore creating a race condition when multiple rustc processes are
modifying shared data such as `search-index.js`. WSL1 does however
support `flock`.

`flock` is supported by all unix-like platforms. The only caveat is that
Linux 2.6.11 or earlier does not support `flock` on NFS mounts, but as
the minimum supported Linux version is 2.6.18, it is not an issue.

Fixes rust-lang#72157

This commit updates the code generation for `#![no_builtins]` to always
produce object files instead of conditionally respecting
`-Clinker-plugin-lto` and sometimes producing bitcode. This is intended
to address rust-lang/cargo#8239.

The issue at hand here is that Cargo has tried to get "smarter" about
codegen in whole crate graph scenarios. When LTO is enabled it attempts
to avoid codegen on as many crates as possible, opting to pass
`-Clinker-plugin-lto` where it can to only generate bitcode. When this
is combined with `-Zbuild-std`, however, it means that
`compiler-builtins` only generates LLVM bitcode instead of object files.
Rustc's own LTO passes then explicitly skip `compiler-builtins` (because
it wouldn't work anyway) which means that LLVM bitcode gets sent to the
linker, which chokes most of the time.

The fix in this PR is to not actually respect `-Clinker-plugin-lto` for
`#![no_builtins]` crates. These crates, even if slurped up by the linker
rather than rustc, will not work with LTO. They define symbols which are
only referenced as part of codegen, so LTO's aggressive internalization
would trivially remove the symbols only to have the linker realize later
that the symbol is undefined. Since pure-bitcode never makes sense for
these libraries, the `-Clinker-plugin-lto` flag is silently ignored.

Fixes rust-lang#68489

When checking two `TokenStreams` to see if they are 'probably equal',
we ignore the `IsJoint` information associated with each `TokenTree`.
However, the `IsJoint` information determines whether adjacent tokens
will be 'glued' (if possible) when construction the `TokenStream` - e.g.
`[Gt Gt]` can be 'glued' to `BinOp(Shr)`.

Since we are ignoring the `IsJoint` information, 'glued' and 'unglued'
tokens are equivalent for determining if two `TokenStreams` are
'probably equal'. Therefore, we need to 'unglue' all tokens in the
stream to avoid false negatives (which cause us to throw out the cached
tokens, losing span information).

Some tokens need to be broken in a loop until we reach
'unbreakable' tokens.

…ge, r=RalfJung

InvalidUndefBytes: Track size of undef region used

This PR adds a size to `UndefinedBehaviorInfo::InvalidUndefBytes`, to keep track of how many undefined bytes in a row were accessed, and changes a few methods to pass this information through. This additional information will eventually be used in Miri to improve diagnostics for this UB error. See also rust-lang/miri#1354 for prior discussion.

I expect Miri will break the next time its submodule is updated, due to this change to the `InvalidUndefBytes`. (The current commit for src/tools/miri predates rust-lang/miri#1354, and thus does not try to destructure the `InvalidUndefBytes` variant) I have a corresponding change prepared for that repository.

r? @RalfJung

Replace fcntl-based file lock with flock

WSL1 does not support `fcntl`-based lock and will always report success,
therefore creating a race condition when multiple rustc processes are
modifying shared data such as `search-index.js`. WSL1 does however
support `flock`.

`flock` is supported by all unix-like platforms. The only caveat is that
Linux 2.6.11 or earlier does not support `flock` on NFS mounts, but as
the minimum supported Linux version is 2.6.18, it is not an issue.

Fixes rust-lang#72157

…petrochenkov

Break tokens before checking if they are 'probably equal'

Fixes rust-lang#68489
Fixes rust-lang#70987

When checking two `TokenStreams` to see if they are 'probably equal',
we ignore the `IsJoint` information associated with each `TokenTree`.
However, the `IsJoint` information determines whether adjacent tokens
will be 'glued' (if possible) when construction the `TokenStream` - e.g.
`[Gt Gt]` can be 'glued' to `BinOp(Shr)`.

Since we are ignoring the `IsJoint` information, 'glued' and 'unglued'
tokens are equivalent for determining if two `TokenStreams` are
'probably equal'. Therefore, we need to 'unglue' all tokens in the
stream to avoid false negatives (which cause us to throw out the cached
tokens, losing span information).

…to, r=nnethercote

Always generated object code for `#![no_builtins]`

This commit updates the code generation for `#![no_builtins]` to always
produce object files instead of conditionally respecting
`-Clinker-plugin-lto` and sometimes producing bitcode. This is intended
to address rust-lang/cargo#8239.

The issue at hand here is that Cargo has tried to get "smarter" about
codegen in whole crate graph scenarios. When LTO is enabled it attempts
to avoid codegen on as many crates as possible, opting to pass
`-Clinker-plugin-lto` where it can to only generate bitcode. When this
is combined with `-Zbuild-std`, however, it means that
`compiler-builtins` only generates LLVM bitcode instead of object files.
Rustc's own LTO passes then explicitly skip `compiler-builtins` (because
it wouldn't work anyway) which means that LLVM bitcode gets sent to the
linker, which chokes most of the time.

The fix in this PR is to not actually respect `-Clinker-plugin-lto` for
`#![no_builtins]` crates. These crates, even if slurped up by the linker
rather than rustc, will not work with LTO. They define symbols which are
only referenced as part of codegen, so LTO's aggressive internalization
would trivially remove the symbols only to have the linker realize later
that the symbol is undefined. Since pure-bitcode never makes sense for
these libraries, the `-Clinker-plugin-lto` flag is silently ignored.