Use lld by default on x64 msvc windows

[Gankra]

This is a metabug, constraining the unbound scope of #39915.

What is lld

A linker that's part of the llvm project, which is desirable for two reasons:

• it's very friendly to cross compilation (hence its emphasis for embedded targets)
• it's very fast (often runs in half the time as Gold -- linking can take several minutes for big projects (rustc, servo, etc.) and linking can be a huge % of the compile with incremental builds, so halving this runtime is a Big Deal.)

Rust currently ships its own copy of lld which it calls rust-lld. This is used by default to link bare-metal targets and wasm.

Goal

The goal of this metabug is to use rust-lld by default when targeting x64 msvc windows (as opposed to e.g. mingw windows). It's possible this will incidentally get other windows targets working, but I'm constraining scope for the sake of focusing efforts.

This configuration is already in a decent state, with backend support already implemented. It's just a matter of making that configuration stable enough to use by default.

To test using rust-lld for msvc targets, use -C linker=lld

Blocking Issues

lld on windows status page

• Linking libtest with lld-link results in undefined symbols #68647 - linking libtest results in undefined symbols
• rust-analyser segfault with lto=thin, linker=lld-link #71504 - rust-analyser segfault with lto=thin, linker=lld-link
• Incorrect compilation / STATUS_ACCESS_VIOLATION when linking with lld with target-cpu set #72145 - LLD/COFF does not correctly align TLS section
• ACCESS_VIOLATION when dereferencing once_cell::Lazy in closure with LTO #81408 - ACCESS_VIOLATION when dereferencing once_cell::Lazy in closure with LTO
• Windows rust-lld does not support the MSVC manifestdependency .drectve #85642 - Windows rust-lld does not support the MSVC manifestdependency .drectve
• STATUS_ACCESS_VIOLATION when running rend3 on Windows MSVC linked with LLD and thin LTO #111480

[luser]

This ought to be a bit more straightforward than using lld with gcc since with MSVC you do conventionally invoke the linker directly, and lld has an lld-link.exe binary that tries to be a drop-in replacement for link.exe. You can also run lld -flavor link and pass MSVC-style commandline arguments after that. I would expect that if you set up the environment correctly you ought to be able to simply invoke lld-link.exe instead of link.exe and have it Just Work.

Note that you still need a Visual C++ install (they have a smaller "Build Tools for Visual Studio" install that doesn't include the IDE) or maybe just a Windows SDK in order to have the system .lib files it wants to link to.

[Gankra]

Yes, as noted in the main comment (which I update to track current status) the infra is already hooked up here for invoking it. It's just a bit buggy, and requires a more thorough investigation of failure-modes and how we can smooth them over, e.g.:

• if you don't have Visual C++ installed what happens?
• do we provide lld-link.exe, or do we look for a system one? both? is
• does it work just as well on win 8 and 7? should we restrict efforts to win 10?
• can we directly invoke the lld-link binary on macos/linux when cross-compiling to windows?
• how important are the missing features on the lld-link status page? can we recover and fallback to link when encountering them?

Once someone's done a quick audit of basic failure modes, we should do some community outreach asking people to test -C linker=lld and report how it affects their build times (clean build, big change incremental, tiny change incremental, etc.).

Definitely looking for someone to champion this effort, since my primary goal was just creating more focused initiatives / clarifying the status confusion in the main lld bug.

[luser]

Ah, apologies, I did read that comment but must have missed that.

if you don't have Visual C++ installed what happens?

This is a good question but I think the better question is "if you try to use lld-link but don't have Visual C++ installed is it any worse than what happens in the default case without Visual C++ installed?" Currently rustc does try to locate MSVC but will apparently just try its best if it can't find it:

rust/src/librustc_codegen_ssa/back/link.rs

Line 164 in f8d394e

Command::new(msvc_tool.as_ref().map(|t| t.path()).unwrap_or(linker))

[mati865]

if you don't have Visual C++ installed what happens?

IIRC lld-link errors listing all Windows SDK libraries it cannot find.

can we directly invoke the lld-link binary on macos/linux when cross-compiling to windows?

You cannot cross-compile to MSVC from other OSes unless you find a way to extract Windows SDK and feed it to the LLD. AFAIK MinGW with ld.lld is the only somewhat supported way to cross compile for Windows.

[luser]

Cross-compiling to Windows is not much different than any other OS these days now that LLVM has good support for generating Windows binaries. You do need all the import libs from MSVC at the very least, and probably a Windows SDK for anything non-trivial, but you can absolutely do it:
https://github.com/ProdriveTechnologies/clang-msvc-sdk

Given that requirement it's probably not worth really worrying about as long as it doesn't error in a way that's any more confusing than trying to cross-compile to Windows without specifying an alternate linker. That fails like this for me on Linux with Rust 1.42.0:

luser@eye7:/build/snippet$ cargo build --target=x86_64-pc-windows-msvc
   Compiling itoa v0.3.4
   Compiling snippet v0.1.4-alpha.0 (/build/snippet)
error: linker `link.exe` not found
  |
  = note: No such file or directory (os error 2)

note: the msvc targets depend on the msvc linker but `link.exe` was not found

note: please ensure that VS 2013, VS 2015, VS 2017 or VS 2019 was installed with the Visual C++ option

error: aborting due to previous error

error: could not compile `snippet`.

To learn more, run the command again with --verbose.

Once #58713 is fixed it is probably worth revisiting that topic since it'll become feasible to write pure-Rust descriptions of Windows import libraries and generate working binaries without having MSVC or a Windows SDK.

[luser]

do we provide lld-link.exe, or do we look for a system one? both?

If Rust is already shipping an lld binary it makes sense to allow using it, but given the myriad ways people build software there will assuredly be a desire to use specific lld/lld-link binaries. Given that -C linker already exists and works and there's a corresponding cargo config setting to set it, and that the -C linker-flavor docs say that passing a path to -C linker will automatically detect the correct flavor it doesn't sound like there's anything else that would need to be done to make it work for those scenarios.

does it work just as well on win 8 and 7? should we restrict efforts to win 10?

This definitely bears some investigation but I suspect it's not a terribly big deal in practice. The LLVM docs about building clang for use in Visual Studio say "Any system that can adequately run Visual Studio 2017 is fine". The Visual Studio 2017 System Requirements documentation says it supports Windows 7 SP1 or newer.

how important are the missing features on the lld-link status page? can we recover and fallback to link when encountering them?

Given that both Firefox and Chrome use lld-link for linking their Windows builds nowadays I'd hazard a guess that any missing features are not incredibly important. If someone were to propose making lld-link the default linker I would worry more about that, but anyone opting-in to using it and also relying on niche linker features can presumably just...not use it?

[matklad]

The issue says "To test using rust-lld for msvc targets, use -C linker=lld". Shouldn't it be -C linker=rust-lld? At least, that's what i need to put in .cargo/config to make this work (my experiments):

[target.x86_64-pc-windows-msvc]
linker = "rust-lld"

[petrochenkov]

Potential blocker - #76398 "Rust + MSVC + LLD = Segfault on attempt to access TLS".
This issue currently prevents bootstrap of rustc on x86_64-pc-windows-msvc using LLD as a linker.
Already mentioned under the "LLD/COFF does not correctly align TLS section" item.

[Gankra]

It looks like all the known issues with lld for this target have been fixed! 🎉

Shall we look into flipping it on by default?

[glandium]

FWIW, lld-link has problems running on Windows 7. https://bugzilla.mozilla.org/show_bug.cgi?id=1699228

[AndreKR]

@glandium Apparently only for some build jobs or maybe for some versions of LLD? I'm using it on Windows 7 without running into that issue.

[glandium]

Yeah, apparently the problem is not happening with the build from llvm.org.

[AndreKR]

@glandium I'm using the build that rustup installs.

[bstrie]

Chromium had the same issue on Windows 7 as Firefox, and their answer was that building requires Windows 10. I don't want to be harsh on people with older hardware, but mainstream support for Windows 7 ended before Rust 1.0 even released, and extended support ended over a year ago... I suspect that LLVM itself will stop supporting Windows 7 within the next year, if their deprecation timeline of Windows XP was any indication.

[moxian]

The firefox lld-link problems mentioned in the comment above (#71520 (comment)) sound extremely similar to those we've had in rust in #81051.
They were fixed in LLVM trunk, but given that firefox (as they mention in the thread) have their own fork of LLVM, it's possible that they didn't adopt the fix (as it happened after LLVM 12 cutoff?)?

I don't have the time to test it right now, but if - as per @AndreKR - things are working currently, then i'm cautiously optimistic that they would keep working in the short-medium term as well.

[pravic]

It looks like all the known issues with lld for this target have been fixed!

Well, there is another one last time I tried:

rust-lld: error: /manifestdependency: is not allowed in .drectve

Looks like lld-link doesn't fully support MSVC linker.

[memoryruins]

In the blocking issues section

#71504 - rust-analyser segfault with lto=thin, linker=lld-link

is checked off as being resolved since the issue went away; however, there is a recent open issue that similarly needs thin-lto and lld to be used together to produce a segfault #81408

[bstrie]

@pravic Can you file a bug with a reproduction so that I can link it from the tracking issue?

[ChrisDenton]

@bstrie @pravic I hope you don't mind but I filled a minimal report as I already had something similar laying around.

[ChrisDenton]

#85642 is now resolved in nightly (Rust 1.60.0).

[pnkfelix]

Visited during T-compiler backlog bonanza.

The work that @lqd is doing for rust-lang/compiler-team#510 are partly blocking this, but I am also not sure whether there are other blocking issues. (It sounds like #81408 may be a blocking issue, for example.)

(Furthermore, we are not certain we even will want lld to be the default on x64 msvc windows.)

@rustbot label: S-tracking-needs-summary

[gulbanana]

link.exe performance seems to have improved a lot in VS2019 and VS2022. in some unscientific testing, i'm not seeing gains from switching to rust-lld.exe.

[FeldrinH]

(Furthermore, we are not certain we even will want lld to be the default on x64 msvc windows.)

What are the arguments against having lld as default on windows? To me just the better cross compilation support and getting rid of the need to manually install the MSVC toolchain just for the linker seem like they make lld the preferred choice.

[slonopotamus]

getting rid of the need to manually install the MSVC toolchain

Not only install, but also buy because MSVC toolchain isn't free under certain circumstances. Is link.exe the only used component from MSVC currently? If this change allows to get rid of MSVC completely so only free Windows SDK is required, it is a very strong reason to move forward IMO.

[ChrisDenton]

MSVC provides more than the linker and Windows SDK. It also provides the C runtime (e.g. startup code), core functions such as memcpy, memcmp, etc as well as the panic handling runtime. Perhaps these all could be replaced but as it stands simply switching out the linker doesn't remove the dependency on MSVC.

[frederikhors]

I'm on Windows 10 x64 with:
rustup 1.25.1 (bb60b1e89 2022-07-12)
rustc 1.66.1 (90743e729 2023-01-10)
MSVC v143 installed with Visual Studio Installer from Microsoft.

If I open a Rust project and change a simple char in code (eg: a variable value from 1 to 2) it re-builds the project (using watchexec) in 12 seconds.

I installed llvm and used this in global cargo config file (C:/Users/<username>/.cargo/config)

[target.x86_64-pc-windows-msvc]
linker = "lld-link.exe"

After a cargo clean and an initial re-build (debug mode, 2 minutes, the same as without lld) the time is the same (maybe even worse than 1 second).

So no change with or without LLD.

Can you confirm or am I wrong?

How to get faster incremental (development) builds?

[Keithcat1]

It might be better to test this using larger projects like RustPython or even Servo?

[Elabajaba]

I found a new STATUS_ACCESS_VIOLATION with MSVC+LLD+thinLTO (in both debug and release builds of rend3), haven't tried to narrow the scope down yet. (on both 1.69 stable and 1.71 nightly)

#111480

[B0ryskart0n]

If I understand correctly from the discussion above, using lld by default is close to being the reality and now it's a question of stability checks and some polishing.
Therefore I have two quesitons:

1. What are the current timeline (and workload) expectations on closing this?
2. How would I locally switch to lld project-wise?

[TRI99ERED]

2. How would I locally switch to lld project-wise?

Create a config.toml file in your project directory.
In the file put linker = "rust-lld" under your target.

In case of Windows it's:

[target.x86_64-pc-windows-msvc]
linker = "rust-lld"