Linking modifiers for native libraries #2951

petrochenkov · 2020-06-27T18:41:31Z

Provide some generic syntax (similar to target features) for linking modifiers used for native libraries.

Provide some specific modifiers to control bundling behavior, whole-archive and the library name mangling.

petrochenkov · 2020-06-27T18:45:25Z

cc @retep998
I finally did what I promised in rust-lang/rust#72274 (comment).
Also note the library name change for raw-dylib.

retep998 · 2020-06-27T18:57:45Z

text/0000-native-link-modifiers.md

+undefined symbols at the point at which it is specified on the command line,
+making it similar to static libraries in this regard.
+
+This modifier translates to `--as-needed` for `ld`-like linkers.


Should add a sentence describing what this does for link.exe.

Will add (there's no MSVC equivalent, so it's ignored).

comex · 2020-06-27T22:14:48Z

Notes from a Darwin (macOS/iOS/etc.) perspective:

Darwin uses ld64, which shares the basic syntax of a Unix linker but doesn't support these GNU ld options. Instead it has these equivalents:

verbatim: None. (Note that both GNU ld and ld64 support linking an arbitrarily-named library by passing the full path as a regular argument, but GNU ld's -l: searches the library path for an arbitrary filename, which ld64 doesn't seem to have any equivalent for.)
whole-archive: -all_load to enable for all, -force_load /path/to/foo.a to enable for a single library. rustc already supports this.
as-needed: -dead_strip_dylibs to enable by default, then (new in macOS 11) -needed-lx / -needed_library /path/to/library / -needed_framework framework to mark a single library as not "as needed".

What about the framework linking kind? What does it mean? What modifiers is it compatible with?

Frameworks are Darwin-specific and are literally identical to dylibs but with a different naming scheme and separate search paths.

-F/some/path is the equivalent of -L, but adding to the framework search paths rather than library search paths.
-framework Foo is the equivalent of -lfoo, but it uses the framework search paths and searches for something named Foo.framework/Foo rather than libfoo.dylib.

It should be compatible with the same modifiers as regular dylibs on Darwin.

petrochenkov · 2020-06-28T07:02:08Z

@comex
Thanks!
"What about frameworks" in unresolved questions should have really been "What about Darwin".
I'll include this into the RFC.

petrochenkov · 2020-06-28T07:07:27Z

Note that both GNU ld and ld64 support linking an arbitrarily-named library by passing the full path as a regular argument

Without : the full path will still be "mangled", i.e. when passed -lmy/arbitrary/path ld will search for libmy/arbitrary/path.a or libmy/arbitrary/path.so.

clarfonthey · 2020-06-30T01:29:45Z

text/0000-native-link-modifiers.md

+[guide-level-explanation]: #guide-level-explanation
+
+This is an advanced feature not expected to be used commonly,
+see the reference-level explanation if you know that you need some of these modifiers.


I generally distinguish between guide and reference level for usage and implementation level, although I understand why you didn't bother here. It's not that big of a deal, but I just figured I'd mention this here in case people viewing get ideas. :p

clarfonthey · 2020-06-30T01:32:30Z

text/0000-native-link-modifiers.md

+on some future edition boundary, and hopefully unblocking its stabilization.
+
+The generic syntax provides a way to add more such modifiers in the future
+without introducing new linking kinds.


IMHO this section doesn't really provide that powerful of a motivation. If you've run into needing these issues before, it seems obvious, but I've personally never had to do any special linking despite having done a lot of work in C before. I feel like examples may be helpful, even if they're contrived.

The GitHub issues linked from this PR provide some examples, I'll try to move them into the RFC.

clarfonthey · 2020-06-30T01:33:38Z

text/0000-native-link-modifiers.md

+
+The modifiers are boolean and applied only to the single library specified with `name`. \
+`+` means enable, `-` means disable, multiple options can be separated by commas,
+the last boolean value specified for the given modifier wins. \


Any reason to do this instead of erroring if multiple are specified?

To mirror target features mostly, there it's needed to override features on command line (rust-lang/rust#56527) so that -C target-feature=+keep,+change -C target-feature=-change becomes -C target-feature=+keep,+change,-change and becomes -C target-feature=+keep,-change.

(With linking modifiers it can also be used in theory, to override modifiers from attributes on command line, like
#[link(name = "mylib", modifiers = "+keep,+change")] + -l:-change=mylib:mylib which becomes -l:+keep,-change=mylib.)

Would it be possible to have a deny-by-default lint in code, then? Command line switches make perfect sense, but if someone includes multiple in the code it's almost certainly a mistake.

Even if the multiple specifier case doesn't make sense, it's still very hard to write it accidentally, so making a new lint for it doesn't pull its weight. (There are plenty opportunities to write some unnatural code that doesn't make any sense but is not linted.)
I guess it's easier to make it a hard error then.

clarfonthey · 2020-06-30T01:35:30Z

text/0000-native-link-modifiers.md

+[RFC 1717](https://github.com/rust-lang/rfcs/pull/1717).
+
+The default for this modifier is currently `+bundle`,
+but it could be changed later on some future edition boundary.


If the whole point of editions is that you can link together code for multiple editions at once, wouldn't that not be a good idea?

I don't think this change would break anything implicitly?
When a crate with #[link] attributes migrates from edition 202X to edition 202X+3 it will need to either add the modifier or change its build and/or distribution strategy. Both such such changes should be ok when a crate does edition bump.

clarfonthey · 2020-06-30T01:37:22Z

text/0000-native-link-modifiers.md

+is now added automatically. \
+If your DLL doesn't have the `.dll` suffix, it can be specified with `+verbatim`.
+
+### `whole-archive`


I feel like this name is a bit weird, because really it seems to be talking more about LTO. When reading "whole archive" my first thought is "why wouldn't the whole library be linked?"

The logic behind both whole-archive and as-needed is to simply mirror the corresponding linker options.
Generally, if someone wants the "whole-archive" linking, then they probably want it to have the same name "whole-archive" for searchability.
On the other hand, if someone doesn't know what "whole-archive" means, then they probably don't need this option.

I'm personally viewing this from the perspective of someone looking at existing code to understand how it works. Documentation will include the options that they translate to for searchability, but I'm not sure someone would need to search for this in code itself.

clarfonthey · 2020-06-30T01:39:08Z

text/0000-native-link-modifiers.md

+
+The default for this modifier is `-whole-archive`.
+
+### `as-needed`


Again, a bit fuzzy on the name here. Seems to be more of a statement about LTO.

#2951 (comment)

retep998 · 2020-07-01T14:55:18Z

text/0000-native-link-modifiers.md


 For `ld`-like linkers `rustc` will use the `-l:filename` syntax (note the colon)
 when passing the library, so the linker won't add any prefixes or suffixes as well. \
 See [`-l namespec`](https://sourceware.org/binutils/docs/ld/Options.html) in `ld` documentation
-for more details.
+for more details. \
+For linkers not supporting any verbatim modifiers (e.g. `link.exe` or `ld64`)


link.exe is inherently verbatim, and it is Rust itself which adds any prefix/suffix. That is a significant difference compared to ld64.

Not entirely, link.exe will add .obj to files without extensions.
Also, the paragraph above talks about prefixes/suffixes added by rustc itself.

My idea here was "rustc passes what you passed to the linker without changes and the linker does what it wants with this input", not sure how to word this better.

petrochenkov · 2020-09-03T20:40:18Z

One more request for --whole-archive - https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Link.20with.20--whole-archive/near/208160840.

nikomatsakis · 2020-12-02T14:59:00Z

We ended up adopting a similar plan in the form of a compiler-team major change proposal, rust-lang/compiler-team#356, so I'm going to close this RFC PR. =)

petrochenkov · 2021-01-28T20:04:30Z

@nikomatsakis
This RFC should have been merged, not closed.
The accepted "similar plan" literally says "merge this RFC".
Motivation: I want to refer to this RFC as a normative text when necessary, without explaining that "it may seem that the RFC was closed, but it was actually accepted, just through a different process" every time.

UPD: Tracking issue - rust-lang/rust#81490.

…trochenkov Implement RFC 2951: Native link modifiers A first attempt at implementing rust-lang/rfcs#2951 / rust-lang/compiler-team#356. Tracking Issue: rust-lang#81490 Introduces feature flags for the general syntax (`native_link_modifiers`) and each modifier (`native_link_modifiers_{as_needed,bundle,verbatim,whole_archive}`). r? `@petrochenkov`

petrochenkov added 2 commits June 27, 2020 21:37

Linking modifiers for native libraries

4287dc2

Add a note on stability

d97434c

jonas-schievink added A-ffi FFI related proposals. A-linkage Proposals relating to the linking step. T-compiler Relevant to the compiler team, which will review and decide on the RFC. labels Jun 27, 2020

This was referenced Jun 27, 2020

Tracking issue for #[link(kind)] connecting libraries on Windows rust-lang/rust#37403

Closed

Linked native libraries are sometimes deduplicated and sometimes not deduplicated rust-lang/rust#73319

Open

RUSTFLAGS Linker Flags Order rust-lang/rust#57837

Closed

retep998 reviewed Jun 27, 2020

View reviewed changes

clarfonthey reviewed Jun 30, 2020

View reviewed changes

Address review comments, round 1

a30650b

retep998 reviewed Jul 1, 2020

View reviewed changes

petrochenkov mentioned this pull request Jul 22, 2020

Distribute libc.a as a separate library instead of bundling it into liblibc.rlib on musl and wasi rust-lang/rust#72274

Closed

4 tasks

petrochenkov mentioned this pull request Sep 4, 2020

Accept RFC 2951 "Linking modifiers for native libraries" rust-lang/compiler-team#356

Closed

3 tasks

nikomatsakis closed this Dec 2, 2020

petrochenkov reopened this Jan 28, 2021

This was referenced Jan 28, 2021

Do not append .lib extension to .res file on cargo:rustc-link-lib=dylib=stuff.res rust-lang/rust#81488

Closed

Tracking Issue for linking modifiers for native libraries rust-lang/rust#81490

Closed

nikomatsakis merged commit 40df2d2 into rust-lang:master Mar 5, 2021

luqmana mentioned this pull request Mar 26, 2021

Implement RFC 2951: Native link modifiers rust-lang/rust#83507

Merged

12101111 mentioned this pull request May 23, 2021

Adjust libstd to make non-Rust dependencies optional rust-lang/wg-cargo-std-aware#54

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Linking modifiers for native libraries #2951

Linking modifiers for native libraries #2951

petrochenkov commented Jun 27, 2020

petrochenkov commented Jun 27, 2020

retep998 Jun 27, 2020

petrochenkov Jun 27, 2020 •

edited

Loading

petrochenkov Jul 1, 2020

comex commented Jun 27, 2020

petrochenkov commented Jun 28, 2020

petrochenkov commented Jun 28, 2020 •

edited

Loading

clarfonthey Jun 30, 2020

clarfonthey Jun 30, 2020

petrochenkov Jun 30, 2020

petrochenkov Jul 1, 2020

clarfonthey Jun 30, 2020

petrochenkov Jul 1, 2020 •

edited

Loading

clarfonthey Jul 9, 2020

petrochenkov Jul 22, 2020

clarfonthey Jun 30, 2020

petrochenkov Jul 1, 2020

clarfonthey Jun 30, 2020

petrochenkov Jul 1, 2020

clarfonthey Jul 9, 2020

clarfonthey Jun 30, 2020

petrochenkov Jul 1, 2020

retep998 Jul 1, 2020

petrochenkov Jul 1, 2020 •

edited

Loading

petrochenkov commented Sep 3, 2020

nikomatsakis commented Dec 2, 2020

petrochenkov commented Jan 28, 2021 •

edited

Loading


		The default for this modifier is `-whole-archive`.

		### `as-needed`

Linking modifiers for native libraries #2951

Linking modifiers for native libraries #2951

Conversation

petrochenkov commented Jun 27, 2020

petrochenkov commented Jun 27, 2020

Choose a reason for hiding this comment

petrochenkov Jun 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

comex commented Jun 27, 2020

petrochenkov commented Jun 28, 2020

petrochenkov commented Jun 28, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

petrochenkov Jul 1, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

petrochenkov Jul 1, 2020 • edited Loading

Choose a reason for hiding this comment

petrochenkov commented Sep 3, 2020

nikomatsakis commented Dec 2, 2020

petrochenkov commented Jan 28, 2021 • edited Loading

petrochenkov Jun 27, 2020 •

edited

Loading

petrochenkov commented Jun 28, 2020 •

edited

Loading

petrochenkov Jul 1, 2020 •

edited

Loading

petrochenkov Jul 1, 2020 •

edited

Loading

petrochenkov commented Jan 28, 2021 •

edited

Loading