[WIP]: raw-dylib support for apple targets

[roblabla]

CC #58713

This PR adds macos support to raw-dylib. This works by generating a .tbd file for each library using LLVM's TextAPI functions, and linking against them. .tbd files are an official Apple format representing the exports of a dylib as YAML, whose format is documented here. LLVM's TextAPI provides an easy-to-use API to produce (and consume) those files.

Apple targets bring several complications to the raw-dylib support:

No runtime dylib search path

dyld, the mach-o loader, does not have a default search path to find the shared libraries (or dylibs). Instead, there are five ways to define a shared lib:

• Absolute path (/usr/lib/libEndpointSecurity.dylib): This is how system deps usually end up in a macho. You get an absolute path like
• Relative path: Those get loaded relative to the current working directory. This is almost always wrong.
• @loader_path/libEndpointSecurity.dylib: These are loaded relative to the entity that caused the shared lib to load (the mach-o binary or intermediate shared lib). This can be useful when storing libs next to the binary, or plugins next to a lib.
• @executable_path/libEndpointSecurity.dylib: Similar to the above, but always relative to the binary.
• @rpath/libEndpointSecurity.dylib: RPATH is the closest thing to a search path, but is hardcoded in the mach-o. Essentially, a mach-o will have an LC_RPATH load command that specifies the list of paths to look into for libs. By default, the linker does not set an RPATH.

Currently, we just use whatever is in link(name). The user may specify the full name in the #[link(name)], e.g.

#[link(name = "/usr/lib/libEndpointSecurity.dylib", kind = "raw-dylib", modifiers = "+verbatim")]
extern "C" { /* ... */ }

An alternative solution could be to have a separate attribute for the directory, like #[link(name = "EndpointSecurity", install_path = "/usr/lib", kind = "raw-dylib")].

ar files

Unlike Windows lib files, which can contain .dll files containing the symbols to import along with the .obj files, macOS (and indeed, linux) ar archives do not have a way to specify external symbol dependencies (at least, not that I'm aware!). Those have to be specified separately to the final linking step.

To work around this issue, we cheat a bit. When building an rlib, the tbd is inserted into the rlib, so it can be extracted and linked against during the final linking step. This is... not exactly elegant. But it works and was the easiest solution I could think of.

Currently, the link(name) attribute is used as the filename to use when storing the ar archive (meaning, our ar archive will have a file named /usr/lib/libEndpointSecurity.dylib.tbd. It may be worth adding a prefix to the filename or something?

Additional attributes

Mach-O additionally encodes two pieces of information with every shared library in its LC_LOAD_DYLIB command: the current_version, and the compatibility_version. compatibility_version in particular is used by dyld to figure out if the dylib on the system is new enough to link properly with the current application.

This PR currently sets both of those to 0 - and that seems to work well enough, at least for my use-cases. But we could imagine adding new attributes for them if they're necessary.

Status

This PR works, but is still kinda prototypical. I need to add some unit tests to ensure it works in a vast variety of scenarios (unfortunately, a lot of the windows tests don't work as-is for macos due to the above limitations). The goal here is mostly to get feedback on the current approach, maybe get some ideas bouncing on if there's another way to approach the issues here.

CC @joshtriplett

[rust-highfive]

r? @fee1-dead

(rust-highfive has picked a reviewer for you, use r? to override)

[joshtriplett]

The concept of this seems great. Thank you for working on it!

I do think we'll likely need support for the compatibility version and current version.

I also don't think we should embed the tbd file in the rlib. I think we should place it alongside the rlib and reference it in the metadata.

[fee1-dead]

r question compiler if you want it to be reviewed or r? @ghost

[bors]

☔ The latest upstream changes (presumably #100732) made this pull request unmergeable. Please resolve the merge conflicts.

[bjorn3]

I think the .tbd file should be generated right before invoking the linker rather than while packaging an rlib.

[bjorn3]

meaning, our ar archive will have a file named /usr/lib/libEndpointSecurity.dylib.tbd.

I'm not quite sure if ar archives are allowed to have directories. In any case having a prefix / will in all likelyhood prevent extracting files from it.

[fee1-dead]

Unassigning myself due to the inactivity. If you want to find a reviewer after you are ready, you can use r? compiler.

[Dylan-DPC]

@roblabla any updates on this?

[Dylan-DPC]

Closing this as inactive. Feel free to reöpen this pr or create a new pr if you get the time to work on this. Thanks