Linking C runtime without the C compiler

[davidlattimore]

This is a continuation of a discussion on #112 with @madsmtm and @mati865. I figured I'd move the discussion here, since it's not clear yet what if anything would need to change in Wild. We might well file a new issue once we have some idea.

It'd be nice if it were possible to integrate Wild into compilers and do linking directly from the compiler. The main impediment to this is that the C compiler knows what object files and flags are needed in order to link the C runtime, whereas compilers for high level languages generally don't.

Running rustc for example and telling it to invoke the linker directly is pretty tricky:

❯ rustc -Clinker=ld main.rs
  = note: ld: cannot find -lgcc_s: No such file or directory

To get a hello-world binary that runs, we likely need something like this:

❯ rustc -Clinker=ld -Clink-args='/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/Scrt1.o /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/11/crtbeginS.o -L/lib/x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/11 --dynamic-linker=/lib64/ld-linux-x86-64.so.2 /usr/lib/gcc/x86_64-linux-gnu/11/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/crtn.o' main.rs

This does run, however suffers from two problems. It's a pain to figure out what arguments to pass. It's also not entirely correct, since objects like Scrt1.o are supposed to be at the start before any other objects, but rustc doesn't seem to give us a way to add linker arguments at the start. This would mostly show up as a problem if some user-supplied object added to the .init. or .fini sections, which these days is pretty unlikely.

It'd be nice if there was a standard way to find all the arguments that need to go at the start and end of the link command on a particular platform. These arguments also depend on the link mode. e.g. static linking will add different objects.

Another option would be for the compiler to decide not to use those bits of the C runtime and provide its own runtime for the target platform. That runtime potentially doesn't need to do very much. e.g. call __libc_start_main, passing a pointer to main. IMO, a library like libc should be usable without the need for a C compiler. i.e. it should be documented and stable as to what symbols need to be provided and what initialisation functions need to be called and when.

[mati865]

My initial idea is to replicate what Clang does, hardcode a list of known locations, and iterate over them to find the right ones: https://github.com/llvm/llvm-project/blob/3f648992bf317a3496c4d137374d2c1532423d1c/clang/lib/Driver/ToolChains/Gnu.cpp#L2199-L2222

So, a binary called cc-wild or whatever would be like any other linker proxy, except it would call wild as a library to avoid spawning additional processes. That shouldn't be hard to achieve.

Your idea sounds cool but probably needs more effort to implement correctly. Perhaps there could be two approaches within the same binary, so if one fails, the user can try the other (or even provide automatic fallback).