offload: automate manual clang-linker-wrapper step

[sgasho]

automate manual clang-linker-wrapper step from https://rustc-dev-guide.rust-lang.org/offload/usage.html
extract bitcode from device.bin and then wraps it into the host module.
needs some refactoring.

r? @ZuseZ4

[ZuseZ4]

Thanks to your last PR, we should ship these in our sysroot. Is it guaranteed to be on our default search path? If not, we might want to add an rpath or -L, or?

View changes since the review

[sgasho]

Yes, looking at walk_native_lib_search_dirs, it is on our default linker search path, so we don't need an additional -L, but we need an rpath for runtime. I also checked this on a UofT server.

[sgasho]

I'm looking into this issue I encountered while testing on a gpu server

shotasugano@gpunode1:/w/333/shotasugano/sample-offload$ ./target/x86_64-unknown-linux-gnu/release/sample-offload 
PluginInterface error: Failure to load binary image 0x62597fffacf0 on device 0: "jit compile failure while processing binary image" failure to jit IR image: Unable to find target for this triple (no targets are registered)
omptarget error: Failed to load image "a provided binary image is malformed" failed to load binary 0x62597fffacf0
omptarget fatal error 0: "the plugin backend is in an invalid or unsupported state" failed to load images on device '0'
Aborted                    ./target/x86_64-unknown-linux-gnu/release/sample-offload

update:
fixing by bringin more logic from ClangLinerWrapper.cpp...

[sgasho]

Succeeded on UofT server, though we still need -rpath for now

shotasugano@gpunode1:/w/333/shotasugano/sample-offload$ RUSTFLAGS="-Ctarget-cpu=sm_86 --emit=llvm-bc,llvm-ir -Zoffload=Device -Csave-temps -Zunstable-options" \
cargo +offload-local build -Zunstable-options -r -v \
  --target nvptx64-nvidia-cuda -Zbuild-std=core
    (omitted...)
    Finished `release` profile [optimized] target(s) in 40.58s

shotasugano@gpunode1:/w/333/shotasugano/sample-offload$ RUSTFLAGS="-C embed-bitcode=yes" \
cargo +offload-local rustc \
  --release \
  --bin sample-offload \
  --target x86_64-unknown-linux-gnu \
  -- \
  -C lto=fat \
  -Zunstable-options \
  "-Zoffload=Host=$PWD/target/nvptx64-nvidia-cuda/release/build/sample-offload/839aa6816b5baeb2/out/device.bin" \
  -C "link-arg=-Wl,-rpath,$(rustc +offload-local --print target-libdir)"
info: `cargo` is unavailable for the active toolchain
info: falling back to "/u/shotasugano/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo"
   Compiling libc v0.2.189
   Compiling sample-offload v0.1.0 (/w/333/shotasugano/sample-offload)
    Finished `release` profile [optimized] target(s) in 8.87s

shotasugano@gpunode1:/w/333/shotasugano/sample-offload$ ./target/x86_64-unknown-linux-gnu/release/sample-offload 
all checks passedshotasugano@gpunode1:/w/333/shotasugano/sample-offload$ 

[sgasho]

Succeeded without rpath

shotasugano@gpunode1:/w/333/shotasugano/sample-offload$ RUSTFLAGS="-Ctarget-cpu=sm_86 --emit=llvm-bc,llvm-ir -Zoffload=Device -Csave-temps -Zunstable-options" \
cargo +offload-local build -Zunstable-options -r -v \
  --target nvptx64-nvidia-cuda -Zbuild-std=core
    (omitted...)
    Finished `release` profile [optimized] target(s) in 40.91s

shotasugano@gpunode1:/w/333/shotasugano/sample-offload$ RUSTFLAGS="-C embed-bitcode=yes" \
cargo +offload-local rustc \
  --release \
  --bin sample-offload \
  --target x86_64-unknown-linux-gnu \
  -- \
  -C lto=fat \
  -Zunstable-options \
  "-Zoffload=Host=$PWD/target/nvptx64-nvidia-cuda/release/build/sample-offload/839aa6816b5baeb2/out/device.bin"
info: `cargo` is unavailable for the active toolchain
info: falling back to "/u/shotasugano/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo"
   Compiling libc v0.2.189
   Compiling sample-offload v0.1.0 (/w/333/shotasugano/sample-offload)
    Finished `release` profile [optimized] target(s) in 2.41s

shotasugano@gpunode1:/w/333/shotasugano/sample-offload$ ./target/x86_64-unknown-linux-gnu/release/sample-offload 
all checks passedshotasugano@gpunode1:/w/333/shotasugano/sample-offload$ 

[ZuseZ4]

Exciting, let me test it on some AMD server as well then.

[sgasho]

Thanks!
Also, I think we'd better test this via tarball from a bors try before merging since I modified dist.rs

[ZuseZ4]

I'll look a little closer later, but this adds clang and a few other binaries, right? My hope was that we could directly call the underlying APIs without adding more binaries. During compilation of nvptx64 I don't think we can get around relying on ptxas, but other than that we should already have the APIs there to compile everything ahead of time.

[sgasho]

I confirmed that the sample offload code compiled successfully on the UofT gpu server with the latest commit, without adding extra binaries.

[ZuseZ4]

@bors r+ rollup p=1 unblocks offload in nightly

[rust-bors]

📌 Commit ff680e2 has been approved by ZuseZ4

It is now in the queue for this repository.

🌲 The tree is currently closed for pull requests below priority 5. This pull request will be tested once the tree is reopened.

Reason for tree closure: spurious failures

[ZuseZ4]

there are a few more things to clean up, but looks good for now. Let's see if we can land it today, so that tomorrows nightly during rustconf already has support.