Feat/singlepass dylib #2802

ptitSeb · 2022-02-23T11:33:48Z

Description

Enable Dylib+singlepass on Linux and macOS. Works fine, but larger wasm file may still fail to link on macOS/Aarch64.
For #2736

Amanieu · 2022-02-23T13:14:59Z

lib/engine-dylib/src/artifact.rs

@@ -384,7 +384,7 @@ impl DylibArtifact {
            // We are explicit on the target when the host system is
            // Apple Silicon, otherwise compilation fails.
            if target_triple_str == "arm64-apple-darwin" {
-                vec![format!("--target={}", target_triple_str)]
+                vec![form at!("--target={}", target_triple_str)]


Amanieu · 2022-02-23T13:15:43Z

lib/engine-dylib/src/artifact.rs

            (OperatingSystem::Linux, Architecture::X86_64) => vec!["-Wl,-z,notext"],
+            (OperatingSystem::MacOSX, Architecture::X86_64) => vec!["-Wl,-read_only_relocs,suppress"],


We actually want to remove both of these options. They currently tell the linker to allow text relocations which we don't want for PIC code.

…ommand on linker that enable write to text section

syrusakbary · 2022-03-07T20:39:46Z

I don't have enough context on this PR to approve myself. So we'll wait for @Amanieu review!

Amanieu · 2022-03-15T12:29:42Z

I tested this branch and it seems to generate a .so file with zero runtime relocations, which is good. However I don't see why the no-text flag is still needed. Additionally, this PR has a lot of unnecessary changes like:

emit_ldr_label and emit_nop that are never used.
Leftover change to traphandlers.c
It still includes the linker flags that we don't want. This should never trigger if we are indeed emitting code with zero relocations.

ptitSeb · 2022-03-16T10:02:14Z

It still includes the linker flags that we don't want. This should never trigger if we are indeed emitting code with zero relocations.

The cross compile test is failling. It compile using dylib-cranelift targetting x86_64-linux-musl and fail with a relocation of type R_X86_64_64 for wasmer_vm_probestack in text. It only happens for musl. It seems to fails if I actually cross compile: if I cange the example to build to x86_64-linux-gnu on an x86_64 linux box, it works, but the same change on an arm64 box the sample fails. It seems to only happens when lld is used. ld/gold don't have the issue.

Amanieu · 2022-03-16T11:14:34Z

Can you dump the object files that are passed to the linker in both the success and failing cases? Also the arguments passed to the linker. The wasmer_vm_probestack symbol should be defined in the same object file, in a separate section that is added by engine-dylib.

ptitSeb · 2022-03-16T11:21:01Z

Yes, I was thinking on the same thing, check the difference, if any, on both object files.

ptitSeb · 2022-03-16T12:21:35Z

generated object file are identical in both case.
The dump show this

RELOCATION RECORDS FOR [.text]:
OFFSET           TYPE              VALUE
0000000000000002 R_X86_64_PC32     __WASMER_TRAMPOLINE0-0x0000000000000004
0000000000000008 R_X86_64_PC32     __WASMER_TRAMPOLINE1-0x0000000000000004
000000000000000e R_X86_64_PC32     __WASMER_TRAMPOLINE2-0x0000000000000004
0000000000000014 R_X86_64_PC32     __WASMER_TRAMPOLINE3-0x0000000000000004
000000000000001a R_X86_64_PC32     __WASMER_TRAMPOLINE4-0x0000000000000004
0000000000000020 R_X86_64_PC32     __WASMER_TRAMPOLINE5-0x0000000000000004
0000000000000026 R_X86_64_PC32     __WASMER_TRAMPOLINE6-0x0000000000000004
000000000000002c R_X86_64_PC32     __WASMER_TRAMPOLINE7-0x0000000000000004
0000000000000032 R_X86_64_PC32     __WASMER_TRAMPOLINE8-0x0000000000000004
0000000000000038 R_X86_64_PC32     __WASMER_TRAMPOLINE9-0x0000000000000004
000000000000003e R_X86_64_PC32     __WASMER_TRAMPOLINE10-0x0000000000000004
0000000000000044 R_X86_64_PC32     __WASMER_TRAMPOLINE11-0x0000000000000004
000000000000004a R_X86_64_PC32     __WASMER_TRAMPOLINE12-0x0000000000000004
0000000000000050 R_X86_64_PC32     __WASMER_TRAMPOLINE13-0x0000000000000004
0000000000000056 R_X86_64_PC32     __WASMER_TRAMPOLINE14-0x0000000000000004
000000000000005c R_X86_64_PC32     __WASMER_TRAMPOLINE15-0x0000000000000004
0000000000000062 R_X86_64_PC32     __WASMER_TRAMPOLINE16-0x0000000000000004
0000000000000068 R_X86_64_PC32     __WASMER_TRAMPOLINE17-0x0000000000000004
000000000000006e R_X86_64_PC32     __WASMER_TRAMPOLINE18-0x0000000000000004
0000000000000074 R_X86_64_PC32     __WASMER_TRAMPOLINE19-0x0000000000000004
000000000000007a R_X86_64_PC32     __WASMER_TRAMPOLINE20-0x0000000000000004
0000000000000080 R_X86_64_PC32     __WASMER_TRAMPOLINE21-0x0000000000000004
0000000000000086 R_X86_64_PC32     __WASMER_TRAMPOLINE22-0x0000000000000004
000000000000008c R_X86_64_PC32     __WASMER_TRAMPOLINE23-0x0000000000000004
0000000000000092 R_X86_64_PC32     __WASMER_TRAMPOLINE24-0x0000000000000004
0000000000000098 R_X86_64_PC32     __WASMER_TRAMPOLINE25-0x0000000000000004
000000000000009e R_X86_64_PC32     __WASMER_TRAMPOLINE26-0x0000000000000004
00000000000000a4 R_X86_64_PC32     __WASMER_TRAMPOLINE27-0x0000000000000004
00000000000000aa R_X86_64_PC32     __WASMER_TRAMPOLINE28-0x0000000000000004
00000000000000b0 R_X86_64_PC32     __WASMER_TRAMPOLINE29-0x0000000000000004
00000000000000b6 R_X86_64_PC32     __WASMER_TRAMPOLINE30-0x0000000000000004
00000000000000c0 R_X86_64_64       wasmer_vm_probestack

so there still is a reloc for the wasmer_vm_probestack.

Amanieu · 2022-03-16T12:58:31Z

Actually the R_X86_64_64 relocation seems strange. That's an absolute 64-bit relocation. I would have expected a R_X86_64_PC32 instead which is used for the offset in a call instruction.

I think that I know what the problem is, can you try applying this commit from #2807: 657bbae

You might also need to apply this fix: a280914

ptitSeb · 2022-03-16T13:23:39Z

Yes, it works with both commit.

ptitSeb · 2022-03-16T13:24:41Z

So I'll wait until the Stack/Corosensei branch is merged...

lib/object/src/module.rs

ptitSeb · 2022-03-17T11:30:25Z

bors r+

bors · 2022-03-17T12:53:43Z

Build succeeded:

ptitSeb added 3 commits February 22, 2022 10:22

Re-enable some singlepass/dylib test on linux and macos

81afc23

Changed how relocations are handled in singlepass/aarch64

b7d705a

Fixed dylib-singlepass on macos/Arm64

4f710ac

ptitSeb requested a review from syrusakbary as a code owner February 23, 2022 11:33

ptitSeb requested a review from Amanieu February 23, 2022 11:33

Try to fix macOS/x86_64 for dylib+singlepass

3e35e82

Amanieu reviewed Feb 23, 2022

View reviewed changes

ptitSeb added 3 commits February 23, 2022 15:21

Change the relocation type for x86_64 on Singlepass and removed the c…

ffc559a

…ommand on linker that enable write to text section

Put back no-text flags, it's still needed

a69e944

Merge branch 'master' into feat/singlepass_dylib

73d78be

ptitSeb added 4 commits March 15, 2022 14:14

Removed debug leftofver

17138fe

Removed unused emit_ldr_label and emit_nop arm64 emiter

80cee07

Merge branch 'master' into feat/singlepass_dylib

fd62d3e

Merge branch 'master' into feat/singlepass_dylib

44faa7e

ptitSeb added 2 commits March 16, 2022 16:04

Merge branch 'master' into feat/singlepass_dylib

dab1d2b

Removed the '-notext' flags to dylib-engine linker command

ed7a672

Amanieu approved these changes Mar 16, 2022

View reviewed changes

lib/object/src/module.rs Outdated Show resolved Hide resolved

Amanieu approved these changes Mar 16, 2022

View reviewed changes

Removed Arm64MovwX relocs

31c9e62

bors bot merged commit 704d935 into master Mar 17, 2022

bors bot deleted the feat/singlepass_dylib branch March 17, 2022 12:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/singlepass dylib #2802

Feat/singlepass dylib #2802

ptitSeb commented Feb 23, 2022

Amanieu Feb 23, 2022

ptitSeb Feb 23, 2022

Amanieu Feb 23, 2022

syrusakbary commented Mar 7, 2022

Amanieu commented Mar 15, 2022

ptitSeb commented Mar 16, 2022

Amanieu commented Mar 16, 2022

ptitSeb commented Mar 16, 2022

ptitSeb commented Mar 16, 2022

Amanieu commented Mar 16, 2022

ptitSeb commented Mar 16, 2022

ptitSeb commented Mar 16, 2022

ptitSeb commented Mar 17, 2022

bors bot commented Mar 17, 2022

		(OperatingSystem::Linux, Architecture::X86_64) => vec!["-Wl,-z,notext"],
		(OperatingSystem::MacOSX, Architecture::X86_64) => vec!["-Wl,-read_only_relocs,suppress"],

Feat/singlepass dylib #2802

Feat/singlepass dylib #2802

Conversation

ptitSeb commented Feb 23, 2022

Description

Amanieu Feb 23, 2022

Choose a reason for hiding this comment

ptitSeb Feb 23, 2022

Choose a reason for hiding this comment

Amanieu Feb 23, 2022

Choose a reason for hiding this comment

syrusakbary commented Mar 7, 2022

Amanieu commented Mar 15, 2022

ptitSeb commented Mar 16, 2022

Amanieu commented Mar 16, 2022

ptitSeb commented Mar 16, 2022

ptitSeb commented Mar 16, 2022

Amanieu commented Mar 16, 2022

ptitSeb commented Mar 16, 2022

ptitSeb commented Mar 16, 2022

ptitSeb commented Mar 17, 2022

bors bot commented Mar 17, 2022