Investigate orphan sections

[ojeda]

Since synchronizing with v5.10 (#60) we are getting:

ld: warning: orphan section `.text.__rust_probestack' from `drivers/char/rust_example/rust_example.o' being placed in section `.text.__rust_probestack'
ld: warning: orphan section `.eh_frame' from `drivers/char/rust_example/rust_example.o' being placed in section `.eh_frame'

See e.g. https://github.com/Rust-for-Linux/linux/runs/1561327749

[ojeda]

A guess is that some kernel structures changed in v5.10 and now, when using the bindings, we have crossed the stack size threshold in some function, thus calling the probe.

[adamrk]

I think this might be related to some actual errors I'm seeing:

ERROR: modpost: "__rust_probestack" [drivers/char/rust_example_4.ko] undefined!

(see the failed runs here: https://github.com/adamrk/linux/pull/1/checks?check_run_id=1711541791).
The symbol is defined in rust/complier_builtins.o and vmlinux:

$ objdump --syms  rust/vmlinux | rg probestack
ffffffff81801393 l    d  .text.__rust_probestack	0000000000000000 .text.__rust_probestack
ffffffff81801393 g     F .text.__rust_probestack	0000000000000039 .hidden __rust_probestack

Could this issue be the .hidden attribute? That seems to be explicitly set in the definition.

I'm happy to dig into this some more, but I'm kind of stuck at this point so any suggestions would be helpful.

[adamrk]

Looks like the modpost error isn't directly related. Opened #73 instead.

[adamrk]

Maybe the issue here is just that .text.__rust_probestack is a non-standard section? Making this change to vmlinux.lds.h takes care of the warning and seems to work fine, but it doesn't seem correct to add logic there just for this special case:

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index b2b3d81b1535..42d46683f188 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -590,7 +590,7 @@
 #define TEXT_TEXT                                                      \
                ALIGN_FUNCTION();                                       \
                *(.text.hot .text.hot.*)                                \
-               *(TEXT_MAIN .text.fixup)                                \
+               *(TEXT_MAIN .text.fixup .text.__rust_probestack)        \
                *(.text.unlikely .text.unlikely.*)                      \
                *(.text.unknown .text.unknown.*)                        \
                NOINSTR_TEXT                                            \

Looks like __rust_probstack is intentionally put in a separate section (source), but I'm not sure why. Maybe if we end up copying needed parts compiler_builtins into the tree (one of the options in #69) we can change it to store __rust_probestack in the regular .text section?

Or is there a way to add a separate linker script file that specifically handles this?

[bjorn3]

The separate section was added as part of moving from a naked function to global_asm! in rust-lang/compiler-builtins#328 to add CFI information to help unwinding inside a debugger. The separate section matches how rustc compiles with -Zfunction-sections=yes by default.

Does Linux have it's own probestack function? You may be able to alias __rust_probestack to it and completely avoid linking to compiler-builtins. Instead using whatever Linux uses for providing gcc/LLVM compiler intrinsics. Alternatively you could wait for rust-lang/rust#80838 to land and switch to inline asm stack probes.

[ojeda]

AFAICS both GCC and Clang generate the probing inline by default, so we should likely wait for rust-lang/rust#80838.

Having said that, I don't see stack clash protection used nor proposed for the kernel yet, in fact I see a related patch that required disabling it explicitly -- @nickdesaulniers @kees?

[nickdesaulniers]

Having said that, I don't see stack clash protection used nor proposed for the kernel yet, in fact I see a related patch that required disabling it explicitly

Maybe a userspace centric feature that can be disabled?

ld: warning: orphan section .text.__rust_probestack' from drivers/char/rust_example/rust_example.o' being placed in section `.text.__rust_probestack'
ERROR: modpost: "__rust_probestack" [drivers/char/rust_example_4.ko] undefined!

So looks like a missing function (symbol) AND missplaced section? Adding that section makes sense if __rust_probestack is required; not sure what that would look like in the kernel.

ld: warning: orphan section .eh_frame' from drivers/char/rust_example/rust_example.o' being placed in section `.eh_frame'

In C compilers, that's controlled by -fno-asynchronous-unwind-info which we generally use in the kernel, except for one unwinder that 32b ARM uses (by default). There should be a corresponding flag for rustc (or, if not, go make it!).

EDIT: sorry for closing/reopened. "Fat fingers"

[bjorn3]

In C compilers, that's controlled by -fno-asynchronous-unwind-info which we generally use in the kernel, except for one unwinder that 32b ARM uses (by default). There should be a corresponding flag for rustc (or, if not, go make it!).

This unwind info is not generated by the compiler. It is part of the assembly that implements __rust_probestack. The compiler doesn't have any control over this.

So looks like a missing function (symbol) AND missplaced section? Adding that section makes sense if __rust_probestack is required; not sure what that would look like in the kernel.

Same applies here.

Maybe a userspace centric feature that can be disabled?

I believe the target spec has an option to disable stackprobes.

[adamrk]

I think this is resolved with the latest rustc version changes.

[ojeda]

Yeah, when compiled with a high enough LLVM (11.0.1), we should now be getting inlined stack probes now since #85. The CI compiles with 11.1.0 at the moment.

However, the warnings went away before, since #80 was merged, since there was no more compiler_builtins.