Implement __morestack for ARM

[brson]

There are some difficulties with implementing __morestack for ARM, but I don't know what they are. @amuxtux can you explain?

For the time being it looks like we're just going to want to disable stack growth completely on ARM. To do this I think we should turn off the LLVM split stack feature for the ARM target (not use no_split_stack #1226).

[amuxtux]

I have little idea at the moment, if someone find this explanation wrong please correct.

1. first of all its about red zone : in x86 rust code, the prologue code (possibly the red zone impl) seems to be using F and G segment registers. in ARM there are no such registers. It may be possible to use other registers like LR and PC, I am not sure about it.
2. second but comparatively less complex problem is to understand and port the x86 version of morestack.S to ARM assembly.
3. third point I am also not sure but I write it in any case. It seems to have something to do with split stack option of gcc also. Unlike normal x86 libcc I did not find that option with android NDK's gcc. It must have something to with morestack call, but again also not quite clear what and how it will have impact. [any help will be highly appreciated ]

[amuxtux]

--- just in addition to above ----
// i captured followings with rustc -S option for both x86 and ARM.

Following seems to be red zone function inside prologue in i386
cmpq %gs:816, %rsp <------ seems GS register
ja LBB0_0
movabsq $8, %r10
movabsq $0, %r11
callq ___morestack
ret
< ......... prologue continues ...........>

Following is the current Prolog for ARM for same simple function: (with out red zone)
push {r11, lr}
mov r11, sp
sub sp, sp, #20 <------- currently static increment of stack

To implement in ARM we should have proper RZ implementation followed by __morestack insertion.

[brson]

The x86 segment registers that the prologue is looking in contain the thread control block, where pthreads stores it's per-thread data structures, in particular the task-local storage slots. The pthreads implementation reserves a number of slots for various platform-specific things and the prologue (and the accessors in record_sp.S) uses assembly to access one of these slots directly.

For the ARM port we want to figure out where that thread control block lives and identify a reserved slot that we can 'steal'. For example, on x86_64 Linux we are using %fs:112 which corresponds to ... (TODO can't locate this information).

There is a big comment about bionic's TLS implementation that might offer some leads. There are apparently some reserved TLS slots that we may be able to use. Finally, there are two different mechanisms for accessing TLS on bionic depending on the hardware/software revision in use. With all this information presumably we can find a free word somewhere to hold the stack pointer.

If you haven't seen it here is the original gcc split stack proposal, containing some useful details.

[brson]

I mentioned this elsewhere but I'll record it here too: Rust doesn't depend on a C++ compiler with -fsplit-stack. All of our split stack support is provided by LLVM and the Rust runtime. We don't currently compile any C++ code with split stacks. It is not going to be a problem that the Android toolchain doesn't have -fsplit-stack.

[brson]

A lot of ARM processors don't have the TLS register and implement TLS with a syscall (or a trap). Making a syscall on every function call is not going to work, so we may need to support a configuration that doesn't do stack growth. My impression is that newer generation processors should have the register.

Some info here: http://elinux.org/Android_on_OMAP#TLS_issue

[amuxtux]

Thanks @brson , I put more effort to understand the problem and what need to be done.

So in order to get started , I first created a morestack.s with simple return for ARM i.e. mov pc, lr and statically compiled linked with rustrt using ndk tool chain.

However LLVM when generate code for ARM even with -segmented-stacks option does not insert __morestack external symbol into the generated code.

From rust side I have verified there is no problem as EnableSegmentedStack option is true while calling LLVM.

so I started implementing adjustForSegmentedStacks method in ARMFrameLowering.cpp.

Since I hardly know that big code so its being really hard and taking lot of time. Any way I believe once it is done it would give a ground to do some experiments to detect bottom of stack in case of arm-android and finally will lead to filling in empty morestack.S

[ILyoan]

I'm working on this and I believe I've almost done with this.
I just supposed that all of the android devices have TLS register so that we can access TLS address via mrc instruction. And I use the last TLS slot(64th slot) to record stack limit because it seems to me that the android kernel tries to allocate the lowest empty TLS slot(except the reserved slot) to users.

My current working branches are
https://github.com/ILyoan/rust/tree/arm_morestack
https://github.com/ILyoan/llvm/tree/arm_splitstack
Building rustc for android is done with CMake. It can be built with legacy make build system after issue #4513 is landed.

[brson]

@ILyoan Wow! Nice work.

I'm glad we can assume the presence of TLS. That makes things easier.

[yichoi]

Being managed without cmake
https://github.com/yichoi/rust/commits/arm_latest (old makefile comes from cross3)
https://github.com/yichoi/rust/commits/arm_morestack (current makefile comes for cross5