Approaches for avoiding fragmentation in code memory allocation

[Keno]

This is moved over from #14623. Currently LLVM's memory allocator will not write to memory that has been protected already, causing some amount of memory fragmentation. It would be good to figure out how to avoid this, while maintaining W^X memory protection guarantees.

[Keno]

Ok, after a lengthy conversation between @yuyichao and myself we've come up with the following plan to address this:

As a generic fallback for Unix, use a MAP_SHARED mmap region to create two mappings to the memory region, the first of which has the appropriate permissions, the second is mapped either PROT_NONE or PROT_WRITE depending on whether we currently need to access it or not. Once we no longer need a page in the second mapping, we can also unmap it.

There are also several operating-system specific parts to take into account:

• On Linux we can be more efficient and avoid the use of MAP_SHARED. The scheme is to map pages as MAP_PRIVATE as usual, but to either use /proc/self/mem or the appropriate userfaultfd ioctls to bypass the memory protection checks and write data to the protected page. Which to use needs to be benchmarked (and sincer the latter is only available in kernels >= 4.2, the former needs to exist as a fallback anyway). This will potentially require an additional copy, but that cost can be mitigated in LLVM (but doesn't need to be in the initial version).
• On OS X we can use the mach_make_memory_entry_64 system call to obtain a reference to the local process's address space and establish a temporary dual mapping that way. It's worth noting that OS X allocated memory in 128MB chunks and you only ever get a reference to one chunk. Also shm_open regions can apparently not be mapped as PROT_EXEC (cf https://bugs.chromium.org/p/nativeclient/issues/detail?id=1872). It's not entirely clear that this code path is worth implementing, but avoiding MAP_SHARED may be desirable.
• For any MAP_SHARED implementation, there need to be special considerations around fork, @yuyichao suggested the following:

if you fork,
on the child, you can just stop allocating in the shared region and create a new shm when you need to jit again
on the parent, you don't need to do anything
I guess rwdata can be a concern?
but rwdata can just use plain mmap
I wasn't planing to use dual map for rwdata anyway....
since they are, well, rw.....
and if you are paranoid about jitting on parent can be seen on the child, we can stop allocating on the parent in the shared page too

• On Windows, one would use the regular CreateFileMapping/MapViewOfFileEx to create the requisite dual mappings.

@yuyichao Please add anything I may have forgotten.

[yuyichao]

For fork support, I guess throwing away the pages in the parent is potentially bad since it can happen fairly often. The fork concern also applies to many other implementations as well (/proc/self/mem and userfaultfd for example, due to shared handles, those handles needs to be reopen'd on fork too.)