Add userfaultfd-based Region backend #492

tyler · 2020-04-08T19:28:18Z

This pull request integrates Fastly's userfaultfd based memory management backend as an alternate Region implementation.

Userfaultfd is a Linux-specific mechanism for handling page faults within userspace. (See https://www.kernel.org/doc/html/latest/admin-guide/mm/userfaultfd.html for a good technical explanation of it.) For Lucet, the use case is that when an Instance is started, none of the linear memory has to be copied initially.

We register the entire region with userfaultfd. When an instance is started, we set up the stacks, metadata, etc as normal, but we leave the pages of the linear memory "missing". When the instance starts and tries to access one of the linear memory pages, this triggers a page fault, as there is no physical memory backing the virtual memory. Since the region is registered with userfaultfd, this triggers a message to be sent to the userfaultfd handler thread. The handler thread determines which instance and module has faulted, and copies the necessary memory into place, before reawakening the instance thread.

At its core what this does is provide a way of reducing startup time and increasing flexibility in how memory is handled at the cost of increased runtime overhead (by was of context switches especially).

(All credit for this actually goes to @acfoltzer.)

acfoltzer · 2020-04-09T23:09:57Z

@tyler there are a few things I'd like to add to the public API surface and documentation to go with this. It would probably be easiest if I did that on a branch and then PRed into this, is that alright with you?

fst-crenshaw · 2020-04-09T23:56:30Z

lucet-runtime/lucet-runtime-internals/src/region/uffd.rs

+                    AddrLocation::Globals | AddrLocation::SigStack => {
+                        tracing::error!("UFFD pagefault at unexpected location: {:?}", loc);
+                        uffd.wake(fault_page as *mut c_void, host_page_size())
+                            .map_err(|e| Error::InternalError(e.into()))?;


One of the things I've been seeing in the mmap-related PRs is a re-examination of errors like these. If there had been a UFFD pagefault at an unexpected location, should the lucet runtime return its generic "InternalError" or should it just panic?

See related PR: #486

This is a good call, it should just panic. I think @acfoltzer wanted to take care of that in a second PR. Though tbh I think we should just do it here.

fst-crenshaw · 2020-04-10T00:38:09Z

lucet-runtime/lucet-runtime-internals/src/region/uffd.rs

+            // zero the sigstack
+            (slot.sigstack, limits.signal_stack_size),
+        ]
+        .into_iter()


I am new to userfaultfd, and I'm wondering about a small difference between its implementation of new_instance_with and the mmap implementation.

lucet/lucet-runtime/lucet-runtime-internals/src/region/mmap.rs

Lines 121 to 122 in 83fd224

// make the stack read/writable

(slot.stack, limits.stack_size),

So yeah, that's intentional here. We're making the allocation of the stack lazy. If you look at the other lines changed in that commit, you'll see where I make page faults in the Stack area zero out the page on demand.

That said, it does point to a lack of documentation here that I should fix.

Added some docs!

fst-crenshaw · 2020-04-10T00:40:33Z

lucet-runtime/lucet-runtime-internals/src/region/uffd.rs

+                .map_err(|e| Error::InternalError(e.into()))?,
+        );
+
+        // map the chunk of virtual memory for all of the slots


I'm guessing the answer to my question could be here. The memory is already marked as READ/WRITE for all the slots.

lucet/lucet-runtime/lucet-runtime-internals/src/region/uffd.rs

Lines 136 to 139 in e7fe5f1

AddrLocation::Stack => unsafe {

uffd.zeropage(fault_page as *mut c_void, host_page_size(), true)

.map_err(|e| Error::InternalError(e.into()))?;

},

is also a key part of why this works

Hm. If I'm reading that correctly, does that mean that the stack isn't zeroed out until it's actually needed?

It may help to refer to the ioctl_userfaultfd(2) man page on this, since "zeropage" does a lot more than what it sounds like: http://man7.org/linux/man-pages/man2/ioctl_userfaultfd.2.html (search for UFFDIO_ZEROPAGE on that page).

Basically zeropage is one of several ways that the uffd worker thread can resolve a page fault in the memory that it has registered. The stack starts off as mprotect read/write but madvise don't need, so accesses will trigger a uffd fault. We then recognize that the fault address is in the stack, and then call zeropage to both zero out the memory and resolve the fault, so that the instance's thread wakes up and can make progress once again.

fst-crenshaw · 2020-05-16T01:57:18Z

Now that CircleCI is running mmap and userfaultfd tests, will we be able to merge this?

acfoltzer · 2020-05-18T16:31:55Z

Now that CircleCI is running mmap and userfaultfd tests, will we be able to merge this?

CircleCI experiments are currently only happening on branches, but I'm hoping to get a proper PR together today.

fst-crenshaw · 2020-05-18T18:23:35Z

So excited!

…rting with entrypoint_test.

…nces.

… a fork-related bug.

…yone.

- Instantiates the suite in `lucet_runtime_internals::alloc::tests` for `UffdRegion - Fixes early return issues in `UffdRegion` similar to #455 - Adds a test to show that the per-instance heap limit applies to runtime expansions, not just initial instantiation - Refactors `validate_runtime_spec` to take the per-instance heap limit as an additional argument. This centralizes the logic for rejecting initially-oversized heap limits, and makes it clearer what's happening in each region's instantiation logic. - Removes the `UffdRegion`'s assertion that signal stack size is a multiple of page size. Since the user can now control this as a parameter, we reject it gracefully when validating `Limits` rather than panicking.

Leaving the question of errors in the handler alone for this commit, since that'll be a more major change.

Notably, this should get us building and running uffd in Linux CI. It turns out to be a tremendous pain to enable a feature flag for just one crate within a workspace. The situation is [being addressed][1], but in the meantime I believe the best route forward is to just have uffd on by default for Linux. [1]: rust-lang/cargo#5364

…on variants

Co-authored-by: Adam C. Foltzer <[email protected]>

…d of a host page.

…ization works as expected.

… should handle it gracefully, rather than imploding.

tyler · 2020-05-19T23:58:43Z

Ready for review :)

fst-crenshaw

Approve approve approve!

fst-crenshaw · 2020-05-20T05:24:44Z

The GitHub Actions tests are failing since they can't run the userfaultfd tests. I thought I'd try a little PR to have the offensive tests get skipped by GitHub Actions, since they are being run by CircleCI. See: #528

fst-crenshaw self-requested a review April 8, 2020 19:46

acfoltzer self-requested a review April 9, 2020 20:48

fst-crenshaw reviewed Apr 9, 2020

View reviewed changes

fst-crenshaw reviewed Apr 10, 2020

View reviewed changes

fst-crenshaw mentioned this pull request Apr 24, 2020

KillSwitch race testing #497

Merged

tyler force-pushed the tyler/uffd-integration branch from 6e60af6 to fc3049a Compare May 1, 2020 23:25

tyler and others added 19 commits May 19, 2020 13:35

Pull in the code for Uffd and make it compile

c284f02

Start converting tests to accept multiple Region implementations. Sta…

5abad26

…rting with entrypoint_test.

Reverse order of key:value mapping in test macro. Just feels better.

08f91d7

Convert globals_test to accept multiple region implementations

bcaa22b

Get UffdRegion up to speed with changes to the memory layout of insta…

80f6ec7

…nces.

Make the guest_fault test suite work with UffdRegion. Included fixing…

595da2e

… a fork-related bug.

Make the host test suite work with mmap and uffd.

ed87baa

Get memory test suite working with uffd and mmap.

05b7729

Make stack test suite work with mmap and uffd.

477fa43

Make start test suite work with uffd and mmap

0a57f8f

Make strcmp test suite work with uffd and mmap

f6862bb

Make c_api and c_api tests work with mmap and uffd.

42ed5aa

Put uffd behind a feature flag so we don't break compilation for ever…

03a70c9

…yone.

Use the crates.io version of userfaultfd package

1d76d9c

Use uffd to make the stack lazily allocated

98881d1

Add some documentation to UffdRegion

4b88270

add some uffd documentation and metadata, panickify a couple errors

e29f743

Leaving the question of errors in the handler alone for this commit, since that'll be a more major change.

acfoltzer and others added 4 commits May 19, 2020 13:35

update userfaultfd dependency so it should run on CI

2cab71c

update Docker base image to bionic

52363a9

disable uffd alloc tests on non-Linux

d666e41

Recover some tests that were lost or broken during rebase

a96cb55

tyler force-pushed the tyler/uffd-integration branch from fc3049a to a96cb55 Compare May 19, 2020 20:39

tyler and others added 7 commits May 19, 2020 13:46

Fix indentation problem from rebase conflict

5f57506

Fix conflict resulting from change to binary build of binaryen

7558673

Add UffdStrategy functionality

36d8011

Adapt tests to use RegionCreate form of 'create' function on all Regi…

1d802e4

…on variants

No record syntax needed on the DefaultUffdStrategy type.

6beaade

Co-authored-by: Adam C. Foltzer <[email protected]>

Add a UFFD Strategy that copies an entire wasm page at a time, instea…

224647f

…d of a host page.

Add uffd_specific memory test that confirms that lazy memory material…

ff1457e

…ization works as expected.

tyler mentioned this pull request May 19, 2020

Allow flexible memory management using Uffd #515

Closed

tyler added 3 commits May 19, 2020 15:51

Switch to RegionCreate syntax in runtime-internal tests for UFFD

a49b47c

Provide more details when an InternalError occurs

39248ca

If the Heap ends earlier than expected, the WasmPageSizedUffdStrategy…

e136d54

… should handle it gracefully, rather than imploding.

fst-crenshaw approved these changes May 20, 2020

View reviewed changes

tyler merged commit c557dd8 into master May 20, 2020

tyler deleted the tyler/uffd-integration branch May 20, 2020 17:18

fst-crenshaw mentioned this pull request May 21, 2020

move mmap into sysdeps #524

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add userfaultfd-based Region backend #492

Add userfaultfd-based Region backend #492

tyler commented Apr 8, 2020 •

edited

Loading

acfoltzer commented Apr 9, 2020

fst-crenshaw Apr 9, 2020

tyler Apr 10, 2020

fst-crenshaw Apr 10, 2020

tyler Apr 10, 2020

tyler Apr 10, 2020

fst-crenshaw Apr 10, 2020

acfoltzer Apr 10, 2020 •

edited

Loading

fst-crenshaw Apr 10, 2020

acfoltzer Apr 10, 2020

fst-crenshaw commented May 16, 2020

acfoltzer commented May 18, 2020

fst-crenshaw commented May 18, 2020

tyler commented May 19, 2020

fst-crenshaw left a comment

fst-crenshaw commented May 20, 2020

	// make the stack read/writable
	(slot.stack, limits.stack_size),

	AddrLocation::Stack => unsafe {
	uffd.zeropage(fault_page as *mut c_void, host_page_size(), true)
	.map_err(\|e\| Error::InternalError(e.into()))?;
	},

Add userfaultfd-based Region backend #492

Add userfaultfd-based Region backend #492

Conversation

tyler commented Apr 8, 2020 • edited Loading

acfoltzer commented Apr 9, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

acfoltzer Apr 10, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fst-crenshaw commented May 16, 2020

acfoltzer commented May 18, 2020

fst-crenshaw commented May 18, 2020

tyler commented May 19, 2020

fst-crenshaw left a comment

Choose a reason for hiding this comment

fst-crenshaw commented May 20, 2020

tyler commented Apr 8, 2020 •

edited

Loading

acfoltzer Apr 10, 2020 •

edited

Loading