Merge #1212

1212: Add support for GDB JIT debugging r=MarkMcCaskey a=MarkMcCaskey

This PR adds support for JIT debugging to Wasmer with the Cranelift backend using a fork of `wasmtime-debug`.

The motivation for this change is partially inspired by the feature in Wasmtime and the implementation is largely derived from Wasmtime's `wasmtime-debug` crate and not included in this PR.  This implementation is currently Cranelift-only (if LLVM has value tracking we can add this there too without too much effort; we'd have to do the value tracking ourselves in Singlepass and I don't have enough context to know how hard that would be) and is based on a generic fork of the `wasmtime-debug` -- which will be published and uploaded in another repo.

This PR started out implementing the [Wasm-DWARF](https://yurydelendik.github.io/webassembly-dwarf/) reading and writing with gimli but after working on it for a few days, reading a chunk of the DWARF spec, seeing that Wasmtime had solved this well, and realizing how long this would likely take, I decided that it didn't make sense to spend the engineering effort there so I made a copy of `wasmtime-debug` and removed some of the less portable Cranelift pieces (very minor changes) and all code relying on data structures from wasmtime.  The resulting crate is completely generic and would work fine with Wasmtime or any other Wasm runtime at the cost of requiring some `transmute`s or a linear pass over the debug data to reconstruct it in terms of the new types exposed by the fork.  Perhaps there's a cleaner way to handle that that I haven't considered.

The integration with the GDB JIT interface is from the LLVM examples (I don't remember if I properly attributed everything in this PR/version of the code -- I still have the other branches locally though which I'll review before shipping this) and some of the code in this PR is from Wasmtime/Cranelift source code such as the sorting of the `ebb`s in `clif_backend::resolver`.  I spent a long time debugging some subtle bugs and ended up using a few things from Wasmtime's integration with `wasmtime-debug` and some bits from `cranelift-wasm` and `cranelift-codegen`.

If there's interest from other people in working on the generic `wasmtime-debug` fork, I'm happy to get other maintainers involved and/or move it to a shared organization.

Special thanks to [Yury Delendik](https://github.com/yurydelendik) and the other `wasmtime-debug` authors for their work on Wasm debugging.  Also shout out to Cranelift for the nice API for tracking variables/data.

### TODO:
- [x] Update attributions file for LLVM, [wasm-dwarf](https://github.com/yurydelendik/wasm-dwarf), and Wasmtime/Cranelift and do another pass over code to make sure we're in compliance with the licenses from the relevant projects and have properly attributed the code used from other projects.
- [x] Adjust API of wasm-debug based on feedback
- [x] Discuss with Nick integration with LLVM
- [x] Discuss with Heyang integration with Singlepass
- [x] Adjust implementation based on feedback from team (traits modified, etc.)
- [x] Clean up some pointer wrangling code
- [x] Add opt-in feature to wasmer-runtime-core to enale wasm-debug so library users who won't use debug info are not affected

# Review

- [x] Add a short description of the the change to the CHANGELOG.md file


Co-authored-by: Mark McCaskey <[email protected]>
Co-authored-by: Mark McCaskey <[email protected]>

ATTRIBUTIONS.md

@@ -1,21 +1,30 @@
 # Wasmer Attributions
 
-Wasmer is a community effort.
-In order to build the best WebAssembly runtime it's our duty to see how other runtimes are approaching the same space
-and get inspired from them on the things that they got right, so Wasmer and its community can benefit from a solid
-foundation.
+Wasmer is a community effort and makes use of code from various other
+projects. Listed below are notable sections of code that are licensed
+from other projects and the relevant license of those projects.
 
-These are the different project that we used as inspiration:
+These are the projects that were used as inspiration and/or that we are using code from:
 
 - [Nebulet](https://github.com/nebulet/nebulet): as the base for creating a great Rust WebAssembly runtime
 - [WAVM](https://github.com/wavm/wavm): for their great integration and testing framework
 - [greenwasm](https://github.com/Kimundi/greenwasm): for their [spectests framework](https://github.com/Kimundi/greenwasm/tree/master/greenwasm-spectest)
-- [wasmtime](https://github.com/CraneStation/wasmtime): for their [mmap implementation](https://github.com/CraneStation/wasmtime/blob/3f24098edc81cd9bf0f877fb7fba018cad0f039e/lib/runtime/src/mmap.rs)
+- [wasmtime](https://github.com/CraneStation/wasmtime):
+  - For their [mmap implementation](https://github.com/CraneStation/wasmtime/blob/3f24098edc81cd9bf0f877fb7fba018cad0f039e/lib/runtime/src/mmap.rs)
+  - For the implementation of the `__jit_debug_register_code` function
+    in Rust, the structure of using Cranelift with the GDB JIT
+    interface including implementation details regarding the structure
+    of generating debug information for each function with Cranelift
+    (for example, the sorting of the extended basic blocks before
+    processing the instructions), and the API for transforming DWARF
+    see [wasm-debug's attribution file](https://github.com/wasmerio/wasm-debug/blob/master/ATTRIBUTIONS.md)
+    for more information
 - [stackoverflow](https://stackoverflow.com/a/45795699/1072990): to create an efficient HashMap with pair keys
 - [Emscripten](https://github.com/kripken/emscripten): for emtests test sources to ensure compatibility
+- [The WebAssembly spec](https://github.com/WebAssembly/spec/tree/master/test): for implementation details of WebAssembly and spectests
 
-We would love to hear from you if you think we can take inspiration from other projects that we haven't covered here.
-😊
+Please let us know if you believe there is an error or omission in
+this list and we will do our best to correct it.
 
 ## Licenses
 

CHANGELOG.md

@@ -2,6 +2,11 @@
 
 ## **[Unreleased]**
 
+- [#1212](https://github.com/wasmerio/wasmer/pull/1212) Add support for GDB JIT debugging:
+  - Add `--generate-debug-info` and `-g` flags to `wasmer run` to generate debug information during compilation. The debug info is passed via the GDB JIT interface to a debugger to allow source-level debugging of Wasm files. Currently only available on clif-backend.
+  - Break public middleware APIs: there is now a `source_loc` parameter that should be passed through if applicable.
+  - Break compiler trait methods such as `feed_local`, `feed_event` as well as `ModuleCodeGenerator::finalize`.
+
 ## 0.14.1 - 2020-02-24
 
 - [#1245](https://github.com/wasmerio/wasmer/pull/1245) Use Ubuntu 16.04 in CI so that we use an earlier version of GLIBC.

Cargo.toml

@@ -80,7 +80,7 @@ serde = { version = "1", features = ["derive"] } # used by the plugin example
 typetag = "0.1" # used by the plugin example
 
 [features]
-default = ["fast-tests", "wasi", "backend-cranelift", "wabt"]
+default = ["fast-tests", "wasi", "backend-cranelift", "wabt", "wasmer-runtime-core/generate-debug-information"]
 "loader-kernel" = ["wasmer-kernel-loader"]
 debug = ["fern", "log/max_level_debug", "log/release_max_level_debug"]
 trace = ["fern", "log/max_level_trace", "log/release_max_level_trace"]
@@ -89,6 +89,7 @@ docs = ["wasmer-runtime/docs"]
 fast-tests = []
 backend-cranelift = [
     "wasmer-clif-backend",
+    "wasmer-clif-backend/generate-debug-information",
     "wasmer-runtime/cranelift",
     "wasmer-middleware-common-tests/clif",
 ]

Makefile

@@ -219,6 +219,7 @@ check: check-bench
 	# builds, test as many combined features as possible with each backend
 	# as default, and test a minimal set of features with only one backend
 	# at a time.
+	cargo check --manifest-path lib/runtime-core/Cargo.toml
 	cargo check --manifest-path lib/runtime/Cargo.toml
 	# Check some of the cases where deterministic execution could matter
 	cargo check --manifest-path lib/runtime/Cargo.toml --features "deterministic-execution"