From e4b056517e6ac4e8f9301b54db466854ca36ad91 Mon Sep 17 00:00:00 2001 From: Wenyong Huang Date: Tue, 25 Jan 2022 00:17:21 +0800 Subject: [PATCH 1/7] Refactor documents --- README.md | 94 +++++++++++++++++++++++++++---------------- doc/build_wamr.md | 62 +++++++++++++++++++++++++--- doc/build_wasm_app.md | 18 ++++++--- 3 files changed, 128 insertions(+), 46 deletions(-) diff --git a/README.md b/README.md index 8ac6735189..8d28314e4b 100644 --- a/README.md +++ b/README.md @@ -1,46 +1,55 @@ WebAssembly Micro Runtime ========================= -[Build WAMR VM core](./doc/build_wamr.md) | [Embed WAMR](./doc/embed_wamr.md) | [Export native function](./doc/export_native_api.md) | [Build WASM applications](./doc/build_wasm_app.md) | [Samples](https://github.com/bytecodealliance/wasm-micro-runtime#samples) +[Build WAMR](./doc/build_wamr.md) | [Build AOT Compiler](./README.md#build-wamrc-aot-compiler) | [Embed WAMR](./doc/embed_wamr.md) | [Export Native API](./doc/export_native_api.md) | [Build WASM Apps](./doc/build_wasm_app.md) | [Samples](./README.md#samples) **A [Bytecode Alliance][BA] project** [BA]: https://bytecodealliance.org/ -WebAssembly Micro Runtime (WAMR) is a standalone WebAssembly (WASM) runtime with a small footprint. It includes a few parts as below: -- The **"iwasm" VM core**, supporting WebAssembly interpreter, ahead of time compilation (AoT) and Just-in-Time compilation (JIT) +WebAssembly Micro Runtime (WAMR) is a lightweight standalone WebAssembly (WASM) runtime with small footprint, high performance and highly configurable features for applications cross from embedded, IoT, edge to Trusted Execution Environment (TEE), smart contract, cloud native applications and so on. It includes a few parts as below: +- The [**"iwasm" VM core**](./README.md#iwasm-vm-core) to run WASM applications, supporting interpreter mode, AOT mode (Ahead of Time compilation) and JIT mode (Just-in-Time compilation) -- The **application framework** and the supporting API's for the WASM applications +- The [**"wamrc" AOT compiler**](./README.md#build-wamrc-aot-compiler) to compile WASM file into AOT file for best performance and smaller footprint, which is run by "iwasm" VM Core -- The **dynamic management** of the WASM applications +- The [**application framework**](./README.md#application-framework) and the supporting APIs for the WASM applications + +- The [**dynamic management**](./README.md#remote-application-management) of the WASM applications + +Getting started +================== +- [Build iwasm VM core](./doc/build_wamr.md) on [Linux](./doc/build_wamr.md#linux), [SGX](./doc/linux_sgx.md), [MacOS](./doc/build_wamr.md#macos) and [Windows](./doc/build_wamr.md#windows), and [Build wamrc AOT compiler](./README.md#build-wamrc-aot-compiler) +- [Embed WAMR into host applications](./doc/embed_wamr.md) +- [Register native APIs for WASM applications](./doc/export_native_api.md) +- [Build WASM applications](./doc/build_wasm_app.md) +- [Port WAMR to a new platform](./doc/port_wamr.md) +- [Benchmarks](./tests/benchmarks) and [Samples](./samples) iwasm VM core ========================= -### key features - -- 100% compliant to the W3C WASM MVP -- Small runtime binary size (85K for interpreter and 50K for AoT) and low memory usage -- Near to native speed by AoT -- Self-implemented module loader enables AoT working cross Linux, SGX and MCU systems -- Choices of WASM application libc support: the built-in libc subset for the embedded environment or [WASI](https://github.com/WebAssembly/WASI) for standard libc -- [Embeddable with the supporting C API's](./doc/embed_wamr.md) -- [The mechanism for exporting native API's to WASM applications](./doc/export_native_api.md) -- [Multiple modules as dependencies](./doc/multi_module.md), ref to [sample](samples/multi-module) -- [Thread management and pthread library](./doc/pthread_library.md), ref to [sample](samples/multi-thread) -- [Linux SGX (Intel Software Guard Extension) support](./doc/linux_sgx.md) -- [Source debugging](./doc/source_debugging.md) -- [XIP (Execution In Place) support](./doc/xip.md) - -### post-MVP features -- [Non-trapping float-to-int conversions](https://github.com/WebAssembly/nontrapping-float-to-int-conversions) -- [Sign-extension operators](https://github.com/WebAssembly/sign-extension-ops) -- [Bulk memory operations](https://github.com/WebAssembly/bulk-memory-operations) -- [Shared memory](https://github.com/WebAssembly/threads/blob/main/proposals/threads/Overview.md#shared-linear-memory) -- [Multi-value](https://github.com/WebAssembly/multi-value) +### Key features + +- Full compliant to the W3C WASM MVP +- Small runtime binary size (~85K for interpreter and ~50K for AOT) and low memory usage +- Near to native speed by AOT and JIT +- Self-implemented AOT module loader to enable AOT working on Linux, Windows, MacOS, Android, SGX and MCU systems +- Choices of WASM application libc support: the built-in libc subset for the embedded environment or [WASI](https://github.com/WebAssembly/WASI) for the standard libc +- [The simple C APIs to embed WAMR into host environment](./doc/embed_wamr.md), see [how to integrate WAMR](./doc/embed_wamr.md) and the [API list](./core/iwasm/include/wasm_export.h) +- [The mechanism to export native APIs to WASM applications](./doc/export_native_api.md), see [how to register native APIs](./doc/export_native_api.md) +- [Multiple modules as dependencies](./doc/multi_module.md), ref to [document](./doc/multi_module.md) and [sample](samples/multi-module) +- [Multi-thread, pthread APIs and thread management](./doc/pthread_library.md), ref to [document](./doc/pthread_library.md) and [sample](samples/multi-thread) +- [Linux SGX (Intel Software Guard Extension) support](./doc/linux_sgx.md), ref to [document](./doc/linux_sgx.md) +- [Source debugging support](./doc/source_debugging.md), ref to [document](./doc/source_debugging.md) +- [WAMR-IDE](./test-tools/wamr-ide) to develop WebAssembly applications with build, run and debug support, ref to [document](./test-tools/wamr-ide) +- [XIP (Execution In Place) support](./doc/xip.md), ref to [document](./doc/xip.md) + +### WASM post-MVP features - [wasm-c-api](https://github.com/WebAssembly/wasm-c-api), ref to [document](doc/wasm_c_api.md) and [sample](samples/wasm-c-api) -- [Tail-call](https://github.com/WebAssembly/tail-call) - [128-bit SIMD](https://github.com/WebAssembly/simd), ref to [samples/workload](samples/workload) - [Reference Types](https://github.com/WebAssembly/reference-types), ref to [document](doc/ref_types.md) and [sample](samples/ref-types) +- [Non-trapping float-to-int conversions](https://github.com/WebAssembly/nontrapping-float-to-int-conversions) +- [Sign-extension operators](https://github.com/WebAssembly/sign-extension-ops), [Bulk memory operations](https://github.com/WebAssembly/bulk-memory-operations) +- [Multi-value](https://github.com/WebAssembly/multi-value), [Tail-call](https://github.com/WebAssembly/tail-call), [Shared memory](https://github.com/WebAssembly/threads/blob/main/proposals/threads/Overview.md#shared-linear-memory) ### Supported architectures and platforms @@ -49,14 +58,13 @@ The iwasm supports the following architectures: - X86-64, X86-32 - ARM, THUMB (ARMV7 Cortex-M7 and Cortex-A15 are tested) - AArch64 (Cortex-A57 and Cortex-A53 are tested) -- MIPS -- XTENSA - RISCV64, RISCV32 (RISC-V LP64 and RISC-V LP64D are tested) +- XTENSA, MIPS, ARC -Following platforms are supported. Refer to [WAMR porting guide](./doc/port_wamr.md) for how to port WAMR to a new platform. +Following platforms are supported, click links below for how to build iwasm on that platform. Refer to [WAMR porting guide](./doc/port_wamr.md) for how to port WAMR to a new platform. - [Linux](./doc/build_wamr.md#linux), [Linux SGX (Intel Software Guard Extension)](./doc/linux_sgx.md), [MacOS](./doc/build_wamr.md#macos), [Android](./doc/build_wamr.md#android), [Windows](./doc/build_wamr.md#windows) -- [Zephyr](./doc/build_wamr.md#zephyr), [AliOS-Things](./doc/build_wamr.md#alios-things), [VxWorks](./doc/build_wamr.md#vxworks), [NuttX](./doc/build_wamr.md#nuttx), [RT-Thread](./doc/build_wamr.md#RT-Thread) +- [Zephyr](./doc/build_wamr.md#zephyr), [AliOS-Things](./doc/build_wamr.md#alios-things), [VxWorks](./doc/build_wamr.md#vxworks), [NuttX](./doc/build_wamr.md#nuttx), [RT-Thread](./doc/build_wamr.md#RT-Thread), [ESP-IDF](./doc/build_wamr.md#esp-idf) ### Build iwasm VM core (mini product) @@ -85,6 +93,27 @@ cmake --build . --config Release # wamrc.exe is generated under .\Release directory ``` +### Performance and Footprint + +- **Performance and footprint data**: checkout [here](./wiki/Performance) for the performance and footprint data. +- **Memory profiling**: checkout [here](./doc/build_wamr.md#enable-memory-profiling-experiment) for how to profile the memory usage. +- **Performance profiling**: checkout [here](./doc/build_wamr.md#enable-performance-profiling-experiment) for how to profile the performance. +- **Benchmarks**: checkout these links for how to run the benchmarks: [PolyBench](./tests/benchmarks/polybench), [CoreMark](./tests/benchmarks/coremark), [Sightglass](./tests/benchmarks/sightglass), [JetStream2](./tests/benchmarks/jetstream). + +### User cases + +WAMR is widely used in a lot areas, here are some cases: +- [Hyperledger Private Data Objects](https://github.com/hyperledger-labs/private-data-objects/blob/main/common/interpreter/wawaka_wasm/README.md) +- [Inclavare Containers](https://github.com/alibaba/inclavare-containers) +- [Fassm](https://github.com/faasm/faasm) +- [Waft](https://developer.aliyun.com/article/787582) +- [Envoy Proxy](https://github.com/envoyproxy/envoy) +- [Apache Teaclave](https://teaclave.apache.org/docs/executing-wasm) + +The 2021 [WebAssembly Open Day](https://www.sofastack.tech/activities/sofa-meetup-13) was hold, check out [here](https://www.sofastack.tech/activities/sofa-meetup-13) for the details, and here are the videos: +- [Video1](https://www.bilibili.com/video/BV1EZ4y1X7md?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video2](https://www.bilibili.com/video/BV1p44y1J7ag?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video3](https://www.bilibili.com/video/BV1j34y1r7CR?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video4](https://www.bilibili.com/video/BV1gY411p7gx?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0) +- [Video5](https://www.bilibili.com/video/BV1gS4y1M7kF?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video6](https://www.bilibili.com/video/BV1NF411B79W?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video7](https://www.bilibili.com/video/BV1dL411j7uB?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0) + Application framework =================================== @@ -150,13 +179,10 @@ exception. See the LICENSE file for details. This license allows you to freely use, modify, distribute and sell your own products based on WAMR. Any contributions you make will be under the same license. - - # More resources Check out the [Wiki documents ](https://github.com/bytecodealliance/wasm-micro-runtime/wiki) for more resources: -- [Performance and footprint data](https://github.com/bytecodealliance/wasm-micro-runtime/wiki/Performance) - [Community news and events](https://github.com/bytecodealliance/wasm-micro-runtime/wiki/Events) - [Roadmap](https://github.com/bytecodealliance/wasm-micro-runtime/wiki/Roadmap) - [WAMR TSC meetings](https://github.com/bytecodealliance/wasm-micro-runtime/wiki/TSC-meeting) diff --git a/doc/build_wamr.md b/doc/build_wamr.md index ff6edf9f44..4698d91bf5 100644 --- a/doc/build_wamr.md +++ b/doc/build_wamr.md @@ -203,13 +203,23 @@ mkdir build cd build cmake .. make +# iwasm is generated under current directory ``` - -By default in Linux, the interpreter, AOT and WASI are enabled, and JIT is disabled. +By default in Linux, the `fast interpreter`, `AOT` and `Libc WASI` are enabled, and JIT is disabled. And the build target is set to X86_64 or X86_32 depending on the platform's bitwidth. -To enable WASM JIT, firstly we should build LLVM: +To run a wasm file with interpreter mode: +```Bash +iwasm +``` +To run a AOT file, firstly please refer to [Build wamrc AOT compiler](../README.md#build-wamrc-aot-compiler) to build wamrc, and then: +```Bash +wamrc -o +iwasm +``` + +To enable the `JIT` mode, firstly we should build LLVM: ``` Bash cd product-mini/platforms/linux/ @@ -222,14 +232,23 @@ Then pass argument `-DWAMR_BUILD_JIT=1` to cmake to enable WASM JIT: mkdir build cd build cmake .. -DWAMR_BUILD_JIT=1 +# or "cmake .. -DWAMR_BUILD_JIT=1 -DWAMR_BUILD_LAZY_JIT=0" to disable LLVM Lazy JIT and enable LLVM MC JIT make ``` -By default, the Lazy JIT is enabled to speedup the lanuching process and reduce the JIT compilation time +By default, the LLVM Orc Lazy JIT is enabled to speedup the lanuching process and reduce the JIT compilation time by creating threads to compile the WASM functions parallely, and for the main thread, the functions in the module will not be compiled until they are firstly called and haven't been compiled by the compilation threads. -To disable it, please pass argument `-DWAMR_BUILD_LAZY_JIT=0` to cmake. +To disable it and enable LLVM MC JIT instead, please pass argument `-DWAMR_BUILD_LAZY_JIT=0` to cmake. +To disable `fast interpreter` and enable `classic interpreter` instead: +``` Bash +mkdir build +cd build +cmake .. -DWAMR_BUILD_FAST_INTERP=0 +make +``` + Linux SGX (Intel Software Guard Extension) ------------------------- @@ -252,8 +271,23 @@ mkdir build cd build cmake .. make +# iwasm is generated under current directory +``` +By default in MacOS, the `fast interpreter`, `AOT` and `Libc WASI` are enabled, and JIT is disabled. +And the build target is set to X86_64 or X86_32 depending on the platform's bitwidth. + +To run a wasm file with interpreter mode: +```Bash +iwasm +``` +To run a AOT file, firstly please refer to [Build wamrc AOT compiler](../README.md#build-wamrc-aot-compiler) to build wamrc, and then: +```Bash +wamrc -o +iwasm ``` Note: +For how to build the `JIT` mode and `classic interpreter` mode, please refer to [Build iwasm on Linux](./build_wamr.md#linux). + WAMR provides some features which can be easily configured by passing options to cmake, please see [WAMR vmcore cmake building configurations](./build_wamr.md#wamr-vmcore-cmake-building-configurations) for details. Currently in MacOS, interpreter, AoT, and builtin libc are enabled by default. Windows @@ -271,8 +305,24 @@ mkdir build cd build cmake .. cmake --build . --config Release +# ./Release/iwasm.exe is generated ``` -The executable file is `build/Release/iwasm.exe` + +By default in Windows, the `fast interpreter`, `AOT` and `Libc WASI` are enabled, and JIT is disabled. + +To run a wasm file with interpreter mode: +```Bash +iwasm.exe +``` +To run a AOT file, firstly please refer to [Build wamrc AOT compiler](../README.md#build-wamrc-aot-compiler) to build wamrc, and then: +```Bash +wamrc.exe -o +iwasm.exe +``` +Note: +For how to build the `JIT` mode and `classic interpreter` mode, please refer to [Build iwasm on Linux](./build_wamr.md#linux). + +WAMR provides some features which can be easily configured by passing options to cmake, please see [WAMR vmcore cmake building configurations](./build_wamr.md#wamr-vmcore-cmake-building-configurations) for details. Currently in Windows, interpreter, AoT, and builtin libc are enabled by default. VxWorks ------------------------- diff --git a/doc/build_wasm_app.md b/doc/build_wasm_app.md index cff4afd2a2..4ed548977b 100644 --- a/doc/build_wasm_app.md +++ b/doc/build_wasm_app.md @@ -1,12 +1,13 @@ +# Build WASM applications - -# Prepare WASM building environments +Prepare WASM building environments +================================== For C and C++, WASI-SDK version 12.0+ is the major tool supported by WAMR to build WASM applications. Also, we can use [Emscripten SDK (EMSDK)](https://github.com/emscripten-core/emsdk), but it is not recommended. And there are some other compilers such as the standard clang compiler, which might also work [here](./other_wasm_compilers.md). To install WASI SDK, please download the [wasi-sdk release](https://github.com/CraneStation/wasi-sdk/releases) and extract the archive to default path `/opt/wasi-sdk`. -The official *wasi-sdk release* doesn't fully support *latest 128-bit SIMD spec* yet. WARM provides a script in [build-wasi-sdk](../test-tools/build-wasi-sdk/) to generate +The official *wasi-sdk release* doesn't fully support *latest 128-bit SIMD spec* yet. WAMR provides a script in [build-wasi-sdk](../test-tools/build-wasi-sdk/) to generate another wasi-sdk with *llvm-13* from source code and installs it at *../test-tools/wasi-sdk*. If you plan to build WASM applications with *latest 128-bit SIMD*, please use it instead of the official release. And [sample workloads](../samples/workload) are using the self-compiled wasi-sdk. @@ -37,7 +38,7 @@ $ cargo build --release --target wasm32-wasi Build WASM applications with wasi-sdk -========================= +===================================== You can write a simple ```test.c``` as the first sample. @@ -165,7 +166,12 @@ Please ref to [pthread library](./pthread_library.md) for more details. ## 4. Build wasm app with SIMD support -Normally we should install emsdk and use its SSE header files, please ref to workload samples, e.g. [bwa CMakeLists.txt](../samples/workload/bwa/CMakeLists.txt) and [wasm-av1 CMakeLists.txt](../samples/workload/wasm-av1/CMakeLists.txt) for more details. +The official *wasi-sdk release* doesn't fully support *latest 128-bit SIMD spec* yet. WARM provides a script in [build-wasi-sdk](../test-tools/build-wasi-sdk/) to generate +another wasi-sdk with *llvm-13* from source code and installs it at *../test-tools/wasi-sdk*. If you plan to build WASM applications with *latest 128-bit SIMD*, please use it instead of the official release. + +And also you can install emsdk and use its SSE header files, please ref to workload samples, e.g. [bwa CMakeLists.txt](../samples/workload/bwa/CMakeLists.txt) and [wasm-av1 CMakeLists.txt](../samples/workload/wasm-av1/CMakeLists.txt) for more details. + +For both wasi-sdk and emsdk, please add the option `-msimd128` for clang or emcc to generate WASM application with SIMD bytecodes. # Build WASM applications with emsdk @@ -326,7 +332,7 @@ Examples: wamrc -o test.aot test.wasm Run WASM app in WAMR mini product build -======================== +======================================= Run the test.wasm or test.aot with WAMR mini product build: ``` Bash From 501c9831bd99438702d1c17d88678833af855ad1 Mon Sep 17 00:00:00 2001 From: Wenyong Huang Date: Tue, 25 Jan 2022 17:33:46 +0800 Subject: [PATCH 2/7] Update document --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 8d28314e4b..20f01671b5 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ WebAssembly Micro Runtime WebAssembly Micro Runtime (WAMR) is a lightweight standalone WebAssembly (WASM) runtime with small footprint, high performance and highly configurable features for applications cross from embedded, IoT, edge to Trusted Execution Environment (TEE), smart contract, cloud native applications and so on. It includes a few parts as below: - The [**"iwasm" VM core**](./README.md#iwasm-vm-core) to run WASM applications, supporting interpreter mode, AOT mode (Ahead of Time compilation) and JIT mode (Just-in-Time compilation) -- The [**"wamrc" AOT compiler**](./README.md#build-wamrc-aot-compiler) to compile WASM file into AOT file for best performance and smaller footprint, which is run by "iwasm" VM Core +- The [**"wamrc" AOT compiler**](./README.md#build-wamrc-aot-compiler) to compile WASM file into AOT file for best performance and smaller runtime footprint, which is run by "iwasm" VM Core - The [**application framework**](./README.md#application-framework) and the supporting APIs for the WASM applications From b658878165730c680f51a3bfb060845f18446363 Mon Sep 17 00:00:00 2001 From: Wenyong Huang Date: Tue, 25 Jan 2022 17:49:43 +0800 Subject: [PATCH 3/7] change AoT to AOT, fix grammar error --- README.md | 12 ++++++------ doc/build_wamr.md | 16 ++++++++-------- doc/build_wasm_app.md | 4 ++-- doc/pthread_library.md | 6 +++--- doc/source_debugging.md | 6 +++--- 5 files changed, 22 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 20f01671b5..cf16c6e8c2 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ WebAssembly Micro Runtime [BA]: https://bytecodealliance.org/ WebAssembly Micro Runtime (WAMR) is a lightweight standalone WebAssembly (WASM) runtime with small footprint, high performance and highly configurable features for applications cross from embedded, IoT, edge to Trusted Execution Environment (TEE), smart contract, cloud native applications and so on. It includes a few parts as below: -- The [**"iwasm" VM core**](./README.md#iwasm-vm-core) to run WASM applications, supporting interpreter mode, AOT mode (Ahead of Time compilation) and JIT mode (Just-in-Time compilation) +- The [**"iwasm" VM core**](./README.md#iwasm-vm-core) to run WASM applications, supporting interpreter mode, AOT mode (Ahead-of-Time compilation) and JIT mode (Just-in-Time compilation) - The [**"wamrc" AOT compiler**](./README.md#build-wamrc-aot-compiler) to compile WASM file into AOT file for best performance and smaller runtime footprint, which is run by "iwasm" VM Core @@ -61,18 +61,18 @@ The iwasm supports the following architectures: - RISCV64, RISCV32 (RISC-V LP64 and RISC-V LP64D are tested) - XTENSA, MIPS, ARC -Following platforms are supported, click links below for how to build iwasm on that platform. Refer to [WAMR porting guide](./doc/port_wamr.md) for how to port WAMR to a new platform. +Following platforms are supported, click each link below for how to build iwasm on that platform. Refer to [WAMR porting guide](./doc/port_wamr.md) for how to port WAMR to a new platform. - [Linux](./doc/build_wamr.md#linux), [Linux SGX (Intel Software Guard Extension)](./doc/linux_sgx.md), [MacOS](./doc/build_wamr.md#macos), [Android](./doc/build_wamr.md#android), [Windows](./doc/build_wamr.md#windows) - [Zephyr](./doc/build_wamr.md#zephyr), [AliOS-Things](./doc/build_wamr.md#alios-things), [VxWorks](./doc/build_wamr.md#vxworks), [NuttX](./doc/build_wamr.md#nuttx), [RT-Thread](./doc/build_wamr.md#RT-Thread), [ESP-IDF](./doc/build_wamr.md#esp-idf) ### Build iwasm VM core (mini product) -WAMR supports building the iwasm VM core only (no app framework) to the mini product. The WAMR mini product takes the WASM application file name or AoT file name as input and then executes it. For the detailed procedure, please see **[build WAMR VM core](./doc/build_wamr.md)** and **[build and run WASM application](./doc/build_wasm_app.md)**. Also we can click the link of each platform above to see how to build iwasm on it. +WAMR supports building the iwasm VM core only (no app framework) to the mini product. The WAMR mini product takes the WASM application file name or AOT file name as input and then executes it. For the detailed procedure, please see **[build WAMR VM core](./doc/build_wamr.md)** and **[build and run WASM application](./doc/build_wasm_app.md)**. Also we can click the link of each platform above to see how to build iwasm on it. -### Build wamrc AoT compiler +### Build wamrc AOT compiler -Both wasm binary file and AoT file are supported by iwasm. The wamrc AoT compiler is to compile wasm binary file to AoT file which can also be run by iwasm. Execute following commands to build **wamrc** compiler for Linux: +Both wasm binary file and AOT file are supported by iwasm. The wamrc AOT compiler is to compile wasm binary file to AOT file which can also be run by iwasm. Execute following commands to build **wamrc** compiler for Linux: ```shell cd wamr-compiler @@ -150,7 +150,7 @@ The WAMR [samples](./samples) integrate the iwasm VM core, application manager a - [**basic**](./samples/basic): Demonstrating how to use runtime exposed API's to call WASM functions, how to register native functions and call them, and how to call WASM function from native function. - **[simple](./samples/simple/README.md)**: The runtime is integrated with most of the WAMR APP libraries, and a few WASM applications are provided for testing the WAMR APP API set. It uses **built-in libc** and executes apps in **interpreter** mode by default. -- **[littlevgl](./samples/littlevgl/README.md)**: Demonstrating the graphic user interface application usage on WAMR. The whole [LittleVGL](https://github.com/lvgl/) 2D user graphic library and the UI application are built into WASM application. It uses **WASI libc** and executes apps in **AoT mode** by default. +- **[littlevgl](./samples/littlevgl/README.md)**: Demonstrating the graphic user interface application usage on WAMR. The whole [LittleVGL](https://github.com/lvgl/) 2D user graphic library and the UI application are built into WASM application. It uses **WASI libc** and executes apps in **AOT mode** by default. - **[gui](./samples/gui/README.md)**: Move the [LittleVGL](https://github.com/lvgl/) library into the runtime and define a WASM application interface by wrapping the littlevgl API. It uses **WASI libc** and executes apps in **interpreter** mode by default. - **[multi-thread](./samples/multi-thread/)**: Demonstrating how to run wasm application which creates multiple threads to execute wasm functions concurrently, and uses mutex/cond by calling pthread related API's. - **[spawn-thread](./samples/spawn-thread)**: Demonstrating how to execute wasm functions of the same wasm application concurrently, in threads created by host embedder or runtime, but not the wasm application itself. diff --git a/doc/build_wamr.md b/doc/build_wamr.md index 4698d91bf5..08cccfc0cd 100644 --- a/doc/build_wamr.md +++ b/doc/build_wamr.md @@ -37,7 +37,7 @@ cmake -DWAMR_BUILD_PLATFORM=linux -DWAMR_BUILD_TARGET=ARM NOTE: the fast interpreter runs ~2X faster than classic interpreter, but consumes about 2X memory to hold the WASM bytecode code. -#### **Configure AoT and JIT** +#### **Configure AOT and JIT** - **WAMR_BUILD_AOT**=1/0, default to enable if not set - **WAMR_BUILD_JIT**=1/0, default to disable if not set @@ -99,7 +99,7 @@ cmake -DWAMR_BUILD_PLATFORM=linux -DWAMR_BUILD_TARGET=ARM > Note: if it is enabled, the call stack will be dumped when exception occurs. > - For interpreter mode, the function names are firstly extracted from *custom name section*, if this section doesn't exist or the feature is not enabled, then the name will be extracted from the import/export sections -> - For AoT/JIT mode, the function names are extracted from import/export section, please export as many functions as possible (for `wasi-sdk` you can use `-Wl,--export-all`) when compiling wasm module, and add `--enable-dump-call-stack` option to wamrc during compiling AoT module. +> - For AOT/JIT mode, the function names are extracted from import/export section, please export as many functions as possible (for `wasi-sdk` you can use `-Wl,--export-all`) when compiling wasm module, and add `--enable-dump-call-stack` option to wamrc during compiling AOT module. #### **Enable memory profiling (Experiment)** - **WAMR_BUILD_MEMORY_PROFILING**=1/0, default to disable if not set @@ -213,7 +213,7 @@ To run a wasm file with interpreter mode: ```Bash iwasm ``` -To run a AOT file, firstly please refer to [Build wamrc AOT compiler](../README.md#build-wamrc-aot-compiler) to build wamrc, and then: +To run an AOT file, firstly please refer to [Build wamrc AOT compiler](../README.md#build-wamrc-aot-compiler) to build wamrc, and then: ```Bash wamrc -o iwasm @@ -280,7 +280,7 @@ To run a wasm file with interpreter mode: ```Bash iwasm ``` -To run a AOT file, firstly please refer to [Build wamrc AOT compiler](../README.md#build-wamrc-aot-compiler) to build wamrc, and then: +To run an AOT file, firstly please refer to [Build wamrc AOT compiler](../README.md#build-wamrc-aot-compiler) to build wamrc, and then: ```Bash wamrc -o iwasm @@ -288,7 +288,7 @@ iwasm Note: For how to build the `JIT` mode and `classic interpreter` mode, please refer to [Build iwasm on Linux](./build_wamr.md#linux). -WAMR provides some features which can be easily configured by passing options to cmake, please see [WAMR vmcore cmake building configurations](./build_wamr.md#wamr-vmcore-cmake-building-configurations) for details. Currently in MacOS, interpreter, AoT, and builtin libc are enabled by default. +WAMR provides some features which can be easily configured by passing options to cmake, please see [WAMR vmcore cmake building configurations](./build_wamr.md#wamr-vmcore-cmake-building-configurations) for details. Currently in MacOS, interpreter, AOT, and builtin libc are enabled by default. Windows ------------------------- @@ -314,7 +314,7 @@ To run a wasm file with interpreter mode: ```Bash iwasm.exe ``` -To run a AOT file, firstly please refer to [Build wamrc AOT compiler](../README.md#build-wamrc-aot-compiler) to build wamrc, and then: +To run an AOT file, firstly please refer to [Build wamrc AOT compiler](../README.md#build-wamrc-aot-compiler) to build wamrc, and then: ```Bash wamrc.exe -o iwasm.exe @@ -322,7 +322,7 @@ iwasm.exe Note: For how to build the `JIT` mode and `classic interpreter` mode, please refer to [Build iwasm on Linux](./build_wamr.md#linux). -WAMR provides some features which can be easily configured by passing options to cmake, please see [WAMR vmcore cmake building configurations](./build_wamr.md#wamr-vmcore-cmake-building-configurations) for details. Currently in Windows, interpreter, AoT, and builtin libc are enabled by default. +WAMR provides some features which can be easily configured by passing options to cmake, please see [WAMR vmcore cmake building configurations](./build_wamr.md#wamr-vmcore-cmake-building-configurations) for details. Currently in Windows, interpreter, AOT, and builtin libc are enabled by default. VxWorks ------------------------- @@ -376,7 +376,7 @@ west espressif install After that set `ESPRESSIF_TOOLCHAIN_PATH` according to the output, for example `~/.espressif/tools/zephyr`. Note: -WAMR provides some features which can be easily configured by passing options to cmake, please see [WAMR vmcore cmake building configurations](./build_wamr.md#wamr-vmcore-cmake-building-configurations) for details. Currently in Zephyr, interpreter, AoT and builtin libc are enabled by default. +WAMR provides some features which can be easily configured by passing options to cmake, please see [WAMR vmcore cmake building configurations](./build_wamr.md#wamr-vmcore-cmake-building-configurations) for details. Currently in Zephyr, interpreter, AOT and builtin libc are enabled by default. AliOS-Things diff --git a/doc/build_wasm_app.md b/doc/build_wasm_app.md index 4ed548977b..a8698e11f1 100644 --- a/doc/build_wasm_app.md +++ b/doc/build_wasm_app.md @@ -270,9 +270,9 @@ $ cmake You will get ```hello_world``` which is the WASM app binary. -# Compile WASM to AoT module +# Compile WASM to AOT module -Please ensure the wamrc was already generated and available in your shell PATH. Then we can use wamrc to compile WASM app binary to WAMR AoT binary. +Please ensure the wamrc was already generated and available in your shell PATH. Then we can use wamrc to compile WASM app binary to WAMR AOT binary. ``` Bash wamrc -o test.aot test.wasm diff --git a/doc/pthread_library.md b/doc/pthread_library.md index b1de6b10ff..5243ba1783 100644 --- a/doc/pthread_library.md +++ b/doc/pthread_library.md @@ -90,9 +90,9 @@ emcc -O3 -mbulk-memory -matomics -s MALLOC="none" \ main.c -o test.wasm ``` -**Build AoT module** +**Build AOT module** -You can build the wasm module into AoT module with pthread support, please pass option `--enable-multi-thread` to wamrc: +You can build the wasm module into AOT module with pthread support, please pass option `--enable-multi-thread` to wamrc: ``` bash wamrc --enable-multi-thread -o test.aot test.wasm ``` @@ -105,7 +105,7 @@ cmake .. -DWAMR_BUILD_LIB_PTHREAD=1 make # Then you can run the wasm module above: ./iwasm test.wasm -# Or the AoT module: +# Or the AOT module: # ./iwasm test.aot ``` diff --git a/doc/source_debugging.md b/doc/source_debugging.md index afb03ae9b3..f10bbb3331 100644 --- a/doc/source_debugging.md +++ b/doc/source_debugging.md @@ -53,9 +53,9 @@ Then you can use lldb commands to debug your applications. Please refer to [lldb > Known issue: `step over` on some function may be treated as `step in`, it will be fixed later. -## Debugging with AoT +## Debugging with AOT -> Note: AoT debugging is experimental and only a few debugging capabilities are supported. +> Note: AOT debugging is experimental and only a few debugging capabilities are supported. 1. Build lldb (assume you have already built llvm) ``` bash @@ -80,7 +80,7 @@ cmake .. -DWAMR_BUILD_DEBUG_AOT=1 make ``` -4. Compile wasm module to AoT module +4. Compile wasm module to AOT module ``` bash wamrc -o test.aot test.wasm ``` From e9b972c5d93f31d750eda09122139c8bf0359b1e Mon Sep 17 00:00:00 2001 From: Wenyong Huang Date: Tue, 25 Jan 2022 18:03:15 +0800 Subject: [PATCH 4/7] Add paper link --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index cf16c6e8c2..92820c829d 100644 --- a/README.md +++ b/README.md @@ -114,6 +114,9 @@ The 2021 [WebAssembly Open Day](https://www.sofastack.tech/activities/sofa-meetu - [Video1](https://www.bilibili.com/video/BV1EZ4y1X7md?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video2](https://www.bilibili.com/video/BV1p44y1J7ag?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video3](https://www.bilibili.com/video/BV1j34y1r7CR?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video4](https://www.bilibili.com/video/BV1gY411p7gx?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0) - [Video5](https://www.bilibili.com/video/BV1gS4y1M7kF?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video6](https://www.bilibili.com/video/BV1NF411B79W?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video7](https://www.bilibili.com/video/BV1dL411j7uB?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0) +Papers +- [A fast WebAssembly Interpreter design in WASM-Micro-Runtime](https://www.intel.cn/content/www/cn/zh/developer/articles/technical/webassembly-interpreter-design-wasm-micro-runtime.html) + Application framework =================================== From 92492f8353a492c3425cf1abc04b2c2947391eab Mon Sep 17 00:00:00 2001 From: Wenyong Huang Date: Tue, 25 Jan 2022 23:10:56 +0800 Subject: [PATCH 5/7] Add memory model and memory tune document --- README.md | 10 ++++------ doc/memory_tune.md | 27 +++++++++++++++++++++++++++ doc/pics/wamr_memory_model.png | Bin 0 -> 61716 bytes 3 files changed, 31 insertions(+), 6 deletions(-) create mode 100644 doc/memory_tune.md create mode 100755 doc/pics/wamr_memory_model.png diff --git a/README.md b/README.md index 92820c829d..c9cfba4fef 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ iwasm VM core - [Multi-thread, pthread APIs and thread management](./doc/pthread_library.md), ref to [document](./doc/pthread_library.md) and [sample](samples/multi-thread) - [Linux SGX (Intel Software Guard Extension) support](./doc/linux_sgx.md), ref to [document](./doc/linux_sgx.md) - [Source debugging support](./doc/source_debugging.md), ref to [document](./doc/source_debugging.md) -- [WAMR-IDE](./test-tools/wamr-ide) to develop WebAssembly applications with build, run and debug support, ref to [document](./test-tools/wamr-ide) +- [WAMR-IDE (Experimental)](./test-tools/wamr-ide) to develop WebAssembly applications with build, run and debug support, ref to [document](./test-tools/wamr-ide) - [XIP (Execution In Place) support](./doc/xip.md), ref to [document](./doc/xip.md) ### WASM post-MVP features @@ -96,7 +96,8 @@ cmake --build . --config Release ### Performance and Footprint - **Performance and footprint data**: checkout [here](./wiki/Performance) for the performance and footprint data. -- **Memory profiling**: checkout [here](./doc/build_wamr.md#enable-memory-profiling-experiment) for how to profile the memory usage. +- **Memory usage tunning**: checkout [here](./doc/memory_tune.md) for the memory model and how to tune the memory usage. +- **Memory usage profiling**: checkout [here](./doc/build_wamr.md#enable-memory-profiling-experiment) for how to profile the memory usage. - **Performance profiling**: checkout [here](./doc/build_wamr.md#enable-performance-profiling-experiment) for how to profile the performance. - **Benchmarks**: checkout these links for how to run the benchmarks: [PolyBench](./tests/benchmarks/polybench), [CoreMark](./tests/benchmarks/coremark), [Sightglass](./tests/benchmarks/sightglass), [JetStream2](./tests/benchmarks/jetstream). @@ -112,10 +113,7 @@ WAMR is widely used in a lot areas, here are some cases: The 2021 [WebAssembly Open Day](https://www.sofastack.tech/activities/sofa-meetup-13) was hold, check out [here](https://www.sofastack.tech/activities/sofa-meetup-13) for the details, and here are the videos: - [Video1](https://www.bilibili.com/video/BV1EZ4y1X7md?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video2](https://www.bilibili.com/video/BV1p44y1J7ag?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video3](https://www.bilibili.com/video/BV1j34y1r7CR?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video4](https://www.bilibili.com/video/BV1gY411p7gx?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0) -- [Video5](https://www.bilibili.com/video/BV1gS4y1M7kF?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video6](https://www.bilibili.com/video/BV1NF411B79W?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video7](https://www.bilibili.com/video/BV1dL411j7uB?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0) - -Papers -- [A fast WebAssembly Interpreter design in WASM-Micro-Runtime](https://www.intel.cn/content/www/cn/zh/developer/articles/technical/webassembly-interpreter-design-wasm-micro-runtime.html) +- [Video5](https://www.bilibili.com/video/BV1gS4y1M7kF?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video6](https://www.bilibili.com/video/BV1NF411B79W?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0), [Video7](https://www.bilibili.com/video/BV1dL411j7uB?from=search&seid=6209423751443457915&spm_id_from=333.337.0.0) Application framework =================================== diff --git a/doc/memory_tune.md b/doc/memory_tune.md new file mode 100644 index 0000000000..5db7916c0f --- /dev/null +++ b/doc/memory_tune.md @@ -0,0 +1,27 @@ +Memory model and memory usage tunning +===================================== + +## The memory model + +The memory model of WAMR can be basically described as below: +![WAMR memory model](./pics/wamr_memory_model.png "WAMR memory model") + +Note: +- **global heap**: the heap to allocate memory for runtime data structures, including wasm module, wasm module instance, exec env, wasm operand stack and so on. It is initialized by `wasm_runtime_init` or `wasm_runtime_full_init`. And for `wasm_runtime_full_init`, developer can specify the memory allocation mode with `RuntimeInitArgs *init_args`: allocate memory from a user defined byte buffer, from user defined allocation function, or from the platform's os_malloc function. Refer to [wasm_export.h](../core/iwasm/include/wasm_export.h#L98-L141) and [Embedding WAMR guideline](doc/embed_wamr.md#the-runtime-initialization) for more details. And developer can use `wasm_runtime_malloc/wasm_runtime_free` to allocate/free memory from/to the global heap. +- **wasm operand stack**: the stack to store the operands required by wasm bytecodes as WebAssembly is based on a stack machine. If the exec_env is created by developer with `wasm_runtime_create_exec_env`, then its size is specified by `wasm_runtime_create_exec_env`, or if the exec_env is created by runtime, e.g. exec_env of sub-thread, the size is specified by `wasm_runtime_instantiate`. +- **linear memory**: a contiguous, mutable array of raw bytes. It is created with an initial size but can be grown dynamically. For most compilers, e.g. wasi-sdk, emsdk, rustc or asc, normally it includes three parts, data area, auxiliary stack area and heap area. For wasi-sdk, the initial/max size can be specified with `-Wl,--initial-memory=n1,--max-memory=n2`, for emsdk, the initial/max size can be specified with `-s INITIAL_MEMORY=n1 -s MAXIMUM_MEMORY=n2 -s ALLOW_MEMORY_GROWTH=1` or `-s TOTAL_MEMORY=n`, and for asc, they can be specified with `--initialMemory` and `--maximumMemory` flags. +- **aux stack**: the auxiliary stack resides in linear memory to store some temporary data when calling wasm functions, for example, calling a wasm function with struct arguments. For wasi-sdk, the size can be specified with `-z stack-size=n`, for emsdk, the size can be specified with `-s TOTAL_STACK=n`. +- **app heap and libc heap**: the heap to allocate memory for wasm app, note that app heap is created only when the malloc/free functions (or __new/__release functions for AssemblyScript) are not exported and runtime can not detect the libc heap. To export the malloc/free functions, for wasi-sdk and emsdk, developer can use `-Wl,--export=malloc -Wl,--export=free` options, for asc, developer can use `--exportRuntime` option. For app heap, the size is specified by `wasm_runtime_instantiate`. It is recommended to export the malloc/free functions and not to create app heap in single thread mode, andf or multi-threading, as the libc heap isn't thread-safe, it is recommended to remove the dlmalloc.o from libc.a for wasi-sdk and use `-s MALLOC=“none` for emsdk, refer to [WAMR pthread library](./pthread_library.md) for more details. And developer can use `wasm_runtime_module_malloc/wasm_runtime_module_free` to allocate/free memory from/to app heap (or libc heap if malloc/free functions are exported). +- **_data_end global and __heap_base global**: two globals exported by wasm application to indicate the end of data area and the base address of libc heap. For WAMR, it is recommended to export them as when there are no possible memory grow operations, runtime will truncat the linear memory to the size indicated by `__heap_base`, so as to reduce the footpring, or at least one page (64KB) is required by linear memory. + +## Tune the memory usage + +Normally there are some methods to tune the memory usage: +- set the global heap size with `wasm_runtime_full_init` +- set the wasm operand stack size with `wasm_runtime_create_exec_env` or `wasm_runtime_instantiate` +- set the linear memory size +- set the auxiliary stack size +- export `malloc/free` functions to use libc heap and not create app heap +- set the app heap size with `wasm_runtime_instantiate` +- use `nostdlib` mode, add `-Wl,--strip-all`: refer to [How to reduce the footprint](./build_wasm_app.md#2-how-to-reduce-the-footprint) of building wasm app for more details +- use XIP mode, refer to [WAMR XIP (Execution In Place) feature introduction](./xip.md) for more details diff --git a/doc/pics/wamr_memory_model.png b/doc/pics/wamr_memory_model.png new file mode 100755 index 0000000000000000000000000000000000000000..c9df2bb0eda1bee9290baf4ee24281a01bb6621e GIT binary patch literal 61716 zcmce-WmFvNw?2r5;6a1Cy9Noa!D-wHF2TJC65PET_oneca0mnl?oNO}aQ7g=VVZO9 zz2~0an)&dbwPrrBdR2ANRqwOq-TQfVMX9UGW1^9u!NI{{D!h}?fP+J#fP+JzeTe`& z!+NU54*P(2*N~Tjs~RKQgB>8*NUBJ}!PO+7-AS{dz7(fKxkTBJA&hMf?J~zXC+YOR@k3jS4pdd^`-R__l zXaBn;1kBXzC<#7(rQ%sphSIC6lv0!>wf+ z6>rW<&s%KObTt3@G3Q~FSMZqVL-n^mLU)ICb7Cvlh$tV%__wny#`v!cHWmYq{kIch zx%-GLV!5|}>KQzL!Va)LLBsy*B;5a-lbu1(-qw|eW1C#yc1qs;CKG44PCa?p2MKX5 zG)<;1fS~rvGe_r-i?7>H!vJE!E9ass-7Lmy!W5WYt!LpaXB_;C??uh<-?aViBOMoi zYTI8>o!Qm1sqP*dZHb<{HV!%Hm2Elu@Mrbnp#|ya+u_OwuUcaFj)xY@%gs7nx;`Rz zJhE$ezpWz29e;P~3q%N4?_LRcb=w&lPW$zG48)Ig7YGbH@Zo!TGDNyP)H>r3av2Jk zKaKZplLqH|$oC6>B=>R}n+}Xykpx{+Y%&DCx_vYX{$ujzg#YbAY|F}{QM2#Qo2Khr z5O-f^5M`&X>d(|AfwccwpTfmR>#zQh+_aKxKfGnhH?b|KAk>-^XLHh%rRV30|8YjTuT+ph~>taJ85Dc@I;ox{B7jjF@@_?K{(aAXj)z7|Z39{k-@ecN9A)B8J1 z!Ht_KCC<1rE6mS=6D}9U@9`1&J6-7EBr9&fD84b;v&#IoE%87>;~ZCWG!>EX=#=TY3BpTh^+}H_nCSJfl^l?F((}VCkImFzt?r~x6jC836PC$-x(&{gk z#YlN~n>4B%ZpS4jHu>5PpK3E20zKFwVvF%A&6u;1SfU2CFT}f4-=2l!$XCcWO-A{u`|o|$)u6*5q?2E6 zG77);WT8atueTERAj&leYBT2lz>oU!lW%cC_vX*Bd?((_{2wic6&ee#i{=hx{NuZ< zuty@TW(eR@1sW^zlfzw&B42tMZ#$E~+z~QjwT68B%e8gf#gLN1@ObR4Av}Nc9v~?y zi`iv|yaCl9Zkkj}=s7qjG@>Hb1(Q~TO6o+OT=$_ww?E8 z;C5(@*G>O8VimCAnL6U(mUP~mV`N!O_5k-g0Wa#M@|{kf%WPcE0U6F@W$hM~2v3|o3%bn3^yZE0-{_)dUR&g52mN#{EmSbiSM%)Nql|@k#b-{?RTwM4$TLRVXIERvS@^ z)hUdffH~76JV4EhSHRszmLXacMv*EqzkWlKphi%zlC#lHd;)RFN0C-d=kQ$+(Y0P8 zN_*@M0$fn2&2$!LwJ0!4QTd2Qn-l)FER>EGvhJfHQpeei=$w%Wnxrd9@Tba+^bHuX z+_dLGBd!1R>Y)dc=2Jg|l%M;1f$!`4!J+BXLdYnDX&bji$w;0>pf1obw^nW>>ir9~ zVqzQSay5PwVO`9J0ruOG;Zm)v+n3>0eCRrQ{K(;}Y#(XmY}Q-UJ}Z0Zfs6A>`do8o zWKYyOgy&zB)A=+^MLRB1q}bBzhl} zrxbA9wRwSlRYSg+|K9p_T(!rfSOGJ$=REq@-g~rm&YS-!PS* zUb%GUl3z81WHYfLeyipFLL-?@Db$1>G5c#fw>(XOTz}A;FYyHP%KND~l6Y(%A=igw9&6i&r71kVz`ecN z?T<0x!-Gb}4v62DKpTDQ(?Ze*&6-X+KXG}LhV%!rBz5c9GR>JjT8BFtN?lq_`qIlr z=Fq~zlXR$oaAwnWE9Hdc=bOY5T1-MG6rkwX;dpJeqLpc)Ngdv$5L%ijsNd)$)jW4~ z6JO$TIIRn|pQ?Vl8_$>L^zLjvsbF9Yb))@ZLG>0%BwUCeE$hzOdm}@|e_T8`;3M=U zge#1Ott|a`1Lse1f?Zch_ky2v9@)!n&7c_W+<4O(aBN8{m~p5JtUEM6%`7@*QJXnT z_ps0yS37`~9;Ag)x+H||7~A#}nZ4F;9%|K~LYonYSC!{_s-oljQx~JnB)A>e@*}klM-)DeObaudov6%zr0-QY}n94 zkjAmE9(xy7%V3XwA?@s+nN3p!n9r{eYANTf0H`x=IU=MVw&`lH0VG>)Cq8^4*_Sul z=dp{b#Rd^+Wf$>f7Z{-z_zWqg4&h>4WagBP1DX#D96nJ{_lkTUWd5|D#tADs$g+K} z@`7wWQS^oRadHT(XDk}Ed~*+2eS%?R`2vzgiVn_WjNj3yElsU-wCYI%5~w;tVz$t`%{?00rNcH59=(|Z=BKF z=TPX=^F2xva6pz20YMJ<*B3$|Ak@21PwU@hNis8cZ2CWs+Vr{k9%AuG8QIfq1irvI~>bb9(a%*{@wMdBH ztQoD=_X}bdX^YYs1%;5+AI_j$ed!^*zu}f&B#2T3&?(<6Z5Lv`4 zJO>=<`u%j%Qlz9p6h(KCQ=pJl(MzcNFZau0Dkk26vT%w&TaV}vF4>mPF>da<_USeN zbvNmXK|WdA{C8)e47qrb2sW@H6f`g6tvN3AiJsL|M~~}uL!9K5f3`dMZ<|b@hn4eW z3C*6rMuv?f4ugQ47i*6vj#XYMP$swzx7DpC^lTub25m0mZY*w2pJ&=}PCy4t1FGUd zi*@80!aVx(>{sh`?h`kwHaZH3?6BMvHX^J(P9}duNM!Kag+EFX94kdXmr1g(jq364 zVPv#i%_z$S|0gr!zVJ6i|6& zC$k%qCCyjA1c3J8q$rA5XLV17)p1$JUqrkwOGBD>_Jv-{-cEK4iet^XkL{yvHF6A% z8hhJ(rIE6Ni)Jj&kEJ`l!zhWuOR$$fEW?}a={ZM)=J)n?H|Hj8_ zy)uL6^=Foz{OsrfTI2jCM?6525vR~}HqNXx66DiIY^`m5+cHKNti!9_6F4lrZr=p+ ziOB8SfdYgcfaac>YQbG>PuHB53JROdiA0je?Ks`l!{m%|Ir6^CoD~+2fnmj82$xw- z?LV3G;G7+&&@pNTk3#GpIhc+$?gwvm7T{Gp5loiT7e8SlY-R0pd8W z_`851coRgEkVTL*VaWGg6Q}$o*#_+K>&>muG7tauQ5(lH#uqu}zZ=*R<%{$R5qh7$ zr=p*kYHTv!T@K7^AmQ*6RV@rs^!Zxjl2MPyD3IkxWFWiC|CSkV4r!QR4@YB!6>`w~ zCT1>Xci1YHTYkIj?VZR{*G1c55272-Q4Fa^u?T8ufxD;?ul}3e;*3dAl6BQ`d4nou z(BfD1@tbXmgjo!n{mSDzVwyQVi*=IH@f;E%<-&J*eSqb&)rE`tyRtURt=T}y?>mx% zQy%5&6b?bY12S=6wL2ep05HGZ|KT0>b;v1WP<@Y1YE-G5iy>;>C;Ezrif671yo2>z zbGsD#lUh=~Px5vG4|bYE?1TRXl0JmxG5B^;QI#fQS%|-zw&3H>MgoqW2lk4Wm_5ny zXWKK9K;a@ow-t{0d8y-%b(;+RbkAoSuAl&ś}yNKj*&C`S}L=BzE9$ znvgU2AOx(KMC);5E_l6Y@1?;Y29)&;#UCEpWXB&Ra9PfKE=N+AF+CTzs10~R9Ain5S0Q}Xkj8#VOCW*#j3g&KGXlAvsiAET=Lc9vM?FV?g zDE7IQv!&kqK32ADW)w6N@@-w$PihW`*_XE@J8v#J3n^z0h!`04{9ye{chJJ*`R4Wr z*DGUiMW+PRMp6`ue#Vi@G<(k-B)~*rPP(PE%lo5krIIPNC5@=mk_Pg{j7RJ9Us-do zC{Uk!-I5cG1KVQWFM_po*Gzx@`3u-VeO4`Y&M&bBvthRRQ;b}uRTdXfhAR$VBe{k& z`OiU7AA+}{FJ3aIf~gHWk?qH7zid;XSj4svz+k@1QFY;jv@e}ufOh)bF50q{A7d9{ z1}l3lL9cJOtjFN|^+I<^tYBATrf#?U-t9Z?8LpPkd=xQuOJ`L=7o}(`+P>8@7=d^z z-=bjkRpR)+w9aveZ882c;J?CqY4u^%i_p9qNjH^H!cZ$a@6~puo`HijEA5!IzsLB( z3!8|Q8=B98-72OB+#`fe7nro8zfKg0Eu3;5+*V+{EWP+Q!QA`fc6jy~CL(vzzy9|e z*2-^WAHPo}^wx*58Y5AwOLKpd8}4vy?El4hpma{binHf~|C5u&&|cmD`T^_c0%DR> z!oTQ~zI0Kd%mOnzv%kMv2>Ji{tB5p0%BSo8CqwtE>x&X`QLoOjv&D{wMOqz;vCNu{ zI9P@!wXhBEgHbbCFmB*%nDc2Qq2sJh+^kjlk0#|!4dvq+B}LY%*3Bg4!(_)FSOs1u zei=RL)%RzO@-Au4VvG1=#+!4#ApR zC4sjkfjhtZA9t}IbD#getmC{c@4APvuUG&El~ed%$TT9RL&HYBXsn+R+kS~LI`O6c z3wb9SyF()$w-P$+PV(<T)8V?HDVBAkE7My9R|z2|epbX5`s`BRGEp z|Mrub`CkT!@0`HER6<{bhh1X*7N@SB3ua}caBEvVgLy2NTfA9}g54hm3N06( zboW%37hGNamj%_Hznp_C=>Pl?UFQye1#TbY0k<>M+W$yH1Nu%QG+NTQ$wd9e@7h1l znD#H<-}mP}6+1J5D;g~nW3D=$9*l4v^l_>?9);6Rj8A;$m{#B}upwVmZ}BNa zeJ_^W$IDEz^7uE?FG1blN%Z~OhsdxwU8ewotek%!JI>6BtDlfj+$tZP>M|l$ilM(? zUg`)m1?P`C+nF8zgIP?gbrTJ{Qx_Y1RY+JUw_*?AbpoY0g?7Zom#ISDG8#7a5vhML z@*ZVlSN#D)M2JBkGxC5QW(U<27&1-F>|!@V*rHE19YN$6Cq8#`Z9C)s=UI6->8h?0 zn3vTLt5;TmKr@ZCy7Umwx#1$S0)`j@r;@JIWr@SA%3fOdqOw2R0Kf~BecGCRB z-*R^Fc+t!6yZ6yFHd8?CaV_TH?(+&6e>Rx3?P%HX8S75QG4tyCRqqJIhmPx`l_5RX z)I&lDD&|&5o$qz-l|Sj@%*4?cC&F6z_y@|HcA^+8;+M1rzWL$HE=-v2K4cd`zSyz` z$}DjjgWNs(v6|dgU#%kB*nXY%^LA}~qEXRP>n+x1$#F3ByG)H@Q078FWJrFkf_XFS zIX4R*{M1z)z~^UaWi~vtL0K`h((YJg-NeeSs8L9Rjf|D+|3^PQ2r=62Wv&QNs^Z#w z(0yF=YIhRi>V%lHtHj`Pvi$9}iBAy2#!|Q~!HO>>V{W)AQCQ;zm4H1Tp|nimw?L}H z-E39GW9Y2U{WUj=3{#%2sXth=#SB9?U)zKxn3$2Hpk4VDI z%LY1c<|LilE8s7|2~bl(j3C<4SNA__-4Po`7R&mRwU0s+*1ln*FD|-=c7=x+ZwECe z7BECLtKXkZD7~=Sn-Q|5|0oWR;}(r24@9r;Wq*U(kAc&Vj9J8hF)G%nk57lpEr2zXeQ(r3oH0iO`mFpI4C9R34 zO(rD=v(lMRY#Mo$&5*wQchGyQ9y$7NTSMTVd+%aw9;fB!i#qkr%EEAm25<%?nN5s} zFO*EJ>XR5)f^_8KnZb0Lu}En~Z{CJ>jFqGCXl1&5Lfk9%RbN&(f0rXf`uS^_;o}q+ zjMW5RP>Lejxz=ky-JJYDtkZPsHvD|TF`q(G-|q@IE+vLTm}(Q9KeDPuDmifCV`hWvsXCrpwNr|l~=J%kgzg23X3a7?s%oDP9a z^!(*)FREh9fGW?;BCVzIuZT(xH%^`bKvsU#7@GaNI-1v^uIqN7ui!P?%hm58fMz-# zt^&VriPOHHZ|5|97T?VF*|A1+yLF|(NuyPhEaCZikH*kv%(J`sLbHM_ zP|oH&><6xAFM)38MsqvaYrcu(w9kGLvhD)Jumt~hDp)J4o|N`myrR`<(KY}ga0m8d zEnxh>X;T*%8F$?*(aZK`x8Q=;SxE|$0OXJFf?sR&CZu2Sd%sb<3^d-bBJ=R{9j3CXCnoy`X9N_hjmp zjN`*hJNPs5#%EDw}~;p%UwbA1pxgd{(nAEO!>Tq!{G1-T@v@Qm88@K7l~zj)Q=Csj3(C{`AwBYFc7quKeyptVi*PzZS=Y4xWi zoVqj$%F0A17pw15IgeyRdwFQv92xD(ygkt1k#87L!#i=Hg{7Bp|1BQf zd0}`5TDs0zP)=4{P^2mUegsKzrNw|IT(NQzjHC$(jtcuF#FV2M>SrfdG2|D-LjrY9 z-X+BUe2Wx+ewl+|0`3HBcOpUjTaWhBTc1O5@!1}83>|hi-_B0%lY;F*VQJnE;NNNE zlmjzc<27Q0`4NYo0tMM>=nE7LER#aO;Tp7B;oWbSANhMId$kws6w&ox#?ns($EFP9 z>$fkT|1jN3a*Je33C;5OQZ%Coq($5&${Vn1Wo{h>mixY6D*ZmAR>k~-e8&BeB7SN; zJs)=oH*4x(H+m!Yq})uotu%5_5ymgeu7#E@X8*9!Je^dm0M)rGGY3$346e&3g)28A z6}$c(>7@x8ip7^R0mh;$?iks!d8d>6R`?$wL6M`3E#Kr1Wv?U-Zf&Z8kg7UEiZN7h zcKtw2`A+U{2&1$%gH@r6WUP%gEpU)$}22SQHaQE5bp~$0jZx`UY z$62Sm6(iwHcGXj3*_58VoxJLF>ofG3$VRELJU5EnUNLpV6+m12?srUJIs<{|Rgf=z zf4+s)vgOVQH))2zB{YuE)^Y@nU_a4hl8U&c{*? zu=VLV796IOzUAdB7dXqRa~XslU_4{aU^&?}yQY_>XKd$fMX$hZx&i}T=&6?f1+0=; zim#tg!)Z){A{URN49OEJjy6%-O!>wu^>tLUqc)Vj)UO5|CP_@WJ|fJP9|=#bzYW|z zpJy^kcq~IM3?A#ahH|=l71XQq-1ha0R|=efcU<$mZ^3vwO|!H>f}$@&!|uTYHEt@B zZ`3LT7wZh~%b1zBi}%)`vUW5v;VE`}#V;W{V(QhKL}z7=pnxt#F8Xjs@XTSu^*fm; zX-UJIQ(Y|j*ukNg)J4wJ;3c^|py^vlI{@zMd>%La8wY7IiSi;4N6N?IX1WsqBjk%D zJgCvi-hY?MZzgnE3;&Eb4AZK4a8ef4GmCWGNR`?1hv6MiU)V>n{$(-%cf!?am?_g> zlHkkU`^;J_$qfAHcl?s$wjscs!Aq3c4=VLP7ZW?e=Ly9@%%$VC+d}CF7+)PVg6Q(J z2-CpGX5iAv-l+&`{SbgjqV#-?_tpXP^Iuiz<~7_=I7gWX}4BJXO)c8+Q~aJ|OKu_kRCbxM_lQeuLf za79lIW8}sC)Kxo^E5a=i(!s)!C3cn^te?@nn~}ni8~4vQ8RS;ZHLyivNQL%rX-2Ud z`5AlsKtd5t3ZH4NdHIarjT$??<=&clqp)3pfXdm(uG2%jLtE+hiY{XjrPU!6p%!NK$4$=u|T}lajfv6I(Y~qAa>2(rP)F{A%v$bC|{-kyko?f-lI`R9{(8893hA zg$5_+>=$8VvQ~e&&U`5b2mv!ZV>lRa0gTkLJ?1(TUAY~tl(1{4{;X;MXZNZ{*H^em zo^LQJji-Fceb1eAXid&6B$&-{c&5nlR))u3${ErY^vsm_3vR2%C3i_U>AQh+GJFm) z!)UU|8Z<9tQ@;|0y+OnF2;&BN!uB2Jam^nc9*dkGte5oUdkcyH_!JFQ9|%9iR=J`j8M5IFr@oN1{R}E%M;(~mZZ)ntEVZ& zT#E_^#xNN)WUszr zb{=%%-s-0ZLW`RTHXH_ZX6w!z^cCzC+?jc%$#SRGiHs|fLIA9hxgR@!$;*=0nUmGA zv8CB_u`6Xry+Njg*iMMFKBv^VB~&qcTh=)IATSpLD^%{ROHMqqR_wJOTi zvfX6Lmt+vz2mqv;)DiAOg+oUl?V&zJgUhxAdbLwucF_`-{5lwB2LcTl3Iz&&90Ie= zvN8oE=F%X*{6vIiw}?L$AKAF(tb`pHkcUe&IY<{P%eW>jjafvq3E1{Twl(tY)D*k7 z(jDoM*x@D{Gg{Eoh|ZKUnSEcvi-ND+t#3c@xj}7Px7FbdRy~8mAyEu!A9z#~j$JC0 znop5WpN9!+A|zth26x@eiV7b=bV9MK+6dkL%$^I4zdr8Wdp9}tpCr6|(i>hy$i?`S z7wC2m2EsO{LtS-liE8&388b#h@IsQR{Ffi1pcgH1fEhs8bv9t61>%Y)hJIzj=Da^oYi{Mo&X)za547{Yy0A2%61l6&Q9 zlH1)SxW}t>kB$7|hD|+5(@X%rU+<*1XM(sIwe8t{q$nz*D-j8k;_~Te!z$k@>wXa7 zkW#3GN8dcx!^=@jOOrt*P5icI3h-;4p{vTS3Jp8oT=ozs_T{euBM)CK4G@7+?;M!#69^E2C4)|R3D_WWK+!$;;$xusMZd%${@Iw6t5i@*Z;r-=Um z%N;j5nOK(lbjL}*SvgC1IvU1nJP>Rk;Y-HR2-C*RuV>_hVB^ zFspS7_>KB@n}a_r&=hlapdgewtC9`f&=S!sv=A<}aAeCK76kB?&}h?MSR4!j5SC~j zXnqygeDh}&tRRbYP$=Q!1A@r}kNv$s6>)#1c^)z_y;~9f4L!UIn&8-AcILz)lXKz@ zlS4Wg*4c&7s)Dep5zhikI!1H%*R3GEsj1K*BH3s7kz!uuvSIDTvq0hh+3XY_4N|EE2&U_8?suK*Y$Wmrb8Z;*~BDfNLTuANrd`n)ea3&zgMx+IG?lb_4lh^eE-3vt#QgM{C~oc{Xf%q>(A7e z*%OY&*Fx#9RXYrW6KE>77wY;E#D`0Dms7M1m< z!p3^8h%nKcE_BYNR__EbrFP#TVidOk)1QRZ>rVouhL-;VIOlr_;m{g|c!0=IT4}EQ z2$Sg+S=R%=w%zmtMCr@ziV+$XJuor!t+2~L3 zj9EOWTbTZjrwXcPGIhbdwUuHq*K%AHQW>oq7kfJ-K+C?%6-XUzK|>9oV)0MyGL4sB zhJ#n<#LV9aUW#5;|5;#%aL(fFcudbuGv>1b%9Kx_aE)g4Y@-}sAZwv2x<*6E0pSq5 zm~_i0=#($m4(B!s(%}|HLE=?4k`m3Cc=NR@q9asTgahCo+Vnd|4l93HQX0#R$dv$L zJz;*cc!M+ykj~(4*{}|r+W@0B7~q5uad5igiHZmJSk-Z zDhmoS75CMNJFJ18tto?o#8SiQgLQ~d{Vr(E3`drb2qlr^A z>hEK59$V}~V3yx6<;W;BUmN5YDSz~$RcU`Oi)FD`@u`7?V)6Uu{x8D+aH%l9c^>!< zU38mvZ5ybT6sdxBh*0jcsF~Lje zNi23m)shXt0R_DUZ#3G_cE5xQcYHwu>pg+JpRvkW{10tl!_2(u zbiki#|IHA}`_*}*k8XT+zb(Z7ScpIDM+C0z1gziwnG-vM1U}e^X|OmpYJQ;%+>-7% zMepEx^I*Rw)CCbZ6A_I9lkNV}oJH*45|!E_6WNfzc{M@(E52*%l??w7=}@!chuRnF zG#Ut5l_N}&WMN8AOGg-J%q54>XfzK9RvBx>vtyt0mF3MqOOYf4mk zFiy3Z66|iLW?y)xyIA-Wbqg)HF#0b1t9g6xkvgeDH`q$cO8E;d%8zoH2D!gcxsL02 zK<~JWnya?!xQUwE+mB0l@GEP|$$$9PapPXX$sVxI7_dIsAF$mNaCv#KSRwvsLHU#= zZu}P{rq>IU;vs7611uL8+DZ2z_Y(bzJfJKV&q!q{5dIU@gqf-^e~+aB-Udr!wi!MO zw|>T?9^?y^>)?eTfp#uBNNo1+o;-T+@1A_h3Khr2-`y}?=_d=QtuK~I_3J34bf^)Y zWq8O^5FO%@!lkHR=oS>pF+DOi$*05`g|>@&g>h|4T|`f+Aa>^jU=9jEZunmK`D^CA z%PJMA0syT0eX|xleA