Merge #1292

1292: Experimental Support for Android (x86_64 and AArch64) r=syrusakbary a=igrep

# Description

Related: #1113

# Current Status

- ~~This pull request is a draft until the next version of libc crate (which should contain rust-lang/libc#1622
    - Now released! 🎉
- I confirmed the tests of wasmer-runtime-core pass on Android x86\_64.
    - The other tests including ones on AArch64 seems too hard so far... 😓 

# Review

- [x] Add a short description of the the change to the CHANGELOG.md file

# Note

I'm happy given any hint to test of Android specific code (esp. in libc I added in rust-lang/libc#1622).


Co-authored-by: Yuji Yamamoto <[email protected]>
Co-authored-by: YAMAMOTO Yuji <[email protected]>
Co-authored-by: Mark McCaskey <[email protected]>

.azure/install-rust.yml

@@ -45,6 +45,10 @@ steps:
       rustup default $RUST_TOOLCHAIN
       rustup target add x86_64-unknown-linux-musl
 
+      if [ -n "$ANDROID" ]; then
+        rustup target add x86_64-linux-android --toolchain $RUST_TOOLCHAIN
+      fi
+
       rustc -Vv
       cargo -V
     displayName: Install Rust

.gitignore

@@ -8,3 +8,7 @@ install/
 capi/
 api-docs/
 api-docs-repo/
+
+# Generated by tests on Android
+/avd
+/core

CHANGELOG.md

@@ -4,6 +4,7 @@
 
 - [#1320](https://github.com/wasmerio/wasmer/pull/1320) Change `custom_sections` field in `ModuleInfo` to be more standards compliant by allowing multiple custom sections with the same name. To get the old behavior with the new API, you can add `.last().unwrap()` to accesses. For example, `module_info.custom_sections["custom_section_name"].last().unwrap()`.
 - [#1303](https://github.com/wasmerio/wasmer/pull/1303) NaN canonicalization for singlepass backend.
+- [#1292](https://github.com/wasmerio/wasmer/pull/1292) Experimental Support for Android (x86_64 and AArch64)
 - [#1305](https://github.com/wasmerio/wasmer/pull/1305) Handle panics from DynamicFunc.
 - [#1301](https://github.com/wasmerio/wasmer/pull/1301) Update supported stable Rust version to 1.41.1.
 - [#1300](https://github.com/wasmerio/wasmer/pull/1300) Add support for multiple versions of WASI tests: wasitests now test all versions of WASI.

Makefile

@@ -172,6 +172,9 @@ test-rest:
 
 test: spectests emtests middleware wasitests test-rest examples
 
+test-android:
+	ci/run-docker.sh x86_64-linux-android --manifest-path=lib/singlepass-backend/Cargo.toml
+	ci/run-docker.sh x86_64-linux-android --manifest-path=lib/runtime-core-tests/Cargo.toml
 
 # Integration tests
 integration-tests: release-clif examples

azure-pipelines.yml

@@ -46,6 +46,10 @@ jobs:
         linux:
           imageName: "ubuntu-16.04"
           rust_toolchain: nightly-2019-12-19
+        android:
+          imageName: "ubuntu-16.04"
+          rust_toolchain: nightly-2019-12-19
+          ANDROID: true
         mac:
           imageName: "macos-10.14"
           rust_toolchain: nightly-2019-12-19
@@ -81,10 +85,13 @@ jobs:
         condition: and(succeeded(), eq(variables['Agent.OS'], 'Darwin'))
       - bash: make test
         displayName: Tests (*nix)
-        condition: and(succeeded(), not(eq(variables['Agent.OS'], 'Windows_NT')))
+        condition: and(succeeded(), not(eq(variables['Agent.OS'], 'Windows_NT')), not(variables['ANDROID']))
+      - bash: make test-android
+        displayName: Tests (Android)
+        condition: and(succeeded(), variables['ANDROID'])
       - bash: make spectests-cranelift
         displayName: Tests (Windows)
-        condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
+        condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'), not(variables['ANDROID']))
 
   - job: Check
     pool:

ci/README.md

@@ -0,0 +1,3 @@
+# About this directory
+
+This directory is originally copied from [rust-lang/libc/ci](https://github.com/rust-lang/libc/tree/master/ci).

ci/android-install-ndk.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env sh
+# Copyright 2016 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+
+NDK=android-ndk-r19c
+curl --retry 20 -O https://dl.google.com/android/repository/${NDK}-linux-x86_64.zip
+unzip -q -d ndk ${NDK}-linux-x86_64.zip
+mv ./ndk/"$NDK"/* ./ndk/
+
+rm -rf ./${NDK}-linux-x86_64.zip

ci/android-install-sdk.sh

@@ -0,0 +1,73 @@
+#!/usr/bin/env sh
+# Copyright 2016 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+
+# Prep the SDK and emulator
+#
+# Note that the update process requires that we accept a bunch of licenses, and
+# we can't just pipe `yes` into it for some reason, so we take the same strategy
+# located in https://github.com/appunite/docker by just wrapping it in a script
+# which apparently magically accepts the licenses.
+
+SDK=4333796
+mkdir sdk
+curl --retry 20 https://dl.google.com/android/repository/sdk-tools-linux-${SDK}.zip -O
+unzip -q -d sdk sdk-tools-linux-${SDK}.zip
+
+case "$1" in
+  arm | armv7)
+    api=24
+    image="system-images;android-${api};google_apis;armeabi-v7a"
+    ;;
+  aarch64)
+    api=24
+    image="system-images;android-${api};google_apis;arm64-v8a"
+    ;;
+  i686)
+    api=28
+    image="system-images;android-${api};default;x86"
+    ;;
+  x86_64)
+    api=28
+    image="system-images;android-${api};default;x86_64"
+    ;;
+  *)
+    echo "invalid arch: $1"
+    exit 1
+    ;;
+esac;
+
+# Try to fix warning about missing file.
+# See https://askubuntu.com/a/1078784
+mkdir -p /root/.android/
+echo '### User Sources for Android SDK Manager' >> /root/.android/repositories.cfg
+echo '#Fri Nov 03 10:11:27 CET 2017 count=0' >> /root/.android/repositories.cfg
+
+# Print all available packages
+# yes | ./sdk/tools/bin/sdkmanager --list --verbose
+
+# --no_https avoids
+# javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: No trusted certificate found
+#
+# | grep -v = || true    removes the progress bar output from the sdkmanager
+# which produces an insane amount of output.
+yes | ./sdk/tools/bin/sdkmanager --licenses --no_https | grep -v = || true
+yes | ./sdk/tools/bin/sdkmanager --no_https \
+        "emulator" \
+        "platform-tools" \
+        "platforms;android-${api}" \
+        "${image}" | grep -v = || true
+
+echo "no" |
+    ./sdk/tools/bin/avdmanager create avd \
+        --name "${1}" \
+        --package "${image}" | grep -v = || true

ci/android-sysimage.sh

@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+
+# Copyright 2017 The Rust Project Developers. See the COPYRIGHT
+# file at the top-level directory of this distribution and at
+# http://rust-lang.org/COPYRIGHT.
+#
+# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
+# http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
+# <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
+# option. This file may not be copied, modified, or distributed
+# except according to those terms.
+
+set -ex
+
+URL=https://dl.google.com/android/repository/sys-img/android
+
+main() {
+    local arch="${1}"
+    local name="${2}"
+    local dest=/system
+    local td
+    td="$(mktemp -d)"
+
+    apt-get install --no-install-recommends e2tools
+
+    pushd "${td}"
+    curl --retry 5 -O "${URL}/${name}"
+    unzip -q "${name}"
+
+    local system
+    system="$(find . -name system.img)"
+    mkdir -p ${dest}/{bin,lib,lib64}
+
+    # Extract android linker and libraries to /system
+    # This allows android executables to be run directly (or with qemu)
+    if [ "${arch}" = "x86_64" ] || [ "${arch}" = "arm64" ]; then
+        e2cp -p "${system}:/bin/linker64" "${dest}/bin/"
+        e2cp -p "${system}:/lib64/libdl.so" "${dest}/lib64/"
+        e2cp -p "${system}:/lib64/libc.so" "${dest}/lib64/"
+        e2cp -p "${system}:/lib64/libm.so" "${dest}/lib64/"
+    else
+        e2cp -p "${system}:/bin/linker" "${dest}/bin/"
+        e2cp -p "${system}:/lib/libdl.so" "${dest}/lib/"
+        e2cp -p "${system}:/lib/libc.so" "${dest}/lib/"
+        e2cp -p "${system}:/lib/libm.so" "${dest}/lib/"
+    fi
+
+    # clean up
+    apt-get purge --auto-remove -y e2tools
+
+    popd
+
+    rm -rf "${td}"
+}
+
+main "${@}"

ci/docker/aarch64-linux-android/Dockerfile

@@ -0,0 +1,53 @@
+FROM ubuntu:19.04
+
+RUN dpkg --add-architecture i386 && \
+    apt-get update && \
+    apt-get install -y --no-install-recommends \
+  file \
+  curl \
+  ca-certificates \
+  python \
+  unzip \
+  expect \
+  openjdk-8-jre \
+  libstdc++6:i386 \
+  libpulse0 \
+  gcc \
+  libc6-dev \
+  make \
+  cmake # cmake is necessary to build wabt
+
+WORKDIR /android/
+COPY android* /android/
+
+ENV ANDROID_ARCH=aarch64
+ENV PATH=$PATH:/android/sdk/tools:/android/sdk/platform-tools
+
+RUN sh /android/android-install-ndk.sh $ANDROID_ARCH
+RUN sh /android/android-install-sdk.sh $ANDROID_ARCH
+ENV ANDROID_NDK_HOME=/android/ndk
+
+RUN mv /root/.android /tmp
+RUN chmod 777 -R /tmp/.android
+RUN chmod 755 /android/sdk/tools/* /android/sdk/emulator/qemu/linux-x86_64/*
+
+ENV PATH=$PATH:/rust/bin:$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin \
+    CARGO_TARGET_AARCH64_LINUX_ANDROID_AR=aarch64-linux-android-ar \
+    CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER=aarch64-linux-android28-clang++ \
+    CC_aarch64_linux_android=aarch64-linux-android28-clang \
+    CXX_aarch64_linux_android=aarch64-linux-android28-clang++ \
+    CARGO_TARGET_AARCH64_LINUX_ANDROID_RUNNER=/tmp/runtest \
+    HOME=/tmp
+
+ADD runtest-android.rs /tmp/runtest.rs
+
+ENTRYPOINT [ \
+  "bash", \
+  "-c", \
+  # set SHELL so android can detect a 64bits system, see
+  # http://stackoverflow.com/a/41789144
+  "SHELL=/bin/dash /android/sdk/emulator/emulator @aarch64 -no-window & \
+   rustc /tmp/runtest.rs -o /tmp/runtest && \
+   exec \"$@\"", \
+  "--" \
+]

ci/docker/x86_64-linux-android/Dockerfile

@@ -0,0 +1,31 @@
+FROM ubuntu:19.04
+
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends \
+  ca-certificates \
+  curl \
+  gcc \
+  libc-dev \
+  python \
+  unzip \
+  make \
+  cmake # cmake is necessary to build wabt
+
+WORKDIR /android/
+ENV ANDROID_ARCH=x86_64
+COPY android-install-ndk.sh /android/
+RUN sh /android/android-install-ndk.sh $ANDROID_ARCH
+ENV ANDROID_NDK_HOME=/android/ndk/
+
+# We do not run x86_64-linux-android tests on an android emulator.
+# See ci/android-sysimage.sh for informations about how tests are run.
+COPY android-sysimage.sh /android/
+RUN bash /android/android-sysimage.sh x86_64 x86_64-24_r07.zip
+
+ENV PATH=$PATH:/rust/bin:$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin \
+    CARGO_TARGET_X86_64_LINUX_ANDROID_AR=x86_64-linux-android-ar \
+    CARGO_TARGET_X86_64_LINUX_ANDROID_LINKER=x86_64-linux-android28-clang++ \
+    CC_x86_64_linux_android=x86_64-linux-android28-clang \
+    CXX_x86_64_linux_android=x86_64-linux-android28-clang++ \
+    LD_LIBRARY_PATH=$ANDROID_NDK_HOME/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/x86_64-linux-android/ \
+    HOME=/tmp

ci/run-docker.sh

@@ -0,0 +1,35 @@
+#!/usr/bin/env sh
+
+# Small script to run tests for a target (or all targets) inside all the
+# respective docker images.
+
+set -e
+
+echo "${HOME}"
+pwd
+
+TARGET="${1}"
+shift
+
+echo "Building docker container for target $target"
+
+# use -f so we can use ci/ as build context
+image_tag=test-"$TARGET"
+docker build -t "$image_tag" -f "ci/docker/${TARGET}/Dockerfile" ci/
+mkdir -p target
+
+set -x
+
+docker run \
+  --rm \
+  --user "$(id -u)":"$(id -g)" \
+  --env CARGO_HOME=/cargo \
+  --env CARGO_TARGET_DIR=/checkout/target \
+  --volume "$(dirname "$(dirname "$(command -v cargo)")")":/cargo \
+  --volume "$(rustc --print sysroot)":/rust:ro \
+  --volume "$(pwd)":/checkout:ro \
+  --volume "$(pwd)"/target:/checkout/target \
+  --init \
+  --workdir /checkout \
+  "$image_tag" \
+  sh -c "HOME=/tmp PATH=\$PATH:/rust/bin exec cargo test --target ${TARGET} $@"