Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] How to run tests on iOS and Android #36297

Merged
merged 20 commits into from
May 15, 2020
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions docs/workflow/testing/libraries/testing-android.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Testing Libraries on Android

To build the tests and run them on Android (devices or emulators) you need the following prerequisites.

- [Android NDK](https://developer.android.com/ndk/downloads)
- [Android SDK](https://developer.android.com/studio)
- OpenJDK

(TODO: provide command-line steps to download them with dependencies)

Once SDKs are downloaded, set `ANDROID_NDK_ROOT`(CLARIFY: looks like `ANDROID_NDK_HOME` should also be set) and `ANDROID_SDK_ROOT`.
EgorBo marked this conversation as resolved.
Show resolved Hide resolved
Example:
```
export ANDROID_SDK_ROOT=/Users/egorbo/Library/Android/sdk
EgorBo marked this conversation as resolved.
Show resolved Hide resolved
export ANDROID_NDK_ROOT=/Users/egorbo/android-ndk-r21b
export ANDROID_NDK_HOME=$ANDROID_NDK_ROOT
```

Next, we need OpenSSL binaries and headers, we haven't properly integrated this dependency yet (it's an ongoing discussion) but there is a workaround:

- Download and unzip https://maven.google.com/com/android/ndk/thirdparty/openssl/1.1.1g-alpha-1/openssl-1.1.1g-alpha-1.aar
- Set these env variables:
```
GOOGLE_OPENSSL=/Users/egorbo/prj/openssl-1.1.1g-alpha-1.aar/prefab/modules
export AndroidOpenSslHeaders="$GOOGLE_OPENSSL/ssl/include"
export AndroidOpenSslCryptoLib="$GOOGLE_OPENSSL/crypto/libs/android.x86_64/libcrypto.so"
export AndroidOpenSslLib="$GOOGLE_OPENSSL/ssl/libs/android.x86_64/libssl.so"
```
**IMPORTANT:** make sure correct ABIs are used in the path, e.g. `-arch x64` -> `android.x86_64` (TODO: auto-detect)
EgorBo marked this conversation as resolved.
Show resolved Hide resolved

Now we're ready to build everything for Android:
```
./build.sh mono+libs -os Android -arch x64
```
and even run tests one by one for each test suite:
```
./build.sh libs.tests -os Android -arch x64 -test
```
Make sure an emulator is booted or a device is plugged and unlocked.
EgorBo marked this conversation as resolved.
Show resolved Hide resolved
**NOTE**: Xharness doesn't run any UI on Android and runs tests using headless testing API so the device/emulator won't show anything (but still must stay active).
EgorBo marked this conversation as resolved.
Show resolved Hide resolved
EgorBo marked this conversation as resolved.
Show resolved Hide resolved

### Running individual test suites
- The following shows how to run tests for a specific library
```
cd src/libraries/System.Numerics.Vectors/tests
~runtime/dotnet.sh build /t:Test src/libraries/System.Numerics.Vectors/tests /p:TargetOS=Android /p:TargetArchitecture=x64
EgorBo marked this conversation as resolved.
Show resolved Hide resolved
```

### Obtaining the logs
XHarness doesn't talk much and only saves tests result at the end to a file but you can also subscribe to live logs via the following command:
```
adb logcat -S DOTNET
```
Or simply open `logcat` window in Android Studio or Visual Stuido.

### Known Issues
EgorBo marked this conversation as resolved.
Show resolved Hide resolved
- We don't support `-os Android` on Windows yet (`WSL` can be used instead)
29 changes: 29 additions & 0 deletions docs/workflow/testing/libraries/testing-apple.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Testing Libraries on iOS and tvOS

In order to build librarires and tests for iOS or tvOS you only need some fresh version of XCode installed (e.g. 11.3 or higher).
EgorBo marked this conversation as resolved.
Show resolved Hide resolved

Build Librarires for iOS:
EgorBo marked this conversation as resolved.
Show resolved Hide resolved
```
./build.sh mono+libs -os iOS -arch x64
```
Run tests one by one for each test suite on a simulator:
```
./build.sh libs.tests -os iOS -arch x64 -test
```
Unlike XHarness-Android, XHarness for iOS is able to boot simulators by its own.
EgorBo marked this conversation as resolved.
Show resolved Hide resolved

### Running individual test suites
- The following shows how to run tests for a specific library on a simulator
```
cd src/libraries/System.Numerics.Vectors/tests
~runtime/dotnet.sh build /t:Test /p:TargetOS=iOS /p:TargetArchitecture=x64
```
for devices you need to specify `DevTeamProvisioning`:
EgorBo marked this conversation as resolved.
Show resolved Hide resolved
```
cd src/libraries/System.Numerics.Vectors/tests
~runtime/dotnet.sh build /t:Test /p:TargetOS=iOS /p:TargetArchitecture=arm64 /p:DevTeamProvisioning=...
```
AppleAppBuilder generates temp Xcode projects you can manually open and resolve provisioning there using native UI and deploy to your devices and start debugging.

### Known Issues
- Most of the test suites crash on devices due to [#35674)(https://github.com/dotnet/runtime/issues/35674)