Hello Alien

In this project I attempt to use SBCL in an android application.

Status: succeeded. (Yay!)

Huge thanks to karlosz, Shinmera, hayley and |3b| who helped me to debug the problem on the #sbcl channel!

Also thanks to fstamour for the idea.

You can check out the .apk in the artifacts folder. It works only on x86_64 and arm64-v8a cpu and it is not signed (but you still can install it without ADB).

Design

SBCL can be built as a shared library (see SBCL news v2.1.10). It must be compiled using Android NDK (see my fork of sbcl).

alien.lisp defines a function hello that simply returns one string. It exports it to the C wrapper with define-alien-callable.

The C wrapper hello-alien.c defines functions initLisp and getAlien that are used from Java.

Finally, the HelloActivity.java program defines the main activity for the app. It is just a button. When you press it, the app loads the C wrapper, initializes lisp and calls getAlien that calls the lisp hello function, and prints the result as the text on the button.

What didn’t work

The application proccess just died when it tries to initialize_lisp. Everything works with a standalone executable (that runs on android) though.

Why it didn’t work

SBCL tried initialize function hello that must be present in the C wrapper around the .core file, but couldn’t find it.

The problem was that SBCL looks for the symbol hello in the main process, which is the Java process instead of the C wrapper.

The solution is to load the libhello-alien.so library during the startup, adding a simple lambda to the *init-hook*.

Setting up Android SDK & Android NDK & emulator.

This section describes how to set up command line tools from the Android SDK without installing Android Studio.

It requires some free space on the disk (about 2-5 GB).

Download

You need to download Android SDK from the Android Studio’s download page. The link is in the bottom of the page.

After the extraction you will also need to restructure your folders. Command line tools are not bundled properly for some reason.

The desired structure is

.
├── android-sdk
│   └── cmdline-tools
│   │   └── tools
|   |   |   └── ...

Here is a little script that might be useful. It automatically downloads and restructures the folders.

mkdir android-sdk && cd android-sdk
# This link might be broken in the future.
# Replace it or download the .zip archive manually
wget https://dl.google.com/android/repository/commandlinetools-linux-9477386_latest.zip
unzip commandlinetools-linux-9477386_latest.zip
rm commandlinetools-linux-9477386_latest.zip
mv cmdline-tools tools
mkdir cmdline-tools
mv tools cmdline-tools

Environment variables

Next step is to setup your environment variables.

Here is a script for that. (Call with source.) It must be placed next to the android-sdk folder.

# setup.sh script
unset $CDPATH
SCRIPT_DIR="$(cd $(dirname ${BASH_SOURCE[0]}) >/dev/null 2>&1 && pwd)"

# These are required for command line tools.
export ANDROID_HOME="$SCRIPT_DIR/android-sdk"
export ANDROID_SDK_ROOT="$ANDROID_HOME"

# These tell emulator to store its data NOT in the $USER_HOME/.android folder.
# Unfortunately, some other tools will still store their data there.
export ANDROID_USER_HOME="$SCRIPT_DIR/.android"
export ANDROID_EMULATOR_HOME="$ANDROID_USER_HOME"
export ANDROID_AVD_HOME="$ANDROID_USER_HOME/avd"

# Used by SBCL / C wrapper build scripts.
# SBCL doesn't work with NDK version 22 and higher.
export NDK="$ANDROID_HOME/ndk/21.4.7075529"

export PATH="$PATH:$ANDROID_HOME/cmdline-tools/tools/bin"  # sdkmanager and avdmanager
export PATH="$PATH:$ANDROID_HOME/platform-tools"  # adb
export PATH="$PATH:$ANDROID_HOME/emulator"  # emulator

# That sets up tools used during the build of the .apk file.
# You might want to replace the version.
export PATH="$PATH:$ANDROID_HOME/build-tools/34.0.0-rc3"

I also put it into my .bashrc file for conveniece.

# in your .bashrc
source "path/to/the/setup.sh"

To install / find packages for Android SDK use sdkmanager program.

Needed packages

You need to install platform-tools, build-tools;30.0.3 and platforms;android-33 packages from sdkmanager:

sdkmanager --install "platform-tools" "platforms;android-33" "build-tools;30.0.3"

NDK

Install it with sdkmanager:

sdkmanager --install "ndk;21.4.7075529"

Later versions of NDK do not work with SBCL for some reason.

Emulator

Install it with sdkmanager:

sdkmanager --install "emulator"

To run the emulator you need to create an Android Virtual Device first.

Install the needed packages with sdkmanager:

# You can choose other versions.
# The SDK version (33 here) must be the same.
# See sdkmanager --list
sdkmanager --install "system-images;android-33;google_apis;x86_64"
sdkmanager --install "platforms;android-33"   

Create the AVD with avdmanager:

# You can use another name (-n flag).
# You can use different device (--device flag), list possible devices with
#   avdmanager list device
avdmanager -s create avd -f -n image \
           -k "system-images;android-33;google_apis;x86_64" \
           -p $ANDROID_AVD_HOME \
           --device "pixel_4"

Run the emulator:

emulator @image

And you can connect to the shell:

adb shell

Compiling the project

Additional requirements.

• Java version 17 It is needed for the gradle 8.1 (used as build system).
• An android device connected by adb. You can use an emulator instead. It is required to build the lisp code (and SBCL) yourself, but it is not required to build the ~.apk~ file.

.apk file

To build the .apk file:

./gradlew assembleDebug

To install through adb:

./gradlew installDebug

Or you can find the .apk file at build/outputs/apk folder.

SBCL / CL code / C wrapper

Use make-all.sh sript. You need to have exactly one android device / emulator connected through adb.

To customize the build you will need to modify the following scripts:

• SBCL Compiled by make-sbcl.sh script. It downloads the fork of the sbcl, compiles it and puts libsbcl.so to the libs/$ABI folder. You can pass additional flags to the make-android.sh invocation to customize SBCL build.
• CL code Compiled by make-core.sh script. The core alien.core must be compiled with the same sbcl as in libsbcl.so.
• C wrapper Compiled by make-c.sh script.

  You can pass -DFAKE flag to the first invocation to build a version that doesn’t initialize lisp.

Executable for adb shell

It was removed to simplify the project. It remained untouched in the with-executable branch/