Cross-platform Compilation Example.

[singku]

This has been asked many times by SDK users, but there is currently no example. I'd like to use Github Pinned Issue feature to pin a cross-platform compilation example I did for more visibility (Compare to Github Wiki page) and easier communication with SDK users.

Host

Amazon EC2

Host OS

Ubuntu 18.0

Target Platform

Mips Linux

Cross-compilation toolchain

g++-multilib-mips-linux-gnu

SDK Version

1.7.235

Steps

• Install g++-multilib-mips-linux-gnu

    sudo apt install g++-multilib-mips-linux-gnu

• Create cross-compiled targets directory

    mkdir /home/ubuntu/cross_output

• Set up environment variables for make to use cross-compile binaries utilities

    export CROSS_COMPILE_BIN_PREFIX=/usr/bin/mips-linux-gnu
    export CROSS_COMPILE_HOME=/usr/mips-linux-gnu
    export CC=$CROSS_COMPILE_BIN_PREFIX-gcc
    export LD=$CROSS_COMPILE_BIN_PREFIX-ld
    export AR=$CROSS_COMPILE_BIN_PREFIX-ar
    export AS=$CROSS_COMPILE_BIN_PREFIX-as
    export NM=$CROSS_COMPILE_BIN_PREFIX-nm
    export RANLIB=$CROSS_COMPILE_BIN_PREFIX-ranlib
    export CPPFLAGS="-I$CROSS_COMPILE_HOME/include/"
    export CROSS_INSTALL_PREFIX=/home/ubuntu/cross_output
    export LDFLAGS="-L$CROSS_COMPILE_HOME/lib/ -L$CROSS_COMPILE_HOME/lib64/ -L$CROSS_INSTALL_PREFIX/lib/"
    export PATH=$CROSS_COMPILE_HOME/bin:$PATH

• Cross compile zlib

    wget https://zlib.net/zlib-1.2.11.tar.gz
    tar xvzf zlib-1.2.11.tar.gz
    cd zlib-1.2.11
    ./configure --prefix=$CROSS_INSTALL_PREFIX
    make
    make install

• Cross compile Openssl

    wget https://www.openssl.org/source/openssl-1.1.1c.tar.gz
    tar xvzf openssl-1.1.1c.tar.gz
    cd openssl-1.1.1c
    ./Configure --prefix=$CROSS_INSTALL_PREFIX linux-mips32
    make -j
    make install

• Cross compile curl

    wget https://curl.haxx.se/download/curl-7.65.1.tar.gz
    tar xvzf curl-7.65.1.tar.gz
    cd curl-7.65.1
    ./configure --prefix=$CROSS_INSTALL_PREFIX --host=mips32-linux
    make -j
    make install

• Create cross compilation toolchain scripts for SDK

    # file name: mips.toolchain.cmake
    SET(CMAKE_SYSTEM_NAME Linux)

    # specify the cross compiler
    SET(CMAKE_C_COMPILER /usr/bin/mips-linux-gnu-gcc)
    SET(CMAKE_CXX_COMPILER /usr/bin/mips-linux-gnu-g++)

    # where is the target environment
    SET(CMAKE_FIND_ROOT_PATH /home/ubuntu/cross_output)

    # search for programs in the build host directories
    SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
    # for libraries and headers in the target directories
    SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
    SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

• Cross-compile SDK

    git clone 
    cmake ../ -DBUILD_ONLY="s3;iam;sts" -DCMAKE_TOOLCHAIN_FILE="mips.toolchain.cmake" - DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=$CROSS_INSTALL_PREFIX -DCMAKE_PREFIX_PATH=$CROSS_INSTALL_PREFIX -DAUTORUN_UNIT_TESTS=OFF
    make -j

[abhijitnathwani]

Hi @singku

Thank you for this example. I have been stuck at cross-compiling the SDK for aarch64-linux.
I followed the steps as mentioned above, however, on running the cmake command, I get the error for pthread.

Host OS:
Ubuntu 16.04

Target
aarch64-linux

Toolchain
gcc-aarch64-linux-gnu

cmake command:

cmake /path/to/aws-iot/aws-sdk-cpp/ -DCMAKE_TOOLCHAIN_FILE=/path/to/aarch64.toolchain.cmake -DBUILD_ONLY="iot" -DBUILD_SHARED_LIBS=ON -DCMAKE_PREFIX_PATH=${CROSS_INSTALL_PREFIX} -DAUTORUN_UNIT_TESTS=OFF

Error:

[ 20%] Performing configure step for 'AwsCCommon'
-- The C compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/aarch64-linux-gnu-gcc
-- Check for working C compiler: /usr/bin/aarch64-linux-gnu-gcc -- works
--
--
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Check if compiler accepts -pthread
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   THREADS_PTHREAD_ARG (advanced)
For details see /path/to/open-curl-zlib-cross-compile/cmake_build_dir/.deps/build/src/AwsCCommon-build/TryRunResults.cmake

If I manually set the THREADS_PTHREAD_ARG to 0, then during make there are several header files issues.

Can you help figure out what is wrong here?

[singku]

@abhijitnathwani I just tried build for aarch64-linux on ubuntu16.04.

• I installed g++aarch64-linux-gnu We need a g++ compiler for CPP SDK
• I installed latest CMake 3.16 to get ride of the TRY_RUN() problem
• Changed linux-mips32 to linux-aarch64 for openssl
• Changed mips32-lnux to aarch64-linux for curl and added --with-ssl=$CROSS_INSTALL_PREFIX --with-zlib=$CROSS_INSTALL_PREFIX

=======================

• There was no problem doing a static build -DBUILD_SHARED_LIBS=OFF
• When doing dynamic build, if I disable testing by -DENABLE_TESTING=OFF it is also build fine. Because this will not link executable.
• There seems to be some problem with aarch64-linux-gnu-ld, which can't find libcurl.so, libssl.so, etc automatically for me when linking an test executable. Normally those libs will be resolved automatically by linker when handling libaws-cpp-sdk-core.so. But I think this won't be a problem because what you need are sos, you can always specify missing libs in your own building scripts when linking your own executable.
• While set CMAKE_INSTALL_RPATH_USE_LINK_PATH to true in SDK can fix this issue.

I have created a PR in our private repo to fix the above issue

[ahmedyarub]

btw, I'm adding VCPKG support for cross-platform compilation of this SDK. Please check my PRs!