Android Build Needs To Be Updated - Doc And Termex #66562

vindicatorr · 2022-03-14T00:16:33Z

Description

The documentation and script for cross-compiling Android seems to be out of date.

Reproduction Steps

$ NDK_DIR="<path_to>/23.1.7779620/" \
__lldb_Dir="<path_to>/23.1.7779620/toolchains/llvm" \
PATH="<path_to>/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/python3/bin/python3.9:$PATH" \
./eng/common/cross/build-android-rootfs.sh arm64 31

Expected behavior

Build on Arch Linux

Actual behavior

Target API level: 31
Target architecture: arm64
NDK location: <path_to>/23.1.7779620/
Target Toolchain location: <path_to>/runtime/.tools/android-rootfs/android-ndk-r21
Download dependencies...
cp: cannot stat '<path_to>/runtime/.tools/android-rootfs/tmp/arm64//data/data/com.termux/files/usr/*': No such file or directory

Regression?

No response

Known Workarounds

No response

Configuration

x64
Arch Linux

Other information

runtime/docs/workflow/building/coreclr/android.md

Line 17 in 51b90cc

cross/init-android-rootfs.sh

cross/build-android-rootfs.sh

runtime/eng/common/cross/build-android-rootfs.sh

Line 68 in 51b90cc

__lldb_Dir="$__CrossDir/lldb"

Is this not part of ndk (23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/bin/lldb)?
If this is the same kind of lldb, then I think it should just reference this if NDK_DIR is set.

runtime/eng/common/cross/build-android-rootfs.sh

Line 110 in 51b90cc

    
           for path in $(wget -qO- http://termux.net/dists/stable/main/binary-$__AndroidArch/Packages |\

https://termux.net/dists/stable/main/binary-aarch64/Packages results in {"code":"PAGE_NOT_FOUND","message":"Page not found"}
If the only packages that are needed are:

__AndroidPackages="libicu"
__AndroidPackages+=" libandroid-glob"
__AndroidPackages+=" liblzma"
__AndroidPackages+=" krb5"
__AndroidPackages+=" openssl"

... is there any reason why the sources can't be downloaded and just built along with the rest of it? What does Termex end up offering instead?

runtime/eng/common/cross/build-android-rootfs.sh

Line 115 in 51b90cc

wget -qO- http://termux.net/$path | dpkg -x - "$__TmpDir"

With dpkg, I'm guessing the script isn't meant for Arch Linux users. It would be good to have a "check" at the start of the script to determine if it should even continue.

EDIT0:

runtime/eng/common/cross/build-android-rootfs.sh

Line 68 in 51b90cc

__lldb_Dir="$__CrossDir/lldb"

I also commented out this line since I set the environment variable for it (IF it's supposed to work like that). I'm now going to go about manually doing the sysroot stuff, and then run the build.sh that the script shows at the bottom.

EDIT1:
https://github.com/termux/termux-packages/wiki/Package-Management
So maybe https://packages.termux.org/apt/termux-main/dists/stable/main/binary-aarch64/Packages now?

EDIT2:
I'm seeing

runtime/eng/common/cross/build-android-rootfs.sh

Line 17 in 51b90cc

    
           echo "By default, the NDK will be downloaded into the cross/android-rootfs/android-ndk-$__NDK_Version directory. If you already have an NDK installation,"

seem to counter with

runtime/eng/common/cross/build-android-rootfs.sh

Line 58 in 51b90cc

__CrossDir="$__ScriptBaseDir/../../../.tools/android-rootfs"

cross vs .tools
In addition to

runtime/eng/common/cross/build-android-rootfs.sh

Line 8 in 51b90cc

echo.

where it looks like it should be echo . //Note the space.

The text was updated successfully, but these errors were encountered:

dotnet-issue-labeler · 2022-03-14T00:16:37Z

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

ghost · 2022-03-14T13:39:02Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

The documentation and script for cross-compiling Android seems to be out of date.

Reproduction Steps

$ NDK_DIR="<path_to>/23.1.7779620/" \
__lldb_Dir="<path_to>/23.1.7779620/toolchains/llvm" \
PATH="<path_to>/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/python3/bin/python3.9:$PATH" \
./eng/common/cross/build-android-rootfs.sh arm64 31

Expected behavior

Build on Arch Linux

Actual behavior

Target API level: 31
Target architecture: arm64
NDK location: <path_to>/23.1.7779620/
Target Toolchain location: <path_to>/runtime/.tools/android-rootfs/android-ndk-r21
Download dependencies...
cp: cannot stat '<path_to>/runtime/.tools/android-rootfs/tmp/arm64//data/data/com.termux/files/usr/*': No such file or directory

Regression?

No response

Known Workarounds

No response

Configuration

x64
Arch Linux

Other information

runtime/docs/workflow/building/coreclr/android.md

Line 17 in 51b90cc

cross/init-android-rootfs.sh

cross/build-android-rootfs.sh

runtime/eng/common/cross/build-android-rootfs.sh

Line 68 in 51b90cc

__lldb_Dir="$__CrossDir/lldb"

Is this not part of ndk (23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/bin/lldb)?
If this is the same kind of lldb, then I think it should just reference this if NDK_DIR is set.

runtime/eng/common/cross/build-android-rootfs.sh

Line 110 in 51b90cc

    
           for path in $(wget -qO- http://termux.net/dists/stable/main/binary-$__AndroidArch/Packages |\

https://termux.net/dists/stable/main/binary-aarch64/Packages results in {"code":"PAGE_NOT_FOUND","message":"Page not found"}
If the only packages that are needed are:

__AndroidPackages="libicu"
__AndroidPackages+=" libandroid-glob"
__AndroidPackages+=" liblzma"
__AndroidPackages+=" krb5"
__AndroidPackages+=" openssl"

... is there any reason why the sources can't be downloaded and just built along with the rest of it? What does Termex end up offering instead?

runtime/eng/common/cross/build-android-rootfs.sh

Line 115 in 51b90cc

wget -qO- http://termux.net/$path | dpkg -x - "$__TmpDir"

With dpkg, I'm guessing the script isn't meant for Arch Linux users. It would be good to have a "check" at the start of the script to determine if it should even continue.

EDIT0:

runtime/eng/common/cross/build-android-rootfs.sh

Line 68 in 51b90cc

__lldb_Dir="$__CrossDir/lldb"

I also commented out this line since I set the environment variable for it (IF it's supposed to work like that). I'm now going to go about manually doing the sysroot stuff, and then run the build.sh that the script shows at the bottom.

EDIT1:
https://github.com/termux/termux-packages/wiki/Package-Management
So maybe https://packages.termux.org/apt/termux-main/dists/stable/main/binary-aarch64/Packages now?

EDIT2:
I'm seeing

runtime/eng/common/cross/build-android-rootfs.sh

Line 17 in 51b90cc

    
           echo "By default, the NDK will be downloaded into the cross/android-rootfs/android-ndk-$__NDK_Version directory. If you already have an NDK installation,"

seem to counter with

runtime/eng/common/cross/build-android-rootfs.sh

Line 58 in 51b90cc

__CrossDir="$__ScriptBaseDir/../../../.tools/android-rootfs"

cross vs .tools
In addition to

runtime/eng/common/cross/build-android-rootfs.sh

Line 8 in 51b90cc

echo.

where it looks like it should be echo . //Note the space.

Author:	vindicatorr
Assignees:	-
Labels:	`area-Infrastructure-coreclr`, `untriaged`
Milestone:	-

janvorli · 2022-03-14T23:13:30Z

@vindicatorr the Android rootfs generation and runtime build was a best effort thing at a time in the past. It is neither maintained nor supported in any way. The android package URLs has been changing a lot in the past, so it keeps breaking from time to time. But fixes from the community are welcome.

Also, in general, the script is meant to be used on Ubuntu or Debian only.

ghost · 2022-07-01T21:43:27Z

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

The documentation and script for cross-compiling Android seems to be out of date.

Reproduction Steps

$ NDK_DIR="<path_to>/23.1.7779620/" \
__lldb_Dir="<path_to>/23.1.7779620/toolchains/llvm" \
PATH="<path_to>/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/python3/bin/python3.9:$PATH" \
./eng/common/cross/build-android-rootfs.sh arm64 31

Expected behavior

Build on Arch Linux

Actual behavior

Target API level: 31
Target architecture: arm64
NDK location: <path_to>/23.1.7779620/
Target Toolchain location: <path_to>/runtime/.tools/android-rootfs/android-ndk-r21
Download dependencies...
cp: cannot stat '<path_to>/runtime/.tools/android-rootfs/tmp/arm64//data/data/com.termux/files/usr/*': No such file or directory

Regression?

No response

Known Workarounds

No response

Configuration

x64
Arch Linux

Other information

runtime/docs/workflow/building/coreclr/android.md

Line 17 in 51b90cc

cross/init-android-rootfs.sh

cross/build-android-rootfs.sh

runtime/eng/common/cross/build-android-rootfs.sh

Line 68 in 51b90cc

__lldb_Dir="$__CrossDir/lldb"

Is this not part of ndk (23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/bin/lldb)?
If this is the same kind of lldb, then I think it should just reference this if NDK_DIR is set.

runtime/eng/common/cross/build-android-rootfs.sh

Line 110 in 51b90cc

    
           for path in $(wget -qO- http://termux.net/dists/stable/main/binary-$__AndroidArch/Packages |\

https://termux.net/dists/stable/main/binary-aarch64/Packages results in {"code":"PAGE_NOT_FOUND","message":"Page not found"}
If the only packages that are needed are:

__AndroidPackages="libicu"
__AndroidPackages+=" libandroid-glob"
__AndroidPackages+=" liblzma"
__AndroidPackages+=" krb5"
__AndroidPackages+=" openssl"

... is there any reason why the sources can't be downloaded and just built along with the rest of it? What does Termex end up offering instead?

runtime/eng/common/cross/build-android-rootfs.sh

Line 115 in 51b90cc

wget -qO- http://termux.net/$path | dpkg -x - "$__TmpDir"

With dpkg, I'm guessing the script isn't meant for Arch Linux users. It would be good to have a "check" at the start of the script to determine if it should even continue.

EDIT0:

runtime/eng/common/cross/build-android-rootfs.sh

Line 68 in 51b90cc

__lldb_Dir="$__CrossDir/lldb"

I also commented out this line since I set the environment variable for it (IF it's supposed to work like that). I'm now going to go about manually doing the sysroot stuff, and then run the build.sh that the script shows at the bottom.

EDIT1:
https://github.com/termux/termux-packages/wiki/Package-Management
So maybe https://packages.termux.org/apt/termux-main/dists/stable/main/binary-aarch64/Packages now?

EDIT2:
I'm seeing

runtime/eng/common/cross/build-android-rootfs.sh

Line 17 in 51b90cc

    
           echo "By default, the NDK will be downloaded into the cross/android-rootfs/android-ndk-$__NDK_Version directory. If you already have an NDK installation,"

seem to counter with

runtime/eng/common/cross/build-android-rootfs.sh

Line 58 in 51b90cc

__CrossDir="$__ScriptBaseDir/../../../.tools/android-rootfs"

cross vs .tools
In addition to

runtime/eng/common/cross/build-android-rootfs.sh

Line 8 in 51b90cc

echo.

where it looks like it should be echo . //Note the space.

Author:	vindicatorr
Assignees:	-
Labels:	`untriaged`, `area-Infrastructure-mono`
Milestone:	-

directhex · 2022-07-06T14:57:12Z

Okay, this is an older community effort to produce a CoreCLR JIT on Android. I think @am11 may have views on it, but I don't know much about it.

janvorli · 2022-07-08T14:59:18Z

@directhex this was not just JIT, but the whole runtime running on Android under Termux. As I've mentioned above, it was brought to functional state for both arm and arm64 long time ago (long time before mono was in the runtime repo) by several community members with my help. But it has never been officially supported. Things mostly just worked due to the fact that the Termux basically mostly behaves like a regular Linux. The problem with the Termux has always been that the native libraries package URLs format / subpath change occasionally. So the rootfs building script eng/common/cross/build-android-rootfs.sh needs to be updated each time someone wants to build the rootfs and runtime on top of it.

am11 · 2022-07-08T15:28:57Z

The setup in build-android-rootfs.sh is indeed something which can be improved and incorporated in unified build-rootfs.sh. This setup goes hand-in-hand with cmake toolschain file (eng/common/cross/toolchain.cmake) with rest of our cross build matrix (./build.sh -cross -os ...). When it was last updated, I was able to build clr+mono+host+libs subsets with it. Things have changed a lot since, as @janvorli mentioned, termux URLs changed as well as the fact that we now explicitly set -p:CrossBuild=false for Android and disregard -cross argument because of the competing (manual) NDK setup used by mono.

It would be nice to consolidate these two parallel setups for Android. My personal preference is to update build-rootfs.sh and use -cross with Android, so it is easier to maintain (as we already have multiple platforms on that plan, including linux-bioinic). I opened #56622 for discussion.

ghost · 2022-08-05T18:04:17Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

The documentation and script for cross-compiling Android seems to be out of date.

Reproduction Steps

$ NDK_DIR="<path_to>/23.1.7779620/" \
__lldb_Dir="<path_to>/23.1.7779620/toolchains/llvm" \
PATH="<path_to>/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/python3/bin/python3.9:$PATH" \
./eng/common/cross/build-android-rootfs.sh arm64 31

Expected behavior

Build on Arch Linux

Actual behavior

Target API level: 31
Target architecture: arm64
NDK location: <path_to>/23.1.7779620/
Target Toolchain location: <path_to>/runtime/.tools/android-rootfs/android-ndk-r21
Download dependencies...
cp: cannot stat '<path_to>/runtime/.tools/android-rootfs/tmp/arm64//data/data/com.termux/files/usr/*': No such file or directory

Regression?

No response

Known Workarounds

No response

Configuration

x64
Arch Linux

Other information

runtime/docs/workflow/building/coreclr/android.md

Line 17 in 51b90cc

cross/init-android-rootfs.sh

cross/build-android-rootfs.sh

runtime/eng/common/cross/build-android-rootfs.sh

Line 68 in 51b90cc

__lldb_Dir="$__CrossDir/lldb"

Is this not part of ndk (23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/bin/lldb)?
If this is the same kind of lldb, then I think it should just reference this if NDK_DIR is set.

runtime/eng/common/cross/build-android-rootfs.sh

Line 110 in 51b90cc

    
           for path in $(wget -qO- http://termux.net/dists/stable/main/binary-$__AndroidArch/Packages |\

https://termux.net/dists/stable/main/binary-aarch64/Packages results in {"code":"PAGE_NOT_FOUND","message":"Page not found"}
If the only packages that are needed are:

__AndroidPackages="libicu"
__AndroidPackages+=" libandroid-glob"
__AndroidPackages+=" liblzma"
__AndroidPackages+=" krb5"
__AndroidPackages+=" openssl"

... is there any reason why the sources can't be downloaded and just built along with the rest of it? What does Termex end up offering instead?

runtime/eng/common/cross/build-android-rootfs.sh

Line 115 in 51b90cc

wget -qO- http://termux.net/$path | dpkg -x - "$__TmpDir"

With dpkg, I'm guessing the script isn't meant for Arch Linux users. It would be good to have a "check" at the start of the script to determine if it should even continue.

EDIT0:

runtime/eng/common/cross/build-android-rootfs.sh

Line 68 in 51b90cc

__lldb_Dir="$__CrossDir/lldb"

I also commented out this line since I set the environment variable for it (IF it's supposed to work like that). I'm now going to go about manually doing the sysroot stuff, and then run the build.sh that the script shows at the bottom.

EDIT1:
https://github.com/termux/termux-packages/wiki/Package-Management
So maybe https://packages.termux.org/apt/termux-main/dists/stable/main/binary-aarch64/Packages now?

EDIT2:
I'm seeing

runtime/eng/common/cross/build-android-rootfs.sh

Line 17 in 51b90cc

    
           echo "By default, the NDK will be downloaded into the cross/android-rootfs/android-ndk-$__NDK_Version directory. If you already have an NDK installation,"

seem to counter with

runtime/eng/common/cross/build-android-rootfs.sh

Line 58 in 51b90cc

__CrossDir="$__ScriptBaseDir/../../../.tools/android-rootfs"

cross vs .tools
In addition to

runtime/eng/common/cross/build-android-rootfs.sh

Line 8 in 51b90cc

echo.

where it looks like it should be echo . //Note the space.

Author:	vindicatorr
Assignees:	-
Labels:	`area-Infrastructure-coreclr`, `area-Infrastructure-mono`
Milestone:	7.0.0

Build and runtime pack for linux-bionic (Android without the Java part). I tried not to regress the existing CoreCLR Android build that is based on a crossrootfs (dotnet#56622) - the if's basically deal with that. Note that it's likely broken anyway (dotnet#66562).

Build and runtime pack for linux-bionic (Android without the Java part). I tried not to regress the existing CoreCLR Android build that is based on a crossrootfs (#56622) - the if's basically deal with that. Note that it's likely broken anyway (#66562).

dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Mar 14, 2022

SingleAccretion added the area-Infrastructure-coreclr label Mar 14, 2022

agocke added area-Infrastructure-mono and removed area-Infrastructure-coreclr labels Jul 1, 2022

steveisok removed the untriaged New issue has not been triaged by the area owner label Jul 5, 2022

steveisok added this to the 7.0.0 milestone Jul 5, 2022

steveisok assigned directhex Jul 5, 2022

steveisok unassigned directhex Aug 5, 2022

steveisok added area-Infrastructure-coreclr and removed area-Infrastructure-mono labels Aug 5, 2022

hoyosjs modified the milestones: 7.0.0, 8.0.0 Aug 10, 2022

MichalStrehovsky mentioned this issue May 26, 2023

Add NativeAOT build and runtime pack for linux-bionic #86781

Merged

truboxl mentioned this issue Jun 11, 2023

Package Request: .NET Core termux/termux-packages#516

Open

agocke modified the milestones: 8.0.0, Future Aug 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Android Build Needs To Be Updated - Doc And Termex #66562

Android Build Needs To Be Updated - Doc And Termex #66562

vindicatorr commented Mar 14, 2022 •

edited

Loading

dotnet-issue-labeler bot commented Mar 14, 2022

ghost commented Mar 14, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

janvorli commented Mar 14, 2022

ghost commented Jul 1, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

directhex commented Jul 6, 2022

janvorli commented Jul 8, 2022

am11 commented Jul 8, 2022

ghost commented Aug 5, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

Android Build Needs To Be Updated - Doc And Termex #66562

Android Build Needs To Be Updated - Doc And Termex #66562

Comments

vindicatorr commented Mar 14, 2022 • edited Loading

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

dotnet-issue-labeler bot commented Mar 14, 2022

ghost commented Mar 14, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

janvorli commented Mar 14, 2022

ghost commented Jul 1, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

directhex commented Jul 6, 2022

janvorli commented Jul 8, 2022

am11 commented Jul 8, 2022

ghost commented Aug 5, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

vindicatorr commented Mar 14, 2022 •

edited

Loading