Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Game crashes in libmonodroid.so Java_mono_android_Runtime_init #1673

Closed
huy2368 opened this issue May 11, 2018 · 4 comments
Closed

Game crashes in libmonodroid.so Java_mono_android_Runtime_init #1673

huy2368 opened this issue May 11, 2018 · 4 comments
Assignees
Milestone

Comments

@huy2368
Copy link

huy2368 commented May 11, 2018

Hi guys
I got a strange issue need your support
We have been developing a game.
We use Xamarin 8.3.99, NDK15c, SDK build tools 27.0.3.
I have an apk run on Samsung Note 10.1 but after I put all data (~30k files - over 930Mb) without compressing () into assets in apk, game crashes
Here is the native log. Im sorry, I will hide game package name.

I/MultiDex(14583): VM with version 2.1.0 has multidex support
I/MultiDex(14583): install
I/MultiDex(14583): VM has multidex support, MultiDex support library is disabled.
W/linker(14583): libmonodroid.so: unused DT entry: type 0x6ffffffe arg 0x5778
W/linker(14583): libmonodroid.so: unused DT entry: type 0x6fffffff arg 0x2
...
W/monodroid(14583): Trying to load sgen from: /data/app/<package_name>-1/lib/arm/libmonosgen-2.0.so
W/linker(14583): libmonosgen-2.0.so: unused DT entry: type 0x6ffffffe arg 0x13828
W/linker(14583): libmonosgen-2.0.so: unused DT entry: type 0x6fffffff arg 0x3
A/libc(14583): Fatal signal 11 (SIGSEGV), code 2, fault addr 0x3d6c4ade in tid 14583
I/DEBUG(2217): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG(2217): Build fingerprint: 'samsung/lt03wifixx/lt03wifi:5.1.1/LMY47X/P600XXUDQA1:user/release-keys'
I/DEBUG(2217): Revision: '5'
I/DEBUG(2217): ABI: 'arm'
I/DEBUG(2217): pid: 14583, tid: 14583, name:  >>> <package_name> <<<
I/DEBUG(2217): signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x3d6c4ade
I/DEBUG(2217):     r0 b4441600  r1 3d6c4ade  r2 00000384  r3 00000000
I/DEBUG(2217):     r4 00000404  r5 b464a2f8  r6 00000404  r7 0000ffff
I/DEBUG(2217):     r8 00000000  r9 00000000  sl 00000004  fp 00000404
I/DEBUG(2217):     ip b6de5648  sp beb335f8  lr b348f421  pc b6d8e4a0  cpsr 20070010
I/DEBUG(2217): backtrace:
I/DEBUG(2217):     #00 pc 000134a0  /system/lib/libc.so (__memcpy_base+96)
I/DEBUG(2217):     #01 pc 0000841d  /data/app/<package_name>-1/lib/arm/libmonodroid.so
I/DEBUG(2217):     #02 pc 0001177b  /data/app/<package_name>-1/lib/arm/libmonodroid.so
I/DEBUG(2217):     #03 pc 00007c43  /data/app/<package_name>-1/lib/arm/libmonodroid.so (monodroid_embedded_assemblies_register_from+142)
I/DEBUG(2217):     #04 pc 0000d181  /data/app/<package_name>-1/lib/arm/libmonodroid.so
I/DEBUG(2217):     #05 pc 0000ca25  /data/app/<package_name>-1/lib/arm/libmonodroid.so (Java_mono_android_Runtime_init+7532)
I/DEBUG(2217):     #06 pc 00324f17  /data/dalvik-cache/arm/data@app@<package_name>[email protected]@classes.dex

I dont know whether there is any limitation from Xamarin with big apk (~960Mb).
Could you guys give us some suggestions for this issue?
Thanks

@jonpryor
Copy link
Member

It would be interesting if you enabled assembly logging:

adb shell setprop debug.mono.log assembly

This will cause gather_bundled_assemblies_from_apk() to print out various log messages related to looking for assemblies within the .apk.

From the backtrace we see that memcpy(3) is called, so we're probably crashing here: https://github.com/xamarin/xamarin-android/blob/f5412f0efcabec586a02e75938b547e61e889c11/src/monodroid/jni/embedded-assemblies.c#L254

That doesn't explain why it's crashing, though. :-(

My guess would be that we're overflowing an integer, and my gut thinks the ((const char*) info->area)+ *offset expression may be problematic, but I'm not entirely sure. Overall, it should be safe, unless monodroid/jni/zip/unzip.c will try to read past the end of the "stream", which actually appears possible: https://github.com/xamarin/xamarin-android/blob/f5412f0efcabec586a02e75938b547e61e889c11/src/monodroid/jni/zip/unzip.c#L365-L370

When ((BUFREADCOMMENT+4) < (uSizeFile-uReadPos) is true -- meaning the current read position is < 4 bytes from the file size -- it'll try to read at least 4 bytes, which may very well be past the end of the file.

We should fix that, but I don't know if that'll fix your crash.

(We should probably also start using long long instead of int, but I also don't know if that'll do anything with your crash.)

@jonpryor jonpryor added this to the d15-9 milestone May 11, 2018
@jonpryor jonpryor self-assigned this May 11, 2018
@huy2368
Copy link
Author

huy2368 commented May 12, 2018

Hi Jonathan
I turned on monolog, made two testcases and got some logs below. Please take a look on it

  1. Apk with uncompressed data (986Mb)
    undoable.txt
I/monodroid-assembly(14915):                        start: 0xffffffff  end: 0x3da16747  len:   1033987912        apk: /data/app/<package_name>-1/base.apk
A/libc(14915): Fatal signal 11 (SIGSEGV), code 1, fault addr 0x3da16343 in tid 14915
A/DEBUG(3657): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
A/DEBUG(3657): Build fingerprint: 'samsung/k3gxx/k3g:6.0.1/MMB29K/G900HXXS1CQD2:user/release-keys'
A/DEBUG(3657): Revision: '10'
A/DEBUG(3657): ABI: 'arm'
A/DEBUG(3657): pid: 14915, tid: 14915, name:  >>> <package_name> <<<
A/DEBUG(3657): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x3da16343
D/hwcutils(3643): MppFactory::MppFactory()
D/(3643): virtual LibMpp* MppFactory::CreateMpp(int, int, int, int) dev(6) mode(0) drm(0), 
D/libexynosgscaler(3643): LibMpp::LibMpp()
A/DEBUG(3657):     r0 ae3b9c00  r1 3da16343  r2 00000384  r3 00000000
A/DEBUG(3657):     r4 00000404  r5 a9ecf7d0  r6 00000404  r7 0000ffff
A/DEBUG(3657):     r8 00000000  r9 00000000  sl 00000004  fp 00000404
A/DEBUG(3657):     ip b6d4d5dc  sp bed98658  lr b31463b5  pc b6cee71c  cpsr 200f0010
D/libexynosgscaler(3643): virtual CGscaler::~CGscaler()
D/libexynosgscaler(3643): virtual LibMpp::~LibMpp()
D/(3643): virtual MppFactory::~MppFactory()
A/DEBUG(3657): backtrace:
A/DEBUG(3657):     #00 pc 0001771c  /system/lib/libc.so (__memcpy_base+96)
A/DEBUG(3657):     #01 pc 000083b1  /data/app/<package_name>-1/lib/arm/libmonodroid.so
A/DEBUG(3657):     #02 pc 00010e77  /data/app/<package_name>-1/lib/arm/libmonodroid.so
A/DEBUG(3657):     #03 pc 00007bd7  /data/app/<package_name>-1/lib/arm/libmonodroid.so (monodroid_embedded_assemblies_register_from+142)
A/DEBUG(3657):     #04 pc 0000c93d  /data/app/<package_name>-1/lib/arm/libmonodroid.so
A/DEBUG(3657):     #05 pc 0000c247  /data/app/<package_name>-1/lib/arm/libmonodroid.so (Java_mono_android_Runtime_init+5626)
A/DEBUG(3657):     #06 pc 00b914ff  /data/app/<package_name>-1/oat/arm/base.odex (offset 0x561000) (void mono.android.Runtime.init(java.lang.String, java.lang.String[], java.lang.String, java.lang.String[], java.lang.ClassLoader, java.lang.String[], java.lang.String[], java.lang.String)+274)
A/DEBUG(3657):     #07 pc 00b90477  /data/app/<package_name>-1/oat/arm/base.odex (offset 0x561000) (void mono.MonoPackageManager.LoadApplication(android.content.Context, android.content.pm.ApplicationInfo, java.lang.String[])+2122)
A/DEBUG(3657):     #08 pc 00b90c07  /data/app/<package_name>-1/oat/arm/base.odex (offset 0x561000) (void mono.MonoRuntimeProvider.attachInfo(android.content.Context, android.content.pm.ProviderInfo)+226)
A/DEBUG(3657):     #09 pc 0376009b  /system/framework/arm/boot.oat (offset 0x2f2f000)
  1. Compress data into data.7z and put it into Assets. Apk size 570Mb.
    runnable.txt
I/monodroid-gc(11004): environment supports jni NewWeakGlobalRef
I/monodroid-assembly(11004):                        start: 0x7a87f000  end: 0x9e29ad84  len:    597802372        apk: /data/app/<package_name>-1/base.apk
I/monodroid-assembly(11004): file-offset: 20b6fc18  start: 0x9b3eec18  end: 0x9b3f0a18  len:         7680  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b71e48  start: 0x9b3f0e48  end: 0x9b3ff048  len:        57856  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b86d98  start: 0x9b405d98  end: 0x9b406b98  len:         3584  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b87c94  start: 0x9b406c94  end: 0x9b407c94  len:         4096  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b88ed8  start: 0x9b407ed8  end: 0x9b408cd8  len:         3584  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b89dc8  start: 0x9b408dc8  end: 0x9b409bc8  len:         3584  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b8acac  start: 0x9b409cac  end: 0x9b40aaac  len:         3584  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b8bb9c  start: 0x9b40ab9c  end: 0x9b40b99c  len:         3584  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b8ca94  start: 0x9b40ba94  end: 0x9b40c894  len:         3584  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b8d984  start: 0x9b40c984  end: 0x9b40d784  len:         3584  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b8e86c  start: 0x9b40d86c  end: 0x9b40e66c  len:         3584  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 20b8f74c  start: 0x9b40e74c  end: 0x9b900d4c  len:      5187072  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 21081d88  start: 0x9b900d88  end: 0x9bcb3b88  len:      3878400  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 215245c4  start: 0x9bda35c4  end: 0x9bdc3dc4  len:       133120  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 21554880  start: 0x9bdd3880  end: 0x9bde7480  len:        80896  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 215684bc  start: 0x9bde74bc  end: 0x9bdec0bc  len:        19456  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 2156e860  start: 0x9bded860  end: 0x9be17060  len:       169984  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 215adbdc  start: 0x9be2cbdc  end: 0x9be4e7dc  len:       138240  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 215dc01c  start: 0x9be5b01c  end: 0x9be9801c  len:       249856  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 21619058  start: 0x9be98058  end: 0x9c1f4458  len:      3523584  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 21975498  start: 0x9c1f4498  end: 0x9c20b298  len:        93696  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 21996e14  start: 0x9c215e14  end: 0x9c276c14  len:       396800  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 219f7c58  start: 0x9c276c58  end: 0x9c27f058  len:        33792  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 21a032f0  start: 0x9c2822f0  end: 0x9c2844f0  len:         8704  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 21a05d88  start: 0x9c284d88  end: 0x9c2f4d88  len:       458752  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 21a97048  start: 0x9c316048  end: 0x9c33cc48  len:       158720  zip-entry:  assemblies/
I/monodroid-assembly(11004): file-offset: 21abdc84  start: 0x9c33cc84  end: 0x9c3d1684  len:       608768  zip-entry:  assemblies/Mono.Android.dll name: Mono.Android.dll [MZ......]
I/monodroid-assembly(11004): file-offset: 21b526bc  start: 0x9c3d16bc  end: 0x9c5ca0bc  len:      2066944  zip-entry:  assemblies/mscorlib.dll name: mscorlib.dll [MZ......]
I/monodroid-assembly(11004): file-offset: 21d4b0f4  start: 0x9c5ca0f4  end: 0x9c6206f4  len:       353792  zip-entry:  assemblies/System.Core.dll name: System.Core.dll [MZ......]
I/monodroid-assembly(11004): file-offset: 21da1728  start: 0x9c620728  end: 0x9c6df928  len:       782848  zip-entry:  assemblies/System.dll name: System.dll [MZ......]
I/monodroid-assembly(11004): file-offset: 21e60960  start: 0x9c6df960  end: 0x9c7c4760  len:       937472  zip-entry:  assemblies/System.Xml.dll name: System.Xml.dll [MZ......]
I/monodroid-assembly(11004): file-offset: 21f457ac  start: 0x9c7c47ac  end: 0x9c7c5fac  len:         6144  zip-entry:  assemblies/System.Runtime.Serialization.dll name: System.Runtime.Serialization.dll [MZ......]
I/monodroid-assembly(11004): file-offset: 21f46fe8  start: 0x9c7c5fe8  end: 0x9c7d99e8  len:        80384  zip-entry:  assemblies/System.Net.Http.dll name: System.Net.Http.dll [MZ......]
I/monodroid-assembly(11004): file-offset: 21f5aa24  start: 0x9c7d9a24  end: 0x9c7e8824  len:        60928  zip-entry:  assemblies/System.Xml.Linq.dll name: System.Xml.Linq.dll [MZ......]
I/monodroid-assembly(11004): file-offset: 21f69868  start: 0x9c7e8868  end: 0x9c7fb868  len:        77824  zip-entry:  assemblies/System.IO.Compression.dll name: System.IO.Compression.dll [MZ......]
I/monodroid-assembly(11004): file-offset: 21f7c8a4  start: 0x9c7fb8a4  end: 0x9c8260a4  len:       174080  zip-entry:  assemblies/Mono.Security.dll name: Mono.Security.dll [MZ......]
I/monodroid-assembly(11004): Package '/data/app/<package_name>-1/base.apk' contains 36 assemblies

@jonpryor
Copy link
Member

Thank you for the debug.mono.log=assembly output.

This is weird:

# ↓↓ bad
I/monodroid-assembly(14915): start: 0xffffffff end: 0x3da16747 len: 1033987912 apk: /data/app/<package_name>-1/base.apk
I/monodroid-assembly(11004): start: 0x7a87f000 end: 0x9e29ad84 len: 597802372 apk: /data/app/<package_name>-1/base.apk
# ↑↑ good

The above message is from here: https://github.com/xamarin/xamarin-android/blob/50bda06ca0d8c74dd950420b84323f16fcbb7002/src/monodroid/jni/embedded-assemblies.c#L374-L378

The start value is the result of calling mmap(2), which is 0xffffffff, which is the value of MAP_FAILED. Ergo, mmap() is failing.

...and we weren't doing any error checking around that. (Though I'm not sure what error checking we could do around that; if mmap() fails, there is currently no fallback path for process startup to find the assemblies to execute.)

Also weird is the len value of 1033987912, or the comma-inserted value of 1,033,987,912. Your "986MB" .apk is over 1GB in size, once installed onto the device.

I thus have an answer: your app is too big -- when running on 32-bit devices -- for our normal app startup code path, because it's too big for mmap to actually load the .apk into the process address space. (Not RAM, mind you; just into the process' address space.)

Furthermore, I can't think of an easy workaround. Using $(BundleAssemblies)=True sounded plausible, except that we always call gather_bundled_assemblies() / monodroid_embedded_assemblies_register_from(), even when assemblies are bundled: https://github.com/xamarin/xamarin-android/blob/50bda06ca0d8c74dd950420b84323f16fcbb7002/src/monodroid/jni/monodroid-glue.c#L2681

Perhaps you can use .obb files, possibly by using the Xamarin.Google.Android.Vending.Expansion.Downloader NuGet package (documentation).

@jonpryor jonpryor modified the milestones: d15-9, d16-1 Dec 15, 2018
mathieubourgeois added a commit to gameloft/xamarin-android that referenced this issue Dec 24, 2018
Xamarin.Android, on launch, needs to do a pass through the files in the apk to register assemblies (.dll/.exe), dll configuration files (.config), symbols (.mdb/.pdb) and bindings typemaps (.jm/.mj). To do this pass, the process was to mmap the apk in the process' adress space, then grab the memory offsets for each file and register them on each system as needed. However, mmap-ing the whole apk has the consequence of taking potentially a lot of adress space. Considering it is legal to have a big apk (even though you can't submit an apk of more than 100 Mb to the Play Store, you can still submit it somewhere else, or embed your data in the apk during the development process for simplicity's sake), having an apk of about ~800 Mb in size automatically crashes on launch in armeabi-v7a because it can't find a contiguous 800 Mb block of adress space. The following log is usually found when that issue hits:
`I/monodroid-assembly(14915): start: 0xffffffff end: 0x3da16747 len: 1033987912 apk: /data/app/<package_name>-1/base.apk`
However, we don't need to mmap the whole apk, only the files that we are actually registering. Therefore, refactor the registration code to not mmap the whole apk. Instead, open the apk regularly, then when we actually need one of the entries, mmap the area of the file in the apk from a page-aligned offset and use those mmap sections for registration instead.
Fixes dotnet#1673
jonpryor pushed a commit that referenced this issue Jan 3, 2019
Fixes: #1673

During process startup, Xamarin.Android looks through the `.apk` for
various files to register within mono and itself, including:

  * Assemblies (`.dll`/`.exe`)
  * Assembly configuration files (`.dll.config`)
  * Debug symbols (`.mdb`/`.pdb`)
  * Type Map files (`.jm`/`.mj`)

Such files are stored *uncompressed* within the `.apk` and are
aligned on 4-byte boundaries via `zipalign`.

In order to *use* these files, the entire `.apk` would be
**mmap**(2)'d into the process address space, so that e.g. the
assemblies could be registered directly with mono without copying the
assembly contents "elsewhere" (e.g. disk, RAM), and instead would be
demand-paged *into* RAM from disk as needed.

There is an unfortunate downside to this approach: `.apk` files can
contain lots of content which *isn't* of interest to mono/etc., such
as Android Assets and Resources (large MP4 video files?), and the
"`mmap()` everything!" approach means that all this *unneeded* data
is *also* `mmap()`ed into the process address space.

Which is what Issue #1673 triggered: a 930MB `.apk` file with lots of
"game assets" could not load within a 32-bit address space, because
there wasn't 930MB of free contiguous address space to use:

	I/monodroid-assembly(14915): start: 0xffffffff end: 0x3da16747 len: 1033987912 apk: /data/app/<package_name>-1/base.apk
	A/libc(14915): Fatal signal 11 (SIGSEGV), code 1, fault addr 0x3da16343 in tid 14915

`mmap()`ing the whole `.apk`, while effective, is overkill.

Update `gather_bundled_assemblies_from_apk()` so that instead of
loading the entire `.apk` with one `mmap()` call, we instead use
separate `mmap()` calls for each distinct file of interest within the
`.apk` -- on OS page-aligned memory -- so that we don't excessively
use process address space.
@jonpryor jonpryor modified the milestones: d16-1, d16-0 Jan 3, 2019
@jonpryor
Copy link
Member

jonpryor commented Jan 3, 2019

This fix will not be part of the forthcoming d16-0 P2 release, but I do want to include this in P3.

jonpryor pushed a commit that referenced this issue Jan 17, 2019
Fixes: #1673

During process startup, Xamarin.Android looks through the `.apk` for
various files to register within mono and itself, including:

  * Assemblies (`.dll`/`.exe`)
  * Assembly configuration files (`.dll.config`)
  * Debug symbols (`.mdb`/`.pdb`)
  * Type Map files (`.jm`/`.mj`)

Such files are stored *uncompressed* within the `.apk` and are
aligned on 4-byte boundaries via `zipalign`.

In order to *use* these files, the entire `.apk` would be
**mmap**(2)'d into the process address space, so that e.g. the
assemblies could be registered directly with mono without copying the
assembly contents "elsewhere" (e.g. disk, RAM), and instead would be
demand-paged *into* RAM from disk as needed.

There is an unfortunate downside to this approach: `.apk` files can
contain lots of content which *isn't* of interest to mono/etc., such
as Android Assets and Resources (large MP4 video files?), and the
"`mmap()` everything!" approach means that all this *unneeded* data
is *also* `mmap()`ed into the process address space.

Which is what Issue #1673 triggered: a 930MB `.apk` file with lots of
"game assets" could not load within a 32-bit address space, because
there wasn't 930MB of free contiguous address space to use:

	I/monodroid-assembly(14915): start: 0xffffffff end: 0x3da16747 len: 1033987912 apk: /data/app/<package_name>-1/base.apk
	A/libc(14915): Fatal signal 11 (SIGSEGV), code 1, fault addr 0x3da16343 in tid 14915

`mmap()`ing the whole `.apk`, while effective, is overkill.

Update `gather_bundled_assemblies_from_apk()` so that instead of
loading the entire `.apk` with one `mmap()` call, we instead use
separate `mmap()` calls for each distinct file of interest within the
`.apk` -- on OS page-aligned memory -- so that we don't excessively
use process address space.
@ghost ghost locked as resolved and limited conversation to collaborators Jun 8, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants