Debug deploy fails if EmbedAssembliesIntoApk=true and AndroidSupportedAbis does not include arm64-v8a

[gkarabin]

Android application type

Classic Xamarin.Android (MonoAndroid12.0, etc.)

Affected platform version

Visual Studio Enterprise 2022 for Mac Version 17.3.8 (build 5) AND Microsoft Visual Studio Enterprise 2022 (64-bit) - Current Version 17.3.6

Description

My team has sample apps for a SDK (set of nugets) for classic Xamarin.Android that we distribute that embed native libraries (.so files) for armeabi-v7a but not arm64-v8a, so we build them with armeabi-v7a;x86;x86_64. We do not include "arm64-v8a" in that set. The apps run on modern ARM64 Android devices just fine.

This combination has worked well in both VS for Windows and Mac 2019 when deploying to a debugger with "fast assembly deployment" disabled. With most versions of VS, fast assembly deployment works, but older versions had issues, so we began disabling fast assembly deployment in the sample apps, leaving it to people who use our nugets to do what they want in their customized apps.

In Visual Studio 2022 (latest releases for Windows and Mac) I see that these apps fail to deploy through the debugger with "fast assembly deployment" disabled. I can replicate this behavior with a simple Xamarin.Android template project where I disable fast assembly deployment and remove "arm64-v8a" from the AndroidSupportedAbis .csproj property.

P.S I could create tickets on developercommunity.visualstudio.com, but given that the issue appears on both Windows and Mac, I expect that tooling in Xamarin.Android is shared and it makes sense to go straight here .

Steps to Reproduce

1. Create a new app in VS for Mac: Android->App->General
2. Remove "arm64-v8a" from AndroidSupportedAbis in the .csproj.
3. See that the app deploys.
4. Disable "fast assembly deployment" (setting EmbedAssembliesIntoApk=true in the .csproj file).
5. See that the app does not deploy.

Did you find any workaround?

Enabling fast deployment is a workaround. But it would be nice to have two working ways to deploy, in case fast assembly deployment breaks.

Relevant log output

Application output window:

Forwarding debugger port 8862
Detecting existing process
> am start -a "android.intent.action.MAIN" -n "com.companyname.abideploytest/crc6478e4fb5c678bec2f.MainActivity"
> Starting: Intent { act=android.intent.action.MAIN cmp=com.companyname.abideploytest/crc6478e4fb5c678bec2f.MainActivity }

[Zygote] Unable to open libbeluga.so: dlopen failed: library "libbeluga.so" not found.
[e.abideploytes] Late-enabling -Xcheck:jni
[e.abideploytes] Unquickening 17 vdex files!
[e.abideploytes] Using default instruction set features for ARM CPU variant (cortex-a9) using conservative defaults
[Perf] Connecting to perf service.
[NetworkSecurityConfig] No Network Security Config specified, using platform default
[NetworkSecurityConfig] No Network Security Config specified, using platform default
[monodroid] Creating public update directory: `/data/user/0/com.companyname.abideploytest/files/.__override__`
[e.abideploytes] Attempt to remove non-JNI local reference, dumping thread
[monodroid-debug] Trying to initialize the debugger with options: --debugger-agent=transport=dt_socket,loglevel=0,address=127.0.0.1:8862,server=y,embedding=1
[monodroid-assembly] Failed to load managed assembly 'mscorlib.dll'. Internal error
[monodroid-assembly] open_from_bundles: failed to load assembly mscorlib.dll
[mono] The assembly mscorlib.dll was not found or could not be loaded.
[mono] It should have been installed in the `/Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/sdks/out/android-armeabi-v7a-release/lib/mono/2.1/mscorlib.dll' directory.

[dellis1972]

@gkarabin

Out of interest what kind of device are you deploying too?

So the way this works is , with Fast Deployment on we ignore the AndroidSupportedAbis and look at the device which is being targeted. We then only include the assemblies for that device.

With Fast Deployment off, you end up with only the ABI's in the AndroidSupportedAbis list. So if you are deploying to a device which does support the abi's you have targeted at all then you should get a deployment error. Not a runtime one.
I tried to repo this using the steps outlined above and it works here some there must be some other different.

Here is the csproj https://gist.github.com/dellis1972/68d33b288753408970cffdda4e978a0e. Can you compare this with your example and see if you can spot the changes? Or upload the example. Also if you can let us know what device you are deploying too that would help us track this down too.

[ghost]

Hi @gkarabin. We have added the "need-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

[gkarabin]

Thanks, @dellis1972. I was using a Honeywell CT45. Dumping out some of the properties from the device's BuildInfo, I see:

Supported32BitAbis: [armeabi-v7a,armeabi], Supported64BitAbis: [arm64-v8a], SupportedAbis: [arm64-v8a,armeabi-v7a,armeabi]

Here is a diff using your example.csproj as the base, and the project I ended up with as the destination:
example-diff.txt. The only interesting differences that I notice are that I'm including x86 and x86_64 support. Removing that doesn't change behavior, though.

Here is the test app source that I'm using (a tiny git repo, shows the template-generated solution, followed by the edits):
abideploytest.zip

[dellis1972]

@gkarabin

Hmm. OK that example works on VSForMac 2022 for me.

I wonder can you get me the diagnostic build log for this particular device? I need to see what our build tasks "thinks" it is.

This command should do the trick. Make sure to uninstall the app first.

msbuild abideploytest.csproj /restore -t:Install -bl

It will produce an msbuild.binlog file which you can zip up and attach to this issue.

[ghost]

Hi @gkarabin. We have added the "need-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

[gkarabin]

Thanks! Here is the binlog: msbuild.binlog.zip. I am going to charge up some devices from other vendors and see if I can find a pattern.

[dellis1972]

so this is weird. arm64-v8a was not in the list of supported package but some native assemblies are being included in the apk.

                       /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/lib/armeabi-v7a/libmono-native.so
                       /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/lib/x86/libmono-native.so
                       /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/lib/arm64-v8a/libmono-native.so
                       /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/lib/x86_64/libmono-native.so
                       /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/lib/armeabi-v7a/libxamarin-debug-app-helper.so
                       /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/lib/x86/libxamarin-debug-app-helper.so
                       /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/lib/arm64-v8a/libxamarin-debug-app-helper.so
                       /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/lib/x86_64/libxamarin-debug-app-helper.so
                       obj/Debug/android/bin/classes.dex
                       obj/Debug/app_shared_libraries/armeabi-v7a/libxamarin-app.so
                       obj/Debug/app_shared_libraries/x86/libxamarin-app.so
                       obj/Debug/app_shared_libraries/x86_64/libxamarin-app.so

So the libmono-native.so is including the arm64-v8a version. So this might be a case where its trying to mix 32bit and 64bit libraries. I'll try to repo here locally and see if we can figure out why we are including the 64 bit ones when we shouldn't be.

[Adam--]

I am experiencing a similar problem and have been following your conversation to try and debug it and find a workaround for the issue. My issue is that when I deploy to my device, I get the "failed to load assembly mscorlib.dll" error when debugging with fast deployment disabled. Unfortunately, using fast deployment is not a viable solution for me because my app is a system app. I also encounter the same errors if I include arm64-v8a (I have tried including only armeabi-v7a and also all the others: arm64-v8a;armeabi-v7a;x86;x86_64). This is a simple Xamarin Forms app built from a template. I am using Windows with VS Enterprise 2022 17.5.0 Preview 2.0 and also Visual Studio Professional 2022 17.4.3, and I am encountering the same issue with both versions.

Here's the error:

[art] Attempt to remove non-JNI local reference, dumping thread
[monodroid-debug] Trying to initialize the debugger with options: --debugger-agent=transport=dt_socket,loglevel=0,address=127.0.0.1:8861,server=y,embedding=1
[monodroid-assembly] Failed to load managed assembly 'mscorlib.dll'. Internal error
[monodroid-assembly] open_from_bundles: failed to load assembly mscorlib.dll
[mono] The assembly mscorlib.dll was not found or could not be loaded.
[mono] It should have been installed in the `/Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/sdks/out/android-armeabi-v7a-release/lib/mono/2.1/mscorlib.dll' directory.

My hardware is pretty custom also. It's a single board computer using an ARM Cortex-A9 running Android 7.1 and it supports armeabi-v7a:

adb shell getprop

[ro.product.cpu.abi]: [armeabi-v7a]
[ro.product.cpu.abi2]: [armeabi]
[ro.product.cpu.abilist]: [armeabi-v7a,armeabi]
[ro.product.cpu.abilist32]: [armeabi-v7a,armeabi]
[ro.product.cpu.abilist64]: []

I ran msbuild abideploytest.csproj /restore -t:Install -bl command and poked around the output.
msbuild.binlog.zip

While I don't really know what I'm looking for, in Xamarin.Android.Common.targets I see

<AndroidSupportedAbis Condition=" '$(AndroidSupportedAbis)' == '' And '$(UsingAndroidNETSdk)' != 'True' ">armeabi-v7a;arm64-v8a</AndroidSupportedAbis>

In Xamarin.Android.Common.Debugging.props I see

<PropertyGroup>
    <BundleAssemblies Condition="'$(BundleAssemblies)' == ''">False</BundleAssemblies>
    <AndroidFastDeploymentType Condition="'$(AndroidFastDeploymentType)' == ''">Assemblies</AndroidFastDeploymentType>
    <AndroidSupportedAbis Condition=" '$(AndroidSupportedAbis)' == '' And '$(BundleAssemblies)' == 'True' And '$(EmbedAssembliesIntoApk)' != 'True' And '$(UsingAndroidNETSdk)' != 'True' ">arm64-v8a;armeabi-v7a;x86</AndroidSupportedAbis>
    <!-- If the user is using the shared runtime, we are going to
        partially override their supported Abis and support x86 as well -->
    <AndroidSupportedAbis Condition=" '$(AndroidSupportedAbis)' == '' And '$(EmbedAssembliesIntoApk)' != 'True' And '$(UsingAndroidNETSdk)' != 'True' ">arm64-v8a;armeabi-v7a;x86;x86_64</AndroidSupportedAbis>

It's interesting to note that if I switch to the new SDK-style project and target net7.0-android33, the issue goes away; however, there's still a lot of libraries out there that are not compatible with net7.0-android yet.

I hope this information helps with the issue. Is there anything else I can do to assist in resolving it?

[dellis1972]

@Adam--

I am curious as to why Fast Dev does not work with system apps? We specifically added support for that almost two years ago based on user feedback. The only restriction was you had to install/run as root but I guess as a system app that is a requirement anyway.

I tried to look at your binlog but the file seems to be corrupted and won't unzip. Any chance you can upload it again?

[Adam--]

I am curious as to why Fast Dev does not work with system apps? We specifically added support for that almost two years ago based on user feedback. The only restriction was you had to install/run as root but I guess as a system app that is a requirement anyway.

Using Fast deployment would be amazing, I'm not sure why it doesn't work, but the error message states it is not supported:

XA0137: The 'run-as' command failed with 'run-as: Package '…' is not an application
'.
Fast Deployment is not currently supported on this device.
Please file an issue with the exact error message using the 'Help->Send Feedback->Report a Problem' menu item in Visual Studio
or 'Help->Report a Problem' in Visual Studio for Mac.
Please set the 'EmbedAssembliesIntoApk' MSBuild property to 'true' to disable Fast Deployment in the Visual Studio project property pages, or edit the project file in a text editor.

To run as a system app, we set android:sharedUserId="android.uid.system" in the AndroidManifest.xml and sign the app with the keys used to sign the OS. Is there another way to do it or am I missing something?

I tried to look at your binlog but the file seems to be corrupted and won't unzip. Any chance you can upload it again?

Sorry about that, here you go:

msbuild.zip

[Adam--]

@dellis1972

I've been searching through your other discussions regarding Fast Deployment and have been trying to follow along there. I don't know if it helps, but I've found the following

• adb shell getprop ro.boot.disable_runas returns empty so it appears it's not disabled
• su 0 executes successfully with no password prompt
• Run-as states that the package is not an application

adb shell
lakeshore:/ # "run-as" "com.companyname.app3" pwd
run-as: Package 'com.companyname.app3' is not an application

• The obj\Debug...\android\AndroidManifest.xml has android:debuggable="true" in it
• I ran msbuild /restore /t:Install /bl /p:_FastDeploymentDiagnosticLogging=true and zipped up the msbuild.binlog:
  fastdeployment-msbuild.zip

It feels like we are straying off this original github issue, would it be better to switch over to new issue on developercommunity.visualstudio.com?

[Adam--]

Hey @dellis1972, I've submitted a new issue at https://developercommunity.visualstudio.com/t/Unable-to-debug-XamarinAndroid-system-a/10247269?

It seemed like the right place to report it, but I'm not too sure.

[dellis1972]

@Adam--

We could open a new issue for the fast dev/system apps bit on github if you like? I prefer GitHub over devcommunity lol.

The log you provided in #7521 (comment) seem to have fast deployment disabled? Was that the intention.

[dellis1972]

@gkarabin nope the fixes I posted look to be related to the origional ticket.

[gkarabin]

Thanks @dellis1972 - the patch is surprisingly small, but I guess that’s often the case!

[gkarabin]

#7658 (comment) puts it all together!

[Adam--]

@dellis1972, I'm having trouble following the fixes for these issues and I may not completely understand how all the parts fit together, so please forgive me if this is a silly question. Will this fix make its way into classic Xamarin.Android or only the newer .NET 7/8 Android (dotnet/runtime)?

[jlinker7]

Has anyone found a work around other than enabling Fast Deployment? Fast deployment does not work on the custom device I am developing for (when it tries to launch the APK it states the apk is corrupt). But, ever since upgrading to VS 2022 I cannot debug on my device. I get the following

[monodroid-debug] Trying to initialize the debugger with options: --debugger-agent=transport=dt_socket,loglevel=0,address=127.0.0.1:8876,server=y,embedding=1
[monodroid-assembly] Failed to load managed assembly 'mscorlib.dll'. Internal error
[monodroid-assembly] open_from_bundles: failed to load assembly mscorlib.dll
[mono] The assembly mscorlib.dll was not found or could not be loaded.
[mono] It should have been installed in the `/Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/sdks/out/android-armeabi-v7a-release/lib/mono/2.1/mscorlib.dll' directory.

I have tried with every combination of the support architectures, but no luck. Once it crashes because of the debugger I can launch the app again just fine, but of course cannot debug it. I have even updated to VS 2022 17.5.0 Preview 6.0 to get the latest and still not working.

[Adam--]

@jlinker7, you might want to try aab (this didn't work for me though). Other than that, there is no other work around besides fast deployment if you can get it to work. Your fast deployment corrupt apk error message is different than the error I had, so the fast deployment issue that @dellis1972 fixed probably won't work for your fast deployment issue. We just need to wait until the mscorlib fix makes its way into VS 2022.

[jlinker7]

@Adam-- Tried with bundle and received same error. Tried with Fast Deployment and bundle which netted a different error. But yes, if I try with apk and Fast Deployment I get...

Install FastDeployment Tools
XA0136: The 'run-as' command failed with 'run-as: Package 'com.lathem.payclock' has corrupt installation
'.
The currently installation of the package in corrupt. Please manually uninstall the
package from all the users on device and try again. If that does not work you can disable Fast Deployment.
Fast Deployment can be disabled in the Visual Studio project property pages or edit the project file in a text editor and set the 'EmbedAssembliesIntoApk' MSBuild property to 'true'.

Done executing task "FastDeploy" -- FAILED.

Don't suppose we have any idea how long it will be before the fix for the original issue makes it into VS 2022 do we? Really don't want to have to work without being able to debug for very long.

[Adam--]

No, I don't.

[dellis1972]

I'm not sure which release the mscorlib fix will be in since it was in the runtime side not the Android side.

@jlinker7 out of interest do you run your app in a multi user environment? The last time I was corrupt installation messages from run-as it was because the device had the same app installed under various different user profiles on the device.

[jlinker7]

@dellis1972 I think it is a Single user system. Its a custom device used as a time clock (our app runs and is the only thing running). When I go to users it shows You (Owner), and then it does show Guest, but it is disabled and I cannot tap on it or do anything with it. So, if it is an active second profile, it gives me no option to remove it. (Swiped down from the top and tapped the user profile, and it had the add guest option, I added a guest and the users switched to displaying the owner and the guest profile and then allowed me to remove guest, so the disabled guest option I am assuming it a place holder)

[dellis1972]

@jlinker7 Do you have any details on which version of android you are running on this device? Is it customized in anyway?
looking though the run-as source [1] I can't find the corrupt installation error message.

[1] https://github.com/aosp-mirror/platform_system_core/blob/master/run-as/run-as.cpp

[jlinker7]

@dellis1972 Yes, it is Android 7.1.1, and it is customized slightly (outsourced, not by us), the only customization is supposed to be the android network settings has a back arrow added to it (they can access that screen from within our app, but we have a back button as our app disabled the nav bar at the bottom).

[jlinker7]

@dellis1972 one other interesting thing I see in the output window is this...

Task Parameter:AllowDeltaInstall=False
Task Parameter:PreserveUserData=True
Task Parameter:FastDevToolPath=C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Xamarin\Android\lib
Task Parameter:ReInstall=False
Task Parameter:DiagnosticLogging=False
Task Parameter:PrimaryCpuAbi=armeabi-v7a

The path for the FastDevToolPath has an extra slash before the lib folder. I am not sure where this value is being set and why it has a double slash there, but I wonder if this could be causing an issue?

[dellis1972]

@jlinker7 The extra slash is not an issue, its been there for a while now. It gets stripped off within the Task.

It might be worth opening a new issues specifically for Fast Deployment on this. Then provide us with some diagnostic output. There is a special flag _FastDeploymentDiagnosticLogging which you can set either on the msbuild command line or in the csproj which will output more detailed diagnostic data.

For example

msbuild foo.csproj /restore -t:Install -p:_FastDeploymentDiagnosticLogging=true -bl

or is you are using .net6+

dotnet build foo.csproj -t:Install -p:_FastDeploymentDiagnosticLogging=true -bl

This will produce a msbuild.binlog file which contains all the data from the deployment. Make sure to do it on a clean build and when the app is not installed on the device. But I think this probably needs to be in a new issue since its not realated to this one (which has a fix on the way).

[jlinker7]

I will try to create a new issue when I get a chance, but if your curiosity gets the better of you before then, here is what was produced...
msbuild.zip

[jlinker7]

@dellis1972 new issue opened here...
#7821

[putto72]

I'm experiencing the same problem, Visual Studio 2022 version 17.5.33516.290. If I enable Fast Debug, I get the run-as problem. If I disable Fast Debug, I get mscorlib problem. I cannot debug anymore using VS 2022! When will this problem be fixed?

[dellis1972]

@grendello Did we fix this mscorlib loading bug? I can't see the fix anywhere. Was it in our stuff or upstream?

@putto72 did run-as work before on the same device? I'd be interested in knowing if we regressed in that area some how?
If that is the case please provide additional data as outlined here https://github.com/xamarin/xamarin-android/wiki/Diagnose-Fast-Deployment-Issues

[putto72]

@putto72 did run-as work before on the same device? I'd be interested in knowing if we regressed in that area some how? If that is the case please provide additional data as outlined here https://github.com/xamarin/xamarin-android/wiki/Diagnose-Fast-Deployment-Issues
Unfortunately I was using Visual Studio 2019 before, so I can't tell you if it worked before...

[dellis1972]

Unfortunately I was using Visual Studio 2019 before, so I can't tell you if it worked before...

Did Fast Deployment work on the same device in VS 2019?

[grendello]

@grendello Did we fix this mscorlib loading bug? I can't see the fix anywhere. Was it in our stuff or upstream?

It was kind of on both sides. It was assumption about MonoVM behavior on our side, and a bug regarding that behavior on MonoVM side. dotnet/runtime#80949 was the fix.

[bobbydharrell]

I see you have a fix merged in, when can we get our hands on it?

[jfichtner]

@grendello, @dellis1972 Would one of you guys mind clarifying what needs to be in place for us to be able to debug again? Is this just a VS2022 update on the horizon, or is the fix only going to apply to .NET 7/8? I'm hoping the former, because my team made an attempt to upgrade to .NET 7 a few months ago and found that it's extremely difficult to upgrade this without also upgrading to .NET MAUI, which (at the time) seemed incredibly buggy and unstable.

[jfichtner]

@grendello , @dellis1972 We still cannot debug our (xamarin-android 11) app using VS2022. Should the known issues above be fixed in the latest version of VS2022 + xamarin-android 11?

[Adam--]

To be honest, I'm not sure how all the pieces and versions fit together, but I was able to update to Visual Studio 2022 version 17.6.5 (I last tried version 17.5.5) and the Fast Deployment workaround now works. I still have the original "Failed to load managed assembly 'mscorlib.dll'" error with Fast Deployment disabled, but it's exciting to be able to use VS 2022 now!

The only thing I could find in the release notes were the following from 17.6.0

.NET MAUI
• Updates .NET MAUI to 7.0.86 (SR6), see https://aka.ms/dotnet-maui-releases for release notes.
• Updates to .NET for Android 33.0.46
• Updates to .NET for iOS 16.4.7060

[jpobst]

With support for Classic Xamarin.Android ended May 1st, 2024, this issue is likely no longer relevant.

If this still persists in .NET 8+, please open a new issue with updated information based on net8.0-android or greater. Please include a link to this issue for context.